Distributed Shared Memory Programming

November 9, 2012 Leave a comment

I spent my summer doing some kind of research on Distributed Shared Memory languages. And as a parallel programming fan, I must say I am impressed with the idea.

Assuming that you know some basic stuff about parallel programming, there are two different and more common models; distributed memory and shared memory model. As can be understood from the names they are used in different architectures, and have advantages and disadvantages accordingly.

Shared memory programming is used for smaller workstations where processing elements can reach any place at the memory with almost same latency. Distributed memory, on the other hand is used where total memory of the computer is distributed over multiple nodes, implying that some PEs can access some parts of the memory faster than the others.

Most common implementation of shared memory model is OpenMP. Programmer can define some block of the code as parallel and give some hints about parallelization and OpenMP handles the rest of the work. As memory is shared there is no need for explicit message passing for data transfer. This leads to a very simple parallelization process. However as this model requires a monolithic memory, computers you can program with this paradigm is relatively small.

Distributed memory, on the other hand does not have any theoretical limit for the number of cores you have or the amount of memory. However, since distributed memory systems are large scale systems which have memory modules at different locations, explicit message passing is required. Programmer must manage the data distribution and arrange the messaging between the nodes as well as the algortihm itself. In that sense, most common implementation of distributed memory, MPI, presents the programmer a local view of the code. Meaning that a program can directly reach only some part of the memory which is specific to some thread.

As can be understood from the other model, DSM tries to find the midpoint between the two most common parallel programming paradigms. To do that DSM presents programmer a global view, where he does not have to deal with message passing and can focus on the algorithm more. However, locality-aware code is required to get the best performance. That means, even though you are not responsible with handling the message passing, you need to know how to distribute the data among nodes and must access that data accordingly.

After you deal with some DSM languages, you get the feeling that previous programming methods you used are like assembly language where you need to fine tune every single detail of the code. However, there is a big drawback for DSM programming and that is of course performance.

Main aim of any type of parallel programming is without a doubt performance. Even though their performance shows serious increase all the time, DSM languages are still far away from being the fastest model. the reason is obvious I believe. Programmer does not have the full control over the code. The bottleneck of most parallel programs, memory access, is done by compiler and all DSM compilers still lack enough optimizations to beat MPI.

Parallel programming is still in its childhood period, and for large scale parallel programming it will still be in childhood for a while, because it is not easy to find resources. However, with the easy concepts presented by DSM model, the process of getting into adulthood will be easier for parallel programming as it makes writing parallel code easier thus reaching a wider community.


Thoughts After a Well-Spent Summer

September 12, 2012 Leave a comment

Well, looking at my last blogpost, one could imagine myriad of blogs posted containing lots of adventurous moments, my feelings about numerous places, recommendations for future tourists and all that sh*t. Even I was imagining about posting at least once a week, whether about usual travel stuff or some geeky topics. However, I did not post anything ever since.

Do I fill regret? Not a bit. Because I now know that I blog whenever I feel like I do not have anything to do -or I feel I am not social enough :D. Anyway, finally I feel like writing something, and hopefully this blogpost will be the start of regular blog postings :P.

About my travel; nothing could have been better. DC is a wonderful place, and according to the vast majority, DC is one of the best places to live in the US. Besides, me being grown up in a tiny city and in a very small community, I admit I liked DC more intimately. If you would ask, how much do I know about the other cities in US, I know none. Except that I have been to New York City twice and I spent 4 days there. Well, I must also admit that, NYC is a great city. But as other great cities, living there is much more complicated than one would think. And frankly, I would never want to live there(this sentence might be subject to deletion if some day, for some reason I must live in NYC). However, everyone who goes to the US must see New York.

About my work; besides my commute which takes 3 hours at least for round-trip, everything was also perfect. Although I leave geeky stuff for another post, I must say that I learned a lot both in terms of new concepts and academic practices.

I know that this post can only be an introduction to what I can say about my summer spent in the US. But leaving the details for some other future posts makes me feel like I am an important person whose fans are out there waiting impatiently for my upcoming posts 😛

Categories: Travel

At Washington, DC. Finally!

June 17, 2012 2 comments

I haven’t written any articles lately. I have been pretty busy with my school and my internship arrangement. Finally, everything(well, most of them) seems right for now, and I passed my first day in Washington, DC.

Starting from the very beginning; I picked Turkish Airlines since it makes a direct flight to DC from Istanbul at a reasonable price. Even though this is my first long flight, I must say, I am really impressed. The trip was comfortable all along, food was good, stewards were awesome.

After I had landed, I had few difficulties about my accommodation, it seems to be solved for now.

My first impressions about DC is pretty good. Actually, there is nothing I can say for now but the only thing I can say is I am happy to be here!

Stay tuned!

Categories: Travel

iPhone Speculators Please Stop Speculating These!

April 2, 2012 Leave a comment

Now that an amount of time has passed after Apple announced new iPad, people can focus on new iPhone rumors. Major topic is the screen size and another one that bugs me is DSLR lens mount, which is not as common. I have been keeping my ideas to myself but after I saw a design that has both of these, I thought that’s enough.

Read more…

Categories: Technology

Unified Communication Standard

March 30, 2012 1 comment

Advances in mobile technologies and people’s increasing addiction to mobile devices has caused lots of different networking technologies in last two decades. When this technologies are coupled with other wireless broadcasts, we see that there are lots and lots of different technologies that somehow connects two devices. But how long will these increase in the number of networking / broadcasting methods last?

Read more…

Categories: Technology

A Computer Architecture Study with SimpleScalar Instructor Kit

March 30, 2012 Leave a comment

SimpleScalar is a tool to simulate programs on different architectures. This tool gives in-depth information about the execution, which we can refer to while analyzing the architecture or the software. In its website, there are instructor benchmarks that are aimed to computer architecture courses. I ran these benchmarks and collected some interesting results, which demonstrates various performance issues on processor architectures.

Read more…

Computer Science, Software Engineering, Computer Engineering …… what the deuce?

March 10, 2012 2 comments

Advances in information technologies led many new specializations and job definitions spawn day by day. Two of the most important ones are arguably computer science(CS) and software engineering(SE). Apparently there are lots of misuses of these two concepts. Well, I don’t know if it is true to use the word “misuse” here, because CS and SE are not clearly as different as black and white.

I will try to find answers for the questions like; what these terms stand for? What each one does? Why do we have different names if they are mostly same and so on. And while doing this brainstorming I will share the things that fly around my brain. Meaning that, what you will read does not have any rock-solid basis 🙂

Read more…

Compiling GCC

March 7, 2012 Leave a comment

I have recently been working on GCC for a kind of research and first problem I encountered was to compile GCC from its source. Having compiled it, now it seems pretty straightforward. However I need to admit that it is really hard to find a complete guide to compile GCC especially for beginners who doesn’t compiled such a huge source before.

Read more…

Categories: Computer Science, How To