⑴
Merge sort is an algorithm for sorting n numbers that I claim is going to be better than selection sort and bubble sort. It's got to be better because that n squared was killing us. We want to get something lower than n squared. So merge sort's pseudocode essentially looks like this.
And this is it. Merge sort says, if you've got n numbers or an array of numbers, sort the left half of them, sort the right half of them. And then merge the sorted halves.
And we'll see what this means in just a moment. But if there's only one number, just quit. So this is my base case because this is recursive. Because if this is a sorting algorithm called merge sort, I'm kind of cheating by using the verb sort in the sorting algorithm. But that just makes it recursive. It doesn't make it wrong. So what do I mean by merging ?
Just to make this clear, here among these numbers are two lists of size 4. And I'm going to scooch them over to the left and the right just to make clear that on the left is one half that is sorted 1346 from samllest to largest. On the right is a second half that's also sorted 0257. So for the sake of discussion, suppose that I'm partway through this algorithm called merge sort. And I've sorted the left half already, clearly, I've sorted the right half already, clearly. Now I need to merge the sorted halves. What do we mean by that ?
Well, that means to essentially, conceptually, point your left hand at the first at the left half. Point your right hand at the right half. And then decide which of these numbers should come first in order to merge or kind of stitch these two lists together in sorted order. Well, which is smaller ?
Obviously, 0. So that last step, merge sorted halves, would have me take this number and put it in an extra empty array up here. Now I move my right hand to the next number in the right half. And I compare the 1 and the 2.
Obviously, 1 comes next. So I put this now up here. Now I compare the 3 and the 2. Obviously the 2 comes next. And so I put this up here - - 3 and 5 - - obviously, 3 - - and 4, 5, 6. And I didn't leave quite enough room for 7 perhaps, and lastly, 7.
So that's all we mean by merging two lists together. If they're already sorted, you just kind of stitch them together by plucking from one or the other the next number that you actually want. And even though I picked up partway through this algorithm those three steps alone would seem to work. So long as you can sort the left half and sort the right half, you can surely them merge the sorted halves. Now I'll go ahead and do his digitally rather than use the physical numbers because clearly, it's a little involved moving them up and down.
But this rack here, this shelving, essentially presents one array with maybe a second array here. And heck, if I really want it, a third and a fourth array. It turns out that with selection sort and bubble sort, we've really been tying our hands because I only allowed myself a constant amount of memory, just one variable in my head, for instance, with selection sort that let me keep track of who was the smallest element. And when we did bubble sort, the only number I kept in mind was i, like i and i plus 1. I didn't allow myself any additional memory. But it turns out in programming and in CS, you can trade off one resource for another. So if you want to spend less time solving a problem, you've got to throw space at it. You've got to spend more space, spend more money, in order to give yourself more space to reduce your time. Conversely, if you're fine with things being slow in terms of time, then you can get away with very little space. So it's kind of like this balance whereby you have to decide which is more important to you, which is more expensive for you, or the like. So let's go ahead then and consider exactly this algorithm as follows.
So suppose that these are the number in question.
So here is our array. It's clearly unsorted. I'd like to sort this. I could use selection sort. But selection sort was not great because it's big O of n squared. And it's omega of n squared, so sort of damned if you do. Damned if you don't. Bubble sort was a little better. I was still big O of n squared. But sometimes, we could get lucky and shave some time off. So it was omega of only n. Let's see if merge sort is fundamentally better. And let's do so by trying to reduce the number of comparisons - - no more looping back and forth and back and forth and back and forth endlessly. So here's how we can draw inspiration from the idea of recursion and divide and conquer as per week zero.
Let's first think of these numbers are indeed in an array contiguously from left and right.
Let's then go ahead and sort the left half.
Because again, the pseudocode that you have to remember throughout this whole algorithm has just three real steps. Sort the left half. Sort the right half. Merge the sorted halves. And then this is sort of a one-off thing. But it's important.
So if we go back to this array on the top shelf, here's the original array.
And I'm going to do so by sort of stealing some more memory. So I can sort of work on a second shelf with just these numbers. So 6341 is now an array of size 4, essentially. How do I sort an array of size 4 ? Well, I've got an algorithm for that - - selection sort. But we decided that's slow. Bubble sort - - that's slow. I'm in the middle of defining merge sort. Let's recursively use the same algorithm by just sorting the left half of the left half. So if you've got an array of size 4, it's only three steps to solve it. Sort left half. Sort right half. Merge.
So let's do that. Let's sort the left half. How do you sort an array of size 2 ? I've got an algorithm for that. Sort the left half. Sort the right half. Merge the two halves.
All right, let's sort the left half. What's now ? So 6 is a list of size 1. What was that special base case then ? So it was quite or just return. Like, I'm already done. So this list of size of 1 is sort of weirdly already sorted. So I'm making progress. Meanwhile, what's the next step after sorting this left half ?
Sort the right half. This is already sorted. But here's the magic. What's the third step for these numbers ? Merge them. So this is like the left hand, right hand thing. Which one comes first ?
Obviously, 3 and then 6. And now we are making progress because now the list of size 2 is sorted. So what have I just done ? I've just finished sorting the left half of the left half. So after I've sorted the left half, what comes next if you rewind in time ?
Sort the right half of that left half. So now I take the rifht half. And how do I sort this right half of size 2 ?
Well, sort it's left half. Done.
Sort it's right half. Done.
Now merge them together. And obviously, the 1 comes first, then the 4. Now where are we ? We're at the point in the story where we have a left left half, sorted and a right left sorted. So what comes next now narratively ?
Merge the two together, so left hand, right hand. So 1346. And where are now in the story ? We're sort of at the beginning because I've now sorted the left half.
And you recall the demo I did physically, this is how I had the left half was already sorted on the second shelf.
But now that I've sorted the left half of the original array, what's the original second step ?
Sort the right half. So it's the same idea. So I'm borrowing some extra memory here. And now I've got an array of size 4. How do I sort an array of size 4 ?
Sort the left half. All right, how do I sort an array of size 2 ?
Sort the left half, done, sort the right half, done.
Merge the two together. And of course, it's 2 and 5. Now, what do I do ? I've sorted the left half of the right half. So now I sort the right half of the right half. How do I sort a list of size 2 ?
Well sort the left half. Sort the right half. Merge them together - - 0 and 7. Where am I in the story ?
I've sorted the left half and the right half of the original right half. So I merge these two together - - 0 and then 2 and then 5 and then 7. Where are we ?
We're at exactly the point in the story where we had the numbers originally on the second shelf. We had a list that's sorted on the left, a list that's sorted on the right. And what I demoed physically was merging those two together. So what happens now ?
- And it seems kind of magical and kind of weird in that I kind of cheated. And when I had these leaves - - these leaf nodes, so to speak - - these singletons - - I was, like, sorted. I wasn't really doing anything. But it's that merging that seems to really be doing the magic of sorting things for us. So that felt like a mouthful. And recursion is general is the kind of thing that bend your brain a little bit. And if that went over your head, like that's fine. It takes time and time and time and practice. But what there was not a lot of with this algorithm was again and again and again and again. I was kind of only doing things once. And then once I fixed a number, I moved on to the next. And that's sort of the essence of this algorithm here.
If it helps you to see it in another visual way. Let me go back to our previous visualization here. Let me re-randomize the array and click merge sort. And this time, notice that merge sort is using more space.
Technically, I was using 1, 2, 3 shelves. But you can actually be slightly more intelligent about it and actually just go back and forth between two shelves just to save a little space. So you're still using twice as much space. But you don't need four times as much space as the diagrams or as the shelves might imply.
So here is merge sort. And you'll notice that we're sort of working in halves - - sometimes, big halves, sometimes smaller halves.
But you can see as the two halves are merged, things seem to happen very quickly.
And so notice that this is the same number of bars as before. But that was way faster, right ? I don't need to stall nearly as much as I have in the past. So why is that ? Well, if we go back to the diagram in question, here's the array that's already sorted.
And if we consider now exactly how much work is done. I'll stipulate already it's not n squared, n squared was slow.
And merge sort is actually much better than that. Let's see how much better it is. So here is the original list - - 63415270. And here are all of the remnants of that algorithm, all of the states that we were in at some point - - sort of leaving a whole bunch of breadcrumbs. How many pieces of work did I do ? So I moved things around like, 24 times, it seems. And how do I actually reason about that ? Well, these are the numbers. This is like my temporary workspace here. And 24 is the literal number. But let's see if we can't do things a little more generically.
Log base 2 of n, recall, is what refers to anything that we're doing in half - - dividing, dividing, dividing. And that's kind of what I was doing here, I took a list of size 8 - - divide it into two of size 4, then 4 of size 2, then eight of size 1. Well, how many times can you do this if you start with eight numbers ? Well, that's log base 2 of 8. And if we do some fancy math, that's just log base 2 of 2 to the third. And remember, the base and the number here can cancel out. So that's actually 3, so log base 2 of 8 is 3, which means that's how many times you can divide a problem of size 8 in half, in half, in half.
But every time we did that per this chart, I had to take the numbers and merge them together, merge them together, merge them together. And so on every row of this postmortem of the algorithm, there are n steps, n steps, n steps. So laterally, there's n steps because I had to merge all of those things back together. But what is the height of these yellow remnants ? Well, it's 3, which is log base 2 of 8, which is 3. So this is technically three times 8, ergo 24 steps. But more generally, this is log n height and n width, so to speak.
So the total running time I claim is actually nlogn, which it's ok if that doesn't quite gel immediately in your mind especially if you're rusty on algorithms.
But and we can throw away the base because that's just a constant factor and with a wave of our hand when we talk about big O notation, merge sort, I claim is in big O of n log n - - that is, n times log n.
Unfortunately, it is also in omega of n log n, which means that, frankly, bubble sort might sometime outperform it, at least when the inputs are already sorted or certainly relatively samll. But that's probably ok, because in general data that we're sorting probably isn't very sorted. And honestly, we could even half merge sort to just do one pass to check initially, is the whole thing sorted and then maybe terminate early. So we can maybe massage the algorithm a little better to be a little smarter. But fundamentally, merge sort is in theta of nlogn is how you would say it. It's on the order of n times log n steps. Now, in terms of that chart, it's strictly higher than linear. But it's strictly lower than quadratic - - n and n squared, respectively. So it clearly seems to be faster. So it's not as good as linear search.
And it's definitely not as good as binary search. But it's way better than selection sort or bubble sort actually were.