⑴
But for now, let's try to formalize the algorithms that both of these volunteers kind of intuitively came up with, first and second. And we'll slap some names on them.
So the first, and I've hinted at this, is what we would call linear search. Anytime you search from left to right or from right to left, it's generally called linear search. Why ? Because you're kind of walking in a line, no matter which direction you're going. But now for today's purpose, let's see if we can't truly formalize what our volunteer's algorithms were by translating them, not necessarily to code yet, but pseudocode. See if we can't map it to English-like syntax that gets the ideas across.
So I dare say, the first algorithm, even though she went from right to left, then from left to right, might look like this. For each door from left to right, she checked if 50 is behind the door as by looking at. If it was behind the door, then she returned true. That didn't happen on the first iteration, though. So she moved on again and again. And now notice the identation here is just as important as it was in week zero.
Notice that only at the very bottom of this algorithm, do I propose returning false. But it's not indented inside of this pseudocode. Why ?
Well, because if I had changed it to be this, what would be the logical bug in this version of that algorithm ? If she had opened the first door, found it to be the wrong number if it says else - - if it's not behind the door, then return false - - that would erroneously conclude the 50 is not there even though it could certainly be among those other doors. So this first version of the code where the return false is sort of left indented, so to speak, and the very last thing you do if you don't previously return true, that just make sure that we're handing all possible cases.
But let's make this maybe a little more technical. This is how a computer scientist or a programmer would likely express this instead. Instead of just drawing it in broad strokes, it's actually fine to kind of steal some of the syntax from languages like C and actually use some of the indices or indexes like 0, 1, 2, 3, 4, 5, 6, to represent the pieces of data we care about. So this a little more precise. For i - - like a variable i - - from the value 0 to n minus 1, so in the case of seven doors, this is like saying, for i starting at 0 going up to 6, do the following. If the number 50 is behind the doors array - - so I'm using array syntax, even though this is technically still pseudocode - - if the i-th location of my doors array has the number 50, return true. Otherwise, if you do that again and again and again - - n total times - - and you still don't find it , you want to return false. So we introduced this. This is just an example of how you can start to borrow ideas from actual code to paint the picture even more precisely of what it is you want a colleague to do, what it is you want your code to do, ultimately, by sort of borrowing these ideas from code and incorporating it into our pseudocode.
But what about the second algorithm here, the second algorithm, whereby he took a divide and conquer approach, starting in the middle and then going right and then going left. Well, it turns out this is generally called binary search. Bi implying two, because you're either going with the left half or the right half again and again. This is literally what we've been talking about since week zero when I searched the phone book. That too was binary search, dividing and dividing and dividing in half and half.
So if we were to draw some pseudocode for this, I would propose that we could do something like this. If 50 is behind the middle door then we got lucky. Just return true. Else if the 50 is less than the value at the middle door - - so if it's smaller than the middle door - - I want to search to the left. So I can say search left half. Else if 50 is greater than the middle door, I want to search to the right. And I think that's almost everything, right ? So taking into account that if there are no doors left or no doors to begin with, we better handle that case. So that we don't induce one of those spinning beach balls, so that the computer doesn't freeze or crash. There's really four possible scenarios in searching for information. It's either in the middle or to the left or to the right or it's just not there at all.
And so I sort of slip that in at the end because technically, that's a question you should ask first because if there's no doors, there's no work to be done. But logically, this is the juicy part - - the other three questions that you might ask yourself.
So this then might be the pesudocode for binary search.
And we could make it more technical. And this is where it kind of escalates quickly syntactically. But I'm just using the same kind of syntax. If doors in my pesudocode represents an array of doors. Well, then, doors bracket middle is just a pseudocode - like way of saying go to the middle door in that array. And then notice, else if 50 is less than - - that middle value - - then search doors bracket 0, so the leftmost one - - through doors middle minus 1. So you don't need to waste time researching the middle door. So I say middle minus 1 so that I scooch over slightly to the left, so to speak. Else if 50 is greater than the value at the middle door, then you want to search 1 over to the right, so middle plus 1 among those doors, through the last door, which is not n, because we start counting at 0, but n minus 1. And the rest of the algorithm is the same. This is just a little more precise. And I dare say, when writing a program in C or any other language, like, honestly, starting in pesudocode like this will generally make it much easier to write the actual code. So in fact, in this and future problem sets, do get into the habit, especially if you're struggling getting started - - just write things out in English and maybe high level.
English a little more like this. Then as a version two, go in with your keyboard or paper, pencil.
And make it a little more precise using some code-like syntax. And then I dare say in version 3, you can now translate this pretty much verbatim to C code. The only headache is going to be rounding issues with integers because if you divide an integer and you get a fraction, it's going to truncate. So all that kind of headache. But you can work through that by just thinking through what's going to get truncated when you round down or up as a solution.
⑵
What is the running time of these algorithm ? What is, that is to say, the efficiency of these algorithms ? And how do we actually measure the efficiency of an algorithm ? Is it with a stopwatch or with some other mechanism ?
Well, I propose that we think back to this picture here, whereby this, again, was representative of both the phone book, example from week zero and theoretically, bug aside, the attendance counting algorithm from earlier today, whereby this same green line theoretically represents how much time it should have taken us as a group to count ourselves. Because if maybe another class comes in and doubles the size of the number of humans in this room, no big deal. That's just one more step or one more iteration of the loop, because half of the people would anyway sit down. So this green algorithm still represents the faster theoretical algorithm today.
And so recall that we described these things more mathematically as n.
So this was one page at a time or one person at a time.
This was two people or two pages at a time. So it's twice as fast , so if n is the number of people or pages and divided by 2 is the total number of steps.
And then this one got a little mathy, but log base 2 of n. And log base 2 of n just means what is the value when you take n and divide it in 2, by 2 again and again and again and again until you're left with just one page or one person standing. But in the world of running times, it turns out that being this precise is not that intellectually interesting. And it sort of devolves into lower level math. It's just not necessary when having discussions about the efficiency of an algorithm or even code that you've written.
So generally, a computer scientist, when asked what's the running time of your algorithm or more generally how good or bad is your algorithm, they'll talk about it being on the order of some number of steps. This is a phrase you'll hear increasingly in computer science, like, the lower level details don't matter that much. All you care about in broad strokes are certain numbers that will add up the most. And in fact, when computer scientists talk about the efficiency of algorithms, they tend to throw away constant factors, so literally, numbers like 2 that might be dividing here or a base here.
So for instance, these two algorithms to a computer scientist would sort be the same. Like, yeah, it's technically twice as fast. But look at the lines, I mean, they're practically the same and this one here too, log base 2. But if you remember from math class, you can change the base of any logarithm from one number to another pretty easily. So let's just generalize it as log of n. It doesn't really matter fundamentally what the numbers actually are.
And honestly, if we zoom out slightly so that the y-axis and the x-axis get even bigger.
Honestly, these first two algorithm really do start to resemble each other closer and closer. And I dare say in your mind's eye, imagine zooming further and further and further out like, that red and yellow line are pretty much - - once n is large enough - - going to be functionally the same. Like, they're practically the same algorithm.
But this one is still doing amazingly because it's a fundamentally different shape. So this is to say when a computer scientist talks about, thinks about the efficiency of algorithms, we just throw away the constant terms that when n gets really large just don't seem to matter as much. They don't add up as much or fundamentally change the picture in a case like this.
So what I'm describing here with this capital letter O has a technical term. This is called big O notation. And this is omnipresent in computer science and often rears its head even in programming specifically when talking about the design of some algorithm.
And this is a little cheat sheet here on the screen now. Very often, algorithms that you write or you use will be describable as being on the order of one of these running times. So n is just representative of the number of things - - number of people - - number of pages - - whatever it is you're actually doing in code. So the mathematical formulas inside of the size of that input how fast or slow the algorithm's going to be.
So this algorithm in the middle here, by O of n, so to speak, means that it takes linear time, in other words. So my first algorithm - - 1, 2, 3, 4 - - or my first algorithm in week zero. That was a linear search. The number of steps it takes is on the order of n because if there's n pages, in the worst case, like, John Harvard's all the way at the end of the phone book, so it takes me n steps. In this case, it's always going to take me n steps to count you all because if I want to point at each and every one of you that is always going to take me n steps. So big O represents an upper bound on the number of steps that you might be counting. And so we often use it to consider the worst case, and the worst case John Harvard, or whoever might be all the way at the end of the phone book. So that linear search is on the order of n.
But what about n squared ? This means n people doing n things. Well, if there's n of you and you've got to shake everyone else's hand, that's technically n time n minus 1. Let's throw away the minus 1. That's n times n or n squared handshakes. That's a lot of handshakes. And so the running time of shaking everyone's hand in the room to introduce yourself would be on the order of n squared.
At the other end of the spectrum, the faster end, big O of 1 doesn't mean that the algorithm takes literally one step. It could take two steps or three or even 1000 steps. But what it means is it's a constant number of steps. So it doesn't matter how many people are in the room, this descirbes something taking just one step total or a constant number of steps total. So for instance, earlier when everyone stood up at the same time, that was constant time because if we had twice as many people comt into the room, it's not going to take us twice as long to stand up if everyone stands up at the same time. That would be a constant time algorithm. For now, just a common list of running times that we might apply to certain algorithms. So linear search, I claim is in big O of n because it's going to take in the worst case n steps. What about binary search ?
How many steps does binary search take on the order of according to this chart ? Log n, because no matter what with binary search, you're dividing in half, half, half, half. But in the worst case, it might be in the last door your check. But you only took log n steps to get there. But it still might be the last one you check. So binary search indeed would be on the order of log n. But sometimes you do get lucky and just fing things quicker. So we don't always want to talk about things in terms of an upper bound, like how many steps in the worst case might this algorithm take. Sometimes it's useful to know in the best case how few steps might an algorithm take.
So for that, we have this capital Greek omega, which is another symbol in computer science. And whereas big O represents upper bound, omega represents lower bound. And it's the exact same idea. It's just a different symbol to represent a different idea, the opposite, in this case.
So here's a similar cheat sheet here. But when you use the omega symbol, that just means that this algorithm might take as few as this many steps, for instance, in the very best case. So by that logic, if I ask about linear search, our first demonstration with the lockers, let's consider the best case. How many steps might it take to search n lockers using linear search in the best case ? You get lucky. Yeah, just one step.
So we could say that linear search is an omega of 1, so to speak. What about binary search ? If you've got n lockers, in the best case, though how few steps might it take us ?
Again, one, because we might just get lucky. And boom, it happens to be right there in the middle. So you could say that both linear search and binary search are an omega of 1. Now, by contrast, my attendance algorithm, the first one I proposed - - 1, 2, 3, 4, 5, 6, 7 - - I claimed a moment ago that that's in big O of n because if there's n people in the room, I've got to point at everyone. But equivalently, if there's n people in the room and I have to point at everyone, what's the fewest number of steps it could take me to take attendance using this linear approach ? So still n, right, unless I guess, which is not an algorithm. Like, unless I guess, I'm not going to get the right answer, so I kind of have to point at everyone. So in both the best case and the worst case, some algorithm still take n steps.
And for this, we have what we'll call theta notation, whereby if big O and omega happen to be the same, which is not always the case, but can be, then you can say that algorithm is the theta of such and such.
So my attendance-talking algorithm, the very first, 1, 2, 3, 4, all the way on up to n would be in theta of n because in both the best case and the worst case, it takes the same number of steps as per my big O and omega analysis. Now, there is a more formal mathematical definition for both of these. And if you take higher level computer science, you'll go more into the weeds of these ideas. But for now, big O, upper bound. Omega, lower bound.