I recently started reading a very interesting book called The Elements of Computing Systems by Noam Nisan and Shimon Schocken. The book is nicknamed “nand2tetris”, so called because the reader starts off with a NAND gate and using it as a building block for more complex systems builds a modern computer(on which he or she implements the game of Tetris).
I’ve only just finished the first chapter and am currently diving into the second, but I can already say that the book is quite remarkable in both what it sets out to do and its pedagogy.
What the book is about
As I mentioned above, the book has you build a computer from scratch. Just so you don’t get the wrong idea, the entire computer is built in a simulator running on your “actual computer”.
The idea here is to learn how computers work by actually building one. In the process, you’ll encounter everything from the low-level, implementing logic gates to writing compilers to high-level object-oriented programming. And of course, the most fun part is that everything from the base will be something you built. I really liked this idea.
The abstraction-implementation paradigm
All in all, the computer you build in the book can be divided into 12 or so “layers”, roughly a chapter is dedicated to each of these layers. Each layer is built on top of the one below it. For example, in the first chapter you build logic gates out of NAND gates. Then, in the second chapter using the gates built in the first chapter, the book has you building an Arithmetic Logic Unit.
This way of layering abstractions is very important in computer science. The idea is to implement each layer such that the layer above it has a clear way of interfacing with it to get the required functionality. The benefit here is that by being able to think about just one level of abstraction at a time, complexity is reduced.
The other big idea I’ve noticed so far in this book is that it tends to be brutal about having the reader do the exercises. Whereas most books which claim to follow a “hands-on” will throw you a problem every once in a while, this book is mostly exercises. For each layer, the book gives you a specification and you are the one who needs to implement each layer.
Of course, some guidance is provided but expect to get stuck at problems. This can be frustrating but the central premise of the book is that you learn by doing things yourself.
Like I said before I’ve only just dipped my toes here so far, but I can already say that this is going to be an interesting experience. Not only will it help fill a gap in my knowledge regarding computer architecture, I think it will be interesting book to study because of the very different pedagogical approach the authors have taken.
My previous post about my new web app- Videodropper, which lets you send Youtube videos to your Dropbox account got quite huge on Hacker News yesterday(At least, much more than I’d anticipated). So, I decided to write another post about Videodropper on how it works.
Videodropper is powered by Python, Flask, Redis and Celery. It is hosted on Epio(as you might have guessed from the ep.io subdomain). All the downloading is handled by youtube-dl and of course, the uploading is done using the Dropbox API.
What happens when you press the “Send to Dropbox” button is that Videodropper gets the Youtube video URL(playlists won’t work, more on that later), and it queues it up on a Celeryd process; when the download starts, another process also starts that monitors the size of the video file so that it doesn’t cross the upload limit. So, that’s two processes for one file download. Currently, Videodropper runs only four processes which means that it can process only two videos at a time. Up until now, these modest resources have served quite well, as the download and upload speed is pretty high. However, the Celery instance, which is limited to 128MB of memory runs out of memory when Videodropper starts getting a lot of requests.
When Celery starts a task which in this case is a “download, then upload”, youtube-dl starts downloading the Youtube video which, as I mentioned above happens quite fast. Then, if the size hasn’t exceeded Dropbox’s upload limit(which is 150MB now, but it should soon be upgraded to 300MB), the upload process begins. That’s basically the core functionality of the app. Oh, and if you choose to optimize the video for iPhone, youtube-dl simply downloads the .mp4 file(Youtube format=18) of the video, no transcoding takes place.
One obvious optimization for Videodropper is the size-monitoring as you’ve seen that it eats up a whole process. One solution would have been to find out the file size before downloading, but I couldn’t find any way to do that(if you happen to know of any please, please tell me about it.)
I’ve got some requests for Videodropper to start supporting playlists which is quite a reasonable request, given that Playlists is a huge part of Youtube. However, at present supporting playlists is just not an option. As I’ve mentioned above, Videodropper runs on a very modest server configuration and downloading playlists would surely cause Videodropper to run out of memory. The limits on the server resources are mostly because Videodropper is a free app, so I am not able to invest much money on it, so if you’re interested in supporting Videodropper you could donate some money on Flattr to help upgrade Videodropper’s server capacity.
In a nutshell, Flask serves the website, Celery queues up the download tasks(and also a size monitoring task) and Redis is used mostly as a backend to Celery, but also for storing the “Recent Downloads” of the user. That’s it. If you have any query or suggestion feel free to shoot them at the comments below.
If you’re constantly troubled by having to wait for Youtube to buffer videos, or want to keep some videos with you so you can watch them again and again without wasting bandwidth, then I’ve just recently released a web service that lets you download videos from Youtube. But there are plenty of websites that already do that, why do I need another one?. What Videodropper(yeah, that’s what its called) actually does is it sends videos to your Dropbox folder.
Why send it to Dropbox?
Syncing- Instead of downloading the video to your computer you’ll be saving it to your personal cloud which will be accessible to you on any PC or mobile device. So sending a Youtube video to your Dropbox folder is basically the same as downloading into your machine, only more awesome.
Bandwidth- Bandwidth is the primary motivation for why I built Videodropper. I have a rather slow internet connection and I get all sort of problems when watching Youtube videos- having to wait for buffering, videos getting stuck(it always happens to me when I resize the player), and the list goes on. Videodropper solves these problems because “Dropbox is smart about using bandwidth.” Downloading happens at the fastest possible speed, but without interfering with your Internet usage, so if you have a slow connection you’ll be able to download Youtube videos and still use the Internet at optimal speed. Also, once its fully downloaded you can watch the full video without any pain.
iOS devices- Dropbox works great on iOS devices and you can optimize videos you send using Videodropper to work on the iPhone or iPod touch. After you’ve synced the videos, you can also watch them later offline(this makes use of Dropbox’s Local Storage on your iPhone/iPod).
How do I use Videodropper?
Using Videodropper is really easy. After linking Videodropper with your Dropbox account, you just copy the URL of the video you want to send, paste it into Videodropper and then send it. A bookmarklet is also being worked on in order to further simplify this process.
Note that Videodropper is still in Beta, so some of your videos may not get through properly, in case you notice any thing strange kindly send an email to email@example.com or reach me on Twitter @samratmansingh
I have been wanting to create a serious web app for some time now, and I think I just did. Well, to be more accurate its a pretty simple web service. Let me introduce to you Instamator, it lets you easily grab and use the photos you’ve liked on Instagram, the popular photo-sharing app(for iOS devices only- but you can check Inkstagram for a web interface).
How does it do that? It produces feeds(JSON, RSS and text) about the photos you liked on Instagram which you can use to easily download all your Instagram “likes” or integrate on your website or blog.
Instamator was inpired by Flickmator, a similar web service built by Thomas Pelletier. It is powered by Flask, Redis and hosted on ep.io.