Scale Cast – A podcast about big data, distributed systems, and scalability

In 2006 we were building distributed applications that needed a master, aka coordinator, aka controller to manage the sub processes of the applications. It was a scenario that we had encountered before and something that we saw repeated over and over again inside and outside of Yahoo!. For example, we have an application that consists […]

The Bloom filter, conceived by Burton H. Bloom in 1970, is a space-efficient probabilistic data structure that is used to test whether an element is a member of a set. False positives are possible, but false negatives are not. Elements can be added to the set, but not removed (though this can be addressed with […]

In this talk we examine how high performance computing has changed over the last 10-year and look toward the future in terms of trends. These changes have had and will continue to have a major impact on our software. A new generation of software libraries and algorithms are needed for the effective and reliable use […]

This talk takes us on a journey through three varied, but interconnected topics. First, our research lab has engaged in a series of disk-based computations extending over five years. Disks have traditionally been used for filesystems, for virtual memory, and for databases. Disk-based computation opens up an important fourth use: an abstraction for multiple disks […]

Today, I’m announcing a new meta podcast about designing scalable systems named Scalecast. I’ve been seeing more and more conference interesting presentations online but I can’t get them to work with my iphone/ipod since they require streaming flash video. I now have a simple script that can fetch the flash video from youtube, transcode it to mp4 video, including AAC audio, and publish it to WordPress. It’s then available for use on any Apple device including legacy ipods and more modern iphones. If you have a suggestion for a video to include in this podcast, just add it as a comment on this post and I’ll try to transcode for you. I primarily did this just for myself. I need to be able to view these videos for my work and my primary mechanism for doing so is my iphone.