Linuxcon was a celebration of twenty years of Linux. I remember colleagues telling me to try the funny little operating system for PCs instead of a BSD variant, their excitement, and making distro ‘floppies’ for . The time has flown, Linux has become a core infrastructure for IT, and many technologies built upon upon Linux has flourished. But could it be better? At least for the database world, things could be better.
Databases are a ‘full body workout’ for Linux according to Josh Berkus. The CEO of PostgreSQL Experts had a session and a BOF at Linuxcon. The goals of these session was to make Linux developers aware of some of the special problems for databases and to get some hints of what database developers can do to better exploit Linux. Linux is the overwhelming OS choice for both the multi-process databases (PostgreSQL and Oracle) and the multi-threaded (MySQL and CouchDB). Both approaches have their own scaling limits. Currently we are stuck with one query for one core. Breaking up a query over multiple cores is inefficient and cache is not shareable between cores.
The wish list from the session included better handling of a large number of processes (1000+), more efficient spin lock checking, and a better affinity setting process for groups of threads. Plus faster creation of network sockets and faster detection of broken sockets would be great. And different files systems optimized for various uses — a write-only, fast append file system for log filess, another for OLTP. and a third for data warehouses.
The kernel developers present were eager to help but did warn that today’s changes take a long time before they become part of popular distributions. So help is on the way but it will take a while.