This gave rise to “lake houses”, which pretty much boil down to the following: (i) dump your data into cloud buckets as files, (ii) adopt a “hammer” for scalable compute, (iii) treat data management as an afterthought by applying “hacks”.
Organizations with large quantities of data prefer storing it in cheap cloud stores in the form of files, effectively separating storage from compute. In the meantime, the cloud has changed data management radically. This data is typically massive and seemingly diverse, and cannot be managed effectively with “traditional” databases. While we have built enormous sophistication over the past 5 decades in relational databases which manage tabular data perfectly, most of the data being generated out there (e.g., by the Sciences) is not tabular.
#UNIVERSAL DATABASE FULL#
The full webinar recording The problem with purpose-built data systems I look forward to hearing your thoughts and initiating a necessary dialogue in our industry.
I am also providing the gist below in text, in case you folks are too busy or bored to listen to some random dude talking about the exact opposite of what cloud vendors are telling you and how the market works. And we built one, so we thought to share how we did it! We call such a system the universal database. We explain that there is huge Engineering overlap across all those data systems and, therefore, it is possible to build a single system that can manage all data types in a unified way, for all applications, in a foundational and future-proof manner. Those create a lot of noise for analysts and scientists dealing with vital data problems across numerous important application domains, making their lives very difficult and slowing down Science for all of us. We argue that the market is saturated with thousands of purpose-built data(base) systems. We hosted a webinar where I shared what we have been working on for the past several years to make this vision a reality. Last week we had a coming out party for our audacious vision at TileDB.
0 kommentar(er)
