What is Azure SQL Data Warehouse? Azure SQL Data Warehouse (SQL DW) is a petabyte-scale MPP analytical data warehouse built on the foundation of SQL Server and run as part of the Microsoft Azure Cloud Computing Platform. Like other Cloud MPP solutions, SQL DW separates storage and compute, billing for Read more…
What is BigQuery? BigQuery is a data warehouse that leverages the massive scale of the Google Cloud architecture to distribute data across thousands of nodes, utilizing as many nodes as are needed to run any query performantly. Unlike other database dialects, where you buy or rent individual machines or space on machines, Read more…
In this article, we describe how we orchestrate Kafka, Dataflow and BigQuery together to ingest and transform a large stream of events. When adding scale and latency constraints, reconciling and reordering them becomes a challenge, here is how we tackle it. Teads for Publisher, one of the webapps powered by Read more…
Zoom scaled from 20 million to 300 million users virtually over night. What’s incredible is from the outside they’ve shown little in the way of apparent growing pains, though on the inside it’s a good bet a lot of craziness is going on. Sure, Zoom has made some design decisions Read more…
Netflix seems so simple. Press play and video magically appears. Easy, right? Not so much. Given our discussion in the What is Cloud Computing? chapter, you might expect Netflix to serve video using AWS. Press play in a Netflix application and video stored in S3 would be streamed from S3, over the Read more…
What is HBase? Apache HBase is an open-source, NoSQL, distributed big data store. It enables random, strictly consistent, real-time access to petabytes of data. HBase is very effective for handling large, sparse datasets. HBase integrates seamlessly with Apache Hadoop and the Hadoop ecosystem and runs on top of the Hadoop Read more…
What is Hive? Apache Hive is a distributed, fault-tolerant data warehouse system that enables analytics at a massive scale. A data warehouse provides a central store of information that can easily be analyzed to make informed, data driven decisions. Hive allows users to read, write, and manage petabytes of data Read more…
What is Presto or PrestoDB? Presto (or PrestoDB) is an open source, distributed SQL query engine, designed from the ground up for fast analytic queries against data of any size. It supports both non-relational sources, such as the Hadoop Distributed File System (HDFS), Amazon S3, Cassandra, MongoDB, and HBase, and relational data Read more…
What is Apache Spark? Apache Spark is an open-source, distributed processing system used for big data workloads. It utilizes in-memory caching, and optimized query execution for fast analytic queries against data of any size. It provides development APIs in Java, Scala, Python and R, and supports code reuse across multiple Read more…
Apache Hadoop is an open source framework that is used to efficiently store and process large datasets ranging in size from gigabytes to petabytes of data. Instead of using one large computer to store and process the data, Hadoop allows clustering multiple computers to analyze massive datasets in parallel more Read more…
