Archives: big data

bigdata OLTP , OLAP No ratings yet.

row-based vs col based db or format row based –> good for OLTP ( transcation),   e.g: cassendra col based –> good for OLAP (? easy to aggreation etc?), druid Parquet hadoop: big data storage, what is the alternatives? S3 on cloud?   pinot vs cassandra druid If your queries ALWAYS constrain • Read More »

bigdata platform with Kubernets or Hadoop No ratings yet.

Hadoop: Hadoop kubernets MapReduce Spark on K8s Flink stream HDFS S3? any better one Resource manager Yarn/Mesos K8s itself   During its evolution phase, Hadoop provided three main functionalities that made it a Big Data-ready solution: a distributed computer mechanism (MapReduce), a robust data storage (HDFS), and a resource manager (YARN/Mesos). But modern technologies now • Read More »

Apache Kafka big picture and quick start No ratings yet.

What is Apache Kafka? ( big picture)  I found the article ( from Jay Kreps) presented a very good big picture on what Kafka suppose to do: you can use Kafka to build a stream data platform. Here the pictures from that article. The big idea is simple: many business processes can be modeled • Read More »