Mary is a Java Champion and a passionate Streaming Developer Advocate at DataStax, a leading data management company that champions Open Source software and specializes in Big Data, DB-as-a-service, Streaming, and Cloud-Native systems. She spent 3.5 years previously as a very effective advocate at IBM, focusing on Java, Jakarta EE, OpenJ9, Open Source, Cloud, and Distributed Systems. She transitioned from Unix/C to Java around 2000 and has never looked back since then. She considers herself a polyglot and loves to continue learning new and better ways to solve real-life problems. She is an active tech community builder outside of her day job, and currently the President of the Chicago Java Users Group (CJUG), as well as a co-organizer for several IBM-sponsored meetup groups in the Greater Chicago area.
Apache Pulsar is a new generation of platform that offers enterprise-grade event streaming and processing capabilities built for today's Cloud Native environment. But what do you do if you want to perform user-facing, ad-hoc, real-time analytics too? That's where Apache Pinot comes in.
Apache Pinot is a realtime distributed OLAP datastore, which is used to deliver scalable real time analytics with low latency. It can ingest data from batch data sources (S3, HDFS, Azure Data Lake, Google Cloud Storage) as well as streaming sources such as Pulsar. Pinot is used extensively at LinkedIn and Uber to power many analytical applications such as Who Viewed My Profile, Ad Analytics, Talent Analytics, Uber Eats and many more serving 100k+ queries per second while ingesting 1Million+ events per second.
Apache Pulsar's highly performant, distributed, fault-tolerant, real-time publish-subscribe as well as queueing messaging platform that operates seamlessly in a Cloud-Native environment with support for geo-replication, multi-tenancy, data warehouse or data lake integrations, and beyond. It is a tried-and-true platform that has major enterprise customers such as Yahoo, Verizon, GM, Comcast, etc.
Best of all, Apache Pulsar and Apache Pinot together represents a blissful union in the #OSS "heaven"!
Come hear the dynamic duo, Mary Grygleski from DataStax, and Karin Wolok, Head of Developer Community at StarTree, on an introduction to both systems and a view of how they work together.
The world is moving at an unprecedented pace and much of it has been powered by the innovations in software and systems. While event handling, messaging, and processing are not necessarily brand new concepts, the recent emergence in hardware such as virtualizations, multi-core processors, and so on, are in fact pushing the envelope in software design and development, elevating it to higher levels of capabilities never seen before. In the case of streaming which very often leverages on the underlying messaging mechanism(s) to bring distributed messaging to higher forms of purposes, such as IoT/IIoT applications, AI/ML data pipelines, or even eCommerce recommendations, event streaming platform has indeed become the “glue” in enabling data to flow through disparate systems in the pipeline and in a very dynamic fashion.
This talk on event streaming is meant for anyone interested in learning about it, and understanding how it fits into the modern software development design and architecture, as well as seeing some of the challenges it faces especially in the Cloud Native environment. We’ll then take a look at an open source platform - Apache Pulsar, which is poised to become the de facto new generation of distributed messaging and streaming platform that will bring joy to developers, and enable systems and applications to be highly responsive with its true real-time capabilities.
With the exploding interests in Machine Learning technology in recent years, they have also introduced many aspects of computing that are required to support such a grand vision in what ML is capable of delivering. On the infrastructural layer, we need to handle the high frequency of data ingestion with low latency, and one of the best mechanisms that we can think of leveraging on is streaming. So what is streaming, and what are the different choices we have as a platform? We will learn about a few options, and zoom in to learn more about what a true next-generation, cloud-native streaming platform such as Apache Pulsar is capable of, beyond what the more common messaging platforms that we have these days.