The Apache Flink® Conference
Stream Processing | Event Driven | Real Time
San Francisco April 9–10, 2018
eBay monitoring platform collects metrics, events and logs from network devices, kubernetes clusters, applications and other monitoring tools, data will be feeding into preprocessing and alerting engine to enrich, normalize, dedupe, alerting, etc. We start by building the first generation preprocessing and alerting engine on top of Storm. Then, migrated from Storm to Flink to embrace streaming/batch unique interface and exactly-one semantics also rich support of windowing. Our monitoring platform aims to provide self-service alert definition which mean tens of thousands of alert rules on streaming data. So we built metadata-driven processing engine on top of Flink. It provides capabilities APIs which translate capabilities to Flink data stream APIs, and provides policies APIs to submit or update users’ process and alerts requests and deploy these requests to shared Flink job for processing without restarting jobs. Also an alert DSL is designed to enable self-service alert definition.
Garrett has been creating software for 10 years, his current role is a senior software engineer on eBay unified monitoring platform team.
Ralph Su is engineer at eBay with various area from cloud to hadoop. He major interests in distributed system and database system. Currently, he is focus on streaming processing of monitoring inside eBay. I’m fond of open-source and committer of apache eagle(id: ralphsu), also contribute to other projects like eBay yidb.