Diting: A Real-time Distributed Feature Serving System for Machine Learning

EasyChair Preprint no. 1336

Abstract

4G networks dramatically boost the speeds and coverage of networks, and there are mounting mobile data produced by data sources of 4G. The generated data can be applied into preventing financial criminal activities, for example, account leakage risks can be prevented with such data instantly, which demands short time delay to predict the fraud, at most within 50 ms. Furthermore, the lower latency, higher data capacity of forthcoming 5G networks will create a new platform for the delivery of services wherever the 5G network exists, paving a way for artificial intelligence-based banking services. Real-time prediction services and data analytics services that can be integrated into business applications have become eagerly demanded in recent years. However, most machine learning and deep learning platforms only provide offline model training&test and model predictions without real-time feature processing. Diting is an easy-to-use distributed intelligent machine learning platform, providing offline model training and low-latency feature serving for predictions in real time. Diting offers incremental feature computing, combines technologies of resource scheduling, rule engines, and Remote Procedure Call (RPC), and builds a real-time distributed computing framework, thus offers low-latency end-to-end prediction services. Diting is also a collaborative, drag-and-drop and virtualization tool. Domain expert users without any programming knowledge can quickly fulfill their business logic. Using real production data for over two months, we show that Diting tremendously improves productivity, and bridges the gap between offline and real-time feature engineering.

Keyphrases: distributed systems, feature engineering, Incremental Aggregation, real-time computing

@Booklet{EasyChair:1336,
  author = {Meng Wan and Chongxin Deng and Siyu Yu},
  title = {Diting: A Real-time Distributed Feature Serving System for Machine Learning},
  howpublished = {EasyChair Preprint no. 1336},

  year = {EasyChair, 2019}}