Google Cloud Dataflow

Google Cloud Dataflow is a fully managed service for executing Apache Beam pipelines within the Google Cloud Platform ecosystem.

Logo of Google Cloud Dataflow

History

Google Cloud Dataflow was announced in June, 2014[1] and released to the general public as an open beta in April, 2015.[2] In January, 2016 Google donated the underlying SDK, the implementation of a local runner, and a set of IOs (data connectors) to access Google Cloud Platform data services to the Apache Software Foundation.[3] The donated code formed the original basis for the Beam project.

gollark: For space-efficiently sending really big programs you might as well use gzip.
gollark: It would probably be possible to serialize to some nice binary format, but not necessarily very useful.
gollark: Good programs ignore nonsense like "physical laws".
gollark: It takes time to execute → there are performance issues.
gollark: Is there some profiling tool you can use to analyze possible performance issues?

References
  1. "Sneak peek: Google Cloud Dataflow, a Cloud-native data processing service". Google Cloud Platform Blog. Retrieved 2018-09-08.
  2. "Google Opens Cloud Dataflow To All Developers, Launches European Zone For BigQuery". TechCrunch. Retrieved 2018-09-08.
  3. "Google wants to donate its Dataflow technology to Apache". Venture Beat. Retrieved 2019-02-21.
This article is issued from Wikipedia. The text is licensed under Creative Commons - Attribution - Sharealike. Additional terms may apply for the media files.