Migrating Apache Hive Workload to Apache Spark: Bridge the Gap Zhan Zhang (Facebook) and Jane Wang (Facebook) from udf Watch Video
Preview(s):
Gallery
Play Video: (Note: The default playback of the video is HD VERSION. If your browser is buffering the video slowly, please play the REGULAR MP4 VERSION or Open The Video below for better experience. Thank you!)
⏲ Duration: 14 min 72 sec ✓ Published: 11-Jun-2018
Description: At Spark Summit 2017, we described our framework to migrate production Hive workload to Spark with minimal user intervention. After a year of migration, Spark now powers an important part of our batch processing workload. The migration framework supports syntax compatibility analysis, offline/online shadowing, and data validation.nnIn this session, we first introduce new features and improvements in the migration framework to support bucketed tables and increase automation. Next, we will deep di
Play Video: (Note: The default playback of the video is HD VERSION. If your browser is buffering the video slowly, please play the REGULAR MP4 VERSION or Open The Video below for better experience. Thank you!)