Techniques for addressing UDF incompatibility between spark and Gluten and Velox - Shiyu Gan

Techniques for addressing UDF incompatibility between spark and Gluten and Velox - Shiyu Gan

19 Lượt nghe
Techniques for addressing UDF incompatibility between spark and Gluten and Velox - Shiyu Gan
In this talk, Shiyu talks through challenges encountered in the productionization of Gluten/Velox around compatibility between Spark’s and Velox’s json and re (regular expression) UDF’s and then presents two techniques in solving these incompatibilities between UDF’s.