Partition-Aware Data Loading in Spark SQL

Originally posted on Knoldus:
Data loading, in Spark SQL, means loading data in memory/cache of Spark worker nodes. For which we use to write following code: val connectionProperties = new Properties() connectionProperties.put(“user”, “username”) connectionProperties.put(“password”, “password”) val jdbcDF = spark.read .jdbc(“jdbc:postgresql:dbserver”, “schema.table”, connectionProperties) In here we are using jdbc function of DataFrameReader API of Spark SQL to load…

Creating A High Order Function From A BiFunction And Predicate In Java8

Originally posted on Knoldus:
In this Blog we will be reading about creating a high order function from a bifunction and predicate in java8.Firstly a brief introduction to bifunction,predicate,high order function in java8 Predicate:Predicate in java 8 is used to apply a filter to a collection of objects.Predicate is a Functional interface and supports Lambda expressions.…