DarkMatter in Cyberspace
  • Home
  • Categories
  • Tags
  • Archives

Read Table Data from MySQL in PySpark


Verified on Spark 2.2.

First copy file mysql-connector-java-5.1.40.jar into folder $SPARK_HOME/jars. Then get the data in table table_name from MySQL service 192.168.110.3:3306/avatar with:

url = "jdbc:mysql://192.168.110.3:3306/avatar"
df = spark_session.read.jdbc(url, table_name, properties={ 'user': username,
        'password': password, 'driver': 'com.mysql.jdbc.Driver' })


Published

Jan 15, 2018

Last Updated

Jan 15, 2018

Category

Tech

Tags

  • jdbc 2
  • mysql 5
  • pyspark 5

Contact

  • Powered by Pelican. Theme: Elegant by Talha Mansoor