Connect Spark to MySql database

As a team of developers, we decided (for the first time) to build a specific business application for one of our customers based on a Spark arhitecture. When the main part of the app was already build, the customer decided to use a MySql database server instead of MS SQL server and that this is not changeable.

So I wonder what are our options now, is it possible to connect with Spark to MySql database? If not, that means that we have to re-writte (at least) a domain layer or find some other solutions to satisfy this need - which is quite a job for this stage of a project.

Thanks.

Martin Hren, Nov 19, 2013

Reply 1

Martin:

Indeed, Spark is written to work with SQL Server. I am not a super expert in MySql but it seems to me that a port to MySql shouldn't be too difficult. The reason for this is that there is very little SQL Server specific code in Spark. The core data access class is named Db and this is where you will need to make some adjustments, particularly the area where new records are inserted and the newly generated Id is retrieved from the database.

To minimize the time required for the port, study the source code of Spark and familiarize yourself with MySql data access. You will also need to add some additional references to MySql specific data assemblies, but that may be all that is needed.

If you decide to pursue this path, please let us know how it goes.
I would be curious to hear if this assessment is correct.

-- Jack

Jack Poorte, Nov 19, 2013