Dofactory.com
Dofactory.com

Data Engineer

Company
Freemind solutions
Pay
$55 - $60 / Hour 
--
Contract
Level
Mid level
Skills

Job Description

Big Data Engineer with Python, Spark

Remote Position

Role: W2

Responsibilities

Integrate data from a variety of data sources (data warehouse, data marts) utilizing on-prem or cloud-based data structures (Azure/AWS); determine new and existing data sources

Develop, implement, and optimize streaming, data lake, and analytics big data solutions

Create and execute testing strategies including unit, integration, and full end-to-end tests of data pipelines

Recommend Kudu, HBase, HDFS, and relational databases based on their strengths

Utilize ETL processes to build data repositories; integrate data into the Hadoop data lake using Sqoop (batch ingest), Kafka (streaming), Spark, Hive, or Impala (transformation)

Adapt and learn new technologies in a quickly changing field

Be creative; evaluate and recommend big data technologies to solve problems and create solutions

Recommend and implement best tools to ensure optimized data performance; perform Data Analysis utilizing Spark, Hive, and Impala

Job Type: Contract

Pay: $55.00 - $60.00 per hour

Experience level:

  • 10 years
  • 11+ years
  • 6 years
  • 7 years
  • 8 years
  • 9 years

Work Location: Remote

 
Ref #
2768900
Posted
30+ days ago
  •  
Last updated 30+ days ago


Stay Inspired!
Join other developers and designers who have already signed up for our mailing list.
Terms     Privacy     Cookies       Do Not Sell       Licensing      
Made with    in Austin, Texas.  - vsn 44.0.0
© Data & Object Factory, LLC.