Data Engineer Resume Samples

Description :

1. Formulated next generation analytics environment, providing self-service, centralized platform for any and all data-centric activities which allows full 360 degree view of customers from product usage to back office transactions.
2. Constructed product-usage SDK data and Siebel data aggregations by using PYSPARK, Scala, Spark SQL and Hive context in partitioned Hive external tables maintained in AWS S3 location for reporting, data science dash boarding and ad-hoc analyses.
3. Built Data virtualization layer (DENODO Base and Derived views), Data visualization using Tableau and accessed aggregations using SQL Clients PostgreSQL & SQL-Workbench.
4. Modernized data analytics environment by using cloud based Hadoop platform QUBOLE, SPLUNK, Version control system GIT, Automatic deployment tool Jenkins and Server-based workflow scheduling system OOZIE.
5. Performed in agile methodology, interacted directly with entire team provided/took feedback on design, Suggested/implemented optimal solutions, and tailored application to meet business requirement and followed Standards.
6. Performed post-implementation troubleshooting of new applications and application upgrades.
7. Conducted staff training and made recommendations to improve technical practices.

Experience

2-5 Years

Level

Junior

Education

Certification In Education

Description :

1. Examples include backlog analysis, capacity planning, production machinery status, pacing, quality, work in process (WIP) and other real-time production reports.
2. Report Development - Interview customers to define current state and guide them to a destination state.
3. Mock-up visuals with Balsamiq or Excel - Locate & vet data sources - Prototype solution and transport into test environment for customer approval and tweaks.
4. Finalize and transport into production environment.
5. Ruby Development - Created a task scheduling application to run in an EC2 environment on multiple servers.
6. For every data source and end point service create a data transformation module that would be executed by the tasking application.
7. Parse and prepare data for exchange using XML & JSON - Created clustered web-site utilizing Sinatra dsl framework with Thin servers behind Amazon load balancers.

Experience

2-5 Years

Level

Junior

Education

BS

Description :

1. Involved in Creation of tables, partitioning tables, Join conditions, correlated sub queries, nested queries, views, sequences, synonyms for the business application development.
2. Extensively involved in writing SQL queries (Sub queries and Join conditions), PL/SQL programming.
3. Understanding the existing business processes and Interacting with Super User and End User to finalize their requirement.
4. Design and Develop of Logical and physical Data Model of Schema Wrote PL/SQL code for data Conversion in there Clearance Strategy Project.
5. Developed database triggers, packages, functions, and stored procedures using PL/SQL and maintained the scripts for various data feeds.
6. Created Indexes for faster retrieval of the customer information and enhance the database performance.
7. Extensively used the advanced features of PL/SQL like collections, nested table, varrays, ref cursors, materialized views and dynamic SQL.

Experience

2-5 Years

Level

Junior

Education

BA In Computer Science

Description :

1. Experience building distributed high-performance systems using Spark and Scala Experience developing Scala applications for loading/streaming data into NoSQL databases (MongoDB) and HDFS.
2. Designed Distributed algorithms for identifying trends in data and processing them effectively.
3. Used Spark and Scala for developing machine learning algorithms which analyses click stream data.
4. Experience in developing machine learning code using spark MLLIB Used Spark SQL for data pre-processing, cleaning and joining very large data sets.
5. Experience in creating data lake using spark which is used for downstream applications Designed and Developed Scala workflows for data pull from cloud based systems and applying transformations on it.
6. Worked on Machine Learning Algorithms Development for analyzing click stream data using Spark and Scala.
7. Database migrations from Traditional Data Warehouses to Spark Clusters.

Experience

2-5 Years

Level

Executive

Education

Diploma

Description :

1. Worked on Recruiting Analytics (RA), a dimensional model designed to analyze the recruiting data in Amazon.
2. RA sources the data from ART (internal recruiting DB) and ties it with the various dimensions from PeopleSoft.
3. To track and analyze myriad of recruiting measures including Activity Counts (Phone screen, interviews, cycle times and Headcount Variance, etc.).
4. Worked on Q1 (PCS statements for all internal employees)/Q3 Performance and Comp reporting, Compliance and Tax audit reporting.
5. Explored the R statistical tool to provide data analysis on peer feedback data on leadership principles.
6. Worked on Payroll Datamart Project (Global) which provided the ability for payroll to view aggregated data across countries.
7. Designed the table structure and reporting format for global reports so that both customers and development team contractors can visualize the final report format.

Experience

2-5 Years

Level

Junior

Education

B. Tech

Description :

1. Lead team to create solar production forecasting engine at variable spatial and temporal resolutions for nationwide fleet of over 200,000 homes.
2. Produce hour ahead and day ahead forecasts based on local irradiance predictions.
3. Delivered to internal and external customers via REST API and csv downloads.
4. Brainstorm new products, validate engineering design, and estimate market acceptance with back of the envelope calculations.
5. Follow up with more detailed modeling leveraging internal customer data and relevant external sources.
6. Collaborate with Django-based reporting team to generate customizable executive reports.
7. Design peak shaving algorithms to reduce commercial customer's peak power consumption with a various energy storage technologies (battery, electric water heater, etc).
8. Validate that battery operation maintains compliance with regulations and battery warranty.

Experience

2-5 Years

Level

Executive

Education

Computer Engineering

Description :

1. Worked with analysts to understand and load big data sets into Accumulo.
2. Created multi-threaded application to enhance the loading capability of the data.
3. Created analytics to allow ad-hoc querying of the data.
4. Cyber Engineer: Worked with analysts to identify patterns in network pcap data.
5. Designed and developed applications to extract and enrich the information and present the results to the system users.
6. Task Lead: Lead a team of software engineers that developed analytical tools and data exploitation techniques that were deployed into multiple enterprise systems.
7. Interfaced with sponsor program management and the user community to develop use cases.
8. Worked with the team to deliver components using agile software development principles.

Experience

10+ Years

Level

Senior

Education

Information Systems

Description :

1. Worked in a team environment to fix data quality issues typically by creating Regular Expression codes to parse the data.
2. Reviewed audit data ingested in to the SIEM tool for accuracy and usability.
3. Collaborated and coordinated with development teams to deploy data quality solutions, while creating and maintaining standard operating procedure documentation.
4. Created trouble tickets for data that could not be parsed.
5. Fixed ingestion issues using Regex and coordinated with System Administrators to verify audit log data.
6. Applied processes improving and re-engineering methodologies to ensure data quality.
7. Utilized the HP ARC Sight Logger to review and analyze collected data from various customers.

Experience

10+ Years

Level

Senior

Education

B.S. In Computer

Description :

1. Worked in close association with the Business Analysts and DBAs for gathering requirements, business analysis, and testing and project coordination and participated in data modeling JAD sessions.
2. Interacted with end users and acquired the reporting needs.
3. Consulted with client management and staff to identify and document business needs and objectives, current operational procedures for creating the logical data model.
4. Created logical, physical and dimension models.
5. Created entity diagrams and relationship diagrams and modeled cascade to maintain referential integrity.
6. Used Erwin to create tables using forward engineering.
7. Reviewed the data model with the functional and technical team.

Experience

2-5 Years

Level

Executive

Education

Diploma

Description :

1. Developed pipelines to pull data from Redshift and send it to downstream systems through S3 and performing Sftp.
2. Created SQL Runner Jobs to load data from S3 into Redshift tables.
3. Expert hands-on in data deduplication, data profiling for many production tables.
4. Importing data from files from S3, and from SQLWorkbench data pumper to Redshift tables.
5. Performance tuning on Table level, updating Distribution Keys and Sort keys on tables.
6. Performance tuning on long running queries using Explain and Analyze commands.
7. Performing DBA activities like performing Vacuum and Analyze for tables, creating tables, views, recovery and cluster monitoring and maintenance.

Experience

2-5 Years

Level

Executive

Education

Bachelor Of Engineering

Description :

1. Responsible for the support data transfer, import-export, reports, user queries, and problems.
2. Generated EDAs using Spotfire and MS Excel for data analysis.
3. Developed various graphs using Spotfire and MS Excel for analyzing the various parameters affecting the project overrun.
4. Worked with the management for the determination and identify the problem.
5. Responsible for the maintenance of secure data transfer.
6. Maintenance and up gradations of technical documents were done regularly.
7. Responsible for the checking of problems, its resolution, modifications, and necessary changes.
8. Analyzed the system and made necessary changes and modifications.

Experience

7-10 Years

Level

Management

Education

Diploma

Description :

1. Create and maintain all bill of materials and specifications with any changes and or revisions that may occur.
2. Ensure all packaging specification data is complete, current and approved.
3. Collaborated with packaging developers to make sure bills of material, specifications and costing were accurate and finalized for a product launch.
4. Team leader in the transition of a data new system.
5. Team lead in company integration, obtaining the all active and historic bill of materials, costing comparisons and specifications.
6. Responsible to pull in depth reports for cost analysis and bill of materials.
7. Maintained the Packaging department's budget.

Experience

2-5 Years

Level

Executive

Education

Business Economics