Senior Data Engineer

Description

The MTA delivers subway, bus and commuter rail service, as well as tolled bridge and tunnel operations, in the New York City metro area. It is the largest public transportation provider in the Western Hemisphere, with millions of daily riders and a multi-billion-dollar budget. Delivering service and maintaining our infrastructure is increasingly data-driven, and so data scientists have become central to the MTA’s operations and decision-making. As a member of the Data & Analytics team, you will work on a dynamic, growing team tasked with helping to define and achieve the MTA’s strategic priorities and address its most pressing challenges. You will get the experience of seeing your work become public data and have it influence critical decisions that affect millions of New Yorkers.

• Develop and operationalize data pipelines, data warehouses, data marts, multi-dimensional cubes, and data lakes to collect, structure, and integrate a wide range of data sources make information available to staff and decision-makers for analysis and consumption.

• Support the development of new data sets, data access, extraction methodologies, algorithms, and processes to improve performance, find solutions to challenges, and take advantage of new opportunities in support of business needs using enterprise-class solutions.

• Develop automated test scripts to monitor and report on data quality, validity, accuracy and usability; conduct root cause analyses in response to issues and implement cost effective resolutions for data anomalies.

• Evaluate existing legacy data, algorithmic, and process solutions; redesign and implement modern data infrastructure to improve efficiency, timeliness, quality, robustness and scalability.

• Create and maintain architecture diagrams, system documentation, data models, mapping documents, business rules, data flow diagrams and other design related artifacts in a manner consistent with team standards to ensure understanding of teammates and, as needed, other audiences, both technical and non-technical.

• Provide input into data governance initiatives and influence the development of sustainable data management and governance practices across the agency.

• In consultation with managers in the Data & Analytics function, design project plans, help carry out project management, update project plans as needed, and communicate with clients and all relevant parties on project status.

• Support the selection and development of teammates within the department. Support a professional environment that respects individual differences, enables all colleagues to develop and contribute to their full potential and to achieve career goals at the MTA and beyond.

• Perform other duties as assigned.

• May need to work outside of regular work hours (i.e., evenings and weekends)

• Travel will be required to other MTA locations or other external sites

• Strong skills in programming, database designs, and data lake architectures for data engineering.
• Ability to work and communicate with data scientists, including understanding basic data science concepts, statistics, and common languages such as Python.
• Ability to work with data of different types — structured, semi-structured, unstructured — as well as from distinct disciplines — transportation, finance, HR, asset maintenance
• Some knowledge of transit/ transportation systems, public management, and associated operating constraints in the area of information management.
• Familiarity with KPI metrics, ability to create algorithms to calculate them, and familiarity with data visualization/ business intelligence tools to render them.
• Experience with data engineering orchestration tools (e.g., Airflow, Dagster, etc.) and DevOps tools (e.g., ADO, Git, Jenkins, Docker etc.) and familiarity with related concepts (e.g., CI/CD).
• Experience in quality assurance techniques and automated testing practices.
• Exceptional ability to read code and understand technical issues, keep up with technical innovation and trends in data engineering.
• Ability to collaborate and provide support to all levels of MTA, both technical and non-technical.
• Ability to deconstruct difficult problems into smaller and simpler pieces.
• Ability to project manage and help lead team-based projects.
• Ability to think at a policy and strategic level.
• Strong written communication skills.

• Bachelor's degree in Computer Science, Information Technology, Engineering, Mathematics, or related field. An equivalent combination of education and experience may be substituted in lieu of a degree.
• A minimum of (3) years of experience working in a data engineering or another position with similar programing and dataset design content.
• A minimum of (3) years of experience building pipelines, automating tasks through scripts, writing database queries and debugging/ maintaining code.
• A minimum of (4 years) experience with both Python and SQL programming.

• A minimum of (4 years) experience with relational databases (e.g., Oracle, Postgres, SQL Server, NoSQL), including writing queries (generally with PL/SQL) to obtain and manipulate data.

Qualified employees can submit an online application by clicking on the 'APPLY NOW' button from either the CAREERS HOME page or the JOB DESCRIPTION page.

For instructions on completing the online application, go to the MTA intranet site and click on the PeopleSoft Information hyperlink. From there select the User Guides hyperlink to gain access to the "Viewing/Applying for Jobs On Line" guide under the Recruiting section of the page.