JOB TITLE: Senior Data Engineer
SALARY RANGE: $82,411-108,165
HAY POINTS: 571
DEPT/DIV: Chief Strategic Initiatives Officer/ Data & Analytics
SUPERVISOR: Manager, Data Engineering
LOCATION: 2 Broadway New York, NY 10004
HOURS OF WORK: 9:00 am - 5:30 pm (7 1/2 hours/day)
The MTA delivers subway, bus and commuter rail service, as well as tolled bridge and tunnel operations, in the New York City metro area. It is the largest public transportation provider in the Western Hemisphere, with millions of daily riders and a multi-billion-dollar budget. Delivering service and maintaining our infrastructure is increasingly data-driven, and so data scientists have become central to the MTA’s operations and decision-making. As a member of the Data & Analytics team, you will work on a dynamic, growing team tasked with helping to define and achieve the MTA’s strategic priorities and address its most pressing challenges. You will get the experience of seeing your work become public data and have it influence critical decisions that affect millions of New Yorkers.
Our work combines new approaches and algorithms with a deep knowledge of our operations to deliver deeper insights into our performance. Our outputs are used by a wide array of people: from the MTA Board and senior execs to front-line managers and our open data customers. If you would like to use your programming skills to work on complex problems to improve public transit in NYC, then we’d like to talk to you.
The incumbent will help lead member of the team that designs, builds, tests, and delivers end-to-end, automated data pipelines over complex on-premises and off-premises platforms. They will work to extract data from multiple source systems containing structured, semi-structured and unstructured data to make it consistent, reliable, available, and usable to colleagues across the MTA and, in support of the agency’s and New York State’s Open Data goals, to external stakeholders and the general public.
They will use languages such as SQL, Python and R and relational database tools such as Oracle, Postgres, and SQL Server to analyze large datasets, build new ones, and design overall data architectures. They will carefully document all work and work closely with colleagues to define needs, problem-solve, support the overall team agenda, and build relationships throughout and at all levels of the agency. They will have to be able to quickly learn the unique features, data constraints and business needs of any part of the MTA. In addition, unlike other data engineering roles, they will support the entire downstream pipeline process and, occasionally, end-users of the data products. In general, they will have to support the MTA’s strategic goals to build data systems and processes that are well-structured and sustainable.
Develop and operationalize data pipelines, data warehouses, data marts, multi-dimensional cubes, and data lakes to collect, structure, and integrate a wide range of data sources make information available to staff and decision-makers for analysis and consumption.
Support the development of new data sets, data access, extraction methodologies, algorithms, and processes to improve performance, find solutions to challenges, and take advantage of new opportunities in support of business needs using enterprise-class solutions.
Develop automated test scripts to monitor and report on data quality, validity, accuracy and usability; conduct root cause analyses in response to issues and implement cost effective resolutions for data anomalies.
Evaluate existing legacy data, algorithmic, and process solutions; redesign and implement modern data infrastructure to improve efficiency, timeliness, quality, robustness and scalability.
Create and maintain architecture diagrams, system documentation, data models, mapping documents, business rules, data flow diagrams and other design related artifacts in a manner consistent with team standards to ensure understanding of teammates and, as needed, other audiences, both technical and non-technical.
Provide input into data governance initiatives and influence the development of sustainable data management and governance practices across the agency.
In consultation with managers in the Data & Analytics function, design project plans, help carry out project management, update project plans as needed, and communicate with clients and all relevant parties on project status.
Support the selection and development of teammates within the department. Support a professional environment that respects individual differences, enables all colleagues to develop and contribute to their full potential and to achieve career goals at the MTA and beyond.
Perform other duties as assigned.
May need to work outside of regular work hours (i.e., evenings and weekends)
Travel will be required to other MTA locations or other external sites
Strong skills in programming, database designs, and data lake architectures for data engineering.
Ability to work and communicate with data scientists, including understanding basic data science concepts, statistics, and common languages such as Python.
Ability to work with data of different types — structured, semi-structured, unstructured — as well as from distinct disciplines — transportation, finance, HR, asset maintenance
Some knowledge of transit/ transportation systems, public management, and associated operating constraints in the area of information management.
Familiarity with KPI metrics, ability to create algorithms to calculate them, and familiarity with data visualization/ business intelligence tools to render them.
Experience with data engineering orchestration tools (e.g., Airflow, Dagster, etc.) and DevOps tools (e.g., ADO, Git, Jenkins, Docker etc.) and familiarity with related concepts (e.g., CI/CD).
Experience in quality assurance techniques and automated testing practices.
Exceptional ability to read code and understand technical issues, keep up with technical innovation and trends in data engineering.
Ability to collaborate and provide support to all levels of MTA, both technical and non-technical.
Ability to deconstruct difficult problems into smaller and simpler pieces.
Ability to project manage and help lead team-based projects.
Ability to think at a policy and strategic level.
Strong written communication skills.
Bachelor's degree in Computer Science, Information Technology, Engineering, Mathematics, or related field. An equivalent combination of education and experience may be substituted in lieu of a degree.
A minimum of (3) years of experience working in a data engineering or another position with similar programing and dataset design content.
A minimum of (3) years of experience building pipelines, automating tasks through scripts, writing database queries and debugging/ maintaining code.
A minimum of (4 years) experience with both Python and SQL programming.
A minimum of (4 years) experience with relational databases (e.g., Oracle, Postgres, SQL Server, NoSQL), including writing queries (generally with PL/SQL) to obtain and manipulate data.
- Master’s degree in Computer Science, Information Technology, Engineering, Mathematics, or a related field.
As an employee of MTA Headquarters, you may be required to complete an annual financial disclosure statement with the State of New York, if your position earns more than $108,368 (this figure is subject to change) per year or if the position is designated as a policy maker.
Qualified employees can submit an online application by clicking on the 'APPLY NOW' button from either the CAREERS HOME page or the JOB DESCRIPTION page.
For instructions on completing the online application, go to the MTA intranet site and click on the PeopleSoft Information hyperlink. From there select the User Guides hyperlink to gain access to the "Viewing/Applying for Jobs On Line" guide under the Recruiting section of the page.
MTA and its subsidiary and affiliated agencies are Equal Opportunity Employers, including with respect to veteran status and individuals with disabilities.
The MTA encourages qualified applicants from diverse backgrounds, experiences, and abilities, including military service members, to apply.