Senior Data and AI Engineer - Finance
- Your analytical curiosity drives you to deeply understand the Databricks platform and explore new AI/ML techniques for finance.
- You enjoy solving abstract problems, which is perfect for designing anomaly detection and automation tools.
- Your logical approach helps you build clean, maintainable code and data models.
- You may struggle with the structured, SOX-compliant change management processes and rigid documentation requirements.
- Frequent stakeholder meetings and requirement gathering with non-technical teams could feel draining and overly repetitive.
About the Role
As a Finance Data and AI Specialist, you will be a hands-on contributor building and maintaining the data pipelines, applications, and AI-powered tools that support Databricks' Finance and Accounting organisation. This is part of the Finance org. You will report to the Senior Manager, Finance Data and AI and work as a core member of a team that combines engineering rigour with Finance domain knowledge.
This role is ideal for someone who is technically strong, eager to go deep on the Databricks Platform, and excited to apply modern data and AI tooling to real Finance problems.
What You Will Do
- Build and maintain finance data pipelines in the Finance DataLake using Databricks Jobs and Lakeflow Lakeflow Spark Declarative Pipelines, including data validation and reconciliation logic
- Develop and iterate on AI-assisted tools for the Finance and Accounting organisation, including automation of manual workflows, anomaly detection, and reporting enhancements
- Contribute to the development of Finance applications (Databricks Apps, Genie Spaces, dashboards) that enable self-service analytics for Finance stakeholders
- Build and maintain reports and dashboards for monthly, quarterly, and executive-level reporting
- Support the implementation of row-level security and data access policies across Finance DataLake datasets
- Work with Accounting, FP&A, Internal Audit, and Procurement teams to gather requirements and deliver well-documented technical solutions
- Follow Git-based version control, pull-request review processes, and CI/CD pipelines (Declarative Automation Bundles, GitHub Actions) to meet SOX change management requirements
- Support financial close by monitoring pipelines, investigating data issues, and escalating as needed
- Participate in UAT for new system integrations and assist with technical documentation
- Contribute to team coding standards and data modelling conventions under the guidance of the Finance Data Lead
What We Look For
- 7+ years of experience in data engineering, analytics engineering, or finance systems
- Proficiency in SQL and Python for pipeline development; hands-on experience with Apache Spark or the Databricks platform is a strong plus
- Experience connecting to and ingesting from financial source systems (NetSuite, Salesforce, Stripe, Zuora, or similar)
- Working knowledge of finance and accounting concepts, including close processes, revenue recognition, and financial reporting
- Ability to translate Finance requirements into clean, maintainable technical solutions with guidance
- Comfortable communicating across technical and non-technical audiences
- Experience contributing to BI dashboards and self-service data products
- Curiosity about AI/ML and interest in applying new techniques to Finance workflows
Nice to Have
- Prior experience at a high-growth SaaS or cloud infrastructure company
- Exposure to AI/BI tools, Genie, or LLM-powered applications
About Databricks
Databricks is the data and AI company. More than 10,000 organizations worldwide — including Comcast, Condé Nast, Grammarly, and over 50% of the Fortune 500 — rely on the Databricks Data Intelligence Platform to unify and democratize data, analytics and AI. Databricks is headquartered in San Francisco, with offices around the globe and was founded by the original creators of Lakehouse, Apache Spark™, Delta Lake and MLflow. To learn more, follow Databricks on Twitter, LinkedIn and Facebook.
Benefits
At Databricks, we strive to provide comprehensive benefits and perks that meet the needs of all of our employees. For specific details on the benefits offered in your region click here.
Our Commitment to Diversity and Inclusion
At Databricks, we are committed to fostering a diverse and inclusive culture where everyone can excel. We take great care to ensure that our hiring practices are inclusive and meet equal employment opportunity standards. Individuals looking for employment at Databricks are considered without regard to age, color, disability, ethnicity, family or marital status, gender identity or expression, language, national origin, physical and mental ability, political affiliation, race, religion, sexual orientation, socio-economic status, veteran status, and other protected characteristics.
Compliance
If access to export-controlled technology or source code is required for performance of job duties, it is within Employer's discretion whether to apply for a U.S. government license for such positions, and Employer may decline to proceed with an applicant on this basis alone.
Best personality fits for this role
Your meticulous attention to detail ensures data pipelines are accurate and validation logic is thorough.
Read the ISTJ career guide →Your strong organizational skills and respect for procedures make you a natural fit for SOX-compliant change management and Git workflows.
Read the ESTJ career guide →Your strategic mindset helps you design robust, scalable data pipelines that align with long-term finance goals.
Read the INTJ career guide →