You will be responsible for the design, engineering, implementation, operation, and continuous improvement of the University’s data platform. This is a hands-on delivery and operations role - the Senior IT Consultant (Data Platform) is expected to build and troubleshoot solutions directly, take ownership of production reliability, and work closely with users to translate business, education, and requirements into practical data products.  

The data platform will support business intelligence, data analytics, data science, machine learning, generative AI, and agentic applications. The primary focus of the role is data engineering and platform capabilities, with a secondary focus on hands-on data analytics, data science, and the responsible use of AI to process and derive value from data. You will collaborate with data owners, stewards, analysts, data scientists, AI engineers, application teams, and security and infrastructure teams, and communicate platform plans, architectural choices, risks, progress, and outcomes clearly to management.

Key Responsibilities

Data Platform Architecture and Engineering

• Design, build, and evolve scalable cloud, hybrid, or on-premises data lakes, warehouses, and lakehouses for structured, semi-structured, and unstructured data
• Define reference architectures, reusable patterns, data contracts, and self-service capabilities from source onboarding to trusted consumption
• Apply cloud-native practices across networking, identity, storage, compute, managed services, resilience, recovery, performance, and cost
• Engineer secure, performant storage, compute, access, and serving layers for BI, analytics, data science, AI, applications, and APIs
• Evaluate technologies and optimise platform reliability, scalability, interoperability, performance, and cost

Data Ingestion, Pipelines and Integration

• Design, build, test, deploy, and maintain production-grade batch, near-real-time, streaming, and change-data-capture pipelines, integrating data from enterprise applications, databases, files, APIs, events, cloud services, and external sources.
• Implement reliable orchestration, dependency management, schema-evolution handling, idempotency, validation, error handling, replay, reconciliation, and recovery mechanisms.
• Develop reusable connectors, ingestion frameworks, integration components, and APIs.
• Monitor and meet agreed data-freshness, availability, performance, and reliability requirements.
• Work with source-system owners to identify and address integration issues and improve data quality at source.

Data Transformation, Modelling and Semantic Layers

• Develop maintainable SQL, Python, Spark, or equivalent ETL/ELT transformations using modular design, version control, automated testing, and code review.
• Design conceptual, logical, and physical data models for operational, dimensional, analytical, and lakehouse workloads.
• Build curated data products and appropriate data layers, including raw, validated, conformed, and consumption-ready data.
• Prepare and serve trusted, governed, and consumption-ready data for business intelligence, analytics, data science, and AI/ML use cases.
• Develop governed semantic models, reusable measures, business rules, and domain-aligned datasets.
• Ensure that data models and semantic layers support consistent reporting, analytics, data science, AI, application, and API consumption.
• Optimise data structures, queries, pipelines, and workloads for performance, maintainability, reliability, and cost.       

Data Governance, Quality, Metadata and Security Controls

• Collaborate with data owners, data stewards, business divisions, and data privacy and governance teams to elicit and formalize business definitions, acceptable data-quality thresholds, data cleansing rules, ownership decisions, and governance policies for the data platform, and implement, operationalise, and monitor the corresponding technical controls within the platform.
• Implement and enforce data-governance controls within the data platform, including controls for data ownership, classification, access, privacy, retention, lineage, auditability, and regulatory compliance.
• Embed profiling, validation, reconciliation, automated quality checks, anomaly detection, observability, and issue-management controls within pipelines and data products.
• Implement and support data catalogue, business glossary, data dictionary, technical metadata, lineage, and searchable documentation for discoverable, trusted data.
• Protect data through ingestion, storage, processing, sharing, analytics, and AI use. Apply discovery, classification, least-privilege access, encryption, key and secrets management, masking, redaction, tokenisation, pseudonymisation or anonymisation, secure sharing, audit logs, and data-loss prevention with security, privacy, and governance teams.

Platform Operations, Reliability and Support

• Take ownership of the operational reliability of the data platform and its production data services.
• Implement monitoring, logging, alerting, tracing, operational dashboards, and automated health checks.
• Troubleshoot production issues and perform incident response, root-cause analysis, remediation, and preventive improvement.
• Manage platform performance, workload optimisation, capacity planning, backup, recovery, availability, and operational resilience.
• Establish and maintain operational procedures, support documentation, runbooks, recovery procedures, and technical knowledge articles.
• Track service quality, platform availability, pipeline reliability, data quality, utilisation, and cost metrics.
• Identify recurring operational issues and implement engineering improvements to reduce manual intervention and reliance on external delivery partners.

DataOps, DevSecOps and Engineering Standards

• Implement secure and automated CI/CD practices for data-platform components, pipelines, infrastructure, configurations, and data models.
• Use Git-based version control, automated testing, deployment controls, code review, environment management, and release governance.
• Apply infrastructure-as-code and policy-as-code practices where appropriate.
• Integrate security, vulnerability, dependency, and configuration controls into engineering and release processes.
• Establish engineering standards, reusable templates, testing frameworks, coding conventions, and documentation practices.
• Promote automation, repeatability, traceability, and separation of duties across development, testing, deployment, and operations.
• Support lifecycle management, monitoring, lineage, and controlled deployment for analytical, machine-learning, and GenAI workloads where these workloads use the data platform.

Technical Delivery and Internal Capability Development

• Lead delivery from requirements discovery and solution design through implementation, testing, migration, deployment, adoption, and production support.
• Translate business needs into practical, supportable, and reusable data-platform solutions.
• Manage technical risks, issues, dependencies, assumptions, and delivery constraints.
• Coordinate the work of vendors and delivery partners where required, while retaining internal understanding and ownership of the delivered solutions.
• Review third-party designs, code, configurations, documentation, and deliverables to ensure compliance with the University’s standards and requirements.
• Ensure that appropriate technical documentation, source code, configurations, operational knowledge, and design rationale are transferred to the University.
• Develop internal data-engineering capability through documentation, mentoring, knowledge sharing, reusable components, and engineering standards.
• Communicate technical choices, trade-offs, progress, risks, service performance, and outcomes clearly to technical and non-technical stakeholders.

Analytics and Data Science Enablement

• Support analysts and data scientists in developing and operationalising repeatable analytical workloads.
• Assist with data exploration, feature preparation, notebooks, dashboards, prototypes, and proof-of-concept activities where required.
• Help implement appropriate controls for reproducibility, evaluation, monitoring, lineage, and lifecycle management.
• Develop selected analytical products or dashboards where these are required to validate or demonstrate data-platform capabilities.

AI and GenAI Data Enablement

• Enable AI solutions through the provision of trusted, reliable, governed, secure, and consumption-ready data for AI, GenAI, and agentic use cases.
• Support the preparation of retrieval-ready datasets and content, including document processing, metadata enrichment, chunking, indexing, embeddings, and vector-search integration where required.
• Provide secure and governed access to data, metadata, knowledge, and context through appropriate data services, APIs, or platform integrations.
• Implement controls for permissions, provenance, freshness, traceability, security, and responsible use of data by AI solutions.
• Work with AI specialists and domain experts to support retrieval quality, grounding, evaluation, and the appropriate use of authoritative data sources.
• Contribute to knowledge bases, taxonomies, ontologies, entity models, relationship models, or knowledge graphs where these are required by an approved solution.

Job Requirements

• Degree in Computer Science, Information Technology, Software Engineering, Data Science, or a related discipline.
• At least 8 years of relevant data-engineering or data-platform experience, with a track record of leading technical delivery and owning production solutions.
• Demonstrated experience designing, building, operating, and continuously improving enterprise data platforms and production data solutions.
• Proven experience building and operating cloud data warehouses, lakehouses, data lakes, or comparable modern data architectures.
• Experience implementing or supporting enterprise data catalogues and data governance platforms.
• Strong hands-on SQL and Python skills, with experience in distributed processing using Apache Spark or an equivalent framework and sound software-engineering practices.
• Strong experience in developing and operating ETL or ELT pipelines, including orchestration, validation, error handling, monitoring, and recovery.
• Experience integrating data from enterprise systems, databases, files, APIs, and cloud services.
• Strong understanding of data modelling, including dimensional, logical, physical, analytical, and semantic modelling.
• Strong understanding of data governance, privacy, and security, including metadata and lineage, access control, and protection of sensitive data across analytics and AI workloads.
• Hands-on DataOps and DevSecOps experience, including Git-based CI/CD, automated testing, infrastructure as code, monitoring, and incident troubleshooting.
• Sound software-engineering practices, including modular design, code review, documentation, testing, and maintainability.
• Experience owning or supporting production data services, including incident management, root-cause analysis, performance optimisation, and operational recovery.
• Ability to work directly with users and technical teams to translate requirements into reliable and supportable data products.
• Ability to explain technical decisions, risks, dependencies, and trade-offs clearly to technical and non-technical stakeholders.
• Experience coordinating vendors or delivery partners.