Description
Platform:
- Real-time API data and third-party specification analysis
- Data profiling and assessment of business value / quality
- Data contract, mapping specification, and lineage creation and maintenance
- Validation and acceptance testing
Skills:
1. Business Process Understanding:
- Ability to connect technical lineage with business processes, understanding how data supports decision-making or business operations.
- Ability to break down complex API specs, understand underlying behaviours, and create data contracts.
2. Knowledge of using JSON Schema for validating the structure of incoming real-time data.
3. API Documentation:
- Familiarity with reading and interpreting detailed API documentation.
4. Data Quality Assessment:
- Understanding the dimensions of data quality, including accuracy, completeness, consistency, timeliness, uniqueness, and validity.
- Proficient use of tools for profiling and cleansing data (like Informatica or SQL-based tools).
5. Understanding of Data Types:
- Familiarity with various data types (structured, semi-structured, unstructured) and how they should be handled in profiling.
Tools & Technologies:
- Proficiency with data profiling tools.
- Strong knowledge of SQL for querying databases, extracting data, and performing basic analysis.
- Clear communication to liaise with both technical teams (e.g., data engineers) and business stakeholders to agree on data definitions, structures, and rules.
- Understanding of logical and physical data models.
Responsibilities:
- Ability to define transformation logic for moving data from one system to another (e.g., mapping columns between source and target systems; applying business rules).
Technical Documentation:
- Skills in documenting the mapping logic clearly so that other data professionals can follow or update the specifications.
Validation & Acceptance Testing
Understanding of Data Flow:
- Ability to visualise and document how data moves across various stages of the pipeline from source to consumption ensuring full traceability.
Metadata Management:
- Understanding how metadata is captured used maintained; Data lineage often relies on detailed metadata to trace its origins transformations.