AI Visual Quality Annotation Specialist
Canva
Job Description
We’re seeking an AI-Native Visual Designer. A new-era hybrid designer with deep visual judgment, strong design fundamentals, and hands-on experience shaping AI systems. You will help train, evaluate, and improve AI-powered creative tools by applying expert annotation, quality assessment, and design-led thinking. This role sits at the intersection of design, data, and product/engineering, ensuring our models deliver visually excellent, brand-aligned, and human-centred output.
1. Visual Evaluation & Annotation
Annotate and label large volumes of visual data (images, UI layouts, compositions, graphics, photos).
Provide high-quality tagging related to style, composition, colour theory, typography, and visual hierarchy.
Evaluate AI-generated output for accuracy, aesthetic quality, usability, and alignment with design principles.
Identify patterns, errors, and edge cases to improve model performance.
2. Design Quality Assessment
Develop visual scoring criteria and rubrics to guide model improvement.
Review outputs across various creative formats (illustration, branding, UI components, marketing visuals).
Validate whether outputs meet user expectations, brand standards, accessibility guidelines, and ethical guidelines.
3. Cross-Functional Collaboration
Work closely with machine learning engineers, research scientists, and product teams to provide design-driven feedback loops.
Translate design concepts into structured feedback that model teams can use (e.g., style attributes, visual defects, composition rules).
Help communicate creative insights into technical documentation.
4. Prototyping & Creative Exploration
Use generative AI tools to create visuals, test hypotheses, and prototype user workflows.
Explore new capabilities and push the boundaries of AI-assisted creativity.
Offer creative direction for improving model outputs and user-facing features.
5. Data & Process Stewardship
Maintain data quality and consistency in annotation projects.
Contribute to the development of taxonomies for visual styles, attributes, and design patterns.
Document annotation methodologies and participate in improving annotation tooling.
Qualifications
Design Foundations
Strong grasp of composition, color theory, typography, layout, visual hierarchy, and storytelling principles.
Portfolio demonstrating visual design excellence across one or more domains (graphic, product, UI/UX, illustration, etc.)
AI & Technical Understanding
Familiarity with generative AI tools (image models, large language models, design AIs).
Understanding of how AI training data, annotation, and quality metrics influence model behavior.
Ability to evaluate model output with precision and articulate actionable feedback.
Collaboration & Communication
Able to translate design critique into structured, repeatable feedback for technical and product partners.
Experience collaborating with engineers, researchers, or technical teams is a plus.
Strong written communication skills and comfort in documentation-heavy workflows.
Bonus Skills
Experience with Figma, Adobe Creative Cloud, or 3D tools.
Prior data labeling, annotation, or ML operations experience.
Basic familiarity with scripting (Python), prompt engineering, or dataset tooling.
Knowledge of design accessibility (WCAG), usability heuristics, or user research methods.
This role is perfect for designers who are:
Curious about the future of AI-driven creativity
Comfortable with both artistic judgment and structured evaluation
Excited to influence how AI models “see” and “create”
Skilled in both visual taste and systematic thinking