Agile - a methodology for AI projects

What is Agile?

Agile is a project management methodology, which is a system or approach used to plan, organise, and manage tasks and resources in a project. 

Project methodologies are used by project teams to: 

• Track Progress: They help keep track of what has been done and what still needs to be done.
• Stay Organised: They ensure that everyone knows their tasks and responsibilities.
• Meet Goals and Deadlines: They help teams deliver the project on time and within budget.

Agile, specifically, emphasises flexibility, collaboration, and customer satisfaction.

Traditional Methods vs. Agile

Traditional methods follow a linear path. This means you do one big task at a time from start to finish. In contrast, Agile is iterative.  It breaks the project into smaller parts and completes them in small, manageable chunks called "sprints". Each “sprint”, or cycle, involves planning, executing, and reviewing a set of tasks, allowing for continuous improvement and adjustments based on feedback.

How Agile Works

1. Sprints: Each sprint typically lasts two to four weeks. During each sprint, a specific set of tasks is completed, resulting in a potentially shippable product increment. This could be a feature, a piece of software, or a working model.

2. Flexibility: After each sprint, the team reviews the work, gets feedback, and makes adjustments before starting the next sprint. This allows for continuous improvement and adaptation to changes.

3. Collaboration: Agile encourages regular communication and collaboration among team members and stakeholders to ensure everyone is aligned and working towards the same goals.

Agile in AI Projects

AI projects, by their nature, are complex and require a high degree of flexibility. Here’s how Agile fits perfectly with AI development:

1. Iterative Development

AI models and systems are built iteratively. Initial versions of an AI model might be simple, but they get refined through multiple iterations based on feedback and additional data. Agile's sprint cycles are ideal for this continuous improvement process.

2. Constant Feedback Loop

AI projects benefit greatly from constant feedback. Whether it's from stakeholders or through testing and validation of AI models, Agile ensures that feedback is integrated continuously, leading to better and more accurate AI systems.

3. Collaboration and Team Dynamics

AI projects often require collaboration between data scientists, software developers, domain experts, and stakeholders. Agile promotes a culture of collaboration and open communication, which is crucial for integrating diverse expertise effectively.

4. Handling Uncertainty and Change

AI projects are inherently uncertain due to the experimental nature of building models. Agile's flexibility allows teams to pivot or make adjustments based on new findings, data, or changes in project scope without significant disruption.

Example: Agile in an AI Project

Imagine you're working on a project to develop an AI-powered chatbot for customer service. Here's how you might apply Agile:

1. Sprint Planning: The team decides to start with a simple version of the chatbot that can handle basic enquiries.
2. First Sprint: In two weeks, the team develops a basic chatbot capable of answering FAQs. They test it and gather user feedback.
3. Sprint Review: The team presents the chatbot to stakeholders, collects feedback, and identifies areas for improvement.
4. Sprint Retrospective: The team discusses what went well, such as successful integration of the chatbot with the existing system, and what could be improved, like enhancing the chatbot's response accuracy.
5. Next Sprint: Based on feedback, the team works on improving the chatbot’s natural language understanding and adding more complex queries.

Agile Key Concepts

Understanding User Stories

• Definition: A user story is a short, simple description of a feature from the perspective of the end user, often following the format: "As a [type of user], I want [an action] so that [a benefit/a reason]."
• Purpose: User stories help clarify what the user needs and why, making it easier to plan and prioritise work.

Product Backlog

• Definition: The product backlog is a list of all the tasks, features, and requirements for the project.
• Maintenance: The backlog is continuously updated and prioritised by the product owner based on feedback and changing requirements.

Scrum Roles

• Product Owner: Represents the stakeholders and ensures the team delivers value to the business.
• Scrum Master: Facilitates the Agile process, removes obstacles, and ensures the team follows Agile practices.
• Development Team: The group of professionals who do the work (e.g., developers, testers, designers).

Agile Tools and Practices

Task Boards (Kanban Boards)

• Purpose: Visual tools that help track the progress of tasks within a sprint.
• Columns: Typically include columns such as "To Do," "In Progress," and "Done."
• Usage: Move tasks across columns as they progress through different stages of completion.

Time Management

• Daily Stand-ups: Short daily meetings (usually 15 minutes) where team members discuss what they did yesterday, what they will do today, and any obstacles they're facing.
• Timeboxing: Allocating a fixed time period to complete a task or activity, helping to keep work focused and manageable.

Sprint Retrospectives

• Purpose: Regular meetings at the end of each sprint where the team reflects on what went well, what didn't, and how to improve.
• Outcome: Actionable insights and adjustments to improve the next sprint.

Agile Planning Principles

Rolling Wave Planning

• Definition: Plans are continually updated as the project progresses. This allows for adjustments based on new information and changing circumstances.
• Purpose: Ensures that planning is flexible and adaptive, allowing the team to focus on the most current and relevant tasks.

Value-Based Prioritisation

• Definition: Work is prioritised based on the value it delivers to the customer. Higher value items are addressed first to maximise the impact of the project.
• Purpose: Ensures that the team is always working on the most important and valuable tasks, delivering the greatest benefit to the customer.

Estimating Techniques

Story Points

• Definition: Abstract units of measure used to estimate the effort required to implement a user story. They are not tied to a specific amount of time but rather to the complexity and effort involved.
• Purpose: Helps teams estimate the relative effort needed for different tasks, making it easier to plan and allocate resources.

Planning Poker

• Definition: A consensus-based technique for estimating user stories using story points. Team members use cards with numbers to represent their estimates and discuss any discrepancies to reach a consensus.
• Purpose: Promotes discussion and ensures that all team members are involved in the estimation process, leading to more accurate and agreed-upon estimates.

User Stories vs. Use Cases

While both user stories and use cases are used in Agile and software development to define requirements, they are not the same. They serve different purposes and are used at different levels of detail.

User Stories

Definition: A user story is a simple, short description of a feature from the perspective of the end user. It focuses on what the user wants to achieve and why.

Structure: User stories are typically written in a specific format:

• "As a [type of user], I want [some goal] so that [some reason].”

‍Example:

• "As a customer, I want to be able to reset my password so that I can access my account if I forget it.”

Purpose:

• High-Level: User stories provide a high-level overview of the requirements without going into detailed specifics. They are used to capture user needs and guide the development process.
• Collaboration: They encourage conversations and collaboration between the development team and stakeholders to clarify requirements and ensure that the end product meets user needs.

Benefits:

• Flexibility: They are easy to write and modify.
• User-Centric: They keep the focus on the user's needs and goals.
• Encourage Discussion: They serve as a starting point for more detailed discussions and planning.

Use Cases

Definition: A use case is a detailed description of how a user interacts with a system to achieve a specific goal. It includes a sequence of steps and covers various scenarios, including alternative paths and exceptions.

Structure: Use cases are more structured and detailed. They often include:

• Title: The name of the use case.
• Primary Actor: The user or system that initiates the interaction.
• Goal: The objective of the use case.
• Preconditions: Conditions that must be true before the use case can start.
• Main Success Scenario: A step-by-step description of the interaction.
• Extensions/Alternative Paths: Other possible interactions or exceptions.

Example:

• Title: Reset Password
• Primary Actor: Customer
• Goal: Customer resets their password successfully.
• Preconditions: Customer is on the login page and has clicked "Forgot Password."
• Main Success Scenario:some text
  1. Customer clicks on "Forgot Password."
  2. System prompts for the registered email address.
  3. Customer enters email address.
  4. System sends a password reset link to the email.
  5. Customer clicks the link and is prompted to enter a new password.
  6. Customer enters and confirms the new password.
  7. System updates the password and confirms success.
• Extensions:some text
  1. If the email address is not registered, the system displays an error message.

Purpose:

• Detailed Requirements: Use cases provide a comprehensive, detailed view of the interactions between the user and the system.
• Documentation: They serve as detailed documentation that can be referred to throughout the development process.
• Clarification: They help clarify all possible scenarios and interactions, ensuring that nothing is overlooked.

Benefits:

• Comprehensive: They cover all possible scenarios, including edge cases.
• Detailed Guidance: They provide detailed guidance for developers, testers, and stakeholders.

Final Thoughts

Agile is all about collaboration, continuous improvement, and flexibility.

‍