Automated LLM Code Reviews with OpenCode and Gitlab CI/CD

Large Language Models (LLMs) are reshaping software engineering, whether we embrace them or approach them with caution. Applications range from practical to controversial, with "vibe coding" sparking debates about the democratization of software engineering, the destruction of the software engineering craftsmanship ethos, or the full-on replacement of the whole profession itself.

Rather than picking sides in this debate, we can focus on harnessing LLMs in ways to augment human expertise. One such way is to utilize the capabilities of LLMs for automated code reviews, freeing engineers to focus on higher-level design and collaboration.

In this post we explore how to integrate an LLM-powered code review agent into a GitLab CI/CD pipeline using OpenCode, an open-source agentic CLI that unifies access to multiple LLM providers.

Why OpenCode?

Most LLM providers offer proprietary coding assistants (e.g., Anthropic’s Claude Code, Mistral’s Vibe, or OpenAI’s Codex).

OpenCode stands out by unifying all LLM providers into one coherent basic agentic "harness," allowing us to easily swap in and out different models of different providers.

OpenCode | The open source AI coding agent

OpenCode - The open source coding agent.

OpenCode

In that, it provides the ideal foundation to experiment with a code review agent, as it grants us the flexibility to exchange models at a whim without much hassle, allowing us to strive for a good cost-to-performance ratio.

Connecting to Jira and GitLab

A useful code review agent must do more than just analyze a code base and generate text about it. It needs to:

• Fetch Context
  • Understand the merge request description, its diffs, and existing discussions around the proposed changes.
  • Retrieve Jira issue information referenced in the MR for additional context.
• Post feedback: Annotate the MR with inline comments and general suggestions.

To achieve this, we’ll build two custom CLI tools, one for GitLab, one for Jira, and expose them to the agent via agent skills in OpenCode.

Overview - Agent Skills

A simple, open format for giving agents new capabilities and expertise.

Agent Skills

The Jira Skill

The Jira integration is intentionally limited to read-only operations (fetching issue details). This ensures the agent can access relevant context without risking unintended modifications.

Implementation

To build the Jira Skill, we need 2 things:

1. A CLI tool (jira-cli) which queries the Jira API using environment variables for configuration (e.g., JIRA_TOKEN, JIRA_EMAIL, JIRA_URL).

# Fetch issue details from JIRA
jira-cli issue details <ISSUE_ID>

2. A SKILL.md file defining the tool’s purpose and usage examples for the agent:

---
name: jira-cli
description: Interact with the JIRA Cloud using the jira-cli tool. Use this skill to view issue details in JIRA projects.
---

# Jira CLI Tool - AI Agent Usage Guide

## Overview

This `jira-cli` tool provides command-line access to Jira issue functionality, allowing AI agents to interact with Jira's API programmatically. 
This guide explains how to use the tool effectively for common issue operations.

## Basic Usage

The tool follows this pattern:
```bash
jira-cli issue <subcommand> [options]
```

## Available Commands

### 1. Get Issue Details

```bash
jira-cli issue details <ISSUE_ID>
```

**Example:**
```bash
jira-cli issue details PROJ-123
```

**Output:**
```
Summary: Implement user authentication
Description: We need to implement a secure user authentication system that supports...
```

The agent can utilize this skill whenever it detects a Jira ticket reference (e.g., PROJ-123) to query for details.

The Gitlab Skill

The GitLab integration is more comprehensive, enabling the agent to:

• Fetch Merge Request details, diffs and existing discussions.
• Post general comments (e.g. "Consider adding tests for this newly introduced service")
• Post inline comments (e.g. "This null check is redundant, please remove it.")

Implementation

1. Again, we need a CLI tool (gitlab-cli) which queries the Gitlab API using environment variables for configuration (e.g., GITLAB_TOKEN, GITLAB_PROJECT_ID, GITLAB_URL).

# Fetch MR Details
gitlab-cli merge-request details <MR_ID>

# Fetch MR Diffs
gitlab-cli merge-request diff <MR_ID>

# Fetch MR Discussions
gitlab-cli merge-request discussions <MR_ID>

# Post a general comment to the MR
gitlab-cli merge-request comment <MR_ID> "Overall well structured MR!"

# Post an inline comment to the MR  
gitlab-cli merge-request comment <MR_ID> "This check is not null-safe! Please add a validation beforehand!" --file "path/to/file" --line 42 --line-type new|old

2. In addition, just like before, we also need a SKILL.md definition.

---
name: gitlab-cli
description: Interact with GitLab Merge Requests (MRs) using the gitlab-cli tool. Use this skill to show MR details, fetch changes, list existing discussions and add comments.
---

# GitLab CLI Tool - AI Agent Usage Guide

## Overview

This `gitlab-cli` tool provides command-line access to GitLab merge request functionality, allowing AI agents to interact with GitLab's API programmatically. 
This guide explains how to use the tool effectively for common merge request operations.

## Basic Usage

The tool follows this pattern:
```bash
gitlab-cli merge-request <subcommand> [options]
```

## Available Commands

### 1. Get Merge Request Details

```bash
gitlab-cli merge-request details <MR_ID>
```

**Example:**
```bash
gitlab-cli merge-request details 42
```

**Output:**
```
Merge Request ID: 42
Project ID: 12345
Title: Fix authentication bug
Description:
This MR fixes the authentication bug that was causing...
```

### 2. Get Merge Request Diffs

```bash
gitlab-cli merge-request diff <MR_ID>
```

**Example:**
```bash
gitlab-cli merge-request diff 42
```

**Output:**
Shows the complete diff of the merge request.

### 3. List Merge Request Discussions

```bash
gitlab-cli merge-request discussions <MR_ID>
```

**Example:**
```bash
gitlab-cli merge-request discussions 42
```

**Output:**
```
---- Comment by @alice ----
  File: src/auth.rs  Line: 42 (new)
This implementation looks good, but we should add more tests.

> Reply by @bob:
> Good point, I'll add them in a follow-up MR.

---- Comment by @charlie ----
  File: README.md
The documentation needs to be updated to reflect these changes.
```

### 4. Add a Comment to a Merge Request

#### General comment (not tied to specific code):
```bash
gitlab-cli merge-request comment <MR_ID> "Your comment text"
```

**Example:**
```bash
gitlab-cli merge-request comment 42 "This looks good to me, just one small suggestion."
```

#### Inline comment (tied to specific code):
```bash
gitlab-cli merge-request comment <MR_ID> "Your comment" --file "path/to/file" --line 42 --line-type new|old
```

**Example (new line):**
```bash
gitlab-cli merge-request comment 42 "This variable name could be more descriptive" --file "src/auth.rs" --line 42 --line-type new
```

**Example (old line):**
```bash
gitlab-cli merge-request comment 42 "This deprecated function should be removed" --file "src/legacy.rs" --line 15 --line-type old
```

It's compact yet obviously a little more elaborate than its Jira pendant.

Docker All The Things

To make the agent reusable across projects and teams, we bundle the CLI tools and their skill definitions into a Docker image.

FROM ghcr.io/anomalyco/opencode:latest

RUN apk update && \
    apk add --no-cache jq bash git coreutils gettext && \
    rm -rf /var/cache/apk/*

COPY gitlab-cli/target/x86_64-unknown-linux-musl/release/gitlab-cli /usr/local/bin/gitlab-cli
COPY gitlab-cli/SKILL.md /root/.config/opencode/skills/gitlab-cli/SKILL.md

COPY jira-cli/target/x86_64-unknown-linux-musl/release/jira-cli /usr/local/bin/jira-cli
COPY jira-cli/SKILL.md /root/.config/opencode/skills/jira-cli/SKILL.md


WORKDIR /project

ENTRYPOINT [""]

This approach has several benefits:

• Portability: The image works in the CI/CD pipeline as well as locally.
• Isolation: Dependencies and configurations are self-contained.
• Reusability: Teams working on various projects can adopt the agent without setup overhead.

How to Review.md?

The agent’s behavior will be defined by a system prompt (REVIEW.md) defined within the respective project using the agent, which should roughly outline:

• The agents role in the process (e.g. "Act as a Senior Java/Python/Rust software engineer reviewing this Merge Request")
• The workflow it should follow:
  • Fetch MR details, diffs and existing comments.
  • Review the changes critically wrt. code quality, correctness and security.
  • Analyze the code base further to explore dependencies.
  • Post feedback and suggestions back to the Gitlab MR.
• Instructions on project-specific rules like referring to coding conventions, tooling, general information about the project and it's structure, etc.
• Information on the MR ($CI_MERGE_REQUEST_IID) such that it can reference the MR and fetch information about it.

# **System Prompt: Automated Code Review Agent**

## **Role**
You are a **Senior Software Engineer** responsible for performing a **detailed and constructive review** of the Merge Request (MR):
- GitLab MR ID: `$CI_MERGE_REQUEST_IID`
- GitLab MR Title: `$CI_MERGE_REQUEST_TITLE`

Your goal is to:
- **Critically assess** the proposed changes for code quality, correctness, and security.
- **Ensure compliance** with the project’s coding standards (see General Rules and Language-Specific Rules).
- **Provide actionable feedback** (general comments + inline suggestions) to improve the MR before merging.
- **Avoid redundant feedback** by incorporating existing MR comments into your review process.
- **Post all feedback to the GitLab MR using the `gitlab-cli` tool** (general comments + inline suggestions).

## **Review Guidelines & Conventions**
Before analyzing the MR, familiarize yourself with the project’s coding standards and conventions by reading:
- **General Rules:** `@AGENTS/rules/general.md`
- **Language-Specific Rules:** `@AGENTS/rules/[language].md` (e.g., `java.md`, `python.md`, etc.)

Ensure all feedback aligns with these guidelines.

## **Review Workflow**
1. **Fetch MR Details and Existing Comments**
   - Use the **`gitlab-cli`** tool to retrieve:
     - MR metadata (title, description, author, textual description of proposed changes).
     - MR diffs (changes introduced).
     - All existing MR comments (both general and inline) to identify feedback that has already been addressed or declined.
   - **Check for Jira references** in the MR description or title (e.g., `MYPROJ-123`).
     - If found, use the **`jira-cli`** tool to retrieve:
       - Linked Jira ticket for additional context (e.g., task title and description).
     - If no Jira reference is found, skip querying Jira.

2. **Analyze Changes**
   - **Critically review** the diffs for:
     - **Code Quality:** Adherence to conventions, readability, and maintainability.
     - **Functional Correctness:** Logic errors, edge cases, or unintended side effects.
     - **Performance:** Inefficient algorithms, unnecessary computations, or resource leaks.
     - **Security:** Vulnerabilities (e.g., injection risks, hardcoded secrets).
     - **Testing:** Adequate test coverage (unit/integration tests).
     - **Documentation:** Updated comments, API docs, or READMEs where necessary.
   - **Cross-reference** with the checked-out Git repository (target branch) to:
     - Verify impacted files/modules.
     - Check for conflicts or inconsistencies with existing code.

3. **Analyze Existing Comments**
   - **Identify declined or resolved feedback** from previous reviews:
     - If a comment was declined (e.g., "Won't fix" or "Intentional"), do not raise the same issue again.
     - If a comment was resolved (e.g., addressed by the author), verify the fix and provide additional feedback if necessary.
   - **Prioritize new feedback** over revisiting old discussions unless the issue persists.

4. **Provide Feedback**
   - **General Comments:** Add MR-level suggestions (e.g., architectural concerns, missing tests).
   - **Inline Comments:** Pinpoint specific lines of code with actionable feedback (e.g., "Consider using a more efficient data structure here").
   - **Prioritize Feedback:**
     - Flag critical issues (e.g., bugs, security risks) vs. nitpicks (e.g., style preferences).
     - Do not repeat declined feedback unless the issue has resurfaced in a new context.
   - **Post all feedback to the GitLab MR using `gitlab-cli`** (general comments + inline suggestions).

5. **Leverage Tools**
   - **`gitlab-cli`:** Fetch MR diffs/changes, show MR details, add comments (inline or general), and retrieve existing comments.
   - **`jira-cli`:** Only used if a Jira ticket reference (e.g., `MYPROJ-123`) is found in the MR.
   - **Bash/Git:** Explore the codebase further (e.g., `git grep`, `git blame`) if context is unclear.

## **Feedback Principles**
- **Be Constructive:** Explain *why* a change is needed (e.g., "This loop could be optimized for better readability").
- **Be Specific:** Avoid vague comments like "This looks bad." Instead, say, "This method violates the Single Responsibility Principle; consider splitting it."
- **Be Respectful:** Frame suggestions as collaborative improvements (e.g., "Would it make sense to...?").
- **Avoid Redundancy:** Do not raise issues that have already been declined or resolved in previous comments.
- **Post all feedback via `gitlab-cli`** (general comments + inline suggestions).

---

### **Example Output (Before Posting to GitLab)**
```markdown
## General Feedback
- **Test Coverage:** The MR adds a new service class but lacks tests. Please add tests for edge cases (e.g., null inputs).
- **Jira Alignment:** This change partially addresses MYPROJ-456; consider also handling the timeout scenario mentioned in the ticket.

## Inline Comments
**File:** `src/main/example/UserService.java`
- **Line 42:** Replace direct comparison with a safer alternative for null safety.
- **Line 78:** This method is too long; refactor into smaller helper methods.
```

### **Notes**
- If the MR is **trivial** (e.g., typo fixes), approve it with minimal feedback and post the approval via `gitlab-cli`.
- For **complex changes**, prioritize high-impact issues (e.g., bugs, security) over style nits.
- Use **emojis sparingly** (e.g., 🚨 for critical issues, ✅ for approvals).
- Always check existing comments to avoid repeating feedback that has already been addressed or declined.
- **All feedback must be posted to the GitLab MR using `gitlab-cli`** (general comments + inline suggestions).

GitLab CI/CD Integration

We put everything together by setting up a Gitlab Pipeline definition to improve and maintain the derived solution so far.

1. We compile the CLI tools
2. We package the tools and skills into a docker image.
3. We execute the agent to review the MR.

stages:
  - build
  - docker
  - run

.build:
  stage: build
  image: rust
  cache:
    key: "$CI_COMMIT_REF_SLUG"
    paths:
      - target/
      - .cargo/
  before_script:
    - apt-get update && apt-get install -y musl-tools
    - rustup target add x86_64-unknown-linux-musl
  script:
    - cd $SKILL_NAME
    - cargo build --release --target x86_64-unknown-linux-musl
  artifacts:
    paths:
      - $SKILL_NAME/target/x86_64-unknown-linux-musl/release/$SKILL_NAME
      - $SKILL_NAME/SKILL.md
    expire_in: 1 hour

build-gitlab-cli:
  extends: .build
  variables:
    SKILL_NAME: gitlab-cli
  rules:
    - if: $CI_MERGE_REQUEST_IID
    - if: $CI_COMMIT_TAG

build-jira-cli:
  extends: .build
  variables:
    SKILL_NAME: jira-cli
  rules:
    - if: $CI_MERGE_REQUEST_IID
    - if: $CI_COMMIT_TAG

.docker:
  stage: docker
  image: docker
  services:
    - docker:dind
  before_script:
    - docker login -u "$DOCKER_HUB_USERNAME" -p "$DOCKER_HUB_PASSWORD"

build-docker-mr:
  extends: .docker
  rules:
    - if: $CI_MERGE_REQUEST_IID
      when: manual
  script:
    - docker build -t "$DOCKER_HUB_REPO:$CI_COMMIT_SHORT_SHA" .
    - docker push "$DOCKER_HUB_REPO:$CI_COMMIT_SHORT_SHA"

build-docker-tag:
  extends: .docker
  rules:
    - if: $CI_COMMIT_TAG
  script:
    - docker build -t "$DOCKER_HUB_REPO:$CI_COMMIT_TAG" -t "$DOCKER_HUB_REPO:latest" .
    - docker push "$DOCKER_HUB_REPO:$CI_COMMIT_TAG"
    - docker push "$DOCKER_HUB_REPO:latest"

run-mr-review:
  stage: run
  image: $DOCKER_HUB_REPO:$CI_COMMIT_SHORT_SHA
  timeout: 15m
  rules:
    - if: $CI_MERGE_REQUEST_IID
      when: manual
  script:
    - envsubst < REVIEW.md
    - opencode run --model mistral/devstral-medium-latest < <(envsubst < REVIEW.md)

This setup defines the workflow for improving, extending and iterating on the agent itself.

Local Usage: Terminal or Web UI

The Docker image we built must not just be used for the CI/CD pipeline; we can also use it locally in one of the two following ways.

Terminal Mode

Via a simple shell script (opencode.sh) we can launch the agent in the terminal.

#!/bin/bash
docker run -it \
    --env-file .env \
    -v "$(pwd):/project/" \
    niggoo/opencode-review-agent:latest \
    "${@:-opencode}"

We can set this up locally on our machine or have it in the projects git.

Web UI Mode

For a ChatGPT-like interface, we can use a docker-compose.yml definition to run OpenCode in web mode.

services:
  opencode-agent:
    image: niggoo/opencode-review-agent:latest
    ports:
      - "1234:1234"
    entrypoint: ["opencode", "web", "--port", "1234", "--hostname", "0.0.0.0"]
    env_file:
      - .env
    volumes:
      - ./:/project/
    restart: unless-stopped

Crucially, the benefits of the combined approach using OpenCode + Skills + Docker becomes clear:

1. Consistency: The same container and agent harness can be used in Gitlab CI/CD as well as locally.
2. Flexbility: Swapping models or providers requires no change in setup, different ones can be used depending on the specific setup and needs.
3. Tool Availability: The tools allowing to connect to Jira and Gitlab are available everywhere in an agnostic and generic way. (Configured via environment variables)

Key Takeaways

Automated code reviews powered by Large Language Models (LLMs) are transforming quality assurance in software engineering. While they effectively streamline the initial review phase, subsequent rounds should arguably still be done on human oversight to ensure thorough validation and refinement.

However, rather than viewing these tools as replacements for human expertise, they serve as collaborative partners that provide consistent, scalable feedback. The system we've built demonstrates how to integrate LLMs in a way that augments engineering workflows without disrupting established processes.

The use of Docker and OpenCode ensures the solution remains portable across different projects and teams, requiring minimal setup to deploy in new environments.

The full source code, including the complete GitLab CI/CD integration alongside scripts for local usage (opencode.sh and docker-compose.yml), is available on Gitlab.