Skip to main content

GitLab

This connector captures data from GitLab repositories and organizations into Flow collections via GitLabs's API V4. It can also work with self-hosted GitLab.

It is available for use in the Flow web application. For local development or open-source workflows, ghcr.io/estuary/source-gitlab:dev provides the latest version of the connector as a Docker image. You can also follow the link in your browser to see past image versions.

This connector is based on an open-source connector from a third party, with modifications for performance in the Flow system. You can find their documentation here, but keep in mind that the two versions may be significantly different.

Supported data resources​

When you configure the connector, you may a list of GitLab Groups or Projects from which to capture data.

From your selection, the following data resources are captured:

Resources​

Each resource is mapped to a Flow collection through a separate binding.

Prerequisites​

There are two ways to authenticate with GitLab when capturing data into Flow: using OAuth2, and manually, by generating a personal access token. Their prerequisites differ.

OAuth is recommended for simplicity in the Flow web app; the access token method is the only supported method using the command line. Which authentication method you choose depends on the policies of your organization. Github has special organization settings that need to be enabled in order for users to be able to access repos that are part of an organization.

Using OAuth2 to authenticate with GitLab in the Flow web app​

  • A GitLab user account with access to the repositories of interest, and which is a member of organizations of interest. How to add a member.

Configuring the connector specification manually using personal access token​

Configuration​

You configure connectors either in the Flow web app, or by directly editing the catalog specification file. See connectors to learn more about using connectors. The values and specification sample below provide configuration details specific to the GitHub source connector.

Setup​

  1. Complete authentication using Oauth or a PAT
  2. Select your start date in the format 2023-08-31T00:00:00
  3. Optionally select Groups and Projects

Sample​

This sample specification reflects the manual authentication method.

captures:
  ${PREFIX}/${CAPTURE_NAME}:
    endpoint:
      connector:
        image: ghcr.io/estuary/source-gitlab:dev
          config:
            credentials:
              option_title: PAT Credentials
              personal_access_token: {secret}
            groups: estuary.dev
            projects: estuary/flow
            start_date: 2022-01-01T00:00:00Z
      bindings:
        - resource:
            stream: branches
            syncMode: full_refresh
          target: ${PREFIX}/assignees
       {...}