Introduction

Packeton - Private PHP package repository for vendors.

Documentation docs.packeton.org

Main Features

• Compatible with Composer API v2, bases on Symfony 6.
• Support update webhook for GitHub, Gitea, Bitbucket and GitLab or custom format.
• Customers user and ACL groups and limit access by vendor and versions.
• Composer Proxies and Mirroring.
• Generic Packeton webhooks
• Allow to freeze updates for the new releases after expire a customers license.
• Mirroring for packages zip files and downloads it's from your host.
• Credentials and Authentication http-basic config or ssh keys.
• Support monolithic repositories, like symfony/symfony
• Pull Request composer.lock change review.
• OAuth2 GitHub, Bitbucket, GitLab/Gitea and Other Integrations.
• Security Monitoring.
• Milty sub repositories.

Compare Private Packagist with Packeton

Install and Run

There is an official packeton image available at https://hub.docker.com/r/packeton/packeton which can be used with the docker-compose file, see docker installation

Installation

Requirements

• PHP 8.1+
• Redis for some functionality (favorites, download statistics, worker queue).
• git/svn/hg depending on which repositories you want to support.
• Supervisor to run a background job worker
• (optional) MySQL or PostgresSQL for the main data store, default SQLite

1. Clone the repository

git clone https://github.com/vtsykun/packeton.git /var/www/packeton/
cd /var/www/packeton/

2. Install dependencies composer install
3. Create .env.local and copy needed environment variables into it. See Configuration
4. IMPORTANT! Don't forget change APP_SECRET
5. Run bin/console doctrine:schema:update --force --complete to setup the DB
6. Create admin user via console.

php bin/console packagist:user:manager username --email=admin@example.com --password=123456 --admin 

7. Setup nginx or any webserver, for example nginx config looks like.

server {
    listen *:443 ssl http2;
    server_name packeton.example.org;
    root /var/www/packeton/public;

    ssl_certificate /etc/nginx/ssl/example.crt;
    ssl_certificate_key /etc/nginx/ssl/example.key;

    ssl_ciphers 'TLS13-CHACHA20-POLY1305-SHA256:TLS13-AES-128-GCM-SHA256:TLS13-AES-256-GCM-SHA384:ECDHE:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-SHA384:ECDHE-RSA-AES128-SHA256:ECDHE-RSA-AES256-SHA:ECDHE-RSA-AES128-SHA:ECDHE-RSA-DES-CBC3-SHA:AES256-GCM-SHA384:AES128-GCM-SHA256:AES256-SHA256:AES128-SHA256:AES256-SHA:AES128-SHA:DES-CBC3-SHA:!aNULL:!eNULL:!EXPORT:!DES:!MD5:!PSK:!RC4';

    ssl_protocols TLSv1.1 TLSv1.2 TLSv1.3;
    ssl_prefer_server_ciphers on;
    ssl_session_cache  builtin:1000  shared:SSL:10m;
    ssl_session_timeout  5m;

    rewrite ^/index\.php/?(.+)$ /$1 permanent;
    try_files $uri @rewriteapp;

    location @rewriteapp {
        rewrite ^(.*)$ /index.php/$1 last;
    }

    access_log off;

    location ~ ^/index\.php(/|$) {
        fastcgi_split_path_info ^(.+\.php)(/.*)$;
        include fastcgi_params;
        fastcgi_param  SCRIPT_FILENAME $document_root$fastcgi_script_name;
        fastcgi_index index.php;
        send_timeout 300;
        fastcgi_read_timeout 300;
        fastcgi_pass unix:/run/php/php8.1-fpm.sock;
    }
}

8. Change cache permission to made it accessible for web server.

chown www-data:www-data -R var/

9. If you get a 500 error in index page packeton.example.org, please check your logs var/log/prod.log or/and webserver log and fix permissions, database config, redis etc.

10. Enable cron tabs and background jobs.

Enable crontab crontab -e -u www-data

* * * * * /var/www/packeton/bin/console --env=prod okvpn:cron >> /dev/null

Setup Supervisor to run worker

sudo apt -y --no-install-recommends install supervisor

Create a new supervisor configuration.

sudo vim /etc/supervisor/conf.d/packagist.conf

Add the following lines to the file.

[program:packeton-workers]
environment =
        HOME=/var/www/
command=/var/www/packeton/bin/console packagist:run-workers
directory=/var/www/packeton/
process_name=%(program_name)s_%(process_num)02d
numprocs=1
autostart=true
autorestart=true
startsecs=0
redirect_stderr=true
priority=1
user=www-data

Configuration

Create a file .env.local and change next options

• APP_SECRET - Must be static, used for encrypt SSH keys in database.
• APP_COMPOSER_HOME - composer home, default /var/www/packeton/var/.composer/
• DATABASE_URL - Database DSN, default sqlite:///%kernel.project_dir%/var/app.db

Example for postgres postgresql://app:pass@127.0.0.1:5432/app?serverVersion=14&charset=utf8 Example for mysql mysql://app:!ChangeMe!@127.0.0.1:3306/app?serverVersion=8&charset=utf8mb4

# .env.local

DATABASE_URL="postgresql://app:!ChangeMe!@127.0.0.1:5432/app?serverVersion=14&charset=utf8"

• PACKAGIST_DIST_PATH - Default %kernel.project_dir%/var/zipball, path to storage zipped artifacts
• REDIS_URL - Redis DB, default redis://localhost
• PACKAGIST_DIST_HOST - Hostname, (auto) default use the current host header in the request.
• TRUSTED_PROXIES - Ips for Reverse Proxy. See Symfony docs
• PUBLIC_ACCESS - Allow anonymous users access to read packages metadata, default: false
• MAILER_DSN - Mailer for reset password, default disabled
• MAILER_FROM - Mailer from

Ssh key access and composer oauth token.

Packagist uses the composer config and global ssh-key to get read access to your repositories, so the supervisor worker packagist:run-workers and web-server must run under the user, that have ssh key or composer config that gives it read (clone) access to your git/svn/hg repositories. For example, if your application runs under www-data and have home directory /var/www, directory structure must be like this.

    └── /var/www/
        └── .ssh/ # ssh keys directory
            ├── config
            ├── id_rsa # main ssh key
            ├── private_key_2 # additional ssh key
            └── private_key_3
        └── packeton/ # project dir
            ├── config APP_COMPOSER_HOME="%kernel.project_dir%/var/.composer"
            ├── public 
            ....
            ├── src 
            └── var
                ├── cache
                ....
                └── .composer # APP_COMPOSER_HOME="%kernel.project_dir%/var/.composer"

By default, composer configuration load from COMPOSER_HOME and it placed at path %kernel.project_dir%/var/.composer. if you want to setup authentication in auth.json need to place this file to composer home, i e. /var/www/packeton/var/.composer/ See Authentication in auth.json

# Example /var/www/packeton/var/.composer/auth.json
{ 
  "http-basic": {
    "git.example.pl": {
      "username": "kastus",
      "password": "489df705a503ac0173256ce01f"
    }
  }
}

Example ssh config for multiple SSH Keys for different github account/repos, see here for details

# ~/.ssh/config - example

Host github-oroinc
	HostName github.com
	User git
	IdentityFile /var/www/.ssh/private_key_2
	IdentitiesOnly yes

Host github-org2
	HostName github.com
	User git
	IdentityFile /var/www/.ssh/private_key_3
	IdentitiesOnly yes

Allow connections to http.

You can create config.json in the composer home (see APP_COMPOSER_HOME env var) or add this option in the UI credentials form.

{
    "secure-http": false
}

Install and Run in Docker

You can use prebuild packeton/packeton image

Quick start

docker run -d --name packeton \
    --mount type=volume,src=packeton-data,dst=/data \
    -p 8080:80 \
    packeton/packeton:latest

After container is running, you may wish to create an admin user via command packagist:user:manager

docker exec -it packeton bin/console packagist:user:manager admin --password=123456 --admin

Docker Environment

All env variables is optional. By default, Packeton uses an SQLite database and build-in redis service, but you can overwrite it by env REDIS_URL and DATABASE_URL. The all app data is stored in the VOLUME /data

• APP_SECRET - Must be static, used for encrypt SSH keys in database. The value is generated automatically, see .env in the data volume.

• APP_COMPOSER_HOME - composer home, default /data/composer

• DATABASE_URL - Database DSN, default sqlite:////data/app.db. Example for postgres "postgresql://app:pass@127.0.0.1:5432/app?serverVersion=14&charset=utf8"

• PACKAGIST_DIST_PATH - Default /data/zipball, path to storage zipped versions

• REDIS_URL - Redis DB, default redis://localhost

• PACKAGIST_DIST_HOST - Hostname, (auto) default use the current host header in the request. Overwrite packagist host (example https://packagist.youcomany.org). Used for downloading the mirroring zip packages. (The host add into dist url for composer metadata). The default value is define dynamically from the header Host.

• TRUSTED_PROXIES - Ips for Reverse Proxy. See Symfony docs

• PUBLIC_ACCESS - Allow anonymous users access to read packages metadata, default: false

• MAILER_DSN - Mailer for reset password, default disabled

• MAILER_FROM - Mailer from

• ADMIN_USER Creating admin account, by default there is no admin user created so you won't be able to login to the packagist. To create an admin account you need to use environment variables to pass in an initial username and password (ADMIN_PASSWORD, ADMIN_EMAIL).

• ADMIN_PASSWORD - used together with ADMIN_USER

• ADMIN_EMAIL - used together with ADMIN_USER

• PRIVATE_REPO_DOMAIN_LIST - Save ssh fingerprints to known_hosts for this domain.

VOLUME

The all app data is stored in the VOLUME /data

User docker compose

The typical example docker-compose.yml

version: '3.6'

services:
    packeton:
        image: packeton/packeton:latest
        container_name: packeton
        hostname: packeton
        environment:
            ADMIN_USER: admin
            ADMIN_PASSWORD: 123456
            ADMIN_EMAIL: admin@example.com
            DATABASE_URL: mysql://app:!ChangeMe!@127.0.0.1:3306/app?serverVersion=8&charset=utf8mb4
        ports:
            - '127.0.0.1:8080:80'
        volumes:
            - .docker:/data

By default, the container starts the supervisor, which is used to run other tasks: nginx, redis, php-fpm, cron, however, you can start one service per container. See docker-compose-prod.yml example:

version: '3.9'

x-volumes: &default-volume
    volumes:
        - app-data:/data
        - app-var:/var/www/packagist/var

x-restart-policy: &restart_policy
    restart: unless-stopped

x-environment: &default-environment
    REDIS_URL: redis://redis
    DATABASE_URL: "postgresql://packeton:pack123@postgres:5432/packeton?serverVersion=14&charset=utf8"
    SKIP_INIT: 1

services:
    redis:
        image: redis:7-alpine
        hostname: redis
        <<: *restart_policy
        volumes:
            - redis-data:/data
 
    postgres:
        image: postgres:14-alpine
        hostname: postgres
        <<: *restart_policy
        volumes:
            - postgres-data:/var/lib/postgresql/data
        environment:
            POSTGRES_USER: packeton
            POSTGRES_PASSWORD: pack123
            POSTGRES_DB: packeton

    php-fpm:
        image: packeton/packeton:latest
        hostname: php-fpm
        command: ['php-fpm', '-F']
        <<: *restart_policy
        <<: *default-volume
        environment:
            <<: *default-environment
            SKIP_INIT: 0
            WAIT_FOR_HOST: 'postgres:5432'
        depends_on:
            - "postgres"
            - "redis"

    nginx:
        image: packeton/packeton:latest
        hostname: nginx
        ports:
            - '127.0.0.1:8088:80'
        <<: *restart_policy
        <<: *default-volume
        command: >
            bash -c 'sed s/_PHP_FPM_HOST_/php-fpm:9000/g < docker/nginx/nginx-tpl.conf > /etc/nginx/nginx.conf && nginx'
        environment:
            <<: *default-environment
            WAIT_FOR_HOST: 'php-fpm:9000'
        depends_on:
            - "php-fpm"

    worker:
        image: packeton/packeton:latest
        hostname: packeton-worker
        command: ['bin/console', 'packagist:run-workers', '-v']
        user: www-data
        <<: *restart_policy
        <<: *default-volume
        environment:
            <<: *default-environment
            WAIT_FOR_HOST: 'php-fpm:9000'
        depends_on:
            - "php-fpm"

    cron:
        image: packeton/packeton:latest
        hostname: packeton-cron
        command: ['bin/console', 'okvpn:cron', '--demand', '--time-limit=3600']
        user: www-data
        <<: *restart_policy
        <<: *default-volume
        environment:
            <<: *default-environment
            WAIT_FOR_HOST: 'php-fpm:9000'
        depends_on:
            - "php-fpm"

volumes:
    redis-data:
    postgres-data:
    app-data:
    app-var:

Build and run docker container with docker-compose

1. Clone repository

git clone https://github.com/vtsykun/packeton.git /var/www/packeton/
cd /var/www/packeton/

2. Run docker-compose build

docker-compose build

# start container.
docker-compose up -d # Run with single supervisor container 
docker-compose up -f docker-compose-prod.yml -d # Or split 

Using a reverse proxy

It is recommended to put a reverse proxy such as nginx, Apache for docker installation.

Reverse-proxy configuration examples

nginx

server {
    listen *:443 ssl http2;

    server_name pkg.example.gob.ar;

    ssl_certificate /etc/nginx/ssl/gob.ar.crt;
    ssl_certificate_key /etc/nginx/ssl/gob.ar.key;
    ssl_ciphers 'ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-SHA384:ECDHE-RSA-AES128-SHA256:ECDHE-RSA-AES256-SHA:ECDHE-RSA-AES128-SHA:ECDHE-RSA-DES-CBC3-SHA:AES256-GCM-SHA384:AES128-GCM-SHA256:AES256-SHA256:AES128-SHA256:AES256-SHA:AES128-SHA:DES-CBC3-SHA:!aNULL:!eNULL:!EXPORT:!DES:!MD5:!PSK:!RC4';

    gzip_vary on;
    gzip_proxied any;
    gzip_comp_level 6;
    gzip_buffers 16 16k;
    gzip_http_version 1.1;
    gzip_min_length 2048;
    gzip_types text/css application/javascript text/javascript application/json;
    access_log  off;

    location / {
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-Proto https;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_pass http://127.0.0.1:8082/;
    }
}

nginx with cloudflare

server {
    listen *:80;

    server_name demo.packeton.org;

    gzip_vary on;
    gzip_proxied any;
    gzip_comp_level 6;
    gzip_buffers 16 16k;
    gzip_http_version 1.1;
    gzip_min_length 2048;
    gzip_types text/css application/javascript text/javascript application/json;

    set_real_ip_from 173.245.48.0/20;
    set_real_ip_from 103.21.244.0/22;
    set_real_ip_from 103.22.200.0/22;
    set_real_ip_from 103.31.4.0/22;
    set_real_ip_from 141.101.64.0/18;
    set_real_ip_from 108.162.192.0/18;
    set_real_ip_from 190.93.240.0/20;
    set_real_ip_from 188.114.96.0/20;
    set_real_ip_from 197.234.240.0/22;
    set_real_ip_from 198.41.128.0/17;
    set_real_ip_from 162.158.0.0/15;
    set_real_ip_from 104.16.0.0/13;
    set_real_ip_from 104.24.0.0/14;
    set_real_ip_from 172.64.0.0/13;
    set_real_ip_from 131.0.72.0/22;
    real_ip_header CF-Connecting-IP;
    add_header Access-Control-Allow-Origin *;

    location / {
        proxy_http_version 1.1;
        proxy_buffering off;
        proxy_set_header Host $http_host;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto https;
        proxy_pass http://127.0.0.1:8082/;
    }
}

Apache

<VirtualHost *:443>
    ServerName pack1.loc.example.ovh
    SSLEngine on
    SSLCertificateFile /etc/nginx/ssl/example.crt
    SSLCertificateKeyFile /etc/nginx/ssl/example.key

    Protocols h2 http/1.1
    ProxyRequests on
    ProxyPreserveHost On
    ProxyPass / http://127.0.0.1:8082/
    ProxyPassReverse / http://127.0.0.1:8082/

    SSLProxyEngine On
    SSLProxyVerify none
    SSLProxyCheckPeerCN off
    SSLProxyCheckPeerName off
    SSLProxyCheckPeerExpire off

    RequestHeader set "X-Forwarded-Proto" expr=%{REQUEST_SCHEME}
</VirtualHost>

Administration

This section contains information on managing your Packeton application.

Table of content

• API Usage
• Outgoing Webhook
  • Intro
  • Examples
    • Telegram notification
    • Slack notification
    • JIRA issue fix version
    • Gitlab setup auto webhook
• Application Roles

Application Roles

• ROLE_USER - minimal customer access level, these users only can read metadata only for selected packages.
• ROLE_FULL_CUSTOMER - Can read all packages metadata.
• ROLE_MAINTAINER - Can submit a new package and read all metadata.
• ROLE_ADMIN - Can create a new customer users, management webhooks and credentials.

You can create a user and then promote to admin or maintainer via console using fos user bundle commands.

php bin/console packagist:user:manager username --email=admin@example.com --password=123456 --admin # create admin user
php bin/console packagist:user:manager user1 --add-role=ROLE_MAINTAINER # Add ROLE_MAINTAINER to user user1

API documentation

About API authorization methods see here

Submit package

POST https://example.com/api/create-package?token=<api_token>
Content-Type: application/json

{
    "repository": {
        "url": "git@github.com:symfony/mime.git"
    }
}

Listing package names

GET https://example.com/packages/list.json?token=<api_token>

# Result
{
  "packageNames": [
    "[vendor]/[package]",
    ...
  ]
}

List packages by vendor

GET https://example.com/packages/list.json?vendor=[vendor]&token=<api_token>

{
  "packageNames": [
    "[vendor]/[package]",
    ...
  ]
}

List packages by type

GET https://example.com/packages/list.json?type=[type]&token=<api_token>

{
  "packageNames": [
    "[vendor]/[package]",
    ...
  ]
}

Get the package git changelog

Get git diff between two commits or tags. WARNING Working only if repository was cloned by git. If you want to use this feature for GitHub you need set composer config flag no-api see here

GET https://example.com/packages/{name}/changelog?token=<api_token>&from=3.1.14&to=3.1.15

{
  "result": [
    "BAP-18660: ElasticSearch 6",
    "BB-17293: Back-office >Wrong height"
  ],
  "error": null,
  "metadata": {
    "from": "3.1.14",
    "to": "3.1.15",
    "package": "okvpn/platform"
  }
}

Getting package data

This is the preferred way to access the data as it is always up-to-date, and dumped to static files so it is very efficient on our end.

You can also send If-Modified-Since headers to limit your bandwidth usage and cache the files on your end with the proper filemtime set according to our Last-Modified header.

There are a few gotchas though with using this method:

• It only provides you with the package metadata but not information about the maintainers, download stats or github info.
• It contains providers information which must be ignored but can appear confusing at first. This will disappear in the future though.

GET https://example.com/p/[vendor]/[package].json?token=<api_token>

{
  "packages": {
    "[vendor]/[package]": {
      "[version1]": {
        "name": "[vendor]/[package],
        "description": [description],
        // ...
      },
      "[version2]": {
        // ...
      }
      // ...
    }
  }
}

Composer v2

GET https://example.com/p2/firebase/php-jwt.json


{
  "minified": "composer/2.0",
  "packages": {
    "[vendor]/[package]": [... list versions ]
  }
}

Using the API

The JSON API for packages gives you all the infos we have including downloads, dependents count, github info, etc. However, it is generated dynamically so for performance reason we cache the responses for twelve hours. As such if the static file endpoint described above is enough please use it instead.

GET https://example.com/packages/[vendor]/[package].json?token=<api_token>

{
  "package": {
    "name": "[vendor]/[package],
    "description": [description],
    "time": [time of the last release],
    "maintainers": [list of maintainers],
    "versions": [list of versions and their dependencies, the same data of composer.json]
    "type": [package type],
    "repository": [repository url],
    "downloads": {
      "total": [numbers of download],
      "monthly": [numbers of download per month],
      "daily": [numbers of download per day]
    },
    "favers": [number of favers]
  }
}

Generic Packeton webhooks

Introduction

Webhooks allow external services to be notified when certain events happen. When the specified events happen, packeton will send a POST request to each of the URLs you provide. Now is supported the next events:

• new_release
• update_release
• delete_release
• push_new_event
• update_new_event
• http_request
• update_repo_failed
• new_repo
• delete_repo

It may be useful for release/deploy process, for example: Automatically create new Jira release when a new version is created in packagist (triggered when new tag is created in bitbucket) and update "fix version" attribute of all the related issues from that release.

To build a custom request payload uses Twig expression language. This allows you to create custom queries. Untrusted template code is evaluate in a Twig sandbox mode, so you will get an error if try to get access for security sensitive information. By default, only admin users can use Webhooks.

Exception (Twig\Sandbox\SecurityNotAllowedMethodError). Calling "setemail" method on a "Packagist\WebBundle\Entity\User" object is not allowed in "__string_template__0d2344b042278505e67568413272d80429f07ecccea43af39cb33608fa747830" at line 1.

Examples

• List of twig variables
• Telegram notification
• Slack notification
• How to use url placeholder
• Interrupt request
• Nesting webhook
• Gitlab setup auto webhook
• JIRA issue fix version
• Accept external request
• Packeton twig function

Twig variables

• package - Package entity.
• versions - Versions[] array of versions.
• webhook - Webhook current webhook entity.
• user - User entity. Only for user login event.
• parentResponse - HookResponse object. Only for nesting webhook.
• request - array Only for http request event.

Telegram notification

POST https://api.telegram.org/bot$TELEGRAM_TOKEN/sendMessage

Options:

{
    "headers": {
        "Content-Type": "application/json"
    }
}

Payload

{% set text = "*New Releases*\n" %}
{% set title = package.name ~ ' (' ~ versions|map(v => "#{v.version}")|join(',') ~ ')' %}
{% set text = text ~ "[" ~ title ~ "](https://pkg.okvpn.org/packages/" ~ package.name ~ ")\n" %}
{% set text = text ~ package.description  %}

{% set request = {
    'chat_id': '-1000006111000',
    'parse_mode': 'Markdown',
    'text': text
} %}

{{ request|json_encode }}

Slack notification

In first you need create a slack app

POST https://slack.com/api/chat.postMessage

Options:

{
    "headers": {
        "Content-Type": "application/json",
        "Authorization": "Bearer xoxp-xxxxxxxxxxxxxxxxxxxxxxxxxx"
    }
}

Payload

{% set text = "*New Releases*\n" %}
{% set title = package.name ~ ' (' ~ versions|map(v => "#{v.version}")|join(',') ~ ')' %}
{% set text = text ~ "<https://pkg.okvpn.org/packages/" ~ package.name ~ "|" ~ title ~ ">\n" %}
{% set text = text ~ package.description  %}

{% set request = {
    'channel': 'jenkins',
    'text': text
} %}

{{ request|json_encode }}

Use url placeholder

The placeholder allow to build URL parameters from twig template.

http://httpbin.org/{{ method }}?repo={{ repoName }}

Syntax

Use placeholder tag

URL: http://httpbin.org/post?repo={{ paramName }}

Variant 1. Send single request.

{% placeholder <paramName> with <string> %}

Variant 2. Send many request for each value from array string[]

{% placeholder <paramName> with <string[]> %}

Example

http://httpbin.org/{{ method }}?repo={{ repoName }}

Payload

{% placeholder method with 'post' %}
{% placeholder repoName with [package.name, 'test/test'] %}

Interrupt request

You can interrupt request if condition is not pass

Syntax

Use interrupt function.

Payload

{% set request = {
    'chat_id': '1555151',
    'parse_mode': 'Markdown',
    'text': 'Text'
} %}

{% if package.name == 'okvpn/mq-insight' %}
    {{ interrupt() }}
{% endif %}

{{ request|json_encode }}

Nesting webhook

You can trigger webhook from twig code. It may be used for send two requests a one event.

Syntax

Use trigger_webhook(hookId: int|Webhook, context: array) function.

Example Payload

{% do trigger_webhook(6, {'project': 'OK', 'version': versions[0].version}) %}

Jira create a new release and set fix version

You need to create two webhook for its, the first must triggers on a new release event, the second will be called from twig code.

Create a new release in JIRA

POST https://jiraserver/rest/api/2/version

Options:

{
    "headers": {
        "Content-Type": "application/json"
    },
    "auth_basic": "jirauser:password"
}

Payload

{% set changeLog = get_changelog(package, null, versions[0].version) %}
{% set ticket = preg_match_all('/((OK|OTEK)-(\\d+))\\s*:/u', changeLog|join(';'), 1) %}

{% set request = {
    'archived': false,
    'releaseDate': versions[0].releasedAt|date('Y-m-d'),
    'name': versions[0].version,
    'released': true,
    'description': 'Packagist auto release',
    'project': 'OK'
} %}

{% if ticket|length == 0 %}
    {{ interrupt('There are not commits with JIRA tiket no.') }}
{% endif %}

{% do trigger_webhook(6, {'project': 'OK', 'ticket': ticket, 'version': versions[0].version}) %}
{{ request|json_encode }}

Update an issue fix version

PUT https://jiraserver/rest/api/2/issue/{{ issue }}

Options:

{
    "headers": {
        "Content-Type": "application/json"
    },
    "auth_basic": "jirauser:password"
}

Payload

{% placeholder issue with ticket %}

{% set request = {
    'fields': {
        'fixVersions': [{'name': version}]
    }
} %}

{{ request|json_encode }}

Http request from code

You can made http request from twig code.

{% set tags = http_request('https://registry.hub.docker.com/v1/repositories/okvpn/orocommerce/tags') %}

{{ tags|json_encode }}

External request

Triggered webhook by HTTP requests to https://PACKEGIST_URL/api/webhook-invoke/{name}

Optional name. If it is specified then this webhook can only be triggered if that name is supplied when invoking https://PACKEGIST_URL/api/webhook-invoke/{name} and name restriction is match.

Example payload:

{% if request.packageName is null %}
    {{ interrupt('package name is not found') }}
{% endif %}

{% set tags = http_request('https://registry.hub.docker.com/v1/repositories/' ~ request.packageName ~'/tags') %}

{{ tags|json_encode }}

Gitlab auto webhook

You can use the event new_repo to add a hook to the specified Gitlab project. It can be useful, if you don't have a Gold Gitlab Plan that allows configure webhooks for your group, so you need add it manually for each a new repository.

POST https://{{ host }}/api/v4/projects/{{ repo }}/hooks?private_token=xxxxxxxxxxxxxxxx

Options:

{
    "headers": {
        "Content-Type": "application/json"
    }
}

Payload

{% set regex = '#^(?:ssh://git@|https?://|git://|git@)?(?P<host>[a-z0-9.-]+)(?::[0-9]+/|[:/])(?P<path>[\\w.-]+(?:/[\\w.-]+?)+)(?:\\.git|/)?$#i' %}
{% set repository = preg_match_all(regex, package.repository) %}

{% if repository.path[0] is null or repository.host[0] is null %}
    {{ interrupt('Regex is not match') }}
{% endif %}

{% set request = {
    'url': 'https://pkg.okvpn.org/api/update-package?token=admin:xxxxxxxxxxxxxxxx',
    'push_events': true,
    'tag_push_events': true
} %}

{% placeholder host with repository.host[0] %}
{% placeholder repo with repository.path[0]|url_encode %}

{{ request|json_encode }}

Here you need replace request.url on your packagist.

New twig functions

See WebhookExtension for details.

Generic Packeton webhooks

Introduction

Webhooks allow external services to be notified when certain events happen. When the specified events happen, packeton will send a POST request to each of the URLs you provide. Now is supported the next events:

• new_release
• update_release
• delete_release
• push_new_event
• update_new_event
• http_request
• update_repo_failed
• new_repo
• delete_repo

It may be useful for release/deploy process, for example: Automatically create new Jira release when a new version is created in packagist (triggered when new tag is created in bitbucket) and update "fix version" attribute of all the related issues from that release.

To build a custom request payload uses Twig expression language. This allows you to create custom queries. Untrusted template code is evaluate in a Twig sandbox mode, so you will get an error if try to get access for security sensitive information. By default, only admin users can use Webhooks.

Exception (Twig\Sandbox\SecurityNotAllowedMethodError). Calling "setemail" method on a "Packagist\WebBundle\Entity\User" object is not allowed in "__string_template__0d2344b042278505e67568413272d80429f07ecccea43af39cb33608fa747830" at line 1.

Examples

• List of twig variables
• Telegram notification
• Slack notification
• How to use url placeholder
• Interrupt request
• Nesting webhook
• Gitlab setup auto webhook
• JIRA issue fix version
• Accept external request
• Packeton twig function

Twig variables

• package - Package entity.
• versions - Versions[] array of versions.
• webhook - Webhook current webhook entity.
• user - User entity. Only for user login event.
• parentResponse - HookResponse object. Only for nesting webhook.
• request - array Only for http request event.

Telegram notification

POST https://api.telegram.org/bot$TELEGRAM_TOKEN/sendMessage

Options:

{
    "headers": {
        "Content-Type": "application/json"
    }
}

Payload

{% set text = "*New Releases*\n" %}
{% set title = package.name ~ ' (' ~ versions|map(v => "#{v.version}")|join(',') ~ ')' %}
{% set text = text ~ "[" ~ title ~ "](https://pkg.okvpn.org/packages/" ~ package.name ~ ")\n" %}
{% set text = text ~ package.description  %}

{% set request = {
    'chat_id': '-1000006111000',
    'parse_mode': 'Markdown',
    'text': text
} %}

{{ request|json_encode }}

Slack notification

In first you need create a slack app

POST https://slack.com/api/chat.postMessage

Options:

{
    "headers": {
        "Content-Type": "application/json",
        "Authorization": "Bearer xoxp-xxxxxxxxxxxxxxxxxxxxxxxxxx"
    }
}

Payload

{% set text = "*New Releases*\n" %}
{% set title = package.name ~ ' (' ~ versions|map(v => "#{v.version}")|join(',') ~ ')' %}
{% set text = text ~ "<https://pkg.okvpn.org/packages/" ~ package.name ~ "|" ~ title ~ ">\n" %}
{% set text = text ~ package.description  %}

{% set request = {
    'channel': 'jenkins',
    'text': text
} %}

{{ request|json_encode }}

Use url placeholder

The placeholder allow to build URL parameters from twig template.

http://httpbin.org/{{ method }}?repo={{ repoName }}

Syntax

Use placeholder tag

URL: http://httpbin.org/post?repo={{ paramName }}

Variant 1. Send single request.

{% placeholder <paramName> with <string> %}

Variant 2. Send many request for each value from array string[]

{% placeholder <paramName> with <string[]> %}

Example

http://httpbin.org/{{ method }}?repo={{ repoName }}

Payload

{% placeholder method with 'post' %}
{% placeholder repoName with [package.name, 'test/test'] %}

Interrupt request

You can interrupt request if condition is not pass

Syntax

Use interrupt function.

Payload

{% set request = {
    'chat_id': '1555151',
    'parse_mode': 'Markdown',
    'text': 'Text'
} %}

{% if package.name == 'okvpn/mq-insight' %}
    {{ interrupt() }}
{% endif %}

{{ request|json_encode }}

Nesting webhook

You can trigger webhook from twig code. It may be used for send two requests a one event.

Syntax

Use trigger_webhook(hookId: int|Webhook, context: array) function.

Example Payload

{% do trigger_webhook(6, {'project': 'OK', 'version': versions[0].version}) %}

Jira create a new release and set fix version

You need to create two webhook for its, the first must triggers on a new release event, the second will be called from twig code.

Create a new release in JIRA

POST https://jiraserver/rest/api/2/version

Options:

{
    "headers": {
        "Content-Type": "application/json"
    },
    "auth_basic": "jirauser:password"
}

Payload

{% set changeLog = get_changelog(package, null, versions[0].version) %}
{% set ticket = preg_match_all('/((OK|OTEK)-(\\d+))\\s*:/u', changeLog|join(';'), 1) %}

{% set request = {
    'archived': false,
    'releaseDate': versions[0].releasedAt|date('Y-m-d'),
    'name': versions[0].version,
    'released': true,
    'description': 'Packagist auto release',
    'project': 'OK'
} %}

{% if ticket|length == 0 %}
    {{ interrupt('There are not commits with JIRA tiket no.') }}
{% endif %}

{% do trigger_webhook(6, {'project': 'OK', 'ticket': ticket, 'version': versions[0].version}) %}
{{ request|json_encode }}

Update an issue fix version

PUT https://jiraserver/rest/api/2/issue/{{ issue }}

Options:

{
    "headers": {
        "Content-Type": "application/json"
    },
    "auth_basic": "jirauser:password"
}

Payload

{% placeholder issue with ticket %}

{% set request = {
    'fields': {
        'fixVersions': [{'name': version}]
    }
} %}

{{ request|json_encode }}

Http request from code

You can made http request from twig code.

{% set tags = http_request('https://registry.hub.docker.com/v1/repositories/okvpn/orocommerce/tags') %}

{{ tags|json_encode }}

External request

Triggered webhook by HTTP requests to https://PACKEGIST_URL/api/webhook-invoke/{name}

Optional name. If it is specified then this webhook can only be triggered if that name is supplied when invoking https://PACKEGIST_URL/api/webhook-invoke/{name} and name restriction is match.

Example payload:

{% if request.packageName is null %}
    {{ interrupt('package name is not found') }}
{% endif %}

{% set tags = http_request('https://registry.hub.docker.com/v1/repositories/' ~ request.packageName ~'/tags') %}

{{ tags|json_encode }}

Gitlab auto webhook

You can use the event new_repo to add a hook to the specified Gitlab project. It can be useful, if you don't have a Gold Gitlab Plan that allows configure webhooks for your group, so you need add it manually for each a new repository.

POST https://{{ host }}/api/v4/projects/{{ repo }}/hooks?private_token=xxxxxxxxxxxxxxxx

Options:

{
    "headers": {
        "Content-Type": "application/json"
    }
}

Payload

{% set regex = '#^(?:ssh://git@|https?://|git://|git@)?(?P<host>[a-z0-9.-]+)(?::[0-9]+/|[:/])(?P<path>[\\w.-]+(?:/[\\w.-]+?)+)(?:\\.git|/)?$#i' %}
{% set repository = preg_match_all(regex, package.repository) %}

{% if repository.path[0] is null or repository.host[0] is null %}
    {{ interrupt('Regex is not match') }}
{% endif %}

{% set request = {
    'url': 'https://pkg.okvpn.org/api/update-package?token=admin:xxxxxxxxxxxxxxxx',
    'push_events': true,
    'tag_push_events': true
} %}

{% placeholder host with repository.host[0] %}
{% placeholder repo with repository.path[0]|url_encode %}

{{ request|json_encode }}

Here you need replace request.url on your packagist.

New twig functions

See WebhookExtension for details.

Generic Packeton webhooks

Introduction

Webhooks allow external services to be notified when certain events happen. When the specified events happen, packeton will send a POST request to each of the URLs you provide. Now is supported the next events:

• new_release
• update_release
• delete_release
• push_new_event
• update_new_event
• http_request
• update_repo_failed
• new_repo
• delete_repo

It may be useful for release/deploy process, for example: Automatically create new Jira release when a new version is created in packagist (triggered when new tag is created in bitbucket) and update "fix version" attribute of all the related issues from that release.

To build a custom request payload uses Twig expression language. This allows you to create custom queries. Untrusted template code is evaluate in a Twig sandbox mode, so you will get an error if try to get access for security sensitive information. By default, only admin users can use Webhooks.

Exception (Twig\Sandbox\SecurityNotAllowedMethodError). Calling "setemail" method on a "Packagist\WebBundle\Entity\User" object is not allowed in "__string_template__0d2344b042278505e67568413272d80429f07ecccea43af39cb33608fa747830" at line 1.

Examples

• List of twig variables
• Telegram notification
• Slack notification
• How to use url placeholder
• Interrupt request
• Nesting webhook
• Gitlab setup auto webhook
• JIRA issue fix version
• Accept external request
• Packeton twig function

Twig variables

• package - Package entity.
• versions - Versions[] array of versions.
• webhook - Webhook current webhook entity.
• user - User entity. Only for user login event.
• parentResponse - HookResponse object. Only for nesting webhook.
• request - array Only for http request event.

Telegram notification

POST https://api.telegram.org/bot$TELEGRAM_TOKEN/sendMessage

Options:

{
    "headers": {
        "Content-Type": "application/json"
    }
}

Payload

{% set text = "*New Releases*\n" %}
{% set title = package.name ~ ' (' ~ versions|map(v => "#{v.version}")|join(',') ~ ')' %}
{% set text = text ~ "[" ~ title ~ "](https://pkg.okvpn.org/packages/" ~ package.name ~ ")\n" %}
{% set text = text ~ package.description  %}

{% set request = {
    'chat_id': '-1000006111000',
    'parse_mode': 'Markdown',
    'text': text
} %}

{{ request|json_encode }}

Slack notification

In first you need create a slack app

POST https://slack.com/api/chat.postMessage

Options:

{
    "headers": {
        "Content-Type": "application/json",
        "Authorization": "Bearer xoxp-xxxxxxxxxxxxxxxxxxxxxxxxxx"
    }
}

Payload

{% set text = "*New Releases*\n" %}
{% set title = package.name ~ ' (' ~ versions|map(v => "#{v.version}")|join(',') ~ ')' %}
{% set text = text ~ "<https://pkg.okvpn.org/packages/" ~ package.name ~ "|" ~ title ~ ">\n" %}
{% set text = text ~ package.description  %}

{% set request = {
    'channel': 'jenkins',
    'text': text
} %}

{{ request|json_encode }}

Use url placeholder

The placeholder allow to build URL parameters from twig template.

http://httpbin.org/{{ method }}?repo={{ repoName }}

Syntax

Use placeholder tag

URL: http://httpbin.org/post?repo={{ paramName }}

Variant 1. Send single request.

{% placeholder <paramName> with <string> %}

Variant 2. Send many request for each value from array string[]

{% placeholder <paramName> with <string[]> %}

Example

http://httpbin.org/{{ method }}?repo={{ repoName }}

Payload

{% placeholder method with 'post' %}
{% placeholder repoName with [package.name, 'test/test'] %}

Interrupt request

You can interrupt request if condition is not pass

Syntax

Use interrupt function.

Payload

{% set request = {
    'chat_id': '1555151',
    'parse_mode': 'Markdown',
    'text': 'Text'
} %}

{% if package.name == 'okvpn/mq-insight' %}
    {{ interrupt() }}
{% endif %}

{{ request|json_encode }}

Nesting webhook

You can trigger webhook from twig code. It may be used for send two requests a one event.

Syntax

Use trigger_webhook(hookId: int|Webhook, context: array) function.

Example Payload

{% do trigger_webhook(6, {'project': 'OK', 'version': versions[0].version}) %}

Jira create a new release and set fix version

You need to create two webhook for its, the first must triggers on a new release event, the second will be called from twig code.

Create a new release in JIRA

POST https://jiraserver/rest/api/2/version

Options:

{
    "headers": {
        "Content-Type": "application/json"
    },
    "auth_basic": "jirauser:password"
}

Payload

{% set changeLog = get_changelog(package, null, versions[0].version) %}
{% set ticket = preg_match_all('/((OK|OTEK)-(\\d+))\\s*:/u', changeLog|join(';'), 1) %}

{% set request = {
    'archived': false,
    'releaseDate': versions[0].releasedAt|date('Y-m-d'),
    'name': versions[0].version,
    'released': true,
    'description': 'Packagist auto release',
    'project': 'OK'
} %}

{% if ticket|length == 0 %}
    {{ interrupt('There are not commits with JIRA tiket no.') }}
{% endif %}

{% do trigger_webhook(6, {'project': 'OK', 'ticket': ticket, 'version': versions[0].version}) %}
{{ request|json_encode }}

Update an issue fix version

PUT https://jiraserver/rest/api/2/issue/{{ issue }}

Options:

{
    "headers": {
        "Content-Type": "application/json"
    },
    "auth_basic": "jirauser:password"
}

Payload

{% placeholder issue with ticket %}

{% set request = {
    'fields': {
        'fixVersions': [{'name': version}]
    }
} %}

{{ request|json_encode }}

Http request from code

You can made http request from twig code.

{% set tags = http_request('https://registry.hub.docker.com/v1/repositories/okvpn/orocommerce/tags') %}

{{ tags|json_encode }}

External request

Triggered webhook by HTTP requests to https://PACKEGIST_URL/api/webhook-invoke/{name}

Optional name. If it is specified then this webhook can only be triggered if that name is supplied when invoking https://PACKEGIST_URL/api/webhook-invoke/{name} and name restriction is match.

Example payload:

{% if request.packageName is null %}
    {{ interrupt('package name is not found') }}
{% endif %}

{% set tags = http_request('https://registry.hub.docker.com/v1/repositories/' ~ request.packageName ~'/tags') %}

{{ tags|json_encode }}

Gitlab auto webhook

You can use the event new_repo to add a hook to the specified Gitlab project. It can be useful, if you don't have a Gold Gitlab Plan that allows configure webhooks for your group, so you need add it manually for each a new repository.

POST https://{{ host }}/api/v4/projects/{{ repo }}/hooks?private_token=xxxxxxxxxxxxxxxx

Options:

{
    "headers": {
        "Content-Type": "application/json"
    }
}

Payload

{% set regex = '#^(?:ssh://git@|https?://|git://|git@)?(?P<host>[a-z0-9.-]+)(?::[0-9]+/|[:/])(?P<path>[\\w.-]+(?:/[\\w.-]+?)+)(?:\\.git|/)?$#i' %}
{% set repository = preg_match_all(regex, package.repository) %}

{% if repository.path[0] is null or repository.host[0] is null %}
    {{ interrupt('Regex is not match') }}
{% endif %}

{% set request = {
    'url': 'https://pkg.okvpn.org/api/update-package?token=admin:xxxxxxxxxxxxxxxx',
    'push_events': true,
    'tag_push_events': true
} %}

{% placeholder host with repository.host[0] %}
{% placeholder repo with repository.path[0]|url_encode %}

{{ request|json_encode }}

Here you need replace request.url on your packagist.

New twig functions

See WebhookExtension for details.

Manage secrets

Before adding sensitive data such as API credentials to your webhook payload strongly recommended to encrypt it. Secret data will not be save to database or shows on webhooks' status. Alternative way you can set own visibility for webhook entity to prevent edit by other admin users.

Encrypt secrets

To add secrets to your webhook, put JSON body to Request options field, for example

{
//     "headers"  more other options 
    "secrets": {
        "allowed-domains": ["api.telegram.org"],
        "TOKEN": "167000000:AAzddkPzfgzkqzzFghiwPutin_khuylo",
        "CHART_ID": "-1000017160005"
    }
}

Once the form is submitted, the secret params will be encrypted and sign and cannot be changed. The sign algo is hmac sha256 with APP_SECRET as key. Digital signature required to prevent modification encrypted data and attack on change allowed-domains

Secrets option

allowed-domains - you can restrict webhook call to untrusted hosts to prevent modify the change URL parameter.

Usage secrets

Use secrets params in request, url or headers options, example:

https://api.telegram.org/bot${secrets.TOKEN}/sendMessage

In body

{% set request = {
    'chat_id': '${secrets.CHART_ID}',
    'text': 'example text'
} %}
{{ request|json_encode }}

But this example will not work.

{% set request = {
    'chat_id': '${secrets.CHART_ID}',
    'text': 'example text'
} %}
{% do log('${secrets.CHART_ID}')

{{ request|json_encode }}

Webhook security

To evaluate expressions uses a Twig sandbox mode.

You will get an error if try to get access for security sensitive information.

SSRF - To prevent SSRF attacks to make HTTP requests to inner private networks uses NoPrivateNetworkHttpClient, so not possible call url like http://10.8.100.1/ etc.

# This code is not works.

{% set text = "*New Releases*\n" %}
{% set title = package.name ~ ' (' ~ versions|map(v => "#{v.version}")|join(',') ~ ')' %}

{% set text = text ~ package.credentials.key  %}

{% set request = {
    'channel': 'jenkins',
    'text': text
} %}

{{ request|json_encode }}

Exception (Twig\Sandbox\SecurityNotAllowedMethodError). Calling "getcredentials" method on a "Packeton\Entity\Package" 
object is not allowed in "__string_template__4b1d9dd7416b75a6c353bd4750fe5490" at line 4.

Block SSRF

Exception (Symfony\Component\HttpClient\Exception\TransportException). IP "127.0.0.1" is blocked for "https://pack.loc.example.org/webhooks".
 * Prev exception (Symfony\Component\HttpClient\Exception\TransportException). IP "127.0.0.1" is blocked for "https://pack.loc.example.org/webhooks".

Test Webhook Twig payload.

You can use test action to check result

How to auto update packages?

You can use GitLab, GitHub, and Bitbucket project post-receive hook to keep your packages up to date every time you push code.

Into

Webhook API request authorization with minimum access level ROLE_MAINTAINER. You can use token query parameter with <username:api_token> to call it.

Also support Packagist.org authorization with username and apiToken query parameters.

Bitbucket Webhooks

To enable the Bitbucket web hook, go to your BitBucket repository, open the settings and select "Webhooks" in the menu. Add a new hook. Y ou have to enter the Packagist endpoint, containing both your username and API token. Enter https://<app>/api/bitbucket?token=user:token as URL. Save your changes and you're done.

GitLab Service

To enable the GitLab service integration, go to your GitLab repository, open the Settings > Integrations page from the menu. Search for Packagist in the list of Project Services. Check the "Active" box, enter your packeton.org username and API token. Save your changes and you're done.

GitLab Group Hooks

Group webhooks will apply to all projects in a group and allow to sync all projects. To enable the Group GitLab webhook you must have the paid plan. Go to your GitLab Group > Settings > Webhooks. Enter https://<app>/api/update-package?token=user:token as URL.

GitHub Webhooks

To enable the GitHub webhook go to your GitHub repository. Click the "Settings" button, click "Webhooks". Add a new hook. Enter https://<app>/api/github?token=user:token as URL.

Gitea Webhooks

To enable the Gitea web hook, go to your Gitea repository, open the settings, select "Webhooks" in the menu and click on 'Add Webhook'. From the dropdown menu select Gitea. You have to enter the Packagist endpoint, containing both your username and API token. Enter https://<app>/api/update-package?token=user:token as URL. The HTTP method has to be POST and content type is application/json. Save your changes and you're done.

Manual hook setup

If you do not use Bitbucket or GitHub there is a generic endpoint you can call manually from a git post-receive hook or similar. You have to do a POST request to https://<app>g/api/update-package?token=user:api_token with a request body looking like this:

{
  "repository": {
    "url": "PACKAGIST_PACKAGE_URL"
  }
}

You can also send a GET request with query parameter composer_package_name

curl 'https://<app>/api/update-package?token=<user:token>&composer_package_name=vender/name'

Customer Users

You can create customers user and manage their packages access. The customer users have may have two users role:

• ROLE_USER - minimal access level, these users only can read metadata only for selected packages
• ROLE_FULL_CUSTOMER - Can read all packages metadata without groups ACL restriction.

For ROLE_USER you will be able to limit packages access by release date too.

To grant access to your packages, need to create ACL group in the first.

Create Customer User.

After creating an ACL group, you may to create a user and grant access to more that one groups. If selected more than one group, then all groups permission will be union together.

SSH Credential and Composer Auth.

Composer provide two why of authentication for privately hosted packages.

You may to setup authentication in auth.json in composer home, i e. /var/www/packeton/var/.composer/ or /data/composer/auth.json for docker installation.

See example usage system auth setup

Using UI Credential Manager

You can overwrite credentials for each repository in UI.

All credentials are encrypted in database by custom DBAL type EncryptedTextType. Encrypted key is APP_SECRET so it must be permanent.

Mirroring and Composer proxies

Packeton can function as a proxy for the Composer repository, including which require authentication. This feature can be used to grant all developers and clients access to private repositories such as Magento. Additionally, it is possible to create ZIP archives from mirrored Git repositories of packages, in cases where HTTP dist is unavailable.

Main Features

• Supports full and lazy synchronization for small and large Composer repositories.
• Supports the Packagist fast metadata-changes-url API.
• Includes Strict Mode and Dependencies Approval functionality.
• Supports Dist/SSH mirroring of source code.

Example metadata with Strict mode and manual dependencies' approval.

{
    "includes": {
        "include-packeton/all$f05f56b8bd12d014a753cdbe6a7d749facd40908.json": {
            "sha1": "f05f56b8bd12d014a753cdbe6a7d749facd40908"
        }
    },
    "mirrors": [
        {
            "dist-url": "/mirror/orocrm/zipball/%package%/%version%/%reference%.%type%",
            "preferred": true
        }
    ],
    "metadata-url": "/mirror/orocrm/p2/%package%.json",
    "available-packages": [
        "romanpitak/dotmailer-api-v2-client",
        "oro/platform-enterprise",
        "oro/crm-enterprise",
        "oro/api-doc-bundle",
        "oro/flotr2",
        "oro/crm-pro-ldap-bundle",
        "oro/multi-host",
        "akeneo/batch-bundle"
    ]
}

Original metadata is:

{
    "packages": [],
    "providers-url": "/p/%package%$%hash%.json",
    "providers": {
        "actualys/drupal-commerce-connector-bundle": {
            "sha256": "4163f3b470b3b824cbcebee5a0d58ea3d516b7b5fa78617ba21120eeec9e494f"
        },
        "agencednd/oro-api-connector-bundle": {
            "sha256": "169c0963fd8442c190f2e9303e0e6fa1fe9ad0c9fb2f6782176d02e65a48eada"
        },
        "akeneo/batch-bundle": {
            "sha256": "4f2c1b9a43124524da45b35236acabd3ee1ad329980b885089e9eb408c1bca01"
        },
    ...
    + 57 packages

For performance if composer user-agent == 1 then includes replaced with providers-lazy-url

Configuration

Example how to enable proxies in your local configuration. To enable proxies in your local configuration, create a file with any name like config/packages/any-name.yaml and add the following configuration:

packeton:
    mirrors:
        packagist:
            url: https://repo.packagist.org
        orocrm:
            url: https://satis.oroinc.com/
            git_ssh_keys:
                git@github.com:oroinc: '/var/www/.ssh/private_key1'
                git@github.com:org2: '/var/www/.ssh/private_key2'
        example:
            url: https://satis.example.com/
            logo: 'https://example.com/logo.png'
            http_basic:
                username: 123
                password: 123
            public_access: true # Allow public access, default false
            sync_lazy: true # default false 
            enable_dist_mirror: false # default true
            available_package_patterns: # Additional restriction, but you can restrict it in UI
                - 'vend1/*' 
            available_packages:
                - 'pack1/name1' # but you can restrict it in UI
            composer_auth: '{"auth.json..."}' # JSON. auth.json to pass composer opts.
            sync_interval: 3600 # default auto.
            info_cmd_message: "\n\u001b[37;44m#Слава\u001b[30;43mУкраїні!\u001b[0m\n\u001b[40;31m#Смерть\u001b[30;41mворогам\u001b[0m" # Info message

The configuration allows you to use multiple SSH key settings for different GitHub accounts.

...
git_ssh_keys:
    git@github.com:oroinc: '/var/www/.ssh/private_key1'
    git@github.com:org2: '/var/www/.ssh/private_key2'

# Or one key
git_ssh_keys: '/var/www/.ssh/private_key1'

Metadata Proxy Specification.

The specification for the metadata proxy depends on the type of repository and the synchronization strategy being used.

Default sync intervals

Commands for Debug

php bin/console packagist:sync:mirrors firegento -vvv

Description:
  Sync mirror repository proxy.

Usage:
  packagist:sync:mirrors [options] [--] [<mirror>]

Arguments:
  mirror                Mirror name in config file.

Options:
      --force           Remote all data and sync again
  -v|vv|vvv, --verbose  Increase the verbosity of messages: 1 for normal output, 2 for more verbose output and 3 for debug

Manual Approval of Dependencies

By default, all new packages are automatically enabled and added to your repository when you run composer update. However, you can enable strict mode to use only approved packages and avoid including untrusted packages in your metadata. This can be useful in preventing dependency confusion attacks, especially if you use a 3rd-party Composer repository like https://satis.oroinc.com/. For more information on preventing dependency hacking, please see dependency confusion

To enable strict mode, go to the Proxy Settings page and select Composer Proxies -> Packagist (or any other name) -> Settings.

Next, go to the View Proxy page and click the "Mass Mirror Packages" button.

Mirror Public Access

Use the following configuration:

packeton:
    mirrors:
        youname:
            url: https://repo.example.org
            public_access: true

Mirroring and Composer proxies

Packeton can function as a proxy for the Composer repository, including which require authentication. This feature can be used to grant all developers and clients access to private repositories such as Magento. Additionally, it is possible to create ZIP archives from mirrored Git repositories of packages, in cases where HTTP dist is unavailable.

Main Features

• Supports full and lazy synchronization for small and large Composer repositories.
• Supports the Packagist fast metadata-changes-url API.
• Includes Strict Mode and Dependencies Approval functionality.
• Supports Dist/SSH mirroring of source code.

Example metadata with Strict mode and manual dependencies' approval.

{
    "includes": {
        "include-packeton/all$f05f56b8bd12d014a753cdbe6a7d749facd40908.json": {
            "sha1": "f05f56b8bd12d014a753cdbe6a7d749facd40908"
        }
    },
    "mirrors": [
        {
            "dist-url": "/mirror/orocrm/zipball/%package%/%version%/%reference%.%type%",
            "preferred": true
        }
    ],
    "metadata-url": "/mirror/orocrm/p2/%package%.json",
    "available-packages": [
        "romanpitak/dotmailer-api-v2-client",
        "oro/platform-enterprise",
        "oro/crm-enterprise",
        "oro/api-doc-bundle",
        "oro/flotr2",
        "oro/crm-pro-ldap-bundle",
        "oro/multi-host",
        "akeneo/batch-bundle"
    ]
}

Original metadata is:

{
    "packages": [],
    "providers-url": "/p/%package%$%hash%.json",
    "providers": {
        "actualys/drupal-commerce-connector-bundle": {
            "sha256": "4163f3b470b3b824cbcebee5a0d58ea3d516b7b5fa78617ba21120eeec9e494f"
        },
        "agencednd/oro-api-connector-bundle": {
            "sha256": "169c0963fd8442c190f2e9303e0e6fa1fe9ad0c9fb2f6782176d02e65a48eada"
        },
        "akeneo/batch-bundle": {
            "sha256": "4f2c1b9a43124524da45b35236acabd3ee1ad329980b885089e9eb408c1bca01"
        },
    ...
    + 57 packages

For performance if composer user-agent == 1 then includes replaced with providers-lazy-url

Configuration

Example how to enable proxies in your local configuration. To enable proxies in your local configuration, create a file with any name like config/packages/any-name.yaml and add the following configuration:

packeton:
    mirrors:
        packagist:
            url: https://repo.packagist.org
        orocrm:
            url: https://satis.oroinc.com/
            git_ssh_keys:
                git@github.com:oroinc: '/var/www/.ssh/private_key1'
                git@github.com:org2: '/var/www/.ssh/private_key2'
        example:
            url: https://satis.example.com/
            logo: 'https://example.com/logo.png'
            http_basic:
                username: 123
                password: 123
            public_access: true # Allow public access, default false
            sync_lazy: true # default false 
            enable_dist_mirror: false # default true
            available_package_patterns: # Additional restriction, but you can restrict it in UI
                - 'vend1/*' 
            available_packages:
                - 'pack1/name1' # but you can restrict it in UI
            composer_auth: '{"auth.json..."}' # JSON. auth.json to pass composer opts.
            sync_interval: 3600 # default auto.
            info_cmd_message: "\n\u001b[37;44m#Слава\u001b[30;43mУкраїні!\u001b[0m\n\u001b[40;31m#Смерть\u001b[30;41mворогам\u001b[0m" # Info message

The configuration allows you to use multiple SSH key settings for different GitHub accounts.

...
git_ssh_keys:
    git@github.com:oroinc: '/var/www/.ssh/private_key1'
    git@github.com:org2: '/var/www/.ssh/private_key2'

# Or one key
git_ssh_keys: '/var/www/.ssh/private_key1'

Metadata Proxy Specification.

The specification for the metadata proxy depends on the type of repository and the synchronization strategy being used.

Default sync intervals

Commands for Debug

php bin/console packagist:sync:mirrors firegento -vvv

Description:
  Sync mirror repository proxy.

Usage:
  packagist:sync:mirrors [options] [--] [<mirror>]

Arguments:
  mirror                Mirror name in config file.

Options:
      --force           Remote all data and sync again
  -v|vv|vvv, --verbose  Increase the verbosity of messages: 1 for normal output, 2 for more verbose output and 3 for debug

Manual Approval of Dependencies

By default, all new packages are automatically enabled and added to your repository when you run composer update. However, you can enable strict mode to use only approved packages and avoid including untrusted packages in your metadata. This can be useful in preventing dependency confusion attacks, especially if you use a 3rd-party Composer repository like https://satis.oroinc.com/. For more information on preventing dependency hacking, please see dependency confusion

To enable strict mode, go to the Proxy Settings page and select Composer Proxies -> Packagist (or any other name) -> Settings.

Next, go to the View Proxy page and click the "Mass Mirror Packages" button.

Mirror Public Access

Use the following configuration:

packeton:
    mirrors:
        youname:
            url: https://repo.example.org
            public_access: true

Mirroring and Composer proxies

Packeton can function as a proxy for the Composer repository, including which require authentication. This feature can be used to grant all developers and clients access to private repositories such as Magento. Additionally, it is possible to create ZIP archives from mirrored Git repositories of packages, in cases where HTTP dist is unavailable.

Main Features

• Supports full and lazy synchronization for small and large Composer repositories.
• Supports the Packagist fast metadata-changes-url API.
• Includes Strict Mode and Dependencies Approval functionality.
• Supports Dist/SSH mirroring of source code.

Example metadata with Strict mode and manual dependencies' approval.

{
    "includes": {
        "include-packeton/all$f05f56b8bd12d014a753cdbe6a7d749facd40908.json": {
            "sha1": "f05f56b8bd12d014a753cdbe6a7d749facd40908"
        }
    },
    "mirrors": [
        {
            "dist-url": "/mirror/orocrm/zipball/%package%/%version%/%reference%.%type%",
            "preferred": true
        }
    ],
    "metadata-url": "/mirror/orocrm/p2/%package%.json",
    "available-packages": [
        "romanpitak/dotmailer-api-v2-client",
        "oro/platform-enterprise",
        "oro/crm-enterprise",
        "oro/api-doc-bundle",
        "oro/flotr2",
        "oro/crm-pro-ldap-bundle",
        "oro/multi-host",
        "akeneo/batch-bundle"
    ]
}

Original metadata is:

{
    "packages": [],
    "providers-url": "/p/%package%$%hash%.json",
    "providers": {
        "actualys/drupal-commerce-connector-bundle": {
            "sha256": "4163f3b470b3b824cbcebee5a0d58ea3d516b7b5fa78617ba21120eeec9e494f"
        },
        "agencednd/oro-api-connector-bundle": {
            "sha256": "169c0963fd8442c190f2e9303e0e6fa1fe9ad0c9fb2f6782176d02e65a48eada"
        },
        "akeneo/batch-bundle": {
            "sha256": "4f2c1b9a43124524da45b35236acabd3ee1ad329980b885089e9eb408c1bca01"
        },
    ...
    + 57 packages

For performance if composer user-agent == 1 then includes replaced with providers-lazy-url

Configuration

Example how to enable proxies in your local configuration. To enable proxies in your local configuration, create a file with any name like config/packages/any-name.yaml and add the following configuration:

packeton:
    mirrors:
        packagist:
            url: https://repo.packagist.org
        orocrm:
            url: https://satis.oroinc.com/
            git_ssh_keys:
                git@github.com:oroinc: '/var/www/.ssh/private_key1'
                git@github.com:org2: '/var/www/.ssh/private_key2'
        example:
            url: https://satis.example.com/
            logo: 'https://example.com/logo.png'
            http_basic:
                username: 123
                password: 123
            public_access: true # Allow public access, default false
            sync_lazy: true # default false 
            enable_dist_mirror: false # default true
            available_package_patterns: # Additional restriction, but you can restrict it in UI
                - 'vend1/*' 
            available_packages:
                - 'pack1/name1' # but you can restrict it in UI
            composer_auth: '{"auth.json..."}' # JSON. auth.json to pass composer opts.
            sync_interval: 3600 # default auto.
            info_cmd_message: "\n\u001b[37;44m#Слава\u001b[30;43mУкраїні!\u001b[0m\n\u001b[40;31m#Смерть\u001b[30;41mворогам\u001b[0m" # Info message

The configuration allows you to use multiple SSH key settings for different GitHub accounts.

...
git_ssh_keys:
    git@github.com:oroinc: '/var/www/.ssh/private_key1'
    git@github.com:org2: '/var/www/.ssh/private_key2'

# Or one key
git_ssh_keys: '/var/www/.ssh/private_key1'

Metadata Proxy Specification.

The specification for the metadata proxy depends on the type of repository and the synchronization strategy being used.

Default sync intervals

Commands for Debug

php bin/console packagist:sync:mirrors firegento -vvv

Description:
  Sync mirror repository proxy.

Usage:
  packagist:sync:mirrors [options] [--] [<mirror>]

Arguments:
  mirror                Mirror name in config file.

Options:
      --force           Remote all data and sync again
  -v|vv|vvv, --verbose  Increase the verbosity of messages: 1 for normal output, 2 for more verbose output and 3 for debug

Manual Approval of Dependencies

By default, all new packages are automatically enabled and added to your repository when you run composer update. However, you can enable strict mode to use only approved packages and avoid including untrusted packages in your metadata. This can be useful in preventing dependency confusion attacks, especially if you use a 3rd-party Composer repository like https://satis.oroinc.com/. For more information on preventing dependency hacking, please see dependency confusion

To enable strict mode, go to the Proxy Settings page and select Composer Proxies -> Packagist (or any other name) -> Settings.

Next, go to the View Proxy page and click the "Mass Mirror Packages" button.

Mirror Public Access

Use the following configuration:

packeton:
    mirrors:
        youname:
            url: https://repo.example.org
            public_access: true

User Authentication

Packeton may support multiple methods of authenticating users. It can additionally be extended to support custom authentication schemes.

Web User authentication

Included in packeton is support for authenticating users via:

• A username and password.
• An email address and password.

But possible to enable LDAP only via configuration, see ldap authentication

Composer API authentication

Packeton is support API authentication only with api token. Password usage is not allowed. You can see api token in thr user profile menu.

Support for authenticating users via:

• HTTP Basic Authentication (username and api token)
• Short query param token = username:apiToken
• Default packagist hook API (query params: username = username, apiToken = apiToken)

Your customer needs to authenticate to access their Composer repository: The simplest way to provide your credentials is providing your set of credentials inline with the repository specification such as:

{
    "repositories": [
        {
            "type": "composer",
            "url": "https://<username>:<api_token>@example.org"
        }
    ]
}

When you don't want to hard code your credentials into your composer.json, you can set up it global.

composer config --global --auth http-basic.example.org username api_token

Example API call.

curl https://example.com/packages/list.json
   -u "username:apiToken"

curl https://example.com/packages/list.json?token=username:apiToken

JWT API Authentication

By default, packeton is storage api tokens in database for each user. But when user loaded from custom user provider, like LDAP need to enable JWT configuration to use API. So Packeton can be configured with a non-standard login type to support JSON Web Tokens.

The JSON Web Token integration in Packeton uses the Firebase library. Also, JWT authentication can be enabled only for API.

Add yaml configuration file to path config/packages/, for example config/packages/jwt.yaml to enable it.

packeton:
    jwt_authentication:
        private_key: '%kernel.project_dir%/var/jwt/eddsa-key.pem'
        public_key: '%kernel.project_dir%/var/jwt/eddsa-public.pem'

Full configurations:

# config/packages/config/packages/jwt.yaml
packeton:
    jwt_authentication:
        private_key: '%kernel.project_dir%/var/jwt/eddsa-key.pem' # required for token sign
        public_key: '%kernel.project_dir%/var/jwt/eddsa-public.pem' # required for token verification
        passphrase: ~
        algo: EdDSA # Sign algo, here libsodium EdDSA

Generate the public/private keys

bin/console packagist:jwt:generate-keypair

bin/console packagist:jwt:generate-keypair --overwrite

Available options:

• --overwrite will overwrite your keys if they already exist.

If keys already exists, a warning message will be raised to prevent you from overwriting your keys accidentally.

JWT Token TTL.

JWT Token is never expire. It was done for compatibility with composer basic HTTP authorization. Each time the api is called, Packeton is checked that the user exists in the database and that he has the same set of permissions and roles.

Digital signatures algos.

We support all algos from Firebase lib: HMAC, OpenSSL RSA, OpenSSL

Rsa, HMAC, EdDSA algorithms generate invariant tokens, i.e. the value of the token will be constant for the same user. It might be convenient as the app does not store the generated tokens.

Example how to change algo:

packeton:
    jwt_authentication:
    ...
        algo: RS256 # RSA 256

Generating keys using OpenSSL

Example, how to generate an RSA private key, key.pem - private key. public.pem - public

openssl genrsa -out key.pem 2048
openssl rsa -in key.pem -outform PEM -pubout -out public.pem

Example, how to generate an ES256 (elliptic curve) key pairs.

openssl ecparam -name prime256v1 -genkey -noout -out key.pem
openssl ec -in key.pem -pubout -out public.pem

Obtain the token

You can run command packagist:user:manager to show the api token:

bin/console packagist:user:manager admin --show-token --token-format=jwt

Or you can found api token on your profile page.

Use the token

Simply use the JWT, like standard API token for composer api.

Cache LDAP user loading.

Since 2.0 composer downloads all packages in parallel, it may run more 12 request at the same time. To prevent calls external LDAP provider each time for JWT token verify, the obtained LDAP user object placed to cache with 60 sec TTL.

Authenticating against an LDAP server

You can enable LDAP authenticating only on configuration level.

Packeton has pre-installed Symfony LDAP component. Add the file config/packages/ldap.yaml to enable LDAP with following content. See LDAP in Symfony Docs

parameters:
    default_login_provider: 'form_login_ldap'
    default_login_options:
        provider: all_users
        login_path: /login
        use_forward: false
        check_path: /login
        failure_path: null
        service: Symfony\Component\Ldap\Ldap
        dn_string: 'uid={username},dc=example,dc=com'

services:
    Symfony\Component\Ldap\Ldap:
        arguments: ['@Symfony\Component\Ldap\Adapter\ExtLdap\Adapter']
        tags:
            - ldap

    Symfony\Component\Ldap\Adapter\ExtLdap\Adapter:
        arguments:
            -   host: ldap.forumsys.com
                port: 389

security:
    providers:
        users_ldap:
            ldap:
                service: Symfony\Component\Ldap\Ldap
                base_dn: dc=example,dc=com
                search_dn: "cn=read-only-admin,dc=example,dc=com"
                search_password: password
                default_roles: ROLE_MAINTAINER
                uid_key: uid

        all_users:
            chain:
                providers: ['packagist', 'users_ldap']

Here is working example where used test ldap.forumsys.com server https://www.forumsys.com/2022/05/10/online-ldap-test-server/

Using LDAP integration does not prevent you from creating user manually from CLI and assign more accessible roles. At the same LDAP password validation will be done on LDAP server side, because CheckLdapCredentialsListener has higher priority loading than default check listener. Therefore, if user is not enable in LDAP - it will not able login to packeton.

User providers priority.

Packeton use Symfony Chain User Provider to lookup users.

If you want to use customer user restriction by vendors and versions, packagist user provider must load before ldap.

security:
    providers:
        users_ldap:
            ldap:
                ... 
 
        all_users:
            chain:
                providers: ['packagist', 'users_ldap'] # Load user/roles form default packagist and if not found - use ldap user
                providers: ['users_ldap', 'packagist'] # packagist users will be ignore 

Load different roles from LDAP.

You can use more 1 user providers:

security:
    providers:
        users_ldap:
            ldap:
                service: Symfony\Component\Ldap\Ldap
                base_dn: dc=example,dc=com
                search_dn: "cn=read-only-admin,dc=example,dc=com"
                filter: "(&(objectclass=groupOfUniqueNames)(ou=scientists)(uniqueMember=uid={username},dc=example,dc=com))"
                search_password: password
                default_roles: ROLE_MAINTAINER
                uid_key: uid

        users_ldap_admin:
            ldap:
                service: Symfony\Component\Ldap\Ldap
                base_dn: dc=example,dc=com
                search_dn: "cn=read-only-admin,dc=example,dc=com"
                filter: "(&(objectclass=groupOfUniqueNames)(ou=mathematicians)(uniqueMember=uid={username},dc=example,dc=com))"
                search_password: password
                default_roles: ROLE_ADMIN
                uid_key: uid

        all_users:
            chain:
                providers: ['packagist', 'users_ldap', 'users_ldap_admin']

Here test example where exists two Groups (ou) that include:

• ou=mathematicians,dc=example,dc=com - assign role ROLE_ADMIN
• ou=scientists,dc=example,dc=com - assign role ROLE_MAINTAINER

API authentication with LDAP users.

By default, packeton is storage api token in database for each user. But if the user was loaded by custom external users' provider, but from the database, you will need enable JWT configuration. See JWT Configuration

Enable LDAP for docker runtime.

You can use docker volume to share own configuration to application.

...
        volumes:
            - .docker:/data
            - ${PWD}/ldap.yaml:/var/www/packagist/config/packages/ldap.yaml

S3 Storage Provider

By default, Packeton stores packages archives on the local filesystem. But you can easily configure the S3 using league/flysystem-bundle.

For docker env, please set env vars.

STORAGE_SOURCE=s3
STORAGE_AWS_BUCKET=packeton-bucket
STORAGE_AWS_PREFIX=packeton
STORAGE_AWS_ARTIFACT_PREFIX=artifact

STORAGE_AWS_ARGS='{"endpoint": "https://s3.waw.io.cloud.ovh.net", "accessKeyId": "xxx", "accessKeySecret": "xxx", "region": "waw"}'

Sometimes for Artifact Repository requires direct access to files from the archive, so to improve performance and reduces count of S3 API requests, the all archives are cached on the local filesystem too.

If you need to use the other provider, like Google Cloud, you may add config file to config/packages or use config.yaml in data docker dir.

flysystem:
    storages:
        s3_v2.storage:
            adapter: 'asyncaws'
            options:
                client: 'packeton.s3.storage'
                bucket: '%env(STORAGE_AWS_BUCKET)%'
                prefix: '%env(STORAGE_AWS_PREFIX)%'

        s3_v2.artifact:
            adapter: 'asyncaws'
            options:
                client: 'packeton.s3.storage'
                bucket: '%env(STORAGE_AWS_BUCKET)%'
                prefix: '%env(STORAGE_AWS_ARTIFACT_PREFIX)%'

        gcloud.storage:
            adapter: 'gcloud'
            options:
                client: 'gcloud_client_service' 
                bucket: 'bucket_name'
                prefix: 'optional/path/prefix'

        gcloud.artifact:
            adapter: 'gcloud'
            options:
                client: 'gcloud_client_service' 
                bucket: 'bucket_name'
                prefix: 'optional/path/artifact'

parameters:
    env(STORAGE_AWS_BUCKET): 'packeton-bucket'
    env(STORAGE_AWS_PREFIX): 'packeton'
    env(STORAGE_AWS_ARGS): '[]'
    env(STORAGE_AWS_ARTIFACT_PREFIX): 'artifact'

services:
    packeton.s3.storage:
        class: AsyncAws\S3\S3Client
        arguments:
            $configuration: '%env(json:STORAGE_AWS_ARGS)%'
            $httpClient: '@Symfony\Contracts\HttpClient\HttpClientInterface'
            $logger: '@logger'
    
    gcloud_client_service:
        class: Google\Cloud\Storage\StorageClient
        arguments:
            - { keyFilePath: 'path/to/keyfile.json' }

STORAGE_SOURCE=s3_v2
STORAGE_SOURCE=gcloud

Custom landing page

If you are distributing packages to your customers, you may want to create a separate domain for Composer metadata-only to hide the default web interface and login page.

Add following lines to you configuration. config.yaml or config/packages/*.yaml

packeton:
    web_protection:
        ## Multi host protection, disable web-ui if host !== app.example.com and ips != 127.0.0.1, 10.9.1.0/24
        ## But the repo metadata will be available for all hosts and ips.
        repo_hosts: ['*', '!app.example.com']
        allow_ips: '127.0.0.1, 10.9.1.0/24'
        status_code: 402
        custom_page: > # Custom landing non-auth page. Path or HTML
            <html>
            <head><title>402 Payment Required</title></head>
            <body>
            <center><h1>402 Payment Required</h1></center>
            <hr><center>nginx</center>
            </body>
            </html>

Where custom_page html content or path to html page.

Here all hosts will be hidden under this page (if ip is not match or host != app.example.com).

app.example.com - this is host for default Web-UI.

Example 2

    web_protection: 
        repo_hosts: ['repo.example.com']

Here Web-UI will be hidden for repo_hosts host repo.example.com.

Security Monitoring

Security Monitoring allow to send notifications when found a security problem in your composer.lock. By default, used packagist.org database.

Packeton is automatically check the main branch of every repository if the composer.lock is exists. You need to configure notifications webhook to receive notification if found a new security issue. Also, you may see list of security advisories in the package page.

Configure Webhook Notifications

Go to Webhook page and click the "Add Webhook". Please fill the form.

For example request payload for telegram. It will send JSON request, because response is array

{% set text = "New security issue *#{package.name}*\n\n" %}
{% for advisory in advisories %}
    {% set text = text ~ "#{advisory.title}\nPackage: *#{advisory.packageName}* #{advisory.version}\n" %}
    {% set text = text ~ (advisory.cve and advisory.link ? "[#{advisory.cve}](#{advisory.link})\n" : "Advisory: #{advisory.advisoryId}\n") %}
    {% set text = text ~ "Reported at: #{advisory.reportedAt}\n\n" %}
{% endfor %}

{% set response = {
    'chat_id': '${secrets.CHART_ID}',
    'text': text,
    'parse_mode': 'Markdown'
} %}

{% return response %}

Twig vars: advisories - list of advisories Composer\Advisory\SecurityAdvisory package - package object.

Where ${secrets.CHART_ID} ${secrets.TOKEN} replace with secrets or hardcode this params. See webhooks docs.

Migrate from Packagist.com / Satis

Packeton import provide interface to fast mass import all private packages from your own composer repository, like Satis/Packagist. also you may use oauth2 integration to import all packages from your VCS hosting

Where

• Glob package filter - List of Glob to filter by package vendor name.

Example input value:

okvpn/*
org1/*

• Select only packages (default all packages in the repository) - used if composer repository does not provide API to fetch list of all packages. Put your composer.json, composer.lock, composer info output or packages names separated by spaces or line break

Example input value:

sebastian/cli-parser               2.0.0   Library for parsing CLI options
sebastian/code-unit                2.0.0   Collection of value objects that represent the PHP code units
sebastian/code-unit-reverse-lookup 3.0.0   Looks up which function or method a line of code belongs to
sebastian/comparator               5.0.1   Provides the functionality to compare PHP values for equality
sebastian/complexity               3.0.1   Library for calculating the complexity of PHP code units
sebastian/diff                     5.0.3   Diff implementation
sebastian/environment              6.0.1   Provides functionality to handle HHVM/PHP environments
sebastian/exporter                 5.0.0   Provides the functionality to export PHP variables for visualization
sebastian/global-state             6.0.1   Snapshotting of global state
sebastian/lines-of-code            2.0.1   Library for counting the lines of code in PHP source code

• Clone preference - used to select URL format. SSH or HTTPs clone.

OAuth2 and Sync integrations

Table of content

• Pull Request review
• Login Restriction
• GitHub Setup
• GitHub App Setup
• GitLab Setup
• Gitea Setup
• Bitbucket Setup

Base configuration reference

To enable OAuth2 integrations, you need to add the following configuration

packeton:
    integrations:
        github: # Alias name 
            allow_login: true # default false 
            allow_register: false # default false 
            default_roles: ['ROLE_USER', 'ROLE_MAINTAINER', 'ROLE_GITLAB']
            
            clone_preference: 'api'
            repos_synchronization: true
            
            disable_hook_repos: false # disabled auto setup webhook
            disable_hook_org: false
            svg_logo: ~ # <svg xmlns= logo
            logo: ~ # png logo
            login_title: Login or Register with GitHub
            description: ~

            login_control_expression: "data['email'] ends with '@packeton.org'" # Restrict logic/register by custom condition.
            login_control_expression_debug: false # help debugging    

            pull_request_review: true # Enable pull request composer.lock review. Default false
            webhook_url: ~ #overwrite host when setup webhooks

            github:
                client_id: 'xxx'
                client_secret: 'xxx'

        gitlab2:  # Alias name - may be any url safe value.
            base_url: 'https://gitlab.production.com/'
            clone_preference: 'clone_https' # Allows [api, clone_https, clone_ssh]
            gitlab: # Provider name: github, gitlab, bitbucket etc 
                client_id: 'xxx'
                client_secret: 'xxx'
                api_version: 'v4' # you may overwrite only for gitlab provider, default v4

        # Use GitHub APP JWT
        # See https://docs.github.com/en/apps/creating-github-apps/authenticating-with-a-github-app/about-authentication-with-a-github-app
        githubapp_main:
            repos_synchronization: true
            pull_request_review: true
            githubapp:
                private_key: '%kernel.project_dir%/var/packeton-private-key.pem'
                passphrase: ~ # private key pass
                app_id: 345472

        gitea:
            allow_login: true
            repos_synchronization: true
            pull_request_review: true
            base_url: 'https://gitea.packeton.com.ar/'
            gitea:
                client_id: '44000000-0000-0000-0000-00000000000'
                client_secret: 'gto_acxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx'

        bitbucket:
            repos_synchronization: true
            pull_request_review: true
            bitbucket:
                key: GA7000000000000000
                secret: 9chxxxxxzxxxxxxxxeexxxxxxxxxxxxx
                api_version: ~ # '/example/rest/v2/' custom api prefix

        google:
            allow_login: true
            google:
                client_id: 'xxxxx.apps.googleusercontent.com'
                client_secret: 'xxxx'    

Where clone_preference:

• api - Use api to get composer info
• clone_https - clone repo with using oauth api token
• clone_ssh - clone repo with system ssh key

repos_synchronization - If enabled, a new package will be automatically created when you will push to a new or exists repo that contains composer.json

Docker env.

To make docker usage more easy, you can use env variables to configure basic settings for each integration without editing the *.yaml configs.

# GitLab
# OAUTH_GITLAB_CLIENT_ID=
# OAUTH_GITLAB_CLIENT_SECRET=
# OAUTH_GITLAB_BASE_URL=

# GitHub
# OAUTH_GITHUB_CLIENT_ID=
# OAUTH_GITHUB_CLIENT_SECRET=
# OAUTH_GITHUB_BASE_URL=

# Gitea 
# OAUTH_GITEA_CLIENT_ID=
# OAUTH_GITEA_CLIENT_SECRET=
# OAUTH_GITEA_BASE_URL=

# Bitbucket 
# OAUTH_BITBUCKET_CLIENT_ID=
# OAUTH_BITBUCKET_CLIENT_SECRET=
# OAUTH_BITBUCKET_BASE_URL=

# Google SSO
# OAUTH_GOOGLE_CLIENT_ID=
# OAUTH_GOOGLE_CLIENT_SECRET=
# OAUTH_GOOGLE_ALLOW_REGISTRATION=

# Additinal vars ${NAME} = GITLAB/GITHUB/ .. 
# OAUTH_*_DISABLE_ORG_HOOK=
# OAUTH_*_DISABLE_REP_HOOK=
# OAUTH_*_ALLOW_LOGIN=
# OAUTH_*_ALLOW_REGISTRATION=

Supported 3-d provider

GitHub

Scopes:

• login: user:email
• repositories: read:org, repo

Redirect Urls:

https://example.com/

GitLab

Scopes:

• login: read_user
• repositories: api

Redirect Urls:

https://example.com/oauth2/{alias}/install
https://example.com/oauth2/{alias}/check

GitLab Groups Webhooks notices

A group webhooks needed for synchronization a new package. They are triggered by events that occur across all projects in the group. This feature is enabled only for Premium / EE / Gold paid plan, but it can be replaced with GitLab Packagist Integration

You must manually set up this integration.

Where token you can find on the packeton integration view page. The token must have whk prefix to find related integration access token.

Gitea

Scopes:

• login: read:user
• repositories: organization, repository, write:issue

Redirect Urls:

https://example.com/oauth2/{alias}/auto

Bitbucket

Scopes:

• repositories & login: account, webhook, team, project, pullrequest

Redirect Urls:

https://example.com/oauth2/{alias}/auto

Pull Request composer lock review

Every time when you create a Pull Request with composer.lock changes, the Packeton add a comment with descriptions of changing dependencies. It detects the next of changes:

• Added a new dependency
• Remove dependency
• Downgrade dependency
• Upgrades dependency.
• Change dist or source url.

Configure Pull Request Review.

In the first you need add oauth integration. It may also GitHub app bot for GitHub hosting, see oauth2.

You must enable pull_request_review on configuration level. Or later you may enable per repository individually in the case then you don't use the integration synchronization.

packeton:
    integrations:
        github:
            pull_request_review: true
            githubapp:
                ...

Now the bot will add comments automatically if you use with integration synchronization. But for enable it manually only for one selected repository you need add webhook (pull_request score) with integration access token.

it may look like. you can found this on integration view page.

https://example.com/api/hooks/gitlab/6?token=whk_810d6b279b3f78b758e09fe01f12378d2bd809c4

GitHub OAuth2 Setup

Go to you GitHub account Settings / Developer settings (https://github.com/settings/developers) and select "New OAuth App"

Use the packeton host as "Callback URL / Redirect URL". For example https://packeton.example.org

Use obtained client_id, client_secret to create configuration in yaml. For docker installation you may use config.yaml file in docker volume.

packeton:
    integrations:
        github: # any alias name 
            allow_login: true 
            ... more options see oauth2 md
            gitlab: # Provider name: github, gitlab, bitbucket etc 
                client_id: 'xxx'
                client_secret: 'xxx'

Now go to Packeton integration page and click Install Integration. You will see a list of available integrations and its Redirect Urls

Click to Connect to set up oauth2 credentials.

GitHub App Integration

GitHub App Integration is alternative of GitHub OAuth2. To create GitHub go to: Settings / Developer settings / GitHub Apps and click New GitHub App.

• Enter a GitHub App name: Private Packeton.
• Enter a homepage url: https://packeton.example.com for example.
• Click Add Callback URL and use the next "Callback URL / Redirect URL".

https://packeton.example.com/oauth2/{alias}/install
https://packeton.example.com/oauth2/{alias}/check

• Uncheck webhook active checkbox

Select Repository permissions:

• Content: read-only
• Webhooks: read-write
• Metadata: read-only (already selected)
• Pull requests read-write

Select Organization permissions:

• Webhooks: read-write
• Members: read (optional)

After creating App go to App view page and find app_id and generate a "Private Key"

The next step is creating of configuration in yaml. For docker installation you may use config.yaml file in docker volume.

packeton:
    integrations:
        github:
            repos_synchronization: true
            pull_request_review: true
            ... 
            githubapp:
                private_key: '/data/packeton-key.pem'
                app_id: 340120

When you must install GitHub App in your GitHub Account.

Go to public App page, like https://github.com/apps/{name} and click configure. Select your organization or own account.

After install you will see installation_id on URL address. For example https://github.com/settings/installations/38069000

Now go to Packeton integration page and click Install Integration. When click to Connect under you github configuration.

To finish setup Go to Packeton integration view page / Settings and provider installation_id in the form

GitLab Integration Setup

Go to you GitLab account preferences (https://gitlab.example.com/-/profile/preferences) and select "Applications"

Redirect Urls:

https://example.com/oauth2/{alias}/install
https://example.com/oauth2/{alias}/check

Select the api, read_user, read_repository scopes and save the new application. The redirect urls you may change later.

Use obtained clinent_id, client_secret to create configuration in yaml. For docker installation you may use config.yaml file in docker volume.

packeton:
    integrations:
        gitlab: # - {alias}
            base_url: 'https://gitlab.example.com/'
            clone_preference: 'clone_https' # Allows [api, clone_https, clone_ssh]
            ... more options see oauth2 md
            gitlab: # Provider name: github, gitlab, bitbucket etc 
                client_id: 'xxx'
                client_secret: 'xxx'
#                api_version: 'v4'

Now go to Packeton integration page and click Install Integration. You will see list of available integrations and its Redirect Urls

Click to Connect to setup oauth2 credentials

Gitea Integration Setup

Go to you Gitea account settings (https://git.example.com/user/settings/applications) and select "Applications".

Redirect Urls:

https://example.com/oauth2/{alias}/auto

The next step is obtain clinent_id, client_secret to creating a configuration in the yaml. For docker installation you may use config.yaml file in docker volume.

packeton:
    integrations:
        gitea:
            allow_login: true
            repos_synchronization: true
            pull_request_review: true
            base_url: 'https://git.example.com/'
            gitea:
                client_id: '44000000-0000-0000-0000-00000000000'
                client_secret: 'gto_acxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx'
#                api_version: 'v1'

Now you can go to the Packeton integration page and click "Install Integration". You will see list of available integrations and its Redirect Urls. Click to Connect to set up oauth2 credentials

Example of pull request review

Bitbucket Integration Setup

Go to you Bitbucket workspace settings (https://bitbucket.org/{name}/workspace/settings/api) and select "Add OAuth consumer".

Redirect Urls:

https://example.com/oauth2/{alias}/auto

The next step is obtain Key, Secret to creating a configuration in the yaml. For docker installation you may use config.yaml file in docker volume.

packeton:
    integrations:
        bitbucket:
            repos_synchronization: true
            pull_request_review: true
            bitbucket:
                key: GA7000000000000000
                secret: 9chxxxxxzxxxxxxxxeexxxxxxxxxxxxx

Now you can go to the Packeton integration page and click "Install Integration". You will see list of available integrations and its Redirect Urls. Click to Connect to set up oauth2 credentials.

Limit login/register with using expression lang

You may limit login with using expression, like symfony expression for access control. For evaluate expression used TWIG engine with customization by this lib okvpn/expression-language. It allows to create a complex expressions where called team/members API to check that user belong to Organization/Repos etc.

Example usage

packeton:
    integrations:
        github:
            allow_login: true
            allow_register: true
            github:
                client_id: 'xxx'
                client_secret: 'xxx'
            login_control_expression: "data['email'] ends with '@packeton.org'"

Example 2. Here check GitLab's groups API.

packeton:
    integrations:
        gitlab:
            allow_login: true
            allow_register: true
            gitlab:
                client_id: 'xx'
                client_secret: 'xx'
            login_control_expression: >
                {% set members = api_cget('/groups/balaba/members') %}
                {% set found = null %}
                {% for member in members %}
                    {% if data['username'] and data['username'] == member['username'] %}
                        {% set found = member %}
                    {% endif %}
                {% endfor %}
    
                {% if found['access_level'] >= 50 %}
                    {% return ['ROLE_ADMIN', 'ROLE_GITLAB'] %}
                {% elseif found['access_level'] >= 40 %}
                    {% return ['ROLE_MAINTAINER', 'ROLE_GITLAB'] %}
                {% elseif found['access_level'] >= 10 %}
                    {% return ['ROLE_USER', 'ROLE_GITLAB'] %}
                {% endif %}
                {% return [] %}

Custom Twig function for expression lang

• api_get(url, query = [], cache = true, app = null) - Call get method
• api_cget(url, query = [], cache = true, app = null) - Call get method with pagination with all pages.

By default, the API call results are cached, but you may overwrite with cache param.

login_control_expression - may return a bool result or list of roles. If returned result is empty - login/register is not allowed.

Debug expressions

You may enable debugging by param

packeton:
    integrations:
        gitlab:
            login_control_expression_debug: true
            login_control_expression: "data['email'] ends with '@packeton.org'"

For localhost, you also can enable symfony dev env. But it's strongly not recommended for prod for security reasons. Then you may use dump action.

APP_ENV=dev

{% set members = api_cget('/groups/balaba/members') %}
{% set found = null %}
{% for member in members %}
    {% if data['username'] and data['username'] == member['username'] %}
        {% set found = member %}
    {% endif %}
{% endfor %}
{% do dump(members) %}
{% do dump(found) %}

{% return [] %}

Example debug panel

When login_control_expression_debug is enabled you may evaluate script from UI.

Contributing

Everyone is welcome to contribute code to https://github.com/vtsykun/packeton.git

1. Development environment.

If you are running Windows, the Windows Subsystem for Linux (WSL) is recommended for development. But the most of the features will work on Windows too. The code of Packeton is written in PHP.

Requirements

• PHP 8.1+
• Redis (or Docker) for some functionality.
• (optional) nginx / php-fpm to run the web server.
• (optional) MySQL or PostgresSQL for the main data store, default SQLite.

2. Get the source.

Make a fork on GitHub, and then create a pull request to provide your changes.

git clone git@github.com:YOUR_GITHUB_NAME/packeton.git
git checkout -b fix/patch-1

3. Install the dependencies

Run composer install

cd packeton
composer install

4. Configure your env vars.

Create a file .env.local with following content.

# .env.local

APP_ENV=dev

# select database, default SQLite
DATABASE_URL="postgresql://postgres:123456@127.0.0.1:5432/packeton?serverVersion=12&charset=utf8"

5. Setup database

bin/console doctrine:schema:update --dump-sql --force

6. Run local webserver.

php -S localhost:8000 -t public/

ENJOY

Configuration Reference

Full packeton configuration:

packeton:
    github_no_api: '%env(bool:GITHUB_NO_API)%' # default true
    rss_max_items: 30
    archive: true
    
    anonymous_access: '%env(bool:PUBLIC_ACCESS)%' # default false
    anonymous_archive_access: '%env(bool:PUBLIC_ACCESS)%' # default false
    archive_options:
        format: zip
        basedir: '%env(resolve:PACKAGIST_DIST_PATH)%'
        endpoint: '%env(PACKAGIST_DIST_HOST)%' # default auto detect by host headers 
        include_archive_checksum: false

    jwt_authentication: # disable by default 
        algo: EdDSA
        private_key: '%kernel.project_dir%/var/jwt/eddsa-key.pem'
        public_key: '%kernel.project_dir%/var/jwt/eddsa-public.pem'
        passphrase: ~

    metadata:
        format: auto # Default, see about metadata.
        info_cmd_message: ~ # Bash logo, example - \u001b[37;44m#StandWith\u001b[30;43mUkraine\u001b[0m

    artifacts:
        support_types: ['gz', 'tar', 'tgz', 'zip']
        allowed_paths:
            - '/data/hdd1/composer'
        # Default path to storage/(local cache for S3) of uploaded artifacts
        artifact_storage: '%composer_home_dir%/artifact_storage'

    integrations: # See oauth2 integrations
        alias_name: # Alias name ()
            allow_login: true # default false 
            allow_register: false # default false 
            default_roles: ['ROLE_USER', 'ROLE_MAINTAINER', 'ROLE_GITLAB']
        
            clone_preference: 'api'
            repos_synchronization: true
        
            disable_hook_repos: false # disabled auto setup webhook
            disable_hook_org: false
            svg_logo: ~ # <svg xmlns= logo
            logo: ~ # png logo
            login_title: Login or Register with GitHub
            description: ~
        
            login_control_expression: "data['email'] ends with '@packeton.org'" # Restrict logic/register by custom condition.
            login_control_expression_debug: false # help debugging    
        
            pull_request_review: true # Enable pull request composer.lock review. Default false
            webhook_url: ~ #overwrite host when setup webhooks
        
            github:
                client_id: 'xxx'
                client_secret: 'xxx'
    
            gitlab:
                client_id: 'xxx'
                client_secret: 'xxx'
                api_version: 'v4'
    
            githubapp:
                private_key: '%kernel.project_dir%/var/packeton-private-key.pem'
                passphrase: ~ # private key pass
                app_id: 345472

            gitea:
                client_id: '44000000-0000-0000-0000-00000000000'
                client_secret: 'gto_acxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx'

            bitbucket:
                key: GA7000000000000000
                secret: 9chxxxxxzxxxxxxxxeexxxxxxxxxxxxx
                api_version: ~ # '/example/rest/v2/' custom api prefix

    # See mirrors section
    mirrors:
        packagist:
            url: https://repo.packagist.org
        orocrm:
            url: https://satis.oroinc.com/
            git_ssh_keys:
                git@github.com:oroinc: '/var/www/.ssh/private_key1'
                git@github.com:org2: '/var/www/.ssh/private_key2'
        example:
            url: https://satis.example.com/
            logo: 'https://example.com/logo.png'
            http_basic:
                username: 123
                password: 123
            public_access: true # Allow public access, default false
            sync_lazy: true # default false 
            enable_dist_mirror: false # default true
            available_package_patterns: # Additional restriction, but you can restrict it in UI
                - 'vend1/*'
            available_packages:
                - 'pack1/name1' # but you can restrict it in UI
            composer_auth: '{"auth.json..."}' # JSON. auth.json to pass composer opts.
            sync_interval: 3600 # default auto.
            info_cmd_message: "\n\u001b[37;44m#Слава\u001b[30;43mУкраїні!\u001b[0m\n\u001b[40;31m#Смерть\u001b[30;41mворогам\u001b[0m" # Info message

    web_protection:
        ## Multi host protection, disable web-ui if host !== app.example.com and ips != 127.0.0.1, 10.9.1.0/24
        ## But the repo metadata will be available for all hosts and ips.
        repo_hosts: ['*', '!app.example.com']
        allow_ips: '127.0.0.1, 10.9.1.0/24'
        status_code: 402
        custom_page: > # Custom landing non-auth page. Path or HTML
            <html>
            <head><title>402 Payment Required</title></head>
            <body>
            <center><h1>402 Payment Required</h1></center>
            <hr><center>nginx</center>
            </body>
            </html>

    web_protection:
        ## Disable web-ui for host = repo.example.com
        repo_hosts: ['repo.example.com']