はじまりの大地
このコミットが含まれているのは:
@@ -0,0 +1,40 @@
|
||||
# Continuous integration
|
||||
|
||||
PeerTube uses Github Actions as a CI platform.
|
||||
CI tasks are described in `.github/workflows`.
|
||||
|
||||
## benchmark.yml
|
||||
|
||||
*Scheduled*
|
||||
|
||||
Run various benchmarks (build, API etc) and upload results on https://builds.joinpeertube.org/peertube-stats/ to be publicly consumed.
|
||||
|
||||
## codeql.yml
|
||||
|
||||
*Scheduled, on push on develop and on pull request*
|
||||
|
||||
Run CodeQL task to throw code security issues in Github. https://lgtm.com/projects/g/Chocobozzz/PeerTube can also be used.
|
||||
|
||||
## docker.yml
|
||||
|
||||
*Scheduled and on push on master*
|
||||
|
||||
Build `chocobozzz/peertube-webserver:latest`, `chocobozzz/peertube:production-...`, `chocobozzz/peertube:v-...` (only latest PeerTube tag) and `chocobozzz/peertube:develop-...` Docker images. Scheduled to automatically upgrade image software (Debian security issues etc).
|
||||
|
||||
## nightly.yml
|
||||
|
||||
*Scheduled*
|
||||
|
||||
Build PeerTube nightly build (`develop` branch) and upload the release on https://builds.joinpeertube.org/nightly.
|
||||
|
||||
## stats.yml
|
||||
|
||||
*On push on develop*
|
||||
|
||||
Create various PeerTube stats (line of codes, build size, lighthouse report) and upload results on https://builds.joinpeertube.org/peertube-stats/ to be publicly consumed.
|
||||
|
||||
## test.yml
|
||||
|
||||
*Scheduled, on push and pull request*
|
||||
|
||||
Run PeerTube lint and tests.
|
||||
@@ -0,0 +1,25 @@
|
||||
# Lib development documentation
|
||||
|
||||
## @peertube/embed-api
|
||||
|
||||
### Build & Publish
|
||||
|
||||
```
|
||||
cd client/src/standalone/embed-player-api/
|
||||
npm run build
|
||||
npm publish --access=public
|
||||
```
|
||||
|
||||
## @peertube/peertube-types
|
||||
|
||||
Typescript definition files generation is controlled by the various `tsconfig.types.json` files.
|
||||
|
||||
The complete types package is generated via:
|
||||
|
||||
```
|
||||
npm run generate-types-package 4.x.x
|
||||
cd packages/types-generator/dist
|
||||
npm publish --access=public
|
||||
```
|
||||
|
||||
> See [scripts/generate-types-package.ts](scripts/generate-types-package.ts) for details.
|
||||
@@ -0,0 +1,35 @@
|
||||
# Application localization documentation
|
||||
|
||||
Source files are in `client/src/locale` and translated files merged from [Weblate](https://weblate.framasoft.org/translate/peertube).
|
||||
|
||||
|
||||
## Generation
|
||||
|
||||
Will generate XLIFF base files for Angular (`angular.xlf`) and JSON files for the player (`player.en-US.json`) and the server (`server.en-US.json`).
|
||||
Then, it will merge new translation keys into localized Angular files (`angular.fr-FR.xlf` etc).
|
||||
|
||||
```
|
||||
npm run i18n:update
|
||||
```
|
||||
|
||||
|
||||
## Upload on Weblate
|
||||
|
||||
Nothing to do here, Github will automatically send a webhook to Weblate that will pull changes.
|
||||
|
||||
|
||||
## Pull translation
|
||||
|
||||
* First, save translations on Weblate so it commits changes.
|
||||
* Then, fetch these commits: `git fetch weblate && git merge weblate/develop`
|
||||
|
||||
|
||||
## Support a new language
|
||||
|
||||
* Add it to [/packages/models/i18n/i18n.ts](/packages/core-utils/src/i18n/i18n.ts)
|
||||
* Add it to [/scripts/build/client.sh](/scripts/build/client.sh)
|
||||
* Add it to [/client/angular.json](/client/angular.json)
|
||||
* Add it to [/scripts/i18n/update.sh](/scripts/i18n/update.sh)
|
||||
* Lock [weblate project](https://weblate.framasoft.org/projects/peertube)
|
||||
* Run `npm run i18n:update`
|
||||
* Build the application and check the new language correctly works
|
||||
@@ -0,0 +1,23 @@
|
||||
# Monitoring
|
||||
|
||||
## Client modules
|
||||
|
||||
To open a report of client build:
|
||||
|
||||
```
|
||||
npm run build -- --analyze-bundle && npm run client-report
|
||||
```
|
||||
|
||||
## API benchmark
|
||||
|
||||
To benchmark the REST API and save result in `benchmark.json`:
|
||||
|
||||
```
|
||||
npm run benchmark-server -- -o benchmark.json
|
||||
```
|
||||
|
||||
You can also grep on a specific test:
|
||||
|
||||
```
|
||||
npm run benchmark-server -- --grep homepage
|
||||
```
|
||||
@@ -0,0 +1,39 @@
|
||||
# Release
|
||||
|
||||
## PeerTube
|
||||
|
||||
* Fix remaining important bugs
|
||||
* Ensure French translation is 100% (for the screens in the JoinPeerTube blog post)
|
||||
* Update [/CHANGELOG.md](/CHANGELOG.md)
|
||||
* Check migrations:
|
||||
```
|
||||
npm run clean:server:test
|
||||
git checkout master && rm -r ./node_modules && yarn install --pure-lockfile && npm run build:server
|
||||
NODE_APP_INSTANCE=6 NODE_ENV=test node dist/server --benchmark-startup
|
||||
git checkout develop && rm -r ./node_modules && yarn install --pure-lockfile && npm run build:server
|
||||
NODE_APP_INSTANCE=6 NODE_ENV=test node dist/server --benchmark-startup
|
||||
```
|
||||
* Run `rm -rf node_modules && rm -rf client/node_modules && yarn install --pure-lockfile && npm run build` to see if all the supported languages compile correctly
|
||||
* Update https://peertube2.cpy.re and check it works correctly
|
||||
* Check CI tests are green
|
||||
* Run BrowserStack **and** local E2E tests
|
||||
* Release: `GITHUB_TOKEN=my_token npm run release -- 1.x.x`
|
||||
* Update `openapi.yaml` version
|
||||
* Upload `tar.xz` on https://builds.joinpeertube.org/release
|
||||
* Create a dedicated branch: `git checkout -b release/1.x.x && git push origin release/1.x.x`
|
||||
* Check the release is okay: https://github.com/Chocobozzz/PeerTube/releases
|
||||
* Update https://peertube3.cpy.re and check it works correctly
|
||||
* Update all other instances and check it works correctly
|
||||
* After a couple of days, update https://joinpeertube.org/api/v1/versions.json
|
||||
|
||||
|
||||
## @peertube/embed-api
|
||||
|
||||
At the root of PeerTube:
|
||||
|
||||
```
|
||||
cd client/src/standalone/embed-player-api
|
||||
npm version patch
|
||||
cd ../../../../
|
||||
npm run release-embed-api
|
||||
```
|
||||
@@ -0,0 +1,82 @@
|
||||
# Server code
|
||||
|
||||
## Database model typing
|
||||
|
||||
Sequelize models contain optional fields corresponding to table joins.
|
||||
For example, `VideoModel` has a `VideoChannel?: VideoChannelModel` field. It can be filled if the SQL query joined with the `videoChannel` table or empty if not.
|
||||
It can be difficult in TypeScript to understand if a function argument expects associations to be filled or not.
|
||||
To improve clarity and reduce bugs, PeerTube defines multiple versions of a database model depending on its associations in `server/core/types/models/`.
|
||||
These models start with `M` and by default do not include any association. `MVideo` for example corresponds to `VideoModel` without any association, where `VideoChannel` attribute doesn't exist. On the other hand, `MVideoWithChannel` is a `MVideo` that has a `VideoChannel` field. This way, a function that accepts `video: MVideoWithChannel` argument expects a video with channel populated. Main PeerTube code should never use `...Model` (`VideoModel`) database type, but always `M...` instead (`MVideo`, `MVideoChannel` etc).
|
||||
|
||||
## Add a new feature walkthrough
|
||||
|
||||
Here's a list of all the parts of the server to update if you want to add a new feature (new API REST endpoints for example) to the PeerTube server.
|
||||
Some of these may be optional (for example your new endpoint may not need to send notifications) but this guide tries to be exhaustive.
|
||||
|
||||
* Configuration:
|
||||
- Add you new configuration key in `config/default.yaml` and `config/production.yaml`
|
||||
- If you configuration needs to be different in dev or tests environments, also update `config/dev.yaml` and `config/test.yaml`
|
||||
- Load your configuration in `server/core/initializers/config.ts`
|
||||
- Check new configuration keys are set in `server/core/initializers/checker-before-init.ts`
|
||||
- You can also ensure configuration consistency in `server/core/initializers/checker-after-init.ts`
|
||||
- If you want your configuration to be available in the client:
|
||||
+ Add your field in `packages/models/src/server/core/server-config.model.ts`
|
||||
+ Update `server/core/lib/server-config-manager.ts` to include your new configuration
|
||||
- If you want your configuration to be updatable by the web admin in the client:
|
||||
+ Add your field in `packages/models/src/server/core/custom-config.model.ts`
|
||||
+ Add the configuration to the config object in the `server/core/controllers/api/config.ts` controller
|
||||
* Controllers:
|
||||
- Create the controller file and fill it with your REST API routes
|
||||
- Import and use your controller in the parent controller
|
||||
* Middlewares:
|
||||
- Create your validator middleware in `server/core/middlewares/validators` that will be used by your controllers
|
||||
- Add your new middleware file `server/core/middlewares/validators/index.ts` so it's easier to import
|
||||
- Create the entry in `server/core/types/express.d.ts` to attach the database model loaded by your middleware to the express response
|
||||
* Validators:
|
||||
- Create your validators that will be used by your middlewares in `server/core/helpers/custom-validators`
|
||||
* Typescript models:
|
||||
- Create the API models (request parameters or response) in `packages/models`
|
||||
- Add your models in `index.ts` of current directory to facilitate the imports
|
||||
* Sequelize model (BDD):
|
||||
- If you need to create a new table:
|
||||
+ Create the Sequelize model in `server/core/models/`:
|
||||
* Create the `@Column`
|
||||
* Add some indexes if you need
|
||||
* Create static methods to load a specific from the database `loadBy...`
|
||||
* Create static methods to load a list of models from the database `listBy...`
|
||||
* Create the instance method `toFormattedJSON` that creates the JSON to send to the REST API from the model
|
||||
+ Add your new Sequelize model to `server/core/initializers/database.ts`
|
||||
+ Create a new file in `server/core/types` to define multiple versions of your Sequelize model depending on database associations
|
||||
+ Add this new file to `server/core/types/*/index.ts` to facilitate the imports
|
||||
+ Create database migrations:
|
||||
* Create the migration file in `server/core/initializers/migrations` using raw SQL (copy the same SQL query as at PeerTube startup)
|
||||
* Update `LAST_MIGRATION_VERSION` in `server/core/initializers/constants.ts`
|
||||
- If updating database schema (adding/removing/renaming a column):
|
||||
+ Update the sequelize models in `server/core/models/`
|
||||
+ Add migrations:
|
||||
* Create the migration file in `initializers/migrations` using Sequelize Query Interface (`.addColumn`, `.dropTable`, `.changeColumn`)
|
||||
* Update `LAST_MIGRATION_VERSION` in `server/core/initializers/constants.ts`
|
||||
* Notifications:
|
||||
- Create the new notification model in `packages/models/src/users/user-notification.model.ts`
|
||||
- Create the notification logic in `server/core/lib/notifier/shared`:
|
||||
+ Email subject has a common prefix (defined by the admin in PeerTube configuration)
|
||||
- Add your notification to `server/core/lib/notifier/notifier.ts`
|
||||
- Create the email template in `server/core/assets/email-templates`:
|
||||
+ A text version is automatically generated from the HTML
|
||||
+ The template usually extends `../common/grettings` that already says "Hi" and "Cheers". You just have to write the title and the content blocks that will be inserted in the appropriate places in the HTML template
|
||||
- If you need to associate a new table with `userNotification`:
|
||||
+ Associate the new table in `UserNotificationModel` (don't forget the index)
|
||||
+ Add the object property in the API model definition (`packages/models/src/users/user-notification.model.ts`)
|
||||
+ Add the object in `UserNotificationModel.toFormattedJSON`
|
||||
+ Handle this new notification type in client (`UserNotificationsComponent`)
|
||||
+ Handle the new object property in client model (`UserNotification`)
|
||||
* Tests:
|
||||
- Create your command class in `packages/server-commands/` that will wrap HTTP requests to your new endpoint
|
||||
- Add your command file in `index.ts` of current directory
|
||||
- Instantiate your command class in `packages/server-commands/src/server/core.ts`
|
||||
- Create your test file in `server/core/tests/api/check-params` to test middleware validators/authentification/user rights (offensive tests)
|
||||
- Add it to `server/core/tests/api/check-params/index.ts`
|
||||
- Create your test file in `server/core/tests/api` to test your new endpoints
|
||||
- Add it to `index.ts` of current directory
|
||||
- Add your notification test in `server/core/tests/api/notifications`
|
||||
* Update REST API documentation in `support/doc/api/openapi.yaml`
|
||||
@@ -0,0 +1,127 @@
|
||||
# Tests
|
||||
|
||||
## Preparation
|
||||
|
||||
Prepare PostgreSQL user so PeerTube can delete/create the test databases:
|
||||
|
||||
```bash
|
||||
sudo -u postgres createuser you_username --createdb --superuser
|
||||
```
|
||||
|
||||
Prepare the databases:
|
||||
|
||||
```bash
|
||||
npm run clean:server:test
|
||||
```
|
||||
|
||||
Build PeerTube:
|
||||
|
||||
```bash
|
||||
npm run build
|
||||
```
|
||||
|
||||
## Server tests
|
||||
|
||||
### Dependencies
|
||||
|
||||
Run docker containers needed by some test files:
|
||||
|
||||
```bash
|
||||
sudo docker run -p 9444:9000 chocobozzz/s3-ninja
|
||||
sudo docker run -p 10389:10389 chocobozzz/docker-test-openldap
|
||||
```
|
||||
|
||||
Ensure you also have these commands:
|
||||
|
||||
```bash
|
||||
exiftool --help
|
||||
parallel --help
|
||||
|
||||
# For transcription tests
|
||||
whisper --help
|
||||
whisper-ctranslate2 --help
|
||||
jiwer --help
|
||||
```
|
||||
|
||||
Otherwise, install the packages. On Debian-based systems (like Debian, Ubuntu or Mint):
|
||||
```bash
|
||||
sudo apt-get install parallel libimage-exiftool-perl
|
||||
sudo pip install -r packages/tests/requirements.txt -r packages/transcription-devtools/requirements.txt
|
||||
```
|
||||
|
||||
### Test
|
||||
|
||||
To run all test suites (can be long!):
|
||||
|
||||
```bash
|
||||
npm run test # See scripts/test.sh to run a particular suite
|
||||
```
|
||||
|
||||
To run a specific test:
|
||||
|
||||
```bash
|
||||
npm run mocha -- --exit --bail packages/tests/src/your-test.ts
|
||||
|
||||
# For example
|
||||
npm run mocha -- --exit --bail packages/tests/src/api/videos/single-server.ts
|
||||
```
|
||||
|
||||
### Configuration
|
||||
|
||||
Some env variables can be defined to disable/enable some tests:
|
||||
|
||||
* `DISABLE_HTTP_IMPORT_TESTS=true`: disable import tests (because of youtube that could rate limit your IP)
|
||||
* `ENABLE_OBJECT_STORAGE_TESTS=true`: enable object storage tests (needs `chocobozzz/s3-ninja` container first)
|
||||
* `AKISMET_KEY`: specify an Akismet key to test akismet external PeerTube plugin
|
||||
* `OBJECT_STORAGE_SCALEWAY_KEY_ID` and `OBJECT_STORAGE_SCALEWAY_ACCESS_KEY`: specify Scaleway API keys to test object storage ACL (not supported by our `chocobozzz/s3-ninja` container)
|
||||
* `ENABLE_FFMPEG_THUMBNAIL_PIXEL_COMPARISON_TESTS=true`: enable pixel comparison on images generated by ffmpeg. Disabled by default because a custom ffmpeg version may fails the tests
|
||||
|
||||
|
||||
### Debug server logs
|
||||
|
||||
While testing, you might want to display a server's logs to understand why they failed:
|
||||
|
||||
```bash
|
||||
NODE_APP_INSTANCE=1 NODE_ENV=test npm run parse-log -- --level debug | less +GF
|
||||
```
|
||||
|
||||
You can also:
|
||||
- checkout only the latest logs (PeerTube >= 5.0):
|
||||
|
||||
```bash
|
||||
tail -n 100 test1/logs/peertube.log | npm run parse-log -- --level debug --files -
|
||||
```
|
||||
|
||||
- continuously print the latests logs (PeerTube >= 5.0):
|
||||
|
||||
```bash
|
||||
tail -f test1/logs/peertube.log | npm run parse-log -- --level debug --files -
|
||||
```
|
||||
|
||||
|
||||
## Client E2E tests
|
||||
|
||||
### Local tests
|
||||
|
||||
To run tests on local web browsers (comment web browsers you don't have in `client/e2e/wdio.local.conf.ts`):
|
||||
|
||||
```bash
|
||||
PEERTUBE2_E2E_PASSWORD=password npm run e2e:local
|
||||
```
|
||||
|
||||
### Browserstack tests
|
||||
|
||||
To run tests on browser stack:
|
||||
|
||||
```bash
|
||||
BROWSERSTACK_USER=your_user BROWSERSTACK_KEY=your_key npm run e2e:browserstack
|
||||
```
|
||||
|
||||
### Add E2E tests
|
||||
|
||||
To add E2E tests and quickly run tests using a local Chrome:
|
||||
|
||||
```bash
|
||||
cd client/e2e
|
||||
../node_modules/.bin/wdio wdio.local-test.conf.ts # you can also add --mochaOpts.grep to only run tests you want
|
||||
```
|
||||
新しい課題から参照
ユーザをブロックする