コミットを比較

..

65 コミット

作成者 SHA1 メッセージ 日付
みてるぞ 2e1b4449ba #399 2026-07-16 00:44:11 +09:00
みてるぞ 4eb483c049 #399 2026-07-16 00:24:24 +09:00
みてるぞ e197f27e9a #399 2026-07-15 23:59:01 +09:00
みてるぞ 34f81325d2 #399 2026-07-15 23:42:05 +09:00
みてるぞ 9af4670c6c #399 2026-07-15 23:29:19 +09:00
みてるぞ 5183b0b354 #399 2026-07-15 23:06:28 +09:00
みてるぞ d0914fce3b #399 2026-07-15 22:51:01 +09:00
みてるぞ 0ac7332458 #399 2026-07-15 22:41:06 +09:00
みてるぞ 07ce19e32d #399 2026-07-15 22:26:58 +09:00
みてるぞ 5f1d619139 #399 2026-07-15 22:04:47 +09:00
みてるぞ 3cd22855b3 #399 2026-07-15 21:42:00 +09:00
みてるぞ 55c69866a7 #399 2026-07-15 21:37:03 +09:00
みてるぞ 04d2f0f5a9 #399 2026-07-15 20:59:00 +09:00
みてるぞ 025f49cbcb #399 2026-07-15 20:51:24 +09:00
みてるぞ d2f1b1ed48 #399 2026-07-15 20:25:51 +09:00
みてるぞ 6e5aa1e30f #399 2026-07-15 19:57:09 +09:00
みてるぞ 21d6c1408a #399 2026-07-15 19:43:39 +09:00
みてるぞ eebbd5d069 #399 2026-07-15 19:33:18 +09:00
みてるぞ 58828597d7 #399 2026-07-15 08:03:25 +09:00
みてるぞ b0c24f319a #399 2026-07-14 20:03:11 +09:00
みてるぞ f9463f383f #399 2026-07-14 19:37:04 +09:00
みてるぞ 6d037192c4 #399 2026-07-14 19:12:23 +09:00
みてるぞ d035da99ad #399 2026-07-14 18:39:24 +09:00
みてるぞ bc660676ef Merge remote-tracking branch 'origin/main' into feature/399 2026-07-14 12:40:32 +09:00
みてるぞ 38650b5671 URL 正規化 (#409) (#410)
Reviewed-on: #410
Co-authored-by: miteruzo <miteruzo@naver.com>
Co-committed-by: miteruzo <miteruzo@naver.com>
2026-07-14 12:38:02 +09:00
みてるぞ 97b132e5ab #399 2026-07-13 01:31:45 +09:00
みてるぞ be68841bd3 #399 2026-07-13 01:17:11 +09:00
みてるぞ ac82adc6b3 #399 2026-07-13 00:58:02 +09:00
みてるぞ aa96ec95d1 #399 2026-07-12 14:03:05 +09:00
みてるぞ df17f20907 #399 2026-07-12 13:14:36 +09:00
みてるぞ 769966648b #399 2026-07-12 12:43:31 +09:00
みてるぞ 7bcb76516c #399 2026-07-12 12:23:53 +09:00
みてるぞ 8970edc59f #399 2026-07-12 11:17:23 +09:00
みてるぞ 9b1ea56e36 #399 2026-07-12 10:27:13 +09:00
みてるぞ 155edfe018 #399 2026-07-12 02:22:36 +09:00
みてるぞ 23f1ffb04b #399 2026-07-12 02:08:32 +09:00
みてるぞ ccfe65a6a4 #399 2026-07-12 02:01:25 +09:00
みてるぞ 06d6e512e4 #399 2026-07-12 01:56:24 +09:00
みてるぞ 5bd097bcfe #399 2026-07-12 01:47:34 +09:00
みてるぞ f636d2a177 #399 2026-07-12 01:39:31 +09:00
みてるぞ 10dc776313 #399 2026-07-12 01:17:45 +09:00
みてるぞ a9e16735f8 #399 2026-07-12 01:11:07 +09:00
みてるぞ 440a6c9961 #399 2026-07-12 00:54:21 +09:00
みてるぞ 4535a9d260 #399 2026-07-12 00:46:53 +09:00
みてるぞ 440d3d38be #399 2026-07-11 23:21:59 +09:00
みてるぞ cb33d9ca25 #399 2026-07-11 23:08:03 +09:00
みてるぞ 19bf24432a #399 2026-07-11 23:07:51 +09:00
みてるぞ ac41385962 Merge remote-tracking branch 'origin/main' into feature/399 2026-07-11 22:36:03 +09:00
みてるぞ 08bf92ff79 #399 2026-07-11 21:46:10 +09:00
みてるぞ c51d7b98ba #399 2026-07-11 21:11:36 +09:00
みてるぞ b7b284c076 スマホでのメニュー表示デザイン・バグ (#405) (#408)
Reviewed-on: #408
Co-authored-by: miteruzo <miteruzo@naver.com>
Co-committed-by: miteruzo <miteruzo@naver.com>
2026-07-11 05:02:23 +09:00
みてるぞ 6c451d260f サムネ取得 API 見直し (#398) (#402)
Reviewed-on: #402
Co-authored-by: miteruzo <miteruzo@naver.com>
Co-committed-by: miteruzo <miteruzo@naver.com>
2026-07-10 01:27:11 +09:00
みてるぞ c1902fbc99 素材管理で新規タグ紐づけ時にタグ履歴が発生しない問題 (#386) (#392)
開発環境で動作確認したらマージする.

Reviewed-on: #392
2026-07-09 06:43:06 +09:00
みてるぞ 2f2b6e2afa タグ D&D 時の表示位置修正 (#268) (#400)
Reviewed-on: #400
Co-authored-by: miteruzo <miteruzo@naver.com>
Co-committed-by: miteruzo <miteruzo@naver.com>
2026-07-08 00:01:07 +09:00
みてるぞ f5b632ed89 設定画面 (#34) (#397)
Reviewed-on: #397
Co-authored-by: miteruzo <miteruzo@naver.com>
Co-committed-by: miteruzo <miteruzo@naver.com>
2026-07-07 00:48:41 +09:00
みてるぞ 15619886f2 サイド・バーの幅を可変に (#385) (#396)
Reviewed-on: #396
Co-authored-by: miteruzo <miteruzo@naver.com>
Co-committed-by: miteruzo <miteruzo@naver.com>
2026-07-05 03:38:01 +09:00
みてるぞ fdf652951a 【再掲】タグの局所記載 (#351) (#380)
#353 を再開できなかったので.

Reviewed-on: #380
Co-authored-by: miteruzo <miteruzo@naver.com>
Co-committed-by: miteruzo <miteruzo@naver.com>
2026-07-03 03:23:36 +09:00
みてるぞ f96ddb65f1 タグが見切れる問題対応 (#76) (#395)
Reviewed-on: #395
2026-07-02 23:40:01 +09:00
みてるぞ 5657fa7e7b “開発モード” の表示 (#389) (#393)
Reviewed-on: #393
2026-07-02 12:47:19 +09:00
みてるぞ 19a185d5b5 上映会タグ表示バグ修正 (#360) (#384)
まだ画面チェックしてゐないのでマージ禁止

Reviewed-on: #384
Co-authored-by: miteruzo <miteruzo@naver.com>
Co-committed-by: miteruzo <miteruzo@naver.com>
2026-07-02 01:37:49 +09:00
みてるぞ acd062f289 タグ循環の禁止 (#332) (#382)
Reviewed-on: #382
2026-06-30 01:20:55 +09:00
みてるぞ 877876b661 サムネつきで広場に追加する際にエラーとなる問題修正 (#352) (#390)
Reviewed-on: #390
Co-authored-by: miteruzo <miteruzo@naver.com>
Co-committed-by: miteruzo <miteruzo@naver.com>
2026-06-30 01:19:23 +09:00
みてるぞ bffd4c422a 過去のタグ名に名称変更できないバグ修正 (#383) (#387)
Reviewed-on: #387
Co-authored-by: miteruzo <miteruzo@naver.com>
Co-committed-by: miteruzo <miteruzo@naver.com>
2026-06-30 01:18:38 +09:00
みてるぞ 776dea87d9 素材管理 (#306) (#381)
開発環境では、**DB を壊さない前提で、migration → API → 画面 → 同期 → ZIP → 履歴**の順に見るのがよいです。今回の差分は素材管理全体に触っているので、単体でチョンチョン見るより、素材の一生を通すのが早いです。

## 0. 先に方針

**やらないこと:**

```sh id="snb3i6"
rails db:drop
rails db:reset
rails db:setup
DISABLE_DATABASE_ENVIRONMENT_CHECK=1 ...
```

これは禁止。
開発 DB に本番データを入れているなら、床板を剥がして耐震確認するようなものです。

---

## 1. migration 確認

まず現在の状態を見る。

```sh id="qsdfpt"
cd backend
RAILS_ENV=development bundle exec rails db:migrate:status
```

その後、通常 migration。

```sh id="a89fir"
RAILS_ENV=development bundle exec rails db:migrate
```

見るポイント:

```txt id="c1u57a"
materials に source_* / normalized_source_key / version_no がある
material_versions に event_type / file snapshot / source snapshot がある
material_export_items がある
material_sync_suppressions がある
material_sync_sources がある
既存 materials に material_versions version_no=1 create が backfill されている
```

確認用:

```sh id="g4vd4m"
RAILS_ENV=development bundle exec rails runner '
puts "materials=#{Material.count}"
puts "versions=#{MaterialVersion.count}"
puts "materials without versions=#{Material.left_joins(:material_versions).where(material_versions: { id: nil }).count}"
puts "sync suppressions table=#{ActiveRecord::Base.connection.table_exists?(:material_sync_suppressions)}"
'
```

ここで `materials without versions=0` になれば、backfill は通っています。

---

## 2. 既存素材一覧の画面確認

フロントを起動して `/materials` を見る。

```sh id="vl28jd"
cd frontend
npm run dev
```

見る観点:

```txt id="i2ovxw"
初期表示で素材が出る
初期表示ではグルーピングがオフ
タグなし素材も出る
カード表示でサムネまたは代替テキストが出る
一覧表示に切り替えられる
q / tag_state / media_kind / sort / direction が効く
```

ここでまず、普通の素材一覧が壊れていないことを確認します。

---

## 3. 左タグバーの確認

`/materials` を開いて左タグバーからタグを選ぶ。

見る観点:

```txt id="n6udul"
URL が tag_id=...&include_descendants=1&group_by=parent_tag になる
選択中タグが左バーで強調される
一覧上部に「選択中」の表示が出る
「タグ選択を解除」で通常表示に戻れる
解除後、tag_id / include_descendants / group_by / page が消える
子タグ・孫タグの素材も一覧に出る
親タググルーピングされる
```

特に重要なのはこれ。

```txt id="d33os0"
親タグ A を選択
  A に直接紐づく素材
  A > B に紐づく素材
  A > B > C に紐づく素材
が同じ一覧に出ること
```

---

## 4. 選択タグから素材追加

左タグからタグを選択した状態で、一覧上部の **このタグに素材を追加** を押す。

見る観点:

```txt id="qdd5uw"
素材追加画面の tag 欄に選択中タグ名が初期入力されている
file または url を指定して保存できる
保存後 return_to で元のタグ選択済み一覧に戻る
戻った一覧に追加した素材が出る
material_versions に create が 1 件できる
```

Rails console でも確認できます。

```sh id="i07rrb"
RAILS_ENV=development bundle exec rails runner '
m = Material.order(id: :desc).first
puts({ id: m.id, tag: m.tag&.name, versions: m.material_versions.count, version_no: m.version_no }.inspect)
'
```

---

## 5. グループ見出しから素材追加

親タググルーピング表示中に、各グループ見出しの **このタグに素材を追加** を押す。

見る観点:

```txt id="fpx8w4"
グループタグ名が tag 欄に初期入力される
保存後、元の一覧に戻る
追加素材がそのグループ内に出る
```

ここは今回の導線の肝です。棚の見出しから直接その棚へ素材を置けるかを見る。

---

## 6. 素材更新と履歴

既存素材の詳細または編集導線から、タグ・URL・ファイル・export path を更新する。

見る観点:

```txt id="w36i61"
更新前 snapshot が無ければ create が補われる
更新後 update version ができる
file_blob_id / file_filename / file_sha256 が material_versions に入る
export_paths_json が履歴に残る
/materials/changes または /materials/versions で履歴が見える
```

console:

```sh id="a3okhv"
RAILS_ENV=development bundle exec rails runner '
m = Material.order(updated_at: :desc).first
puts m.material_versions.order(:version_no).map { |v|
  [v.version_no, v.event_type, v.tag_name, v.file_filename, v.file_sha256, v.export_paths_hash]
}.inspect
'
```

---

## 7. サムネイル

画像素材を追加して、一覧にサムネイルが出るか確認。

動画素材があるなら、`ffmpeg` が入っている環境で backfill。

```sh id="qxm8fj"
cd backend
RAILS_ENV=development bundle exec rails materials:thumbnails:backfill
```

見る観点:

```txt id="s9a1vf"
画像は 180x180 のサムネが付く
動画はフレームからサムネが作られる
非対応ファイルは代替テキスト表示になる
ログに result が出る
```

---

## 8. ZIP export

export path がある素材を用意して、ブラウザで確認。

```txt id="aq7f7o"
/materials/download.zip?profile=legacy_drive
```

見る観点:

```txt id="h4rve5"
ZIP が落ちる
entry path が material_export_items.export_path になる
disabled な export item は入らない
ファイル実体が欠けている場合は 422 と missing_files が返る
```

---

## 9. 抑止

`/materials/suppressions` で path prefix 抑止を追加する。

見る観点:

```txt id="w80jbu"
member で作成できる
guest は forbidden / unauthorized
google_drive_path_prefix で既存素材が discard される
discard 履歴が material_versions に残る
同期時に同じ source_path 配下が再作成されない
```

---

## 10. Google Drive 同期

開発環境では、まず小さいフォルダでやるのがよいです。
いきなり本番素材集フォルダを食わせると、ログが藪になります 🌿

必要な ENV:

```sh id="kzbb7f"
GOOGLE_DRIVE_SERVICE_ACCOUNT_EMAIL=...
GOOGLE_DRIVE_PRIVATE_KEY_PATH=...
MATERIAL_SYNC_SOURCE_KIND=google_drive_path
MATERIAL_SYNC_SOURCE_FILE_ID=<folder_id>
MATERIAL_SYNC_SOURCE_NAME=dev-small-folder
MATERIAL_SYNC_SOURCE_PROFILE=legacy_drive
```

seed で source 作成、または console で作成。

```sh id="sec40c"
RAILS_ENV=development bundle exec rails db:seed
RAILS_ENV=development bundle exec rails materials:sync
```

見る観点:

```txt id="a5ltqu"
imported / updated / unchanged / suppressed / failed がログに出る
2 回目実行で unchanged が増える
tag は nil のままでも保存できる
人手で tag / url を付けた既存同期素材が、再同期で消えない
Google native file は skip される
download 後 sha256 block が効く
```

---

## 11. schema.rb は別途確認

これはテストというより merge gate です。
今回まだ怪しいので、差分に以下が混ざっていないことを確認します。

```txt id="s3k6da"
wiki_assets 削除
wiki_pages.next_asset_no 削除
素材管理と無関係な CHECK constraint 削除
素材管理と無関係な index order 消失
```

ここが残るなら、機能テストが通っても merge は止めた方がいいです。

---

## 最小テスト順

時間がないなら、この順で十分です。

```txt id="c1k2pw"
1. db:migrate
2. materials without versions = 0 を確認
3. /materials 初期表示
4. 左タグ選択 → 子孫込み表示 → グルーピング
5. タグ選択解除
6. 選択タグから素材追加 → return_to で戻る
7. グループ見出しから素材追加
8. 更新して material_versions を確認
9. ZIP export
10. 小さい Drive folder で materials:sync を 2 回
```

これで、今回の差分の主要な導線はほぼ踏めます。

Reviewed-on: #381
Co-authored-by: miteruzo <miteruzo@naver.com>
Co-committed-by: miteruzo <miteruzo@naver.com>
2026-06-28 06:35:53 +09:00
みてるぞ ec2b3d2254 タグ “廃止” 追加 (#378) (#379)
Reviewed-on: #379
Co-authored-by: miteruzo <miteruzo@naver.com>
Co-committed-by: miteruzo <miteruzo@naver.com>
2026-06-22 08:40:06 +09:00
241個のファイルの変更24033行の追加3270行の削除
+980 -21
ファイルの表示
ファイル差分が大きすぎるため省略します 差分を読込み
+186
ファイルの表示
@@ -67,6 +67,192 @@ pass or the remaining failure is clearly blocked.
Before changing behavior, inspect the matching route, controller, model, Before changing behavior, inspect the matching route, controller, model,
service, representation, and spec. service, representation, and spec.
## Shared backend systems
Before adding backend behaviour, search the existing backend first. At minimum,
check these locations:
- `app/controllers`
- `app/controllers/concerns`
- `app/models`
- `app/models/concerns`
- `app/representations`
- `app/services`
- `app/services/*`
- `app/jobs`
- `lib`
- `lib/tasks`
- `config/initializers`
Do not infer commonality from directory names alone. Read the actual
responsibility and representative usage sites.
### Controller reuse
Before adding logic to a controller, inspect:
- `ApplicationController` authentication, authorization, BAN, and IP BAN
- existing render and validation-error helpers
- existing param parsing
- controller concerns
- the controller for the same resource
- existing services
- existing representations
Keep controllers focused on:
- authentication and authorization
- parameter intake
- service and model invocation
- HTTP status selection
- representation selection
Do not reimplement these per controller when an existing path already owns
them:
- authentication and role checks
- validation error JSON
- URL normalisation
- tag normalisation
- thumbnail handling
- version recording
- complex transactions
- external HTTP fetching
- response representation assembly
### Authentication, authorization, and BAN
Treat these as the canonical backend entrypoints:
- `ApplicationController#authenticate_user`
- `current_user`
- `X-Transfer-Code`
- `reject_banned_ip_address!`
- `reject_banned_user!`
- `gte_member?`
- `admin?`
Do not create feature-local permission services, role comparisons, or header
parsing when the existing authentication boundary already owns the behaviour.
If the current boundary is insufficient, extend it minimally instead of adding
another permission path.
### Representations
If an endpoint for the same resource already uses `app/representations`, do not
assemble a separate JSON shape directly inside the controller without first
checking the existing representation contract.
Inspect at least:
- `PostRepr`
- `TagRepr`
- `MaterialRepr`
- `TheatreRepr`
- `UserRepr`
- `WikiPageRepr`
- `DeerjikistRepr`
When a lightweight response is genuinely different in purpose, keep it
deliberate and compatible with the surrounding contracts. Do not force every
identifier list into a large representation, but do not fork the same resource
shape casually either.
### Domain services
When work touches multiple models, transactions, external APIs, file handling,
history creation, or multi-step workflow, search `app/services` first.
At minimum, search for existing services in these responsibility areas:
- version recorder and versioning
- wiki commit
- YouTube or Google Drive API client
- material sync or ZIP export
- similarity calculation
- theatre selection or skip finalisation
- metadata, thumbnail, or file processing
- URL normaliser or sanitisation
- import or export
- preview safety or HTTP fetch
Do not create a same-responsibility service under another namespace or another
name. If an existing service is close, extend that API minimally instead of
wrapping it in a feature-local service.
### Versioning
When a feature writes history, snapshots, or restore roots, search the existing
versioning path first. At minimum, inspect:
- `VersionRecorder`
- `PostVersionRecorder`
- `TagVersionRecorder`
- `TagVersioning`
- `MaterialVersionRecorder`
- `NicoTagVersionRecorder`
- `WikiVersionRecorder`
Do not implement history writes in controllers, callbacks, or ad hoc feature
services when the recorder layer already owns the transaction boundary and
meaning.
### Normalisation, sanitisation, and parsing
For URLs, tag names, times, video durations, identifiers, and paths, search the
existing normaliser, sanitisation rule, parser, and model-callback path first.
Do not let frontend, controller, service, and model each invent different rules
for the same value. Use one canonical normalisation path and keep input
validation distinct from pre-persistence normalisation.
### External HTTP and URL safety
When fetching external URLs, reuse the existing preview-safety stack. Search at
least for:
- URL safety
- redirect validation
- response size limits
- timeouts
- network failure mapping
- HTML metadata extraction
- known-site extraction
- thumbnail fetching
Do not add direct `Net::HTTP`, `Faraday`, or equivalent feature-local HTTP code
that reimplements SSRF checks, redirect restrictions, size limits, or timeouts.
If the current fetcher is insufficient, extend its existing safety contract.
### Storage, files, and Active Storage
When handling files, thumbnails, ZIP output, object storage, or Active Storage
blobs, inspect existing storage helpers, exporters, thumbnail generators, and
checksum helpers first. Do not reimplement the same attach, export path,
download, resize, or checksum flow in a controller or one-off service.
### Concerns
Do not create controller or model concerns merely because some code is shared.
Use a concern only when multiple classes share the same lifecycle, macro,
callback, or tightly cohesive behaviour. Utility collections belong in explicit
objects or services, not in `CommonConcern`, `SharedMethods`, or `Utils`.
### Model boundaries
Model-specific invariants, associations, validations, and normalisation may
live in the model. Multi-model workflow, external access, complex transaction
flow, and feature orchestration belong in services. Do not hide feature
workflow in model callbacks.
### Transactions, locking, and race handling
If transactions, locking, idempotency, or race recovery already exist in a
service or model method, do not add a second implementation in a controller or
new service. Inspect the existing transaction boundary first, avoid wrapping
the same operation in needless nested transactions, and handle unique-constraint
races according to the target constraint's business meaning.
## Ruby style ## Ruby style
- Prefer precise, minimal changes. - Prefer precise, minimal changes.
+49
ファイルの表示
@@ -0,0 +1,49 @@
class MaterialSyncSuppressionsController < ApplicationController
def index
return head :unauthorized unless current_user
return head :forbidden unless current_user.gte_member?
suppressions =
MaterialSyncSuppression
.includes(:created_by_user)
.order(created_at: :desc, id: :desc)
render json: { suppressions: suppressions.map { |s| repr(s) } }
end
def create
return head :unauthorized unless current_user
return head :forbidden unless current_user.gte_member?
suppression =
MaterialSyncSuppressionRegistrar.create!(suppression_params,
created_by_user: current_user)
render json: repr(suppression), status: :created
rescue ActiveRecord::RecordInvalid => e
render_validation_error e.record
end
private
def suppression_params
params.permit(:source_kind,
:source_uri,
:drive_path,
:drive_file_id,
:reason)
end
def repr suppression
{ id: suppression.id,
source_kind: suppression.source_kind,
source_uri: suppression.source_uri,
drive_path: suppression.drive_path,
drive_file_id: suppression.drive_file_id,
normalized_source_key: suppression.normalized_source_key,
reason: suppression.reason,
created_by_user: suppression.created_by_user&.as_json(UserRepr::BASE),
created_at: suppression.created_at,
updated_at: suppression.updated_at }
end
end
+54
ファイルの表示
@@ -0,0 +1,54 @@
class MaterialVersionsController < ApplicationController
def index
page = (params[:page].presence || 1).to_i
limit = (params[:limit].presence || 20).to_i
page = 1 if page < 1
limit = 1 if limit < 1
offset = (page - 1) * limit
q = MaterialVersion.includes(:created_by_user)
q = q.where(material_id: params[:material_id]) if params[:material_id].present?
q = q.where(tag_name: params[:tag].to_s.strip) if params[:tag].present?
q = q.where(event_type: params[:event_type]) if params[:event_type].present?
count = q.except(:order, :limit, :offset).count
versions =
q
.order(Arel.sql('material_versions.created_at DESC, material_versions.id DESC'))
.limit(limit)
.offset(offset)
render json: { versions: serialise_versions(versions), count: }
end
private
def serialise_versions rows
rows.map do |row|
{ id: row.id,
material_id: row.material_id,
version_no: row.version_no,
event_type: row.event_type,
tag_id: row.tag_id,
tag_name: row.tag_name,
tag_category: row.tag_category,
url: row.url,
source_kind: row.source_kind,
source_uri: row.source_uri,
source_path: row.source_path,
source_file_id: row.source_file_id,
normalized_source_key: row.normalized_source_key,
file_blob_id: row.file_blob_id,
file_filename: row.file_filename,
file_content_type: row.file_content_type,
file_byte_size: row.file_byte_size,
file_sha256: row.file_sha256,
export_paths_json: row.export_paths_hash,
discarded_at: row.discarded_at,
created_by_user: row.created_by_user&.as_json(UserRepr::BASE),
created_at: row.created_at }
end
end
end
+472 -34
ファイルの表示
@@ -1,4 +1,11 @@
require 'set'
class MaterialsController < ApplicationController class MaterialsController < ApplicationController
rescue_from MaterialZipExporter::EmptyExportError, with: :render_zip_empty
rescue_from MaterialZipExporter::DuplicatePathError, with: :render_zip_duplicate_path
rescue_from MaterialZipExporter::MissingFileError, with: :render_zip_missing_file
def index def index
page = (params[:page].presence || 1).to_i page = (params[:page].presence || 1).to_i
limit = (params[:limit].presence || 20).to_i limit = (params[:limit].presence || 20).to_i
@@ -8,54 +15,98 @@ class MaterialsController < ApplicationController
offset = (page - 1) * limit offset = (page - 1) * limit
tag_id = params[:tag_id].presence filters = material_index_filters
parent_id = params[:parent_id].presence tag_graph = material_index_tag_graph(filters)
q = Material.includes(:material_export_items,
thumbnail_attachment: :blob,
file_attachment: :blob,
tag: :tag_name)
q = q.where(tag_id: nil) if filters[:tag_state] == 'untagged'
q = q.where.not(tag_id: nil) if filters[:tag_state] == 'tagged'
q = apply_material_tag_filter(q, filters, tag_graph)
q = q.where('materials.created_at >= ?', filters[:created_from]) if filters[:created_from]
q = q.where('materials.created_at <= ?', filters[:created_to]) if filters[:created_to]
q = q.where('materials.updated_at >= ?', filters[:updated_from]) if filters[:updated_from]
q = q.where('materials.updated_at <= ?', filters[:updated_to]) if filters[:updated_to]
q = Material.includes(:tag, :created_by_user).with_attached_file q = material_index_join_tag_name(q) if material_index_needs_tag_name?(filters)
q = q.where(tag_id:) if tag_id q = material_index_join_file_blob(q) if material_index_needs_file_blob?(filters)
q = q.where(parent_id:) if parent_id q = apply_material_query(q, filters[:q]) if filters[:q].present?
q = apply_material_media_kind(q, filters[:media_kind])
count = q.count count = q.distinct.count(:id)
materials = q.order(created_at: :desc, id: :desc).limit(limit).offset(offset) materials =
q
.order(Arel.sql(material_index_order_sql(filters)))
.limit(limit)
.offset(offset)
.to_a
render json: { materials: MaterialRepr.many(materials, host: request.base_url), count: count } response = { materials: MaterialRepr.list_many(materials, host: request.base_url),
count: }
if filters[:tag_id]
response[:tag_scope] = material_index_tag_scope(filters, tag_graph)
end
if filters[:group_by] == 'parent_tag'
response[:groups] = material_index_groups(materials, tag_graph)
end
render json: response
end end
def show def show
material = material =
Material Material
.includes(:tag) .includes(:tag, :material_export_items)
.with_attached_thumbnail
.with_attached_file .with_attached_file
.find_by(id: params[:id]) .find_by(id: params[:id])
return head :not_found unless material return head :not_found unless material
wiki_page_body = material.tag.tag_name.wiki_page&.current_revision&.body wiki_page_body = material.tag&.tag_name&.wiki_page&.current_revision&.body
render json: MaterialRepr.base(material, host: request.base_url).merge(wiki_page_body:) render json: MaterialRepr.base(material, host: request.base_url).merge(wiki_page_body:)
end end
def create def create
return head :unauthorized unless current_user return head :unauthorized unless current_user
return head :forbidden unless current_user.gte_member?
tag_name_raw = params[:tag].to_s.strip tag_name_raw = params[:tag].to_s.strip
file = params[:file] file = params[:file]
file_sha256 = MaterialFileSha256.from_upload(file)
url = params[:url].to_s.strip.presence url = params[:url].to_s.strip.presence
return render_unprocessable_entity('タグは必須です.', field: :tag) if tag_name_raw.blank? return render_unprocessable_entity('タグは必須です.', field: :tag) if tag_name_raw.blank?
if file.blank? && url.blank? if file.blank? && url.blank?
return render_validation_error fields: { file: ['ファイルまたは URL は必須です.'], return render_validation_error fields: { file: ['ファイルまたは URL は必須です.'],
url: ['ファイルまたは URL は必須です.'] } url: ['ファイルまたは URL は必須です.'] }
end end
block = MaterialImportBlockMatcher.match_for_sha256(file_sha256)
return render_material_import_block(block) if block
tag_name = TagName.find_undiscard_or_create_by!(name: tag_name_raw) uploaded_blob = build_uploaded_material_blob!(file, file_sha256)
tag = tag_name.tag material = nil
tag = Tag.create!(tag_name:, category: :material) unless tag
material = Material.new(tag:, url:, begin
created_by_user: current_user, Material.transaction do
updated_by_user: current_user) tag = resolve_material_tag!(tag_name_raw)
material.file.attach(file) material = Material.new(tag:, url:,
created_by_user: current_user,
updated_by_user: current_user)
material.file.attach(uploaded_blob) if uploaded_blob
material.save!
TagVersioning.record_tag_snapshot!(tag, created_by_user: current_user)
upsert_export_paths!(material)
MaterialVersionRecorder.record!(material:, event_type: :create,
created_by_user: current_user)
end
rescue StandardError
uploaded_blob&.purge_later
raise
end
if material.save if material
log_thumbnail_generation(material, MaterialThumbnailGenerator.generate!(material))
render json: MaterialRepr.base(material, host: request.base_url), status: :created render json: MaterialRepr.base(material, host: request.base_url), status: :created
else else
render_validation_error material render_validation_error material
@@ -71,29 +122,42 @@ class MaterialsController < ApplicationController
tag_name_raw = params[:tag].to_s.strip tag_name_raw = params[:tag].to_s.strip
file = params[:file] file = params[:file]
url = params[:url].to_s.strip.presence file_sha256 = MaterialFileSha256.from_upload(file)
url = params.key?(:url) ? params[:url].to_s.strip.presence : material.url
return render_unprocessable_entity('タグは必須です.', field: :tag) if tag_name_raw.blank? return render_unprocessable_entity('タグは必須です.', field: :tag) if tag_name_raw.blank?
if file.blank? && url.blank? if file.blank? && url.blank? && !material.file.attached?
return render_validation_error fields: { file: ['ファイルまたは URL は必須です.'], return render_validation_error fields: { file: ['ファイルまたは URL は必須です.'],
url: ['ファイルまたは URL は必須です.'] } url: ['ファイルまたは URL は必須です.'] }
end end
block = MaterialImportBlockMatcher.match_for_sha256(file_sha256)
return render_material_import_block(block) if block
tag_name = TagName.find_undiscard_or_create_by!(name: tag_name_raw) uploaded_blob = build_uploaded_material_blob!(file, file_sha256)
tag = tag_name.tag
tag = Tag.create!(tag_name:, category: :material) unless tag
material.update!(tag:, url:, updated_by_user: current_user) begin
if file Material.transaction do
material.file.attach(file) MaterialVersionRecorder.ensure_snapshot!(material, created_by_user: current_user)
else tag = resolve_material_tag!(tag_name_raw)
material.file.purge material.assign_attributes(tag:, url:, updated_by_user: current_user)
if uploaded_blob
material.file.attach(uploaded_blob)
elsif params.key?(:url) && url.present? && file.blank?
material.file.detach
end
material.save!
TagVersioning.record_tag_snapshot!(tag, created_by_user: current_user)
upsert_export_paths!(material)
MaterialVersionRecorder.record!(material:, event_type: :update,
created_by_user: current_user)
end
rescue StandardError
uploaded_blob&.purge_later
raise
end end
if material.save log_thumbnail_generation(material, MaterialThumbnailGenerator.generate!(material))
render json: MaterialRepr.base(material, host: request.base_url)
else render json: MaterialRepr.base(material, host: request.base_url)
render_validation_error material
end
end end
def destroy def destroy
@@ -103,7 +167,381 @@ class MaterialsController < ApplicationController
material = Material.find_by(id: params[:id]) material = Material.find_by(id: params[:id])
return head :not_found unless material return head :not_found unless material
material.discard Material.transaction do
MaterialVersionRecorder.ensure_snapshot!(material, created_by_user: current_user)
material.discard!
MaterialVersionRecorder.record!(material:, event_type: :discard,
created_by_user: current_user)
end
head :no_content head :no_content
end end
def download
zip = MaterialZipExporter.new(profile: params[:profile],
tag_id: params[:tag_id]).export
profile = params[:profile].presence || 'legacy_drive'
send_data zip,
type: 'application/zip',
disposition: 'attachment',
filename: "btrc-materials-#{ profile }.zip"
end
private
def material_index_filters
tag_state = params[:tag_state].to_s.presence
tag_state = 'untagged' if bool?(:unclassified)
tag_state = 'all' unless ['all', 'tagged', 'untagged'].include?(tag_state)
media_kind = params[:media_kind].to_s.presence
unless ['all', 'image', 'video', 'audio', 'file_other', 'url_only'].include?(media_kind)
media_kind = 'all'
end
sort = params[:sort].to_s.presence
unless ['created_at', 'updated_at', 'tag_name', 'media_kind', 'file_byte_size',
'version_no', 'id'].include?(sort)
sort = 'created_at'
end
direction = params[:direction].to_s.downcase
direction = 'desc' unless ['asc', 'desc'].include?(direction)
group_by = params[:group_by].to_s.presence
group_by = 'none' unless ['none', 'parent_tag'].include?(group_by)
tag_id = params[:tag_id].to_i
tag_id = nil if tag_id <= 0
{ q: params[:q].to_s.strip.presence,
tag_state:,
media_kind:,
tag_id:,
include_descendants: bool?(:include_descendants),
group_by:,
created_from: parse_time_param(:created_from),
created_to: parse_time_param(:created_to),
updated_from: parse_time_param(:updated_from),
updated_to: parse_time_param(:updated_to),
sort:,
direction: }
end
def parse_time_param name
value = params[name].to_s.strip
return nil if value.blank?
Time.zone.parse(value)
rescue ArgumentError
nil
end
def resolve_material_tag! tag_name_raw
tag_name = TagName.find_undiscard_or_create_by!(name: tag_name_raw)
tag = tag_name.tag
tag || Tag.create!(tag_name:, category: :material)
end
def material_index_needs_tag_name? filters
filters[:q].present? || filters[:sort] == 'tag_name'
end
def material_index_needs_file_blob? filters
filters[:q].present? ||
filters[:media_kind] != 'all' ||
['media_kind', 'file_byte_size'].include?(filters[:sort])
end
def material_index_join_tag_name q
q.left_joins(tag: :tag_name)
end
def material_index_join_file_blob q
q.joins(<<~SQL.squish)
LEFT JOIN active_storage_attachments material_file_attachments
ON material_file_attachments.record_type = 'Material'
AND material_file_attachments.record_id = materials.id
AND material_file_attachments.name = 'file'
LEFT JOIN active_storage_blobs material_file_blobs
ON material_file_blobs.id = material_file_attachments.blob_id
SQL
end
def apply_material_query q, term
like = "%#{ ActiveRecord::Base.sanitize_sql_like(term) }%"
q.where('tag_names.name LIKE :q OR materials.url LIKE :q OR ' \
'material_file_blobs.filename LIKE :q',
q: like)
end
def apply_material_media_kind q, media_kind
case media_kind
when 'image'
q.where('material_file_blobs.content_type LIKE ?', 'image/%')
when 'video'
q.where('material_file_blobs.content_type LIKE ?', 'video/%')
when 'audio'
q.where('material_file_blobs.content_type LIKE ?', 'audio/%')
when 'file_other'
q.where('material_file_attachments.id IS NOT NULL')
.where.not('material_file_blobs.content_type LIKE ?', 'image/%')
.where.not('material_file_blobs.content_type LIKE ?', 'video/%')
.where.not('material_file_blobs.content_type LIKE ?', 'audio/%')
when 'url_only'
q.where('material_file_attachments.id IS NULL').where.not(url: [nil, ''])
else
q
end
end
def apply_material_tag_filter q, filters, tag_graph
return q unless filters[:tag_id]
tag_ids =
if tag_graph
tag_graph[:scope_tag_ids]
else
[filters[:tag_id]]
end
q.where(tag_id: tag_ids)
end
def material_index_tag_graph filters
return nil unless filters[:tag_id]
children_by_parent_id = Hash.new { |h, k| h[k] = [] }
TagImplication.pluck(:parent_tag_id, :tag_id).each do |parent_id, child_id|
children_by_parent_id[parent_id] << child_id
end
scope_tag_ids =
if filters[:include_descendants]
collect_material_descendant_ids(filters[:tag_id], children_by_parent_id)
else
Set.new([filters[:tag_id]])
end
tags_by_id =
Tag
.joins(:tag_name)
.where(id: scope_tag_ids)
.pluck('tags.id', 'tag_names.name', 'tags.category', 'tags.deprecated_at')
.each_with_object({ }) do |(id, name, category, deprecated_at), hash|
hash[id] = { id:, name:, category:, deprecated: deprecated_at.present? }
end
selected_tag = tags_by_id[filters[:tag_id]]
return nil unless selected_tag
group_tag_id_by_tag_id = { }
children_by_parent_id[filters[:tag_id]].each do |child_tag_id|
assign_material_group_branch(child_tag_id,
children_by_parent_id:,
tags_by_id:,
group_tag_id_by_tag_id:,
current_group_tag_id: nil,
seen: Set.new([filters[:tag_id]]))
end
{ selected_tag_id: filters[:tag_id],
selected_tag:,
scope_tag_ids: scope_tag_ids.to_a,
tags_by_id:,
group_tag_id_by_tag_id: }
end
def collect_material_descendant_ids selected_tag_id, children_by_parent_id
ids = Set.new([selected_tag_id])
stack = [selected_tag_id]
until stack.empty?
tag_id = stack.pop
children_by_parent_id[tag_id].each do |child_tag_id|
next if ids.include?(child_tag_id)
ids << child_tag_id
stack << child_tag_id
end
end
ids
end
def assign_material_group_branch tag_id, children_by_parent_id:, tags_by_id:,
group_tag_id_by_tag_id:, current_group_tag_id:, seen:
return if seen.include?(tag_id)
tag = tags_by_id[tag_id]
return unless tag
seen = seen.dup << tag_id
next_group_tag_id = current_group_tag_id
next_group_tag_id = tag_id if next_group_tag_id.nil? && !tag[:deprecated]
group_tag_id_by_tag_id[tag_id] = next_group_tag_id if next_group_tag_id
children_by_parent_id[tag_id].each do |child_tag_id|
assign_material_group_branch(child_tag_id,
children_by_parent_id:,
tags_by_id:,
group_tag_id_by_tag_id:,
current_group_tag_id: next_group_tag_id,
seen:)
end
end
def material_index_groups materials, tag_graph
return [] unless tag_graph
groups_by_key = { }
materials.each do |material|
next unless material.tag_id
group_tag_id =
if material.tag_id == tag_graph[:selected_tag_id]
tag_graph[:selected_tag_id]
else
tag_graph[:group_tag_id_by_tag_id][material.tag_id] || tag_graph[:selected_tag_id]
end
group_tag = tag_graph[:tags_by_id][group_tag_id]
next unless group_tag
key = "tag:#{ group_tag_id }"
group =
groups_by_key[key] ||= { key:,
tag: group_tag_repr(group_tag),
material_ids: [],
count: 0 }
group[:material_ids] << material.id
group[:count] += 1
end
groups_by_key.values.sort_by { |group| group[:tag][:name] }
end
def material_index_tag_scope filters, tag_graph
return nil unless tag_graph
{
tag: group_tag_repr(tag_graph[:selected_tag]),
include_descendants: filters[:include_descendants]
}
end
def group_tag_repr group_tag
{ id: group_tag[:id],
name: group_tag[:name],
category: group_tag[:category] }
end
def material_index_order_sql filters
direction = filters[:direction] == 'asc' ? 'ASC' : 'DESC'
sort_sql =
case filters[:sort]
when 'tag_name'
'tag_names.name'
when 'media_kind'
material_media_kind_sql
when 'file_byte_size'
'material_file_blobs.byte_size'
when 'updated_at'
'materials.updated_at'
when 'version_no'
'materials.version_no'
when 'id'
'materials.id'
else
'materials.created_at'
end
"#{ sort_sql } #{ direction }, materials.id #{ direction }"
end
def material_media_kind_sql
"CASE " \
"WHEN material_file_attachments.id IS NULL AND materials.url IS NOT NULL THEN 5 " \
"WHEN material_file_blobs.content_type LIKE 'image/%' THEN 1 " \
"WHEN material_file_blobs.content_type LIKE 'video/%' THEN 2 " \
"WHEN material_file_blobs.content_type LIKE 'audio/%' THEN 3 " \
"WHEN material_file_attachments.id IS NOT NULL THEN 4 " \
"ELSE 6 END"
end
def upsert_export_paths! material
raw = params[:export_paths]
return if raw.blank?
export_paths = raw.respond_to?(:to_unsafe_h) ? raw.to_unsafe_h : raw.to_h
export_paths.each do |profile, export_path|
profile = profile.to_s
export_path = export_path.to_s.strip
item = material.material_export_items.find_or_initialize_by(profile:)
if export_path.blank?
item.destroy! if item.persisted?
next
end
item.export_path = export_path
item.enabled = true
item.created_by_user ||= current_user
item.save!
end
end
def render_zip_empty
render_unprocessable_entity('ZIP export 対象の素材がありません.')
end
def render_zip_duplicate_path error
render_unprocessable_entity("ZIP export path が重複してゐます: #{ error.message }")
end
def render_zip_missing_file error
missing_files = error.missing_files.map do |missing_file|
{ material_id: missing_file.material_id,
export_path: missing_file.export_path,
blob_id: missing_file.blob_id,
filename: missing_file.filename }
end
render json: { type: 'validation_error',
message: 'ZIP export に必要な素材ファイルが欠損しています.',
errors: { },
base_errors: ['ZIP export に必要な素材ファイルが欠損しています.'],
missing_files: },
status: :unprocessable_entity
end
def build_uploaded_material_blob! file, file_sha256
return nil unless file
file.tempfile.rewind
blob = ActiveStorage::Blob.create_and_upload!(
io: file.tempfile,
filename: file.original_filename,
content_type: file.content_type,
)
MaterialFileSha256.assign_metadata_sha256!(blob, file_sha256)
blob
ensure
file.tempfile.rewind if file&.tempfile
end
def render_material_import_block block
render_validation_error fields: { file: ["抑止された素材です: #{ block.reason }"] }
end
def log_thumbnail_generation material, result
material.reload
Rails.logger.info(
"Material thumbnail generation: material_id=#{ material.id } " \
"result=#{ result } " \
"thumbnail_attached=#{ material.thumbnail.attached? } " \
"content_type=#{ material.content_type }")
end
end end
+44
ファイルの表示
@@ -0,0 +1,44 @@
class PostImportsController < ApplicationController
before_action :require_member!
def preview
rows = PostImportPreviewer.new.preview_rows(
rows: PostImportUrlListParser.parse(params[:source]))
render json: { rows: }
rescue ArgumentError => e
render_bad_request e.message
end
def validate
rows = normalised_import_rows allow_warning_fields: true
changed_row = Integer(params[:changed_row], exception: false)
result =
PostImportPreviewer.new.preview_rows(rows:,
fetch_metadata: changed_row,
metadata_cache: { })
render json: { rows: result }
rescue ArgumentError => e
render_bad_request e.message
end
def create
result = PostImportRunner.new(actor: current_user,
rows: normalised_import_rows).run
render json: result, status: result[:created].positive? ? :created : :ok
rescue ArgumentError => e
render_bad_request e.message
end
private
def require_member!
return head :unauthorized unless current_user
return if current_user.gte_member?
head :forbidden
end
def normalised_import_rows allow_warning_fields: false
PostImportRowNormaliser.normalise!(params[:rows], allow_warning_fields:)
end
end
+5
ファイルの表示
@@ -25,6 +25,7 @@ class PostVersionsController < ApplicationController
SQL SQL
.select('post_versions.*', 'prev.title AS prev_title', 'prev.url AS prev_url', .select('post_versions.*', 'prev.title AS prev_title', 'prev.url AS prev_url',
'prev.thumbnail_base AS prev_thumbnail_base', 'prev.tags AS prev_tags', 'prev.thumbnail_base AS prev_thumbnail_base', 'prev.tags AS prev_tags',
'prev.video_ms AS prev_video_ms',
'prev.original_created_from AS prev_original_created_from', 'prev.original_created_from AS prev_original_created_from',
'prev.original_created_before AS prev_original_created_before') 'prev.original_created_before AS prev_original_created_before')
q = q.where('post_versions.post_id = ?', post_id) if post_id q = q.where('post_versions.post_id = ?', post_id) if post_id
@@ -74,6 +75,10 @@ class PostVersionsController < ApplicationController
current: row.thumbnail_base, current: row.thumbnail_base,
prev: row.attributes['prev_thumbnail_base'] prev: row.attributes['prev_thumbnail_base']
}, },
video_ms: {
current: row.video_ms,
prev: row.attributes['prev_video_ms']
},
tags: build_version_tags(cur_tags, prev_tags), tags: build_version_tags(cur_tags, prev_tags),
original_created_from: { original_created_from: {
current: row.original_created_from&.iso8601, current: row.original_created_from&.iso8601,
+138 -51
ファイルの表示
@@ -1,6 +1,10 @@
class PostsController < ApplicationController class PostsController < ApplicationController
Event = Struct.new(:post, :tag, :user, :change_type, :timestamp, keyword_init: true) Event = Struct.new(:post, :tag, :user, :change_type, :timestamp, keyword_init: true)
class VideoMsParseError < ArgumentError
;
end
def index def index
url = params[:url].presence url = params[:url].presence
title = params[:title].presence title = params[:title].presence
@@ -45,7 +49,9 @@ class PostsController < ApplicationController
.joins("LEFT JOIN (#{ pt_max_sql }) pt_max ON pt_max.post_id = posts.id") .joins("LEFT JOIN (#{ pt_max_sql }) pt_max ON pt_max.post_id = posts.id")
.reselect('posts.*', Arel.sql("#{ updated_at_all_sql } AS updated_at_all")) .reselect('posts.*', Arel.sql("#{ updated_at_all_sql } AS updated_at_all"))
.preload(:uploaded_user, :parents, :children, .preload(:uploaded_user, :parents, :children,
tags: [:deerjikists, :materials, { tag_name: :wiki_page }]) active_post_tags: [:sections,
{ tag: [:deerjikists, :materials,
{ tag_name: :wiki_page }] }])
.with_attached_thumbnail .with_attached_thumbnail
q = q.where('posts.url LIKE ?', "%#{ url }%") if url q = q.where('posts.url LIKE ?', "%#{ url }%") if url
@@ -97,7 +103,9 @@ class PostsController < ApplicationController
def random def random
post = filtered_posts.preload(:uploaded_user, :parents, :children, post = filtered_posts.preload(:uploaded_user, :parents, :children,
tags: [:deerjikists, :materials, { tag_name: :wiki_page }]) active_post_tags: [:sections,
{ tag: [:deerjikists, :materials,
{ tag_name: :wiki_page }] }])
.with_attached_thumbnail .with_attached_thumbnail
.order('RAND()') .order('RAND()')
.first .first
@@ -110,7 +118,9 @@ class PostsController < ApplicationController
post = post =
Post Post
.includes(:uploaded_user, :parents, :children, .includes(:uploaded_user, :parents, :children,
tags: [:deerjikists, :materials, { tag_name: :wiki_page }]) active_post_tags: [:sections,
{ tag: [:deerjikists, :materials,
{ tag_name: :wiki_page }] }])
.with_attached_thumbnail .with_attached_thumbnail
.find_by(id: params[:id]) .find_by(id: params[:id])
return head :not_found unless post return head :not_found unless post
@@ -125,41 +135,21 @@ class PostsController < ApplicationController
child_posts:, child_posts:,
sibling_posts:, sibling_posts:,
related:) related:)
.merge(tags: build_tag_tree_for(post.tags)) .merge(tags: build_tag_tree_for(post))
end end
def create def create
return head :unauthorized unless current_user return head :unauthorized unless current_user
return head :forbidden unless current_user.gte_member? return head :forbidden unless current_user.gte_member?
# TODO: サイトに応じて thumbnail_base 設定 post = PostCreator.new(actor: current_user,
title = params[:title].presence attributes: { title: params[:title], url: params[:url],
url = params[:url] thumbnail: params[:thumbnail], tags: params[:tags],
thumbnail = params[:thumbnail] original_created_from: params[:original_created_from],
tag_names = params[:tags].to_s.split original_created_before: params[:original_created_before],
original_created_from = params[:original_created_from] parent_post_ids: parse_parent_post_ids,
original_created_before = params[:original_created_before] video_ms: params[:video_ms],
parent_post_ids = parse_parent_post_ids duration: params[:duration] }).create!
post = Post.new(title:, url:, thumbnail_base: nil, uploaded_user: current_user,
original_created_from:, original_created_before:)
post.thumbnail.attach(thumbnail) if thumbnail.present?
ApplicationRecord.transaction do
post.save!
tags = Tag.normalise_tags!(tag_names, deny_deprecated: true)
TagVersioning.record_tag_snapshots!(tags, created_by_user: current_user)
tags = Tag.expand_parent_tags(tags).reject(&:deprecated?)
sync_post_tags!(post, tags)
sync_parent_posts!(post, parent_post_ids)
post.resized_thumbnail!
PostVersionRecorder.record!(post:, event_type: :create, created_by_user: current_user)
end
post.reload post.reload
render json: PostRepr.base(post), status: :created render json: PostRepr.base(post), status: :created
@@ -167,6 +157,12 @@ class PostsController < ApplicationController
render_validation_error fields: { tags: 'ニコニコ・タグは直接指定できません.' } render_validation_error fields: { tags: 'ニコニコ・タグは直接指定できません.' }
rescue Tag::DeprecatedTagNormalisationError rescue Tag::DeprecatedTagNormalisationError
render_unprocessable_entity '廃止済みタグは付与できません.', field: :tags render_unprocessable_entity '廃止済みタグは付与できません.', field: :tags
rescue Tag::SectionLiteralParseError
render_validation_error fields: { tags: ['タグ区間の記法が不正です.'] }
rescue PostCreator::VideoMsParseError
render_validation_error fields: { video_ms: ['動画時間の記法が不正です.'] }
rescue MiniMagick::Error
render_validation_error fields: { thumbnail: ['サムネイル画像の変換に失敗しました.'] }
rescue ArgumentError => e rescue ArgumentError => e
render_validation_error fields: { parent_post_ids: [e.message] } render_validation_error fields: { parent_post_ids: [e.message] }
rescue ActiveRecord::RecordInvalid => e rescue ActiveRecord::RecordInvalid => e
@@ -223,6 +219,8 @@ class PostsController < ApplicationController
original_created_from:, original_created_from:,
original_created_before:, original_created_before:,
tag_names:, tag_names:,
video_ms_param: params[:video_ms],
duration_param: params[:duration],
parent_post_ids:) parent_post_ids:)
snapshot_to_apply = snapshot_to_apply =
@@ -253,12 +251,16 @@ class PostsController < ApplicationController
post.reload post.reload
json = PostRepr.base(post, current_user) json = PostRepr.base(post, current_user)
json['tags'] = build_tag_tree_for(post.tags) json['tags'] = build_tag_tree_for(post)
render json:, status: :ok render json:, status: :ok
rescue Tag::NicoTagNormalisationError rescue Tag::NicoTagNormalisationError
render_validation_error fields: { tags: ['ニコニコ・タグは直接指定できません.'] } render_validation_error fields: { tags: ['ニコニコ・タグは直接指定できません.'] }
rescue Tag::DeprecatedTagNormalisationError rescue Tag::DeprecatedTagNormalisationError
render_unprocessable_entity '廃止済みタグは付与できません.', field: :tags render_unprocessable_entity '廃止済みタグは付与できません.', field: :tags
rescue Tag::SectionLiteralParseError
render_validation_error fields: { tags: ['タグ区間の記法が不正です.'] }
rescue VideoMsParseError
render_validation_error fields: { video_ms: ['動画時間の記法が不正です.'] }
rescue ArgumentError => e rescue ArgumentError => e
render_validation_error fields: { parent_post_ids: [e.message] } render_validation_error fields: { parent_post_ids: [e.message] }
rescue ActiveRecord::RecordInvalid => e rescue ActiveRecord::RecordInvalid => e
@@ -357,7 +359,7 @@ class PostsController < ApplicationController
def tagged_post_ids_for(name) = def tagged_post_ids_for(name) =
Post.joins(tags: :tag_name).where(tag_names: { name: }).select(:id) Post.joins(tags: :tag_name).where(tag_names: { name: }).select(:id)
def sync_post_tags! post, desired_tags def sync_post_tags! post, desired_tags, sections
desired_tags.each do |t| desired_tags.each do |t|
t.save! if t.new_record? t.save! if t.new_record?
end end
@@ -376,13 +378,21 @@ class PostsController < ApplicationController
end end
end end
PostTagSection.where(post_id: post.id).destroy_all
sections.each do |tag_id, ranges|
ranges.each do |begin_ms, end_ms|
PostTagSection.create!(post_id: post.id, tag_id:, begin_ms:, end_ms:)
end
end
PostTag.where(post_id: post.id, tag_id: to_remove.to_a).kept.find_each do |pt| PostTag.where(post_id: post.id, tag_id: to_remove.to_a).kept.find_each do |pt|
pt.discard_by!(current_user) pt.discard_by!(current_user)
end end
end end
def build_tag_tree_for tags def build_tag_tree_for post
tags = tags.reject(&:deprecated?).to_a post_tags = post.active_post_tags.reject { |post_tag| post_tag.tag.deprecated? }
tags = post_tags.map(&:tag)
tag_ids = tags.map(&:id) tag_ids = tags.map(&:id)
implications = TagImplication.where(parent_tag_id: tag_ids, tag_id: tag_ids) implications = TagImplication.where(parent_tag_id: tag_ids, tag_id: tag_ids)
@@ -397,6 +407,9 @@ class PostsController < ApplicationController
root_ids = tag_ids - child_ids root_ids = tag_ids - child_ids
tags_by_id = tags.index_by(&:id) tags_by_id = tags.index_by(&:id)
sections_by_tag_id = post_tags.to_h { |post_tag|
[post_tag.tag_id, post_tag.sections.as_json(only: [:begin_ms, :end_ms])]
}
memo = { } memo = { }
@@ -404,8 +417,10 @@ class PostsController < ApplicationController
tag = tags_by_id[tag_id] tag = tags_by_id[tag_id]
return nil unless tag return nil unless tag
sections = sections_by_tag_id.fetch(tag_id, [])
if path.include?(tag_id) if path.include?(tag_id)
return TagRepr.inline(tag).merge(children: []) return TagRepr.inline(tag).merge(children: [], sections:)
end end
if memo.key?(tag_id) if memo.key?(tag_id)
@@ -417,7 +432,7 @@ class PostsController < ApplicationController
children = child_ids.filter_map { |cid| build_node.(cid, new_path) } children = child_ids.filter_map { |cid| build_node.(cid, new_path) }
memo[tag_id] = TagRepr.inline(tag).merge(children:) memo[tag_id] = TagRepr.inline(tag).merge(children:, sections:)
end end
root_ids.filter_map { |id| build_node.call(id, []) } root_ids.filter_map { |id| build_node.call(id, []) }
@@ -486,6 +501,7 @@ class PostsController < ApplicationController
def post_snapshot_from_version version def post_snapshot_from_version version
{ title: version.title, { title: version.title,
video_ms: version.respond_to?(:video_ms) ? version.video_ms : nil,
original_created_from: snapshot_time(version.original_created_from), original_created_from: snapshot_time(version.original_created_from),
original_created_before: snapshot_time(version.original_created_before), original_created_before: snapshot_time(version.original_created_before),
tag_names: editable_tag_names_from_version(version), tag_names: editable_tag_names_from_version(version),
@@ -498,6 +514,7 @@ class PostsController < ApplicationController
def post_snapshot_from_record post def post_snapshot_from_record post
{ title: post.title, { title: post.title,
video_ms: post.video_ms,
original_created_from: snapshot_time(post.original_created_from), original_created_from: snapshot_time(post.original_created_from),
original_created_before: snapshot_time(post.original_created_before), original_created_before: snapshot_time(post.original_created_before),
tag_names: editable_tag_names_from_post(post), tag_names: editable_tag_names_from_post(post),
@@ -505,16 +522,41 @@ class PostsController < ApplicationController
end end
def editable_tag_names_from_post post def editable_tag_names_from_post post
post.tags.not_nico.where(deprecated_at: nil) post
.joins(:tag_name).order('tag_names.name').pluck('tag_names.name') .post_tags
.kept
.joins(tag: :tag_name)
.merge(Tag.not_nico)
.merge(Tag.where(deprecated_at: nil))
.includes(:sections, tag: :tag_name)
.order('tag_names.name')
.map do |post_tag|
name = post_tag.tag.tag_name.name
sections = post_tag.sections.sort_by(&:begin_ms)
next name if sections.empty?
"#{ name }#{ sections.map { Post.section_literal(_1) }.join }"
end
end end
def post_incoming_snapshot title:, original_created_from:, original_created_before:, def post_incoming_snapshot title:, original_created_from:, original_created_before:,
tag_names:, parent_post_ids: tag_names:, video_ms_param:, duration_param:, parent_post_ids:
Tag.normalise_tags!(tag_names, with_tagme: false, deny_deprecated: true,
with_sections: true) =>
{ tags:, sections: }
tags = Tag.expand_parent_tags(tags).reject(&:deprecated?)
video_ms = normalise_video_ms(tags, video_ms_param:, duration_param:)
validate_video_sections!(video_ms, sections)
{ title:, { title:,
video_ms:,
original_created_from: snapshot_time(original_created_from), original_created_from: snapshot_time(original_created_from),
original_created_before: snapshot_time(original_created_before), original_created_before: snapshot_time(original_created_before),
tag_names: incoming_tag_names_for_snapshot(tag_names), tag_names: tags.uniq(&:id).map { |tag|
"#{ tag.name }#{ sections[tag.id].to_a.map { section_literal(_1) }.join }"
}.sort,
parent_post_ids: parent_post_ids.sort } parent_post_ids: parent_post_ids.sort }
end end
@@ -537,11 +579,8 @@ class PostsController < ApplicationController
value.to_s value.to_s
end end
def incoming_tag_names_for_snapshot raw_tag_names def section_literal section
tags = Tag.normalise_tags!(raw_tag_names, with_tagme: false, "[#{ Post.ms_to_time(section[0]) }-#{ section[1] ? Post.ms_to_time(section[1]) : '' }]"
deny_deprecated: true)
Tag.expand_parent_tags(tags).reject(&:deprecated?).map(&:name).uniq.sort
end end
def post_conflict_json post:, base_version_no:, base_snapshot:, def post_conflict_json post:, base_version_no:, base_snapshot:,
@@ -562,6 +601,8 @@ class PostsController < ApplicationController
def post_snapshot_changes base_snapshot, current_snapshot, incoming_snapshot def post_snapshot_changes base_snapshot, current_snapshot, incoming_snapshot
[scalar_snapshot_change(:title, 'タイトル', [scalar_snapshot_change(:title, 'タイトル',
base_snapshot, current_snapshot, incoming_snapshot), base_snapshot, current_snapshot, incoming_snapshot),
scalar_snapshot_change(:video_ms, '動画時間',
base_snapshot, current_snapshot, incoming_snapshot),
scalar_snapshot_change(:original_created_from, 'オリジナルの作成日時(以降)', scalar_snapshot_change(:original_created_from, 'オリジナルの作成日時(以降)',
base_snapshot, current_snapshot, incoming_snapshot), base_snapshot, current_snapshot, incoming_snapshot),
scalar_snapshot_change(:original_created_before, 'オリジナルの作成日時(より前)', scalar_snapshot_change(:original_created_before, 'オリジナルの作成日時(より前)',
@@ -625,11 +666,14 @@ class PostsController < ApplicationController
PostVersionRecorder.ensure_snapshot!(post, created_by_user: current_user) PostVersionRecorder.ensure_snapshot!(post, created_by_user: current_user)
post.update!(title: snapshot[:title], post.update!(title: snapshot[:title],
video_ms: snapshot[:video_ms],
original_created_from: snapshot[:original_created_from], original_created_from: snapshot[:original_created_from],
original_created_before: snapshot[:original_created_before]) original_created_before: snapshot[:original_created_before])
editable_tags = Tag.normalise_tags!(snapshot[:tag_names], with_tagme: false, Tag.normalise_tags!(snapshot[:tag_names], with_tagme: false,
deny_deprecated: true) deny_deprecated: true,
with_sections: true) =>
{ tags: editable_tags, sections: }
TagVersioning.record_tag_snapshots!(editable_tags, created_by_user: current_user) TagVersioning.record_tag_snapshots!(editable_tags, created_by_user: current_user)
readonly_tags = post.tags.nico.to_a readonly_tags = post.tags.nico.to_a
@@ -637,14 +681,17 @@ class PostsController < ApplicationController
tags = readonly_tags + editable_tags tags = readonly_tags + editable_tags
tags = Tag.expand_parent_tags(tags).reject(&:deprecated?) tags = Tag.expand_parent_tags(tags).reject(&:deprecated?)
sync_post_tags!(post, tags) post.video_ms = tags.any? { _1.id == Tag.video.id } ? snapshot[:video_ms] : nil
validate_video_sections!(post.video_ms, sections)
post.save!
sync_post_tags!(post, tags, sections)
sync_parent_posts!(post, snapshot[:parent_post_ids]) sync_parent_posts!(post, snapshot[:parent_post_ids])
PostVersionRecorder.record!(post:, event_type: :update, created_by_user: current_user) PostVersionRecorder.record!(post:, event_type: :update, created_by_user: current_user)
end end
def merge_post_snapshots base_snapshot, current_snapshot, incoming_snapshot def merge_post_snapshots base_snapshot, current_snapshot, incoming_snapshot
[:title, :original_created_from, :original_created_before].map { [:title, :video_ms, :original_created_from, :original_created_before].map {
[_1, merge_scalar_snapshot_value(base_snapshot[_1], [_1, merge_scalar_snapshot_value(base_snapshot[_1],
current_snapshot[_1], current_snapshot[_1],
incoming_snapshot[_1])] incoming_snapshot[_1])]
@@ -688,4 +735,44 @@ class PostsController < ApplicationController
render_validation_error record render_validation_error record
end end
end end
def normalise_video_ms tags, video_ms_param: params[:video_ms], duration_param: params[:duration]
return nil unless tags.any? { _1.id == Tag.video.id }
if video_ms_param.present?
video_ms = Integer(video_ms_param, exception: false)
raise VideoMsParseError unless video_ms&.positive?
return video_ms
end
return nil if duration_param.blank?
video_ms = Tag.time_to_ms!(duration_param.to_s, tag_name: '動画時間')
raise VideoMsParseError unless video_ms.positive?
video_ms
rescue Tag::SectionLiteralParseError
raise VideoMsParseError
end
def validate_video_sections! video_ms, sections
return unless video_ms
sections.each_value do |ranges|
ranges.each do |begin_ms, end_ms|
if begin_ms >= video_ms
post = Post.new
post.errors.add :video_ms, 'タグ区間の開始が動画時間以上です.'
raise ActiveRecord::RecordInvalid, post
end
if end_ms && end_ms > video_ms
post = Post.new
post.errors.add :video_ms, 'タグ区間の終端が動画時間を超えてゐます.'
raise ActiveRecord::RecordInvalid, post
end
end
end
end
end end
+44 -37
ファイルの表示
@@ -1,50 +1,57 @@
class PreviewController < ApplicationController class PreviewController < ApplicationController
before_action :require_member!
def title def title
# TODO: # 既知サイトなら決まったフォーマットで title 取得するやぅに. return render_bad_request('URL は必須です.') if params[:url].blank?
return head :unauthorized unless current_user
url = params[:url] render json: { title: Preview::ThumbnailFetcher.title(params[:url]) }
return render_bad_request('URL は必須です.') unless url.present? rescue Preview::UrlSafety::UnsafeUrl => e
unless url.start_with?(/http(s)?:\/\//)
url = 'http://' + url
end
html = URI.open(url, open_timeout: 5, read_timeout: 5).read
doc = Nokogiri::HTML.parse(html)
title = doc.at('title')&.text&.strip
render json: { title: title }
rescue => e
render_bad_request(e.message) render_bad_request(e.message)
rescue Preview::HttpFetcher::FetchTimeout => e
render_preview_error(e.message, :gateway_timeout)
rescue Preview::HttpFetcher::ResponseTooLarge => e
render_preview_error(e.message, :payload_too_large)
rescue Preview::HttpFetcher::FetchFailed => e
render_preview_error(e.message, :bad_gateway)
end end
def thumbnail def thumbnail
# TODO: 既知ドメインであれば指定のアドレスからサムネールを取得するやぅにする. return render_bad_request('URL は必須です.') if params[:url].blank?
image = MiniMagick::Image.read(Preview::ThumbnailFetcher.fetch(params[:url]))
image.auto_orient
image.resize '180x180>'
image.format 'png'
width, height = image.dimensions
raise Preview::ThumbnailFetcher::GenerationFailed, 'サムネール画像の変換に失敗しました.' if width > 180 || height > 180
send_data image.to_blob, type: 'image/png', disposition: 'inline'
rescue Preview::UrlSafety::UnsafeUrl => e
render_bad_request(e.message)
rescue Preview::HttpFetcher::FetchTimeout => e
render_preview_error(e.message, :gateway_timeout)
rescue Preview::HttpFetcher::ResponseTooLarge => e
render_preview_error(e.message, :payload_too_large)
rescue Preview::HttpFetcher::FetchFailed => e
render_preview_error(e.message, :bad_gateway)
rescue Preview::ThumbnailFetcher::GenerationFailed, MiniMagick::Error => e
render_unprocessable_entity(e.message)
end
private
def require_member!
return head :unauthorized unless current_user return head :unauthorized unless current_user
return if current_user.gte_member?
url = params[:url] head :forbidden
return render_bad_request('URL は必須です.') if url.blank? end
unless url.start_with?(/http(s)?:\/\//) def render_preview_error(message, status)
url = 'http://' + url render json: { type: status.to_s,
end message:,
errors: { },
path = Rails.root.join('tmp', "thumb_#{ SecureRandom.hex }.png") base_errors: [message] },
system("node #{ Rails.root }/lib/screenshot.js #{ Shellwords.escape(url) } #{ path }") status:
if File.exist?(path)
image = MiniMagick::Image.open(path)
image.resize '180x180'
File.delete(path) rescue nil
send_file image.path, type: 'image/png', disposition: 'inline'
else
render json: { type: 'internal_server_error',
message: 'サムネールを生成できませんでした.',
errors: { },
base_errors: ['サムネールを生成できませんでした.'] },
status: :internal_server_error
end
end end
end end
+169 -42
ファイルの表示
@@ -82,8 +82,9 @@ class TagsController < ApplicationController
def with_depth def with_depth
parent_tag_id = params[:parent].to_i parent_tag_id = params[:parent].to_i
parent_tag_id = nil if parent_tag_id <= 0 parent_tag_id = nil if parent_tag_id <= 0
material_filter = material_filter_param(default: 'any')
graph = build_with_depth_graph graph = build_with_depth_graph(material_filter)
tag_ids = tag_ids =
if parent_tag_id if parent_tag_id
@@ -92,19 +93,9 @@ class TagsController < ApplicationController
visible_root_tag_ids(graph) visible_root_tag_ids(graph)
end end
tags = render json: tag_ids
Tag .sort_by { |tag_id| graph[:tags_by_id][tag_id][:name] }
.joins(:tag_name) .map { |tag_id| with_depth_lightweight_repr(tag_id, graph) }
.includes(:tag_name, :materials, tag_name: :wiki_page)
.where(id: tag_ids)
.order('tag_names.name')
.distinct
.to_a
render json: tags.map { |tag|
TagRepr.base(tag).merge(has_children: visible_child_tag_ids(tag.id, graph).present?,
children: [])
}
end end
def autocomplete def autocomplete
@@ -229,13 +220,16 @@ class TagsController < ApplicationController
def materials_by_name def materials_by_name
name = params[:name].to_s.strip name = params[:name].to_s.strip
return render_bad_request('name は必須です.') if name.blank? return render_bad_request('name は必須です.') if name.blank?
material_filter = material_filter_param(default: 'any')
tag = Tag.joins(:tag_name) tag = Tag.joins(:tag_name)
.includes(:tag_name, :materials, tag_name: :wiki_page) .includes(:tag_name, :materials, tag_name: :wiki_page)
.find_by(tag_names: { name: }) .find_by(tag_names: { name: })
return head :not_found unless tag return head :not_found unless tag
render json: build_tag_children(tag) graph = build_with_depth_graph(material_filter)
render json: build_tag_children(tag, graph:)
end end
def update_all def update_all
@@ -251,10 +245,7 @@ class TagsController < ApplicationController
return render_unprocessable_entity('カテゴリは必須です.', field: :category) if category.blank? return render_unprocessable_entity('カテゴリは必須です.', field: :category) if category.blank?
return render_unprocessable_entity '廃止状態は必須です.', field: :deprecated unless params.key?(:deprecated) return render_unprocessable_entity '廃止状態は必須です.', field: :deprecated unless params.key?(:deprecated)
if (name != tag.name && return unless validate_tag_rename(tag, name)
tag.in?([Tag.tagme, Tag.bot, Tag.no_deerjikist, Tag.video, Tag.niconico]))
return render_unprocessable_entity 'システム・タグの名称は変更できません.', field: :name
end
alias_names = params[:aliases].to_s.split.uniq alias_names = params[:aliases].to_s.split.uniq
parent_names = params[:parent_tags].to_s.split.uniq parent_names = params[:parent_tags].to_s.split.uniq
@@ -280,7 +271,7 @@ class TagsController < ApplicationController
else else
tag.update!(category:, deprecated_at: deprecated ? Time.current : nil) tag.update!(category:, deprecated_at: deprecated ? Time.current : nil)
end end
tag.tag_name.update!(name:) rename_tag_name!(tag, name) if name_changed
alias_names << old_name if name_changed alias_names << old_name if name_changed
alias_names.delete(name) alias_names.delete(name)
@@ -316,6 +307,8 @@ class TagsController < ApplicationController
return render_unprocessable_entity 'ニコタグは廃止できません.', field: :deprecated return render_unprocessable_entity 'ニコタグは廃止できません.', field: :deprecated
end end
return unless validate_tag_rename(tag, name)
if tag.nico? || (category.present? && category == 'nico') if tag.nico? || (category.present? && category == 'nico')
return render_unprocessable_entity 'ニコタグは変更できません.', field: :category return render_unprocessable_entity 'ニコタグは変更できません.', field: :category
end end
@@ -327,7 +320,7 @@ class TagsController < ApplicationController
name_changed = name.present? && name != old_name name_changed = name.present? && name != old_name
wiki_page = tag.tag_name.wiki_page if name_changed wiki_page = tag.tag_name.wiki_page if name_changed
tag.tag_name.update!(name:) if name.present? rename_tag_name!(tag, name) if name_changed
tag.update!(category:) if category.present? tag.update!(category:) if category.present?
if deprecated_given && tag.deprecated? != deprecated if deprecated_given && tag.deprecated? != deprecated
tag.update!(deprecated_at: deprecated ? Time.current : nil) tag.update!(deprecated_at: deprecated ? Time.current : nil)
@@ -348,7 +341,14 @@ class TagsController < ApplicationController
private private
def build_with_depth_graph def material_filter_param default:
value = params[:material_filter].to_s.presence
return default unless ['present', 'missing', 'any'].include?(value)
value
end
def build_with_depth_graph material_filter
children_by_parent_id = Hash.new { |h, k| h[k] = [] } children_by_parent_id = Hash.new { |h, k| h[k] = [] }
parent_ids_by_child_id = Hash.new { |h, k| h[k] = [] } parent_ids_by_child_id = Hash.new { |h, k| h[k] = [] }
@@ -361,23 +361,37 @@ class TagsController < ApplicationController
parent_ids_by_child_id.keys + parent_ids_by_child_id.keys +
Tag.where(category: ['meme', 'character', 'material']).pluck(:id)).uniq Tag.where(category: ['meme', 'character', 'material']).pluck(:id)).uniq
tags_by_id = Tag.where(id: tag_ids) material_tag_ids = Material.unscoped.kept.where.not(tag_id: nil).distinct.pluck(:tag_id).to_set
.pluck(:id, :category, :deprecated_at)
.each_with_object({ }) do |(id, category, deprecated_at), h| tags_by_id = Tag.joins(:tag_name)
h[id] = { category:, deprecated: deprecated_at.present? } .where(id: tag_ids)
.pluck('tags.id', 'tag_names.name', 'tags.category', 'tags.deprecated_at')
.each_with_object({ }) do |(id, name, category, deprecated_at), h|
h[id] = { name:, category:, deprecated: deprecated_at.present?,
has_material: material_tag_ids.include?(id) }
end end
{ children_by_parent_id:, parent_ids_by_child_id:, tags_by_id:, { children_by_parent_id:, parent_ids_by_child_id:, tags_by_id:,
visible_child_tag_ids_by_parent_id: { } } visible_child_tag_ids_by_parent_id: { },
visible_subtree_by_tag_id: { },
material_filter: }
end end
def visible_root_tag_ids graph def visible_root_tag_ids graph
graph[:tags_by_id].filter_map do |tag_id, attrs| root_ids = Set.new
next unless with_depth_visible_tag?(attrs)
next unless visible_root_tag?(tag_id, graph)
tag_id graph[:tags_by_id].each do |tag_id, attrs|
next unless visible_root_tag?(tag_id, graph)
next unless visible_subtree?(tag_id, graph)
if attrs[:deprecated]
collect_visible_child_tag_ids(tag_id, graph, root_ids, Set.new)
else
root_ids << tag_id
end
end end
root_ids.to_a
end end
def visible_root_tag? tag_id, graph def visible_root_tag? tag_id, graph
@@ -428,25 +442,77 @@ class TagsController < ApplicationController
return return
end end
visible_ids << tag_id if with_depth_visible_tag?(tag) if with_depth_visible_tag?(tag, graph[:material_filter])
visible_ids << tag_id
return
end
visible_ids << tag_id if graph[:children_by_parent_id][tag_id].any? { |child_tag_id|
visible_subtree?(child_tag_id, graph)
}
end end
def with_depth_visible_tag? tag def with_depth_visible_tag? tag, material_filter
tag[:category].in?(['meme', 'character', 'material']) && !tag[:deprecated] return false unless tag[:category].in?(['meme', 'character', 'material']) && !tag[:deprecated]
case material_filter
when 'present'
tag[:has_material]
when 'missing'
!tag[:has_material]
else
true
end
end end
def build_tag_children tag def visible_subtree? tag_id, graph
cache = graph[:visible_subtree_by_tag_id]
return cache[tag_id] if cache.key?(tag_id)
tag = graph[:tags_by_id][tag_id]
return cache[tag_id] = false unless tag
if tag[:deprecated]
return cache[tag_id] =
graph[:children_by_parent_id][tag_id].any? { |child_tag_id|
visible_subtree?(child_tag_id, graph)
}
end
cache[tag_id] =
with_depth_visible_tag?(tag, graph[:material_filter]) ||
graph[:children_by_parent_id][tag_id].any? { |child_tag_id|
visible_subtree?(child_tag_id, graph)
}
end
def with_depth_lightweight_repr tag_id, graph
tag = graph[:tags_by_id][tag_id]
{ id: tag_id,
name: tag[:name],
category: tag[:category],
deprecated: tag[:deprecated],
has_material: tag[:has_material],
has_children: visible_child_tag_ids(tag_id, graph).present?,
children: [] }
end
def build_tag_children tag, graph: nil
material = tag.materials.first material = tag.materials.first
file = nil tag_graph = graph && graph[:tags_by_id][tag.id]
content_type = nil material =
if material&.file&.attached? nil if tag_graph && !with_depth_visible_tag?(tag_graph, graph[:material_filter])
file = rails_storage_proxy_url(material.file, only_path: false)
content_type = material.file.blob.content_type children = tag.children.sort_by(&:name)
if graph
children = children.filter { |child_tag| visible_subtree?(child_tag.id, graph) }
end end
TagRepr.base(tag).merge( TagRepr.base(tag).merge(
children: tag.children.sort_by { _1.name }.map { build_tag_children(_1) }, children: children.map { build_tag_children(_1, graph:) },
material: material.as_json&.merge(file:, content_type:)) has_material: tag_graph ? tag_graph[:has_material] : material.present?,
material: material && MaterialRepr.base(material, host: request.base_url))
end end
def record_tag_version! tag, event_type:, created_by_user:, name_changed: false, wiki_page: nil def record_tag_version! tag, event_type:, created_by_user:, name_changed: false, wiki_page: nil
@@ -468,6 +534,67 @@ class TagsController < ApplicationController
created_by_user:) created_by_user:)
end end
def validate_tag_rename tag, name
return true if name.blank? || name == tag.name
if tag.in?([Tag.tagme, Tag.bot, Tag.no_deerjikist, Tag.video, Tag.niconico])
render_unprocessable_entity 'システム・タグの名称は変更できません.', field: :name
return false
end
target_tag_name = TagName.with_discarded.find_by(name:)
return true if target_tag_name.nil?
return true if target_tag_name.canonical_id?
render_unprocessable_entity 'その名前は既に使はれてゐます.', field: :name
false
end
def rename_tag_name! tag, name
return if name == tag.name
current_tag_name = tag.tag_name
target_tag_name = TagName.with_discarded.find_by(name:)
if target_tag_name.nil?
current_tag_name.update!(name:)
return
end
promote_tag_alias!(
tag,
current_tag_name:,
promoted_tag_name: target_tag_name)
end
def promote_tag_alias! tag, current_tag_name:, promoted_tag_name:
old_owner_tag = promoted_tag_name.canonical&.tag
if old_owner_tag && old_owner_tag != tag
TagVersioning.ensure_snapshot!(old_owner_tag, created_by_user: current_user)
end
promoted_tag_name.undiscard! if promoted_tag_name.discarded?
promoted_tag_name.update!(canonical: nil)
TagName.with_discarded
.where(canonical_id: current_tag_name.id)
.where.not(id: promoted_tag_name.id)
.find_each do |alias_tag_name|
alias_tag_name.update!(canonical: promoted_tag_name)
end
current_tag_name.wiki_page&.update!(tag_name: promoted_tag_name)
tag.update!(tag_name: promoted_tag_name)
current_tag_name.association(:wiki_page).reset
current_tag_name.association(:tag).reset
current_tag_name.reload.update!(canonical: promoted_tag_name)
return unless old_owner_tag && old_owner_tag != tag
record_tag_version!(old_owner_tag.reload, event_type: :update, created_by_user: current_user)
end
def update_aliases! tag, alias_names def update_aliases! tag, alias_names
alias_names = alias_names.uniq alias_names = alias_names.uniq
+61
ファイルの表示
@@ -0,0 +1,61 @@
class UserSettingsController < ApplicationController
wrap_parameters false
def show
return head :unauthorized unless current_user
render json: current_setting.serializable_hash
end
def update
return head :unauthorized unless current_user
raw_attributes = editable_raw_attributes
field_errors = validate_raw_attributes(raw_attributes)
return render_validation_error fields: field_errors if field_errors.present?
setting = current_setting
setting.assign_attributes(raw_attributes)
if setting.save
render json: setting.serializable_hash, status: :ok
else
render_validation_error setting
end
end
private
def current_setting
Setting.find_or_create_by!(user: current_user) do |setting|
setting.assign_attributes(Setting.defaults)
end
rescue ActiveRecord::RecordNotUnique
Setting.find_by!(user: current_user)
end
def editable_raw_attributes
request.request_parameters.slice(*Setting::EDITABLE_ATTRIBUTES)
end
def validate_raw_attributes raw_attributes
raw_attributes.each_with_object({ }) do |(key, value), errors|
next if value_matches_type?(key, value)
errors[key.to_sym] = ['値の型が不正です.']
end
end
def value_matches_type? key, value
case Setting::TYPE_BY_ATTRIBUTE.fetch(key)
when :string
value.is_a?(String)
when :integer
value.is_a?(Integer)
when :boolean
value == true || value == false
else
false
end
end
end
+38
ファイルの表示
@@ -0,0 +1,38 @@
class UserThemeSlotsController < ApplicationController
wrap_parameters false
def index
return head :unauthorized unless current_user
render json: current_user
.theme_slots
.order(:base_theme, :slot_no)
.map { |slot| slot.serializable_hash }
end
def update
return head :unauthorized unless current_user
base_theme = params[:base_theme].to_s
slot_no = params[:slot_no].to_i
tokens = params[:tokens]
unless UserThemeSlot::BASE_THEMES.include?(base_theme)
return render_validation_error fields: { base_theme: ['値が不正です.'] }
end
unless UserThemeSlot::SLOT_NOS.include?(slot_no)
return render_validation_error fields: { slot_no: ['値が不正です.'] }
end
unless tokens.is_a?(ActionController::Parameters) || tokens.is_a?(Hash)
return render_validation_error fields: { tokens: ['JSON object で指定してください.'] }
end
slot = UserThemeSlot.find_or_initialize_by(user: current_user,
base_theme:,
slot_no:)
slot.tokens = tokens.is_a?(ActionController::Parameters) ? tokens.to_unsafe_h : tokens
slot.save!
render json: slot.serializable_hash, status: :ok
end
end
+43 -2
ファイルの表示
@@ -1,6 +1,9 @@
class Material < ApplicationRecord class Material < ApplicationRecord
include MyDiscard include MyDiscard
SOURCE_KINDS = ['uri', 'google_drive_path', 'google_drive_file',
'legacy_drive_path'].freeze
default_scope -> { kept } default_scope -> { kept }
belongs_to :parent, class_name: 'Material', optional: true belongs_to :parent, class_name: 'Material', optional: true
@@ -10,9 +13,18 @@ class Material < ApplicationRecord
belongs_to :created_by_user, class_name: 'User', optional: true belongs_to :created_by_user, class_name: 'User', optional: true
belongs_to :updated_by_user, class_name: 'User', optional: true belongs_to :updated_by_user, class_name: 'User', optional: true
has_one_attached :file, dependent: :purge has_many :material_versions, dependent: :destroy
has_many :material_export_items, dependent: :destroy
validates :tag_id, presence: true, uniqueness: true has_one_attached :file, dependent: :purge
has_one_attached :thumbnail, dependent: :purge
before_validation :assign_normalized_source_key
validates :tag_id, uniqueness: true, allow_nil: true
validates :source_kind,
inclusion: { in: SOURCE_KINDS },
allow_blank: true
validate :file_must_be_attached validate :file_must_be_attached
validate :tag_must_be_material_category validate :tag_must_be_material_category
@@ -23,8 +35,37 @@ class Material < ApplicationRecord
file.blob.content_type file.blob.content_type
end end
def file_byte_size
return nil unless file&.attached?
file.blob.byte_size
end
def file_filename
return nil unless file&.attached?
file.blob.filename.to_s
end
def snapshot_export_paths
material_export_items.order(:profile).pluck(:profile, :export_path).to_h
end
private private
def assign_normalized_source_key
if source_kind.blank?
self.normalized_source_key = nil
return
end
self.normalized_source_key =
MaterialSyncSuppression.normalize_source_key(source_kind:,
source_uri:,
drive_path: source_path,
drive_file_id: source_file_id)
end
def file_must_be_attached def file_must_be_attached
return if url.present? || file.attached? return if url.present? || file.attached?
+48
ファイルの表示
@@ -0,0 +1,48 @@
class MaterialExportItem < ApplicationRecord
VALID_PROFILES = ['legacy_drive'].freeze
belongs_to :material
belongs_to :created_by_user, class_name: 'User', optional: true
validates :profile, presence: true, inclusion: { in: VALID_PROFILES }
validates :export_path, presence: true, uniqueness: { scope: :profile }
validates :material_id, uniqueness: { scope: :profile }
validate :export_path_must_be_relative_safe_path
scope :enabled, -> { where(enabled: true) }
private
def export_path_must_be_relative_safe_path
return if export_path.blank?
if export_path.start_with?('/')
errors.add(:export_path, '絶対パスは使へません.')
end
if export_path.match?(/\A[A-Za-z]:\//)
errors.add(:export_path, '絶対パスは使へません.')
end
if export_path.include?('\\')
errors.add(:export_path, '/ 区切りで指定してください.')
end
if export_path.include?("\0")
errors.add(:export_path, 'NUL は使へません.')
end
parts = export_path.split('/')
if export_path.include?('//')
errors.add(:export_path, '空の path segment は使へません.')
end
if parts.any? { |part| part.in?(['.', '..']) }
errors.add(:export_path, '.. は使へません.')
end
if export_path.end_with?('/')
errors.add(:export_path, 'directory path は使へません.')
end
end
end
+29
ファイルの表示
@@ -0,0 +1,29 @@
class MaterialImportBlock < ApplicationRecord
MATCH_KINDS = ['sha256', 'exact_path', 'path_prefix', 'manual'].freeze
REASONS = [
'copyright_high_risk',
'copyright_takedown',
'adult_or_sensitive',
'personal_information',
'malware_or_dangerous_file',
'duplicate_or_low_quality',
'source_owner_request',
'other'
].freeze
belongs_to :created_by_user, class_name: 'User', optional: true
validates :match_kind, presence: true, inclusion: { in: MATCH_KINDS }
validates :reason, presence: true, inclusion: { in: REASONS }
validates :sha256, length: { is: 64 }, allow_blank: true
validate :match_value_must_be_present
private
def match_value_must_be_present
return if match_kind == 'manual'
return if sha256.present? || external_path_pattern.present?
errors.add(:base, 'sha256 または external_path_pattern は必須です.')
end
end
+45
ファイルの表示
@@ -0,0 +1,45 @@
class MaterialSyncSource < ApplicationRecord
SOURCE_KINDS = ['uri', 'google_drive_path', 'google_drive_file', 'legacy_drive_path'].freeze
belongs_to :created_by_user, class_name: 'User', optional: true
belongs_to :updated_by_user, class_name: 'User', optional: true
validates :name, presence: true
validates :source_kind,
presence: true,
inclusion: { in: SOURCE_KINDS }
validates :profile, presence: true, inclusion: { in: MaterialExportItem::VALID_PROFILES }
validate :source_value_must_be_present
scope :enabled, -> { where(enabled: true) }
def normalized_source_key
MaterialSyncSuppression.normalize_source_key(source_kind:,
source_uri:,
source_path:,
source_file_id:)
end
private
def source_value_must_be_present
return if source_value_present?
errors.add(:base, '同期元を指定してください.')
end
def source_value_present?
case source_kind
when 'uri'
source_uri.present?
when 'google_drive_file'
source_file_id.present? || source_uri.present?
when 'google_drive_path'
source_file_id.present? || source_uri.present?
when 'legacy_drive_path'
source_path.present?
else
normalized_source_key.present?
end
end
end
+105
ファイルの表示
@@ -0,0 +1,105 @@
class MaterialSyncSuppression < ApplicationRecord
SOURCE_KINDS = [
'uri',
'google_drive_path',
'google_drive_path_prefix',
'google_drive_file',
'legacy_drive_path',
'legacy_drive_path_prefix'
].freeze
REASONS = [
'copyright_high_risk',
'copyright_takedown',
'adult_or_sensitive',
'personal_information',
'malware_or_dangerous_file',
'duplicate_or_low_quality',
'source_owner_request',
'other'
].freeze
belongs_to :created_by_user, class_name: 'User', optional: true
before_validation :assign_normalized_source_key
validates :source_kind, presence: true, inclusion: { in: SOURCE_KINDS }
validates :reason, presence: true, inclusion: { in: REASONS }
validates :normalized_source_key, presence: true, uniqueness: true
validate :source_value_must_be_present
validate :drive_path_must_be_safe_relative_path
def self.normalize_source_key source_kind:,
source_uri: nil,
drive_path: nil,
drive_file_id: nil,
source_path: nil,
source_file_id: nil
kind = source_kind.to_s
value =
case kind
when 'uri'
source_uri.to_s.strip
when 'google_drive_path', 'google_drive_path_prefix',
'legacy_drive_path', 'legacy_drive_path_prefix'
(source_path || drive_path).to_s.strip.gsub(%r{/+}, '/')
when 'google_drive_file'
(source_file_id || drive_file_id).to_s.strip
else
''
end
return nil if kind.blank? || value.blank?
"#{ kind }:#{ value }"
end
private
def assign_normalized_source_key
normalize_path_fields!
self.normalized_source_key =
self.class.normalize_source_key(source_kind:,
source_uri:,
drive_path:,
drive_file_id:)
end
def source_value_must_be_present
return if normalized_source_key.present?
errors.add(:base, '同期元を指定してください.')
end
def drive_path_must_be_safe_relative_path
return unless source_kind.in?(
['google_drive_path', 'google_drive_path_prefix',
'legacy_drive_path', 'legacy_drive_path_prefix'])
value = drive_path.to_s
return if value.blank?
if value.start_with?('/') || value.match?(/\A[A-Za-z]:\//)
errors.add(:drive_path, '相対 path を指定してください.')
end
if value.start_with?('My Drive/', 'マイドライブ/')
errors.add(:drive_path, '同期元フォルダからの相対 path を指定してください.')
end
errors.add(:drive_path, 'NUL は使へません.') if value.include?("\0")
errors.add(:drive_path, '連続スラッシュは使へません.') if value.include?('//')
errors.add(:drive_path, '末尾スラッシュは使へません.') if value.end_with?('/')
parts = value.split('/')
if parts.any? { |part| part.in?(['.', '..']) }
errors.add(:drive_path, '. や .. は使へません.')
end
end
def normalize_path_fields!
return unless source_kind.in?(
['google_drive_path', 'google_drive_path_prefix',
'legacy_drive_path', 'legacy_drive_path_prefix'])
self.drive_path = drive_path.to_s.tr('\\', '/').strip.presence
end
end
+17
ファイルの表示
@@ -0,0 +1,17 @@
class MaterialVersion < ApplicationRecord
EVENT_TYPE_MAP = { create: 'create',
update: 'update',
discard: 'discard',
restore: 'restore' }.freeze
include VersionRecord
belongs_to :material
belongs_to :tag, optional: true
belongs_to :parent, class_name: 'Material', optional: true
belongs_to :updated_by_user, class_name: 'User', optional: true
def export_paths_hash
export_paths_json || {}
end
end
+55 -19
ファイルの表示
@@ -1,5 +1,19 @@
class Post < ApplicationRecord class Post < ApplicationRecord
require 'mini_magick' require 'mini_magick'
require 'stringio'
def self.resized_thumbnail_attachment(upload)
upload.rewind
image = MiniMagick::Image.read(upload.read)
image.resize '180x180'
image.format 'jpg'
{ io: StringIO.new(image.to_blob),
filename: 'resized_thumbnail.jpg',
content_type: 'image/jpeg' }
ensure
upload.rewind
end
belongs_to :uploaded_user, class_name: 'User', optional: true belongs_to :uploaded_user, class_name: 'User', optional: true
@@ -43,9 +57,10 @@ class Post < ApplicationRecord
attribute :version_no, :integer, default: 1 attribute :version_no, :integer, default: 1
before_validation :normalise_url before_validation :normalise_url, if: :will_save_change_to_url?
validates :url, presence: true, uniqueness: true validates :url, presence: true, uniqueness: true, length: { maximum: 768 }
validates :video_ms, numericality: { only_integer: true, greater_than: 0 }, allow_nil: true
validate :validate_original_created_range validate :validate_original_created_range
validate :url_must_be_http_url validate :url_must_be_http_url
@@ -69,7 +84,42 @@ class Post < ApplicationRecord
super(options).merge(thumbnail: nil) super(options).merge(thumbnail: nil)
end end
def snapshot_tag_names = tags.joins(:tag_name).order('tag_names.name').pluck('tag_names.name') def snapshot_tag_names
post_tags
.kept
.joins(tag: :tag_name)
.includes(:sections, tag: :tag_name)
.order('tag_names.name')
.map do |post_tag|
name = post_tag.tag.tag_name.name
sections = post_tag.sections.sort_by(&:begin_ms)
next name if sections.empty?
"#{ name }#{ sections.map { Post.section_literal(_1) }.join }"
end
end
def self.section_literal section
"[#{ Post.ms_to_time(section.begin_ms) }-#{ section.end_ms ? Post.ms_to_time(section.end_ms) : '' }]"
end
def self.ms_to_time ms
total_s = ms / 1_000
s = total_s % 60
min = (total_s / 60) % 60
h = total_s / 3_600
remainder_ms = ms % 1_000
base =
if h.positive?
'%d:%02d:%02d' % [h, min, s]
else
'%d:%02d' % [min, s]
end
remainder_ms.positive? ? "#{ base }.#{ remainder_ms.to_s.rjust(3, '0') }" : base
end
def snapshot_parent_post_ids = parents.order(:id).pluck(:id) def snapshot_parent_post_ids = parents.order(:id).pluck(:id)
@@ -87,12 +137,7 @@ class Post < ApplicationRecord
def resized_thumbnail! def resized_thumbnail!
return unless thumbnail.attached? return unless thumbnail.attached?
image = MiniMagick::Image.read(thumbnail.download) thumbnail.attach(self.class.resized_thumbnail_attachment(StringIO.new(thumbnail.download)))
image.resize '180x180'
thumbnail.purge
thumbnail.attach(io: File.open(image.path),
filename: 'resized_thumbnail.jpg',
content_type: 'image/jpeg')
end end
private private
@@ -128,15 +173,6 @@ class Post < ApplicationRecord
def normalise_url def normalise_url
return if url.blank? return if url.blank?
self.url = url.strip self.url = PostUrlNormaliser.normalise(url) || url.strip
u = URI.parse(url)
return unless u in URI::HTTP
u.host = u.host.downcase if u.host
u.path = u.path.sub(/\/\Z/, '') if u.path.present?
self.url = u.to_s
rescue URI::InvalidURIError
;
end end
end end
+6
ファイルの表示
@@ -10,6 +10,12 @@ class PostTag < ApplicationRecord
belongs_to :created_user, class_name: 'User', optional: true belongs_to :created_user, class_name: 'User', optional: true
belongs_to :deleted_user, class_name: 'User', optional: true belongs_to :deleted_user, class_name: 'User', optional: true
has_many :sections, -> { order(:begin_ms) }, class_name: 'PostTagSection',
foreign_key: [:post_id, :tag_id],
primary_key: [:post_id, :tag_id],
dependent: :delete_all,
inverse_of: :post_tag
validates :post_id, presence: true validates :post_id, presence: true
validates :tag_id, presence: true validates :tag_id, presence: true
validates :post_id, uniqueness: { validates :post_id, uniqueness: {
+20
ファイルの表示
@@ -0,0 +1,20 @@
class PostTagSection < ApplicationRecord
self.primary_key = :post_id, :tag_id, :begin_ms
belongs_to :post
belongs_to :tag
belongs_to :post_tag, -> { kept }, foreign_key: [:post_id, :tag_id],
primary_key: [:post_id, :tag_id],
inverse_of: :sections,
optional: true
validates :post_id, presence: true
validates :tag_id, presence: true
validates :begin_ms, presence: true,
numericality: { only_integer: true, greater_than_or_equal_to: 0 }
validates :end_ms, numericality: { only_integer: true, greater_than: :begin_ms },
allow_nil: true
end
+163
ファイルの表示
@@ -0,0 +1,163 @@
class PostUrlSanitisationRule < ApplicationRecord
include Discard::Model
class InvalidUrlError < StandardError
attr_reader :invalid_rows
def initialize(invalid_rows)
@invalid_rows = invalid_rows
ids = invalid_rows.map { _1.fetch(:post_id) }.join(', ')
super("post URL sanitisation produced invalid URLs for posts #{ ids }")
end
end
class UrlConflictError < StandardError
attr_reader :conflicts
def initialize(conflicts)
@conflicts = conflicts
urls = conflicts.map { _1.fetch(:url) }.uniq.join(', ')
super("post URL sanitisation conflicts detected for #{ urls }")
end
end
self.primary_key = :priority
default_scope -> { kept }
validates :source_pattern, presence: true, uniqueness: true
validate :source_pattern_must_be_regexp
class << self
def sanitise(url) = sanitise_with_rules(url, rules)
def apply!
rewrites = nil
Post.transaction do
compiled_rules = rules
rewrites = Post.order(:id)
.lock('FOR UPDATE')
.pluck(:id, :url)
.map do |post_id, original_url|
{ post_id:,
original_url:,
sanitised_url: sanitise_with_rules(original_url, compiled_rules) }
end
invalid_rows = rewrites.filter { invalid_sanitised_url?(_1.fetch(:sanitised_url)) }
.map { { post_id: _1.fetch(:post_id),
original_url: _1.fetch(:original_url),
sanitised_url: _1.fetch(:sanitised_url) } }
raise InvalidUrlError.new(invalid_rows) if invalid_rows.present?
conflicts = build_conflicts(rewrites)
raise UrlConflictError.new(conflicts) if conflicts.present?
changed = rewrites.filter { _1.fetch(:original_url) != _1.fetch(:sanitised_url) }
return if changed.empty?
token = SecureRandom.hex(6)
changed.each do |row|
Post.where(id: row.fetch(:post_id))
.update_all(url: temporary_url_for(row.fetch(:post_id), token))
end
changed.each do |row|
Post.where(id: row.fetch(:post_id))
.update_all(url: row.fetch(:sanitised_url))
end
end
rescue ActiveRecord::RecordNotUnique => error
conflicts = build_persisted_conflicts(rewrites)
raise error if conflicts.empty?
raise UrlConflictError.new(conflicts), cause: error
end
private
def rules = kept.order(:priority).map { |r| [Regexp.new(r.source_pattern), r.replacement] }
def sanitise_with_rules(url, compiled_rules)
compiled_rules.reduce(url.dup) do |value, (pattern, replacement)|
value.sub(pattern, replacement)
end
end
def temporary_url_for(post_id, token) =
"https://post-url-sanitising.invalid/#{ token }/#{ post_id }"
def invalid_sanitised_url?(url)
return true if url.blank?
return true if url.length > 768
parsed = URI.parse(url)
return true if !(parsed in URI::HTTP)
return true if parsed.host.blank?
false
rescue URI::InvalidURIError
true
end
def build_conflicts(rewrites)
rewrites
.group_by { _1.fetch(:sanitised_url).downcase }
.values
.filter { _1.size > 1 }
.flatten
.map { { url: _1.fetch(:sanitised_url),
post_id: _1.fetch(:post_id),
original_url: _1.fetch(:original_url) } }
end
def build_persisted_conflicts(rewrites)
return [] if rewrites.blank?
target_rows = rewrites.filter { _1.fetch(:original_url) != _1.fetch(:sanitised_url) }
target_keys = target_rows.map { _1.fetch(:sanitised_url).downcase }.uniq
return [] if target_keys.empty?
target_pairs = target_rows.to_h do |row|
[row.fetch(:post_id), row.fetch(:sanitised_url).downcase]
end
persisted_rows = Post.order(:id)
.where('LOWER(url) IN (?)', target_keys)
.pluck(:id, :url)
.reject { |post_id, original_url| target_pairs[post_id] == original_url.downcase }
.map { |post_id, original_url|
{ url: original_url,
post_id:,
original_url:,
conflict_key: original_url.downcase }
}
target_conflicts = target_rows.map { { url: _1.fetch(:sanitised_url),
post_id: _1.fetch(:post_id),
original_url: _1.fetch(:original_url),
conflict_key: _1.fetch(:sanitised_url).downcase } }
(persisted_rows + target_conflicts)
.group_by { _1.fetch(:conflict_key) }
.values
.filter { _1.size > 1 }
.flatten
.map { _1.except(:conflict_key) }
end
end
private
def source_pattern_must_be_regexp
return if source_pattern.blank?
Regexp.new(source_pattern)
rescue RegexpError
errors.add :source_pattern, '変な正規表現だね〜(笑)'
end
end
+1
ファイルの表示
@@ -5,6 +5,7 @@ class PostVersion < ApplicationRecord
belongs_to :parent, class_name: 'Post', optional: true belongs_to :parent, class_name: 'Post', optional: true
validates :url, presence: true validates :url, presence: true
validates :video_ms, numericality: { only_integer: true, greater_than: 0 }, allow_nil: true
validate :validate_original_created_range validate :validate_original_created_range
+47 -2
ファイルの表示
@@ -1,7 +1,52 @@
class Setting < ApplicationRecord class Setting < ApplicationRecord
THEMES = ['system', 'light', 'dark'].freeze
# These remain in the typed settings schema to preserve existing backend
# work. The common `/users/settings` page currently surfaces only `theme`.
AUTO_FETCH_MODES = ['auto', 'manual', 'off'].freeze
WIKI_EDITOR_MODES = ['split', 'write', 'preview'].freeze
STRING_ATTRIBUTES = [
'theme',
'auto_fetch_title',
'auto_fetch_thumbnail',
'wiki_editor_mode',
].freeze
INTEGER_ATTRIBUTES = [].freeze
BOOLEAN_ATTRIBUTES = [].freeze
EDITABLE_ATTRIBUTES =
(STRING_ATTRIBUTES + INTEGER_ATTRIBUTES + BOOLEAN_ATTRIBUTES).freeze
TYPE_BY_ATTRIBUTE = {
'theme' => :string,
'auto_fetch_title' => :string,
'auto_fetch_thumbnail' => :string,
'wiki_editor_mode' => :string,
}.freeze
belongs_to :user belongs_to :user
validates :user_id, presence: true validates :user_id, presence: true
validates :key, presence: true, length: { maximum: 255 } validates :user_id, uniqueness: true
validates :value, presence: true
validates :theme, inclusion: { in: THEMES }
validates :auto_fetch_title, inclusion: { in: AUTO_FETCH_MODES }
validates :auto_fetch_thumbnail, inclusion: { in: AUTO_FETCH_MODES }
validates :wiki_editor_mode, inclusion: { in: WIKI_EDITOR_MODES }
def self.defaults
{
theme: 'system',
auto_fetch_title: 'manual',
auto_fetch_thumbnail: 'manual',
wiki_editor_mode: 'split',
}
end
def self.serializable_attributes
EDITABLE_ATTRIBUTES.map(&:to_sym)
end
def serializable_hash(options = nil)
super({ only: self.class.serializable_attributes }.merge(options || { }))
end
end end
+112 -3
ファイルの表示
@@ -17,6 +17,16 @@ class Tag < ApplicationRecord
end end
end end
class SectionLiteralParseError < ArgumentError
attr_reader :tag_name, :literal
def initialize tag_name, literal
@tag_name = tag_name
@literal = literal
super("invalid section literal for tag #{ tag_name }: #{ literal }")
end
end
has_many :post_tags, inverse_of: :tag has_many :post_tags, inverse_of: :tag
has_many :active_post_tags, -> { kept }, class_name: 'PostTag', inverse_of: :tag has_many :active_post_tags, -> { kept }, class_name: 'PostTag', inverse_of: :tag
has_many :post_tags_with_discarded, -> { with_discarded }, class_name: 'PostTag' has_many :post_tags_with_discarded, -> { with_discarded }, class_name: 'PostTag'
@@ -105,26 +115,60 @@ class Tag < ApplicationRecord
def self.normalise_tags! tag_names, with_tagme: true, def self.normalise_tags! tag_names, with_tagme: true,
with_no_deerjikist: true, with_no_deerjikist: true,
deny_nico: true, deny_nico: true,
deny_deprecated: false deny_deprecated: false,
with_sections: false
if deny_nico && tag_names.any? { |n| n.downcase.start_with?('nico:') } if deny_nico && tag_names.any? { |n| n.downcase.start_with?('nico:') }
raise NicoTagNormalisationError raise NicoTagNormalisationError
end end
sections = { }
tags = tag_names.map do |name| tags = tag_names.map do |name|
raw_name = name
pf, cat = CATEGORY_PREFIXES.find { |p, _| name.downcase.start_with?(p) } || ['', nil] pf, cat = CATEGORY_PREFIXES.find { |p, _| name.downcase.start_with?(p) } || ['', nil]
name = TagName.canonicalise(name.sub(/\A#{ pf }/i, '')).first
name = name.sub(/\A#{ pf }/i, '')
sections_by_tag = []
while (match = name.match(/\A(\S*?)\[([^\[\]\s]*)-([^\[\]\s]*)\](\S*)\z/))
name = "#{ match[1] }#{ match[4] }"
next if match[2].empty? && match[3].empty?
sections_by_tag << normalise_section_range!(
begin_raw: match[2],
end_raw: match[3],
tag_name: name)
end
if name.include?('[') || name.include?(']')
raise SectionLiteralParseError.new(raw_name, raw_name)
end
name = TagName.canonicalise(name).first
find_or_create_by_tag_name!(name, category: (cat || :general)).tap do |tag| find_or_create_by_tag_name!(name, category: (cat || :general)).tap do |tag|
if deny_deprecated && tag.deprecated? if deny_deprecated && tag.deprecated?
raise DeprecatedTagNormalisationError, [tag.name] raise DeprecatedTagNormalisationError, [tag.name]
end end
tag.update!(category: cat) if cat && tag.category != cat tag.update!(category: cat) if cat && tag.category != cat
next if sections_by_tag.empty?
sections[tag.id] ||= []
sections[tag.id].concat(sections_by_tag)
sections[tag.id] = merge_section_ranges(sections[tag.id])
sections.delete(tag.id) if sections[tag.id] == [[0, nil]]
end end
end end
tags << Tag.tagme if with_tagme && tags.size < 10 && tags.none?(Tag.tagme) tags << Tag.tagme if with_tagme && tags.size < 10 && tags.none?(Tag.tagme)
tags << Tag.no_deerjikist if with_no_deerjikist && tags.all? { |t| !(t.deerjikist?) } tags << Tag.no_deerjikist if with_no_deerjikist && tags.all? { |t| !(t.deerjikist?) }
tags.uniq(&:id) tags.uniq!(&:id)
if with_sections
{ tags:, sections: }
else
tags
end
end end
def self.expand_parent_tags tags def self.expand_parent_tags tags
@@ -150,6 +194,45 @@ class Tag < ApplicationRecord
(result + tags).uniq { |t| t.id } (result + tags).uniq { |t| t.id }
end end
def self.normalise_section_range! begin_raw:, end_raw:, tag_name:
begin_ms = begin_raw.empty? ? 0 : time_to_ms!(begin_raw, tag_name:)
end_ms = end_raw.empty? ? nil : time_to_ms!(end_raw, tag_name:)
if end_ms
begin_ms, end_ms = end_ms, begin_ms if begin_ms > end_ms
end_ms = begin_ms + 1 if begin_ms == end_ms
end
[begin_ms, end_ms]
end
def self.merge_section_ranges ranges
sorted_ranges = ranges.sort_by { |begin_ms, end_ms| [begin_ms, end_ms || Float::INFINITY] }
merged = []
sorted_ranges.each do |begin_ms, end_ms|
if merged.empty?
merged << [begin_ms, end_ms]
next
end
last_begin_ms, last_end_ms = merged[-1]
if last_end_ms.nil? || begin_ms <= last_end_ms
merged[-1] = [last_begin_ms, merge_section_end(last_end_ms, end_ms)]
else
merged << [begin_ms, end_ms]
end
end
merged
end
def self.merge_section_end left_end_ms, right_end_ms
return nil if left_end_ms.nil? || right_end_ms.nil?
[left_end_ms, right_end_ms].max
end
def self.find_or_create_by_tag_name! name, category: def self.find_or_create_by_tag_name! name, category:
tn = TagName.find_undiscard_or_create_by!(name: name.to_s.strip) tn = TagName.find_undiscard_or_create_by!(name: name.to_s.strip)
tn = tn.canonical if tn.canonical_id? tn = tn.canonical if tn.canonical_id?
@@ -246,6 +329,32 @@ class Tag < ApplicationRecord
end end
end end
def self.time_to_ms! str, tag_name:
match =
case str
when /\A(?<seconds>\d+)(?:\.(?<ms>\d{1,3}))?\z/
{ hours: nil, minutes: nil, seconds: Regexp.last_match[:seconds],
ms: Regexp.last_match[:ms] }
when /\A(?<minutes>\d+):(?<seconds>[0-5]?\d)(?:\.(?<ms>\d{1,3}))?\z/
{ hours: nil, minutes: Regexp.last_match[:minutes],
seconds: Regexp.last_match[:seconds],
ms: Regexp.last_match[:ms] }
when /\A(?<hours>\d+):(?<minutes>[0-5]?\d):(?<seconds>[0-5]?\d)(?:\.(?<ms>\d{1,3}))?\z/
{ hours: Regexp.last_match[:hours],
minutes: Regexp.last_match[:minutes],
seconds: Regexp.last_match[:seconds],
ms: Regexp.last_match[:ms] }
end
raise SectionLiteralParseError.new(tag_name, str) unless match
total_s = match[:seconds].to_i
total_s += match[:minutes].to_i * 60 if match[:minutes]
total_s += match[:hours].to_i * 3_600 if match[:hours]
total_s * 1_000 + match[:ms].to_s.ljust(3, '0')[0, 3].to_i
end
def nico_tags_cannot_be_deprecated def nico_tags_cannot_be_deprecated
if nico? && deprecated_at.present? if nico? && deprecated_at.present?
errors.add :deprecated_at, 'ニコタグは廃止できません.' errors.add :deprecated_at, 'ニコタグは廃止できません.'
+24
ファイルの表示
@@ -6,6 +6,7 @@ class TagImplication < ApplicationRecord
validates :parent_tag_id, presence: true validates :parent_tag_id, presence: true
validate :parent_tag_mustnt_be_itself validate :parent_tag_mustnt_be_itself
validate :parent_tag_mustnt_create_cycle
private private
@@ -14,4 +15,27 @@ class TagImplication < ApplicationRecord
errors.add :parent_tag_id, '親タグは子タグと同一であってはなりません.' errors.add :parent_tag_id, '親タグは子タグと同一であってはなりません.'
end end
end end
def parent_tag_mustnt_create_cycle
return if tag_id.blank? || parent_tag_id.blank?
return if errors[:parent_tag_id].present?
seen = { }
stack = [parent_tag_id]
until stack.empty?
current_id = stack.pop
next if seen[current_id]
seen[current_id] = true
if current_id == tag_id
errors.add :parent_tag_id, '親タグに子孫タグを指定すると循環します.'
errors.add :base, 'タグの親子関係が循環します.'
return
end
stack.concat(TagImplication.where(tag_id: current_id).pluck(:parent_tag_id))
end
end
end end
+2 -1
ファイルの表示
@@ -7,7 +7,8 @@ class User < ApplicationRecord
has_many :created_posts, has_many :created_posts,
class_name: 'Post', foreign_key: :uploaded_user_id, dependent: :nullify class_name: 'Post', foreign_key: :uploaded_user_id, dependent: :nullify
has_many :settings has_one :setting, dependent: :destroy
has_many :theme_slots, class_name: 'UserThemeSlot', dependent: :destroy
has_many :user_ips, dependent: :destroy has_many :user_ips, dependent: :destroy
has_many :ip_addresses, through: :user_ips has_many :ip_addresses, through: :user_ips
has_many :user_post_views, dependent: :destroy has_many :user_post_views, dependent: :destroy
+27
ファイルの表示
@@ -0,0 +1,27 @@
class UserThemeSlot < ApplicationRecord
BASE_THEMES = ['light', 'dark'].freeze
SLOT_NOS = [1, 2, 3].freeze
belongs_to :user
validates :user_id, presence: true
validates :base_theme, presence: true, inclusion: { in: BASE_THEMES }
validates :slot_no, presence: true, inclusion: { in: SLOT_NOS }
validates :tokens, presence: true
validates :user_id, uniqueness: { scope: [:base_theme, :slot_no] }
validate :tokens_must_be_object
def serializable_hash(options = nil)
hash = { only: [:base_theme, :slot_no, :tokens, :created_at, :updated_at] }
super(hash.merge(options || {}))
end
private
def tokens_must_be_object
return if tokens.is_a?(Hash)
errors.add(:tokens, 'は JSON object で指定してください.')
end
end
+12 -4
ファイルの表示
@@ -1,15 +1,23 @@
module VersionRecord module VersionRecord
extend ActiveSupport::Concern extend ActiveSupport::Concern
DEFAULT_EVENT_TYPE_MAP = { create: 'create',
update: 'update',
discard: 'discard',
restore: 'restore' }.freeze
def readonly? = persisted? def readonly? = persisted?
included do included do
event_type_map = if const_defined?(:EVENT_TYPE_MAP, false)
const_get(:EVENT_TYPE_MAP)
else
DEFAULT_EVENT_TYPE_MAP
end
belongs_to :created_by_user, class_name: 'User', optional: true belongs_to :created_by_user, class_name: 'User', optional: true
enum :event_type, { create: 'create', enum :event_type, event_type_map, prefix: true, validate: true
update: 'update',
discard: 'discard',
restore: 'restore' }, prefix: true, validate: true
validates :version_no, presence: true, numericality: { only_integer: true, greater_than: 0 } validates :version_no, presence: true, numericality: { only_integer: true, greater_than: 0 }
validates :event_type, presence: true validates :event_type, presence: true
+87 -2
ファイルの表示
@@ -2,7 +2,9 @@
module MaterialRepr module MaterialRepr
BASE = { only: [:id, :url, :created_at, :updated_at], BASE = { only: [:id, :url, :version_no, :source_kind, :source_uri,
:source_path, :source_file_id, :normalized_source_key,
:created_at, :updated_at],
methods: [:content_type], methods: [:content_type],
include: { tag: TagRepr::BASE, include: { tag: TagRepr::BASE,
created_by_user: UserRepr::BASE, created_by_user: UserRepr::BASE,
@@ -15,10 +17,93 @@ module MaterialRepr
file: if material.file.attached? file: if material.file.attached?
Rails.application.routes.url_helpers.rails_storage_proxy_url( Rails.application.routes.url_helpers.rails_storage_proxy_url(
material.file, host:) material.file, host:)
end) end,
thumbnail: thumbnail_url(material, host:),
thumbnail_fallback_text: thumbnail_fallback_text(material),
thumbnail_fallback_kind: thumbnail_fallback_kind(material),
media_kind: media_kind(material),
file_byte_size: material.file_byte_size,
export_paths: export_paths(material),
export_items: export_items(material))
end end
def many materials, host: def many materials, host:
materials.map { |m| base(m, host:) } materials.map { |m| base(m, host:) }
end end
def list material, host:
{ id: material.id,
version_no: material.version_no,
url: material.url,
source_kind: material.source_kind,
source_uri: material.source_uri,
source_path: material.source_path,
source_file_id: material.source_file_id,
normalized_source_key: material.normalized_source_key,
tag: compact_tag(material.tag),
thumbnail: thumbnail_url(material, host:),
thumbnail_fallback_text: thumbnail_fallback_text(material),
thumbnail_fallback_kind: thumbnail_fallback_kind(material),
media_kind: media_kind(material),
content_type: material.content_type,
file_byte_size: material.file_byte_size,
created_at: material.created_at,
updated_at: material.updated_at,
export_paths: export_paths(material),
export_items: export_items(material) }
end
def list_many materials, host:
materials.map { |m| list(m, host:) }
end
def export_paths material
material.material_export_items.each_with_object({ }) do |item, hash|
hash[item.profile] = item.enabled ? item.export_path : ''
end
end
def export_items material
material.material_export_items.map do |item|
{ id: item.id,
profile: item.profile,
export_path: item.export_path,
enabled: item.enabled }
end
end
def thumbnail_url material, host:
return nil unless material.thumbnail.attached?
Rails.application.routes.url_helpers.rails_storage_proxy_url(
material.thumbnail, host:)
end
def thumbnail_fallback_text material
material.tag&.name || material.created_at&.strftime('%Y-%m-%d')
end
def thumbnail_fallback_kind material
material.tag.present? ? 'tag_name' : 'created_at'
end
def media_kind material
return 'url_only' unless material.file.attached?
content_type = material.file.blob.content_type.to_s
return 'image' if content_type.start_with?('image/')
return 'video' if content_type.start_with?('video/')
return 'audio' if content_type.start_with?('audio/')
'file_other'
end
def compact_tag tag
return nil unless tag
{ id: tag.id,
name: tag.name,
category: tag.category,
deprecated_at: tag.deprecated_at }
end
end end
+12 -3
ファイルの表示
@@ -8,6 +8,7 @@ module PostRepr
:url, :url,
:title, :title,
:thumbnail_base, :thumbnail_base,
:video_ms,
:original_created_from, :original_created_from,
:original_created_before, :original_created_before,
:created_at, :created_at,
@@ -18,7 +19,7 @@ module PostRepr
def base post, current_user = nil def base post, current_user = nil
json = common(post) json = common(post)
json['tags'] = tag_json(post.tags) json['tags'] = tag_json(post)
json['uploaded_user'] = post.uploaded_user && UserRepr.base(post.uploaded_user) json['uploaded_user'] = post.uploaded_user && UserRepr.base(post.uploaded_user)
json['viewed'] = current_user ? current_user.viewed?(post) : false json['viewed'] = current_user ? current_user.viewed?(post) : false
json json
@@ -52,8 +53,16 @@ module PostRepr
.merge('thumbnail' => thumbnail_url(post)) .merge('thumbnail' => thumbnail_url(post))
end end
def tag_json tags def tag_json post
tags.reject(&:deprecated?).map { |tag| TagRepr.inline(tag) } post
.active_post_tags
.reject { _1.tag.deprecated? }
.sort_by { _1.tag.name }
.map { |post_tag|
TagRepr.inline(post_tag.tag).merge(
'children' => [],
'sections' => post_tag.sections.as_json(only: [:begin_ms, :end_ms]))
}
end end
def thumbnail_url post def thumbnail_url post
+233
ファイルの表示
@@ -0,0 +1,233 @@
require 'json'
require 'jwt'
require 'net/http'
require 'openssl'
require 'tempfile'
require 'uri'
module GoogleDrive
class ApiClient
DRIVE_ENDPOINT = 'https://www.googleapis.com/drive/v3'
TOKEN_ENDPOINT = 'https://oauth2.googleapis.com/token'
FOLDER_MIME_TYPE = 'application/vnd.google-apps.folder'
NATIVE_FILE_MIME_TYPE_PREFIX = 'application/vnd.google-apps.'
DRIVE_SCOPE = 'https://www.googleapis.com/auth/drive.readonly'
def initialize service_account_email: ENV['GOOGLE_DRIVE_SERVICE_ACCOUNT_EMAIL'],
private_key: ENV['GOOGLE_DRIVE_PRIVATE_KEY'],
private_key_path: ENV['GOOGLE_DRIVE_PRIVATE_KEY_PATH'],
subject: ENV['GOOGLE_DRIVE_SUBJECT']
@service_account_email = service_account_email.to_s
@private_key = load_private_key(private_key, private_key_path)
@subject = subject.to_s.presence
@access_token = nil
end
def list_material_files_under_folder folder_id
files = []
each_material_file_under_folder(folder_id) { |entry| files << entry }
files
end
def each_material_file_under_folder folder_id
return enum_for(__method__, folder_id) unless block_given?
walk_folder(folder_id, nil) do |entry, relative_path|
next if entry['mimeType'] == FOLDER_MIME_TYPE
next if native_file?(entry['mimeType'])
yield build_file_entry(entry, relative_path)
end
end
def fetch_material_file file_id
metadata = get_file(file_id)
return nil if native_file?(metadata['mimeType'])
build_file_entry(metadata, metadata['name'])
end
def download_to_tempfile file_id, filename:
tempfile = Tempfile.new(['material-sync', File.extname(filename.to_s)])
tempfile.binmode
request_binary("/files/#{ file_id }",
{ alt: 'media', supportsAllDrives: true }) do |chunk|
tempfile.write(chunk.b)
end
tempfile.rewind
tempfile
rescue StandardError
tempfile&.close!
raise
end
def extract_file_id value
raw = value.to_s.strip
return nil if raw.blank?
return raw unless raw.include?('/')
uri = URI.parse(raw)
return uri.query.to_s[%r{(?:^|&)id=([^&]+)}, 1] if uri.query.present?
return uri.path[%r{/folders/([^/]+)}, 1] if uri.path.include?('/folders/')
uri.path[%r{/d/([^/]+)}, 1]
rescue URI::InvalidURIError
nil
end
private
def walk_folder folder_id, prefix, &block
list_children(folder_id).each do |entry|
relative_path = MaterialSyncExportPath.build(prefix:,
relative_path: entry['name'])
if entry['mimeType'] == FOLDER_MIME_TYPE
walk_folder(entry['id'], relative_path, &block)
next
end
block.call(entry, relative_path)
end
end
def build_file_entry entry, relative_path
{ id: entry['id'],
name: entry['name'],
mime_type: entry['mimeType'],
relative_path: MaterialSyncExportPath.normalize_path(relative_path),
sha256_checksum: entry['sha256Checksum'],
web_view_link: entry['webViewLink'],
web_content_link: entry['webContentLink'] }
end
def list_children folder_id
files = []
page_token = nil
loop do
response =
request_json('/files', {
q: "'#{ folder_id }' in parents and trashed = false",
fields: 'nextPageToken,files(id,name,mimeType,sha256Checksum,' \
'webViewLink,webContentLink)',
orderBy: 'folder,name',
pageSize: 1000,
supportsAllDrives: true,
includeItemsFromAllDrives: true,
pageToken: page_token }.compact)
files.concat(response.fetch('files'))
page_token = response['nextPageToken']
break if page_token.blank?
end
files
end
def get_file file_id
request_json("/files/#{ file_id }",
fields: 'id,name,mimeType,sha256Checksum,webViewLink,webContentLink',
supportsAllDrives: true)
end
def request_json path, params = {}
response = request(:get, path, params:)
unless response.is_a?(Net::HTTPSuccess)
raise "Google Drive API error: #{ response.code } #{ response.body }"
end
JSON.parse(response.body)
end
def request_binary path, params = {}
request(:get, path, params:) do |response|
unless response.is_a?(Net::HTTPSuccess)
raise "Google Drive download error: #{ response.code } #{ response.body }"
end
response.read_body do |chunk|
yield chunk.b
end
end
end
def request method, path, params: nil
uri = URI(DRIVE_ENDPOINT + path)
uri.query = URI.encode_www_form(params) if params.present?
klass =
case method
when :get
Net::HTTP::Get
else
raise ArgumentError, "Unsupported Google Drive request method: #{ method }"
end
request = klass.new(uri)
request['Authorization'] = "Bearer #{ access_token }"
Net::HTTP.start(uri.host,
uri.port,
use_ssl: true,
open_timeout: 10,
read_timeout: 60) do |http|
if block_given?
http.request(request) do |response|
return yield response
end
else
http.request(request)
end
end
end
def access_token
@access_token ||= fetch_access_token
end
def fetch_access_token
raise 'GOOGLE_DRIVE_SERVICE_ACCOUNT_EMAIL is required' if @service_account_email.blank?
raise 'Google Drive private key is required' if @private_key.blank?
payload = { iss: @service_account_email,
scope: DRIVE_SCOPE,
aud: TOKEN_ENDPOINT,
exp: 1.hour.from_now.to_i,
iat: Time.current.to_i }
payload[:sub] = @subject if @subject.present?
assertion = JWT.encode(payload, @private_key, 'RS256')
uri = URI(TOKEN_ENDPOINT)
request = Net::HTTP::Post.new(uri)
request.set_form_data(
grant_type: 'urn:ietf:params:oauth:grant-type:jwt-bearer',
assertion:)
response =
Net::HTTP.start(uri.host,
uri.port,
use_ssl: true,
open_timeout: 10,
read_timeout: 30) do |http|
http.request(request)
end
unless response.is_a?(Net::HTTPSuccess)
raise "Google OAuth error: #{ response.code } #{ response.body }"
end
JSON.parse(response.body).fetch('access_token')
end
def load_private_key private_key, private_key_path
raw = private_key.to_s
raw = raw.gsub('\n', "\n") if raw.present?
raw = File.read(private_key_path) if raw.blank? && private_key_path.present?
return nil if raw.blank?
OpenSSL::PKey::RSA.new(raw)
end
def native_file? mime_type
mime_type.to_s.start_with?(NATIVE_FILE_MIME_TYPE_PREFIX)
end
end
end
+57
ファイルの表示
@@ -0,0 +1,57 @@
require 'digest'
require 'json'
class MaterialFileSha256
def self.blob_metadata blob
metadata = blob.metadata
return metadata if metadata.is_a?(Hash)
return { } if metadata.blank?
JSON.parse(metadata)
rescue JSON::ParserError
{ }
end
def self.metadata_sha256 blob
blob_metadata(blob)['sha256'].presence
end
def self.assign_metadata_sha256! blob, sha256
return if sha256.blank?
metadata = blob_metadata(blob)
metadata['sha256'] = sha256
blob.metadata = metadata
blob.save! if blob.changed?
end
def self.from_blob blob, allow_download: false
sha256 = metadata_sha256(blob)
return sha256 if sha256.present?
return nil unless allow_download
begin
blob.open do |file|
sha256 = Digest::SHA256.file(file.path).hexdigest
assign_metadata_sha256!(blob, sha256)
sha256
end
rescue ActiveStorage::FileNotFoundError, ArgumentError => error
Rails.logger.warn(
"MaterialFileSha256.from_blob failed for blob_id=#{blob.id}: " \
"#{error.class}: #{error.message}",
)
nil
end
end
def self.from_upload upload
tempfile = upload&.tempfile
return nil unless tempfile
tempfile.rewind
Digest::SHA256.file(tempfile.path).hexdigest.tap do
tempfile.rewind
end
end
end
+7
ファイルの表示
@@ -0,0 +1,7 @@
class MaterialImportBlockMatcher
def self.match_for_sha256 sha256
return nil if sha256.blank?
MaterialImportBlock.find_by(match_kind: 'sha256', sha256:)
end
end
+56
ファイルの表示
@@ -0,0 +1,56 @@
class MaterialSyncExportPath
class << self
def build prefix: nil, relative_path: nil, filename: nil, source_file_id: nil
path = normalize_path(relative_path)
path = fallback_filename(filename, source_file_id) if path.blank?
normalized_prefix = normalize_path(prefix)
return path if normalized_prefix.blank?
return normalized_prefix if path.blank?
[normalized_prefix, path].join('/')
end
def normalize_path path
raw = path.to_s.delete("\0").tr('\\', '/').strip
raw = raw.sub(%r{\A[A-Za-z]:/+}, '')
raw = raw.sub(%r{\A/+}, '')
return nil if raw.blank?
segments = raw.split('/').filter_map do |segment|
normalized = normalize_segment(segment)
normalized.presence
end
return nil if segments.empty?
segments.join('/')
end
def uniquify path, source_file_id
normalized = normalize_path(path)
return normalized if normalized.blank? || source_file_id.blank?
ext = File.extname(normalized)
base = ext.present? ? normalized.delete_suffix(ext) : normalized
"#{ base }--#{ source_file_id }#{ ext }"
end
private
def fallback_filename filename, source_file_id
name = normalize_path(filename)
return name if name.present?
return nil if source_file_id.blank?
source_file_id.to_s
end
def normalize_segment segment
cleaned = segment.to_s.delete("\0").tr('\\/', '_').strip
return nil if cleaned.blank? || cleaned == '.'
return '__' if cleaned == '..'
cleaned
end
end
end
+259
ファイルの表示
@@ -0,0 +1,259 @@
require 'digest'
class MaterialSyncImporter
Result = Struct.new(:material, :action, :suppressed, :suppression, keyword_init: true)
def self.import! attributes
new(attributes).import!
end
def initialize attributes
@attributes = attributes
end
def import!
source = source_attributes
key = normalized_source_key(source)
raise ArgumentError, 'normalized_source_key is required for material sync' if key.blank?
suppression = suppression_for(source)
if suppression
return Result.new(material: nil,
action: :suppressed,
suppressed: true,
suppression:)
end
material =
Material
.unscoped
.find_or_initialize_by(normalized_source_key: key)
if unchanged?(material, source)
return Result.new(material:,
action: :unchanged,
suppressed: false,
suppression: nil)
end
event_type =
if material.new_record?
:create
elsif material.discarded?
:restore
else
:update
end
uploaded_blob = uploaded_blob!
if @import_block
return Result.new(material: nil,
action: :suppressed,
suppressed: true,
suppression: @import_block)
end
Material.transaction do
if material.persisted?
MaterialVersionRecorder.ensure_snapshot!(material,
created_by_user: @attributes[:updated_by_user])
end
material.assign_attributes(material_attributes_for(material).merge(source))
material.discarded_at = nil if material.respond_to?(:discarded_at=)
material.file.attach(uploaded_blob) if uploaded_blob
material.save!
upsert_export_item!(material)
MaterialVersionRecorder.record!(material:,
event_type:,
created_by_user: @attributes[:updated_by_user])
end
MaterialThumbnailGenerator.generate!(material)
Result.new(material:,
action: event_type == :create ? :imported : :updated,
suppressed: false,
suppression: nil)
rescue StandardError
uploaded_blob&.purge_later
raise
ensure
close_tempfile!
end
private
def source_attributes
{ source_kind: @attributes[:source_kind],
source_uri: @attributes[:source_uri],
source_path: @attributes[:source_path],
source_file_id: @attributes[:source_file_id] }
end
def normalized_source_key source
MaterialSyncSuppression.normalize_source_key(source_kind: source[:source_kind],
source_uri: source[:source_uri],
source_path: source[:source_path],
source_file_id: source[:source_file_id])
end
def material_attributes_for material
attrs =
@attributes
.except(:source_kind, :source_uri, :source_path, :source_file_id,
:file_blob, :file_tempfile, :file_downloader,
:filename, :content_type,
:file_sha256, :export_path, :profile, :tag, :url)
if material.new_record?
attrs[:tag] = @attributes[:tag] if @attributes.key?(:tag)
attrs[:url] = @attributes[:url] if @attributes.key?(:url)
return attrs
end
if @attributes.key?(:tag) && @attributes[:tag].present?
attrs[:tag] = @attributes[:tag]
end
if @attributes.key?(:url) && @attributes[:url].present?
attrs[:url] = @attributes[:url]
end
attrs
end
def suppression_for source
if source[:source_kind] == 'google_drive_file' && source[:source_path].present?
return MaterialSyncSuppressionMatcher.match_google_drive_candidate(
drive_file_id: source[:source_file_id],
relative_path: source[:source_path])
end
MaterialSyncSuppressionMatcher.match(**source)
end
def uploaded_blob!
return @attributes[:file_blob] if @attributes[:file_blob]
tempfile = upload_tempfile
return nil unless tempfile
tempfile.rewind
file_sha256 = @attributes[:file_sha256] || Digest::SHA256.file(tempfile.path).hexdigest
@attributes[:file_sha256] = file_sha256
@import_block = MaterialImportBlockMatcher.match_for_sha256(file_sha256)
if @import_block
tempfile.rewind
return nil
end
blob = ActiveStorage::Blob.create_and_upload!(
io: tempfile,
filename: @attributes[:filename],
content_type: @attributes[:content_type])
MaterialFileSha256.assign_metadata_sha256!(blob, file_sha256)
tempfile.rewind
blob
end
def upload_tempfile
return @upload_tempfile if defined?(@upload_tempfile)
@upload_tempfile = @attributes[:file_tempfile]
if @upload_tempfile.blank? && @attributes[:file_downloader]
@upload_tempfile = @attributes[:file_downloader].call
end
@upload_tempfile
end
def upsert_export_item! material
export_path = resolved_export_path(material)
return if export_path.blank?
item = material.material_export_items.find_or_initialize_by(profile: effective_profile)
item.export_path = export_path
item.enabled = true
item.created_by_user ||= @attributes[:created_by_user]
item.save!
end
def export_path_taken_by_other? material, profile, export_path
MaterialExportItem
.where(profile:, export_path:)
.where.not(material_id: material.id)
.exists?
end
def close_tempfile!
tempfile = upload_tempfile
return unless tempfile
tempfile.close! unless tempfile.closed?
rescue StandardError
nil
end
def unchanged? material, source
return false if material.new_record? || material.discarded?
return false unless same_source_attributes?(material, source)
return false unless same_export_path?(material)
return false unless same_material_attributes?(material)
same_file_snapshot?(material)
end
def same_source_attributes? material, source
material.source_kind == source[:source_kind] \
&& material.source_uri == source[:source_uri] \
&& material.source_path == source[:source_path] \
&& material.source_file_id == source[:source_file_id]
end
def same_export_path? material
expected = resolved_export_path(material)
current = material.material_export_items.find { |item| item.profile == effective_profile }
return current.blank? if expected.blank?
current&.enabled && current.export_path == expected
end
def same_material_attributes? material
same_tag_attribute?(material) && same_url_attribute?(material)
end
def same_file_snapshot? material
expected_sha256 = @attributes[:file_sha256].to_s.presence
if expected_sha256.present?
return false unless material.file.attached?
return MaterialFileSha256.metadata_sha256(material.file.blob) == expected_sha256
end
!material.file.attached?
end
def effective_profile
@attributes[:profile].presence || 'legacy_drive'
end
def resolved_export_path material
export_path = @attributes[:export_path].to_s.strip
return export_path if export_path.blank?
return export_path unless export_path_taken_by_other?(material, effective_profile, export_path)
MaterialSyncExportPath.uniquify(export_path, @attributes[:source_file_id])
end
def same_tag_attribute? material
return true unless @attributes.key?(:tag)
return true if @attributes[:tag].blank?
material.tag_id == @attributes[:tag].id
end
def same_url_attribute? material
return true unless @attributes.key?(:url)
return true if @attributes[:url].blank?
material.url == @attributes[:url]
end
end
+251
ファイルの表示
@@ -0,0 +1,251 @@
class MaterialSyncRunner
Result = Struct.new(:imported, :updated, :unchanged, :suppressed, :failed,
:errors, keyword_init: true)
def self.sync_enabled!
results = MaterialSyncSource.enabled.order(:id).map { |source| new(source).sync! }
Result.new(imported: results.sum(&:imported),
updated: results.sum(&:updated),
unchanged: results.sum(&:unchanged),
suppressed: results.sum(&:suppressed),
failed: results.sum(&:failed),
errors: results.flat_map(&:errors))
end
def initialize source
@source = source
end
def sync!
result = Result.new(imported: 0, updated: 0, unchanged: 0,
suppressed: 0, failed: 0, errors: [])
if @source.source_kind == 'google_drive_path'
sync_google_drive_path!(result)
else
candidates.each do |candidate|
next if candidate.blank?
sync_candidate!(candidate, result)
end
end
@source.update!(last_synced_at: Time.current)
log_result(result)
result
rescue NotImplementedError, StandardError => e
result.failed += 1
result.errors << { source_id: @source.id, error: e.message }
log_result(result)
result
end
private
def candidates
case @source.source_kind
when 'uri'
[uri_candidate]
when 'google_drive_file'
[google_drive_file_candidate]
when 'legacy_drive_path'
raise NotImplementedError, 'legacy_drive_path material sync is not implemented'
else
raise NotImplementedError, "Unsupported material sync source_kind: #{ @source.source_kind }"
end
end
def uri_candidate
{ source_kind: @source.source_kind,
source_uri: @source.source_uri,
source_path: @source.source_path,
source_file_id: @source.source_file_id,
url: @source.source_uri,
tag: nil,
created_by_user: @source.created_by_user,
updated_by_user: @source.updated_by_user || @source.created_by_user }
end
def sync_candidate! candidate, result
block = MaterialImportBlockMatcher.match_for_sha256(candidate[:file_sha256])
if block
result.suppressed += 1
Rails.logger.info(
material_sync_log(action: 'suppressed',
reason: block.reason,
normalized_source_key: candidate_normalized_source_key(candidate)))
return
end
suppression = suppression_for(candidate)
if suppression
result.suppressed += 1
Rails.logger.info(
material_sync_log(action: 'suppressed',
normalized_source_key: suppression.normalized_source_key))
return
end
import = MaterialSyncImporter.import!(candidate)
result.public_send("#{ import.action }=", result.public_send(import.action) + 1)
rescue StandardError => e
result.failed += 1
result.errors << { source_id: @source.id,
normalized_source_key: candidate_normalized_source_key(candidate),
error: e.message }
Rails.logger.warn(
material_sync_log(action: 'failed',
error: e.message,
normalized_source_key: candidate_normalized_source_key(candidate)))
ensure
close_candidate_tempfile(candidate)
end
def google_drive_path_candidates
folder_id = google_drive_folder_id
Enumerator.new do |entries|
drive_client.each_material_file_under_folder(folder_id).each do |entry|
entries << build_google_drive_candidate(entry)
end
end
end
def sync_google_drive_path! result
folder_id = google_drive_folder_id
scanned_count = 0
drive_client.each_material_file_under_folder(folder_id) do |entry|
scanned_count += 1
sync_candidate!(build_google_drive_candidate(entry), result)
log_google_drive_progress(folder_id, scanned_count, result) if progress_log_scan_count?(scanned_count)
end
log_google_drive_progress(folder_id, scanned_count, result, summary: true)
end
def google_drive_file_candidate
entry = drive_client.fetch_material_file(google_drive_file_id)
return nil unless entry
build_google_drive_candidate(entry, single_file: true)
end
def build_google_drive_candidate entry, single_file: false
relative_path =
if single_file
nil
else
entry[:relative_path]
end
export_path =
MaterialSyncExportPath.build(prefix: @source.export_path_prefix,
relative_path:,
filename: entry[:name],
source_file_id: entry[:id])
{ source_kind: 'google_drive_file',
source_uri: entry[:web_view_link] || google_drive_file_url(entry[:id]),
source_path: relative_path.presence || MaterialSyncExportPath.normalize_path(entry[:name]),
source_file_id: entry[:id],
filename: entry[:name],
content_type: entry[:mime_type],
file_downloader: lambda {
drive_client.download_to_tempfile(entry[:id], filename: entry[:name])
},
file_sha256: entry[:sha256_checksum],
export_path:,
profile: @source.profile,
tag: nil,
url: nil,
created_by_user: @source.created_by_user,
updated_by_user: @source.updated_by_user || @source.created_by_user }
end
def candidate_normalized_source_key candidate
MaterialSyncSuppression.normalize_source_key(source_kind: candidate[:source_kind],
source_uri: candidate[:source_uri],
source_path: candidate[:source_path],
source_file_id: candidate[:source_file_id])
end
def suppression_for candidate
if candidate[:source_kind] == 'google_drive_file' && candidate[:source_path].present?
return MaterialSyncSuppressionMatcher.match_google_drive_candidate(
drive_file_id: candidate[:source_file_id],
relative_path: candidate[:source_path])
end
MaterialSyncSuppressionMatcher.match(source_kind: candidate[:source_kind],
source_uri: candidate[:source_uri],
source_path: candidate[:source_path],
source_file_id: candidate[:source_file_id])
end
def drive_client
@drive_client ||= GoogleDrive::ApiClient.new
end
def google_drive_folder_id
google_drive_target_id.tap do |id|
if id.blank?
raise ArgumentError, 'google_drive_path source_file_id or source_uri is required'
end
end
end
def google_drive_file_id
google_drive_target_id.tap do |id|
if id.blank?
raise ArgumentError, 'google_drive_file source_file_id or source_uri is required'
end
end
end
def google_drive_target_id
@source.source_file_id.presence || drive_client.extract_file_id(@source.source_uri)
end
def google_drive_file_url file_id
"https://drive.google.com/file/d/#{ file_id }/view"
end
def close_candidate_tempfile candidate
tempfile = candidate[:file_tempfile]
return unless tempfile
tempfile.close! unless tempfile.closed?
rescue StandardError
nil
end
def log_result result
Rails.logger.info(material_sync_log(imported: result.imported,
updated: result.updated,
unchanged: result.unchanged,
suppressed: result.suppressed,
failed: result.failed))
end
def progress_log_scan_count? scanned_count
scanned_count == 1 || (scanned_count % 50).zero?
end
def log_google_drive_progress folder_id, scanned_count, result, summary: false
Rails.logger.info(
material_sync_log(folder_id:,
scanned_count:,
imported: result.imported,
updated: result.updated,
unchanged: result.unchanged,
suppressed: result.suppressed,
failed: result.failed,
progress: summary ? 'summary' : 'scan'))
end
def material_sync_log fields
{ material_sync_source_id: @source.id,
material_sync_source_name: @source.name }.merge(fields).to_json
end
end
+68
ファイルの表示
@@ -0,0 +1,68 @@
class MaterialSyncSuppressionMatcher
def self.match source_kind:,
source_uri: nil,
drive_path: nil,
drive_file_id: nil,
source_path: nil,
source_file_id: nil
key = MaterialSyncSuppression.normalize_source_key(source_kind:,
source_uri:,
drive_path:,
drive_file_id:,
source_path:,
source_file_id:)
return nil if key.blank?
MaterialSyncSuppression.find_by(normalized_source_key: key)
end
def self.match_google_drive_candidate drive_file_id:, relative_path:
normalized_path = validate_relative_path!(relative_path)
exact = [
normalize_key('google_drive_file', drive_file_id),
normalize_key('google_drive_path', normalized_path),
normalize_key('legacy_drive_path', normalized_path),
].compact
suppression =
MaterialSyncSuppression.find_by(normalized_source_key: exact)
return suppression if suppression
MaterialSyncSuppression
.where(source_kind: ['google_drive_path_prefix', 'legacy_drive_path_prefix'])
.where('drive_path = :path OR :path LIKE CONCAT(drive_path, "/%")', path: normalized_path)
.order(:id)
.first
end
def self.suppressed?(...)
match(...).present?
end
def self.validate_relative_path! relative_path
raw = relative_path.to_s.delete("\0").tr('\\', '/').strip
raise ArgumentError, 'relative_path is required' if raw.blank?
raise ArgumentError, 'relative_path must be relative' if raw.start_with?('/')
raise ArgumentError, 'relative_path must be relative' if raw.match?(/\A[A-Za-z]:\//)
if raw.start_with?('My Drive/', 'マイドライブ/')
raise ArgumentError, 'relative_path must be relative to source folder'
end
raise ArgumentError, 'relative_path must not contain //' if raw.include?('//')
raise ArgumentError, 'relative_path must not end with /' if raw.end_with?('/')
parts = raw.split('/')
if parts.any? { |part| part.in?(['.', '..']) }
raise ArgumentError, 'relative_path must not contain dot segments'
end
raw
end
def self.normalize_key source_kind, value
return nil if value.blank?
MaterialSyncSuppression.normalize_source_key(source_kind:, source_path: value,
source_file_id: value)
end
end
+51
ファイルの表示
@@ -0,0 +1,51 @@
class MaterialSyncSuppressionRegistrar
def self.create! attributes, created_by_user:
new(attributes, created_by_user:).create!
end
def initialize attributes, created_by_user:
@attributes = attributes
@created_by_user = created_by_user
end
def create!
suppression = nil
MaterialSyncSuppression.transaction do
suppression = MaterialSyncSuppression.create!(
@attributes.merge(created_by_user: @created_by_user))
discard_existing_materials!(suppression)
end
suppression
end
private
def discard_existing_materials! suppression
matching_materials(suppression).find_each do |material|
MaterialVersionRecorder.ensure_snapshot!(material, created_by_user: @created_by_user)
material.discard!
MaterialVersionRecorder.record!(material:,
event_type: :discard,
created_by_user: @created_by_user)
end
end
def matching_materials suppression
materials = Material.unscoped.kept
case suppression.source_kind
when 'google_drive_path', 'legacy_drive_path'
materials.where(source_path: suppression.drive_path)
when 'google_drive_path_prefix', 'legacy_drive_path_prefix'
path = suppression.drive_path.to_s
materials.where(
'source_path = :path OR source_path LIKE :prefix',
path:,
prefix: "#{ path }/%")
else
materials.where(normalized_source_key: suppression.normalized_source_key)
end
end
end
+126
ファイルの表示
@@ -0,0 +1,126 @@
# frozen_string_literal: true
require 'mini_magick'
require 'open3'
require 'tempfile'
class MaterialThumbnailGenerator
SIZE = '180x180'
class << self
def generate! material
new(material).generate!
end
end
def initialize material
@material = material
@ffmpeg_stderr = []
end
def generate!
return log_result(:no_file) unless @material.file.attached?
return log_result(:unsupported_content_type) unless image? || video?
@material.file.blob.open do |file|
thumbnail = image? ? image_thumbnail(file.path) : video_thumbnail(file.path)
return log_result(:generation_failed) unless thumbnail
return attach_thumbnail(thumbnail)
end
rescue ActiveStorage::FileNotFoundError => e
log_result(:file_not_found, error: e)
rescue MiniMagick::Error => e
log_result(:mini_magick_error, error: e)
rescue ArgumentError, StandardError => e
log_result(:generation_failed, error: e)
end
private
def image? = content_type.start_with?('image/')
def video? = content_type.start_with?('video/')
def content_type
return nil unless @material.file.attached?
@material.file.blob.content_type.to_s
end
def image_thumbnail path
image = MiniMagick::Image.open(path)
image.resize(SIZE)
image.format('jpg')
image
end
def video_thumbnail path
[1, 0].each do |seconds|
tempfile = Tempfile.new(['material-thumbnail', '.jpg'])
tempfile.close
ok = extract_video_frame(path, tempfile.path, seconds)
next unless ok && File.size?(tempfile.path)
return image_thumbnail(tempfile.path)
ensure
tempfile&.unlink
end
nil
end
def extract_video_frame input_path, output_path, seconds
_stdout, stderr, status =
Open3.capture3('ffmpeg',
'-y',
'-ss', seconds.to_s,
'-i', input_path,
'-frames:v', '1',
'-f', 'image2',
output_path)
@ffmpeg_stderr << stderr if stderr.present?
status.success?
rescue Errno::ENOENT => e
@ffmpeg_stderr << "ffmpeg unavailable: #{ e.message }"
false
end
def attach_thumbnail image
blob = nil
File.open(image.path) do |io|
blob = ActiveStorage::Blob.create_and_upload!(
io:,
filename: 'material-thumbnail.jpg',
content_type: 'image/jpeg')
end
@material.thumbnail.attach(blob)
log_result(:attached)
rescue StandardError => e
blob&.purge_later
log_result(:attach_failed, error: e)
end
def log_result result, error: nil
log_payload = { material_id: @material.id,
file_blob_id: file_blob_id,
content_type:,
result:,
error_class: error&.class&.name,
error_message: error&.message,
ffmpeg_stderr: @ffmpeg_stderr.join("\n").presence }
if [:attached, :no_file, :unsupported_content_type].include?(result)
Rails.logger.info("Material thumbnail generation: #{ log_payload.to_json }")
else
Rails.logger.warn("Material thumbnail generation: #{ log_payload.to_json }")
end
result
end
def file_blob_id
return nil unless @material.file.attached?
@material.file.blob.id
end
end
+73
ファイルの表示
@@ -0,0 +1,73 @@
class MaterialVersionRecorder < VersionRecorder
EVENT_TYPES = ['create', 'update', 'discard', 'restore'].freeze
def self.record! material:, event_type:, created_by_user:, file_snapshot: nil
new(material:, event_type:, created_by_user:, file_snapshot:).record!
end
def initialize material:, event_type:, created_by_user:, file_snapshot: nil
@file_snapshot = file_snapshot
super(record: material, event_type:, created_by_user:)
end
def self.ensure_snapshot! material, created_by_user:
return if material.material_versions.exists?
record!(material:, event_type: :create,
created_by_user: material.created_by_user || created_by_user)
end
private
def version_class = MaterialVersion
def version_association = :material_versions
def record_key = :material
def snapshot_attributes
blob = @record.file.attached? ? @record.file.blob : nil
file_snapshot = build_file_snapshot(blob)
{ url: @record.url,
parent: @record.parent,
tag: @record.tag,
tag_name: @record.tag&.name,
tag_category: @record.tag&.category,
source_kind: @record.source_kind,
source_uri: @record.source_uri,
source_path: @record.source_path,
source_file_id: @record.source_file_id,
normalized_source_key: @record.normalized_source_key,
export_paths_json: @record.snapshot_export_paths,
discarded_at: @record.discarded_at,
file_blob_id: file_snapshot[:file_blob_id],
file_filename: file_snapshot[:file_filename],
file_content_type: file_snapshot[:file_content_type],
file_byte_size: file_snapshot[:file_byte_size],
file_checksum: file_snapshot[:file_checksum],
file_sha256: file_snapshot[:file_sha256] }
end
def build_file_snapshot blob
return @file_snapshot if @file_snapshot
return empty_file_snapshot unless blob
{ file_blob_id: blob.id,
file_filename: blob.filename.to_s,
file_content_type: blob.content_type,
file_byte_size: blob.byte_size,
file_checksum: blob.checksum,
file_sha256: MaterialFileSha256.metadata_sha256(blob) }
end
def empty_file_snapshot
{ file_blob_id: nil,
file_filename: nil,
file_content_type: nil,
file_byte_size: nil,
file_checksum: nil,
file_sha256: nil }
end
def event_types = self.class::EVENT_TYPES
end
+148
ファイルの表示
@@ -0,0 +1,148 @@
require 'stringio'
require 'zlib'
# Initial implementation keeps every file payload and the final ZIP in memory.
# Keep this service boundary stable so job/cached export paths can replace it later.
class MaterialZipExporter
Entry = Struct.new(:path, :data, :mtime, keyword_init: true)
MissingFile = Struct.new(:material_id, :export_path, :blob_id, :filename, keyword_init: true)
class EmptyExportError < StandardError; end
class DuplicatePathError < StandardError; end
class MissingFileError < StandardError
attr_reader :missing_files
def initialize missing_files
@missing_files = missing_files
super("Missing files: #{missing_files.map(&:export_path).join(', ')}")
end
end
def initialize profile: 'legacy_drive', tag_id: nil
@profile = profile.presence || 'legacy_drive'
@tag_id = tag_id.presence
end
def export
entries = build_entries
raise EmptyExportError if entries.empty?
ZipWriter.write(entries)
end
private
def build_entries
rows = MaterialExportItem
.enabled
.includes(material: { file_attachment: :blob })
.joins(:material)
.merge(Material.kept)
.where(profile: @profile)
.order(:export_path)
rows = rows.where(materials: { tag_id: @tag_id }) if @tag_id
missing_files = []
entries = rows.filter_map do |item|
material = item.material
next unless material.file.attached?
data = download_blob(item, missing_files)
next unless data
Entry.new(path: item.export_path,
data:,
mtime: material.updated_at || Time.current)
end
raise MissingFileError.new(missing_files) if missing_files.any?
paths = entries.map(&:path)
duplicated = paths.find { |path| paths.count(path) > 1 }
raise DuplicatePathError, duplicated if duplicated
entries
end
def download_blob item, missing_files
blob = item.material.file.blob
blob.download
rescue ActiveStorage::FileNotFoundError
missing_files << MissingFile.new(
material_id: item.material_id,
export_path: item.export_path,
blob_id: blob.id,
filename: blob.filename.to_s,
)
nil
end
class ZipWriter
VERSION_NEEDED = 20
GP_FLAG = 0x0800
COMPRESSION_STORE = 0
def self.write entries
new(entries).write
end
def initialize entries
@entries = entries
@central_directory = []
end
def write
io = StringIO.new(''.b)
@entries.each do |entry|
write_entry(io, entry)
end
central_start = io.pos
@central_directory.each { |header| io.write(header) }
central_size = io.pos - central_start
io.write([0x06054b50, 0, 0, @entries.size, @entries.size,
central_size, central_start, 0].pack('VvvvvVVv'))
io.string
end
private
def write_entry io, entry
path = entry.path.b
data = entry.data.b
crc32 = Zlib.crc32(data)
dos_time, dos_date = dos_timestamp(entry.mtime)
offset = io.pos
local_header = [0x04034b50, VERSION_NEEDED, GP_FLAG, COMPRESSION_STORE,
dos_time, dos_date, crc32, data.bytesize, data.bytesize,
path.bytesize, 0].pack('VvvvvvVVVvv')
io.write(local_header)
io.write(path)
io.write(data)
@central_directory << central_header(path:, crc32:, size: data.bytesize,
dos_time:, dos_date:, offset:)
end
def central_header path:, crc32:, size:, dos_time:, dos_date:, offset:
[0x02014b50, VERSION_NEEDED, VERSION_NEEDED, GP_FLAG, COMPRESSION_STORE,
dos_time, dos_date, crc32, size, size, path.bytesize, 0, 0, 0, 0, 0,
offset].pack('VvvvvvvVVVvvvvvVV') + path
end
def dos_timestamp time
local = time.to_time
dos_time = (local.hour << 11) | (local.min << 5) | (local.sec / 2)
dos_date = ((local.year - 1980) << 9) | (local.month << 5) | local.day
[dos_time, dos_date]
end
end
end
+122
ファイルの表示
@@ -0,0 +1,122 @@
class PostCreator
class VideoMsParseError < ArgumentError; end
def initialize actor:, attributes:
@actor = actor
@attributes = attributes.symbolize_keys
end
def create!
post = Post.new(title: @attributes[:title].presence,
url: @attributes[:url],
thumbnail_base: @attributes[:thumbnail_base].presence,
uploaded_user: @actor,
original_created_from: @attributes[:original_created_from].presence,
original_created_before: @attributes[:original_created_before].presence)
thumbnail = @attributes[:thumbnail]
post.thumbnail.attach(Post.resized_thumbnail_attachment(thumbnail)) if thumbnail.present?
ApplicationRecord.transaction do
post.save!
Tag.normalise_tags!(tag_names, deny_deprecated: true, with_sections: true) =>
{ tags:, sections: }
TagVersioning.record_tag_snapshots!(tags, created_by_user: @actor)
tags = Tag.expand_parent_tags(tags).reject(&:deprecated?)
post.video_ms = normalise_video_ms(tags)
validate_video_sections!(post.video_ms, sections)
post.save!
sync_post_tags!(post, tags, sections)
sync_parent_posts!(post, parent_post_ids)
PostVersionRecorder.record!(post:, event_type: :create, created_by_user: @actor)
end
post
end
private
def tag_names = @attributes[:tags].to_s.split
def parent_post_ids
Array(@attributes[:parent_post_ids]).flat_map { _1.to_s.split }.map { |token|
id = Integer(token, exception: false)
raise ArgumentError, "親投稿 Id. が不正です: #{ token }" if id.nil? || id <= 0
id
}.uniq
end
def normalise_video_ms tags
return nil unless tags.any? { _1.id == Tag.video.id }
video_ms = @attributes[:video_ms]
if video_ms.present?
value = Integer(video_ms, exception: false)
raise VideoMsParseError unless value&.positive?
return value
end
duration = @attributes[:duration]
return nil if duration.blank?
return duration.to_i if duration.is_a?(Numeric) && duration.to_i.positive?
value = Tag.time_to_ms!(duration.to_s, tag_name: '動画時間')
raise VideoMsParseError unless value.positive?
value
rescue Tag::SectionLiteralParseError
raise VideoMsParseError
end
def validate_video_sections! video_ms, sections
return unless video_ms
sections.each_value do |ranges|
ranges.each do |begin_ms, end_ms|
post = Post.new
if begin_ms >= video_ms
post.errors.add :video_ms, 'タグ区間の開始が動画時間以上です.'
raise ActiveRecord::RecordInvalid, post
end
if end_ms && end_ms > video_ms
post.errors.add :video_ms, 'タグ区間の終端が動画時間を超えてゐます.'
raise ActiveRecord::RecordInvalid, post
end
end
end
end
def sync_post_tags! post, desired_tags, sections
desired_ids = desired_tags.map(&:id).to_set
current_ids = post.tags.pluck(:id).to_set
Tag.where(id: desired_ids - current_ids).find_each do |tag|
PostTag.create_or_find_by!(post:, tag:, created_user: @actor)
end
PostTagSection.where(post_id: post.id).destroy_all
sections.each do |tag_id, ranges|
ranges.each do |begin_ms, end_ms|
PostTagSection.create!(post_id: post.id,
tag_id:,
begin_ms:,
end_ms:)
end
end
PostTag.where(post_id: post.id,
tag_id: (current_ids - desired_ids).to_a).kept.find_each do |post_tag|
post_tag.discard_by!(@actor)
end
end
def sync_parent_posts! post, ids
if ids.include?(post.id)
post.errors.add :parent_post_ids, '自分自身を親投稿にはできません.'
raise ActiveRecord::RecordInvalid, post
end
missing = ids - Post.where(id: ids).pluck(:id)
if missing.present?
post.errors.add :parent_post_ids, "存在しない親投稿 Id. があります: #{ missing.join(' ') }"
raise ActiveRecord::RecordInvalid, post
end
ids.each { |parent_post_id| PostImplication.create_or_find_by!(post:, parent_post_id:) }
end
end
+485
ファイルの表示
@@ -0,0 +1,485 @@
require 'time'
require 'timeout'
class PostImportPreviewer
FIELDS = [
'title',
'thumbnail_base',
'original_created_from',
'original_created_before',
'duration',
'tags',
'parent_post_ids'].freeze
FETCH_WARNING_FIELDS = ['url', 'title', 'thumbnail_base'].freeze
TITLE_FETCH_WARNING = 'タイトルを取得できませんでした.'.freeze
THUMBNAIL_FETCH_WARNING = 'サムネールを取得できませんでした.'.freeze
METADATA_FETCH_WARNING = 'メタデータを取得できませんでした.'.freeze
def preview_rows rows:, fetch_metadata: true, metadata_cache: { }
prepared_rows = rows.map { prepare_row(_1) }
url_counts = prepared_rows.filter_map { _1[:normal_url] }.tally
existing_posts =
Post.where(url: prepared_rows.map { _1[:normal_url] }.compact.uniq).index_by(&:url)
existing_parent_ids = preload_parent_ids(prepared_rows)
preload_metadata!(
prepared_rows,
fetch_metadata,
metadata_cache,
existing_posts,
url_counts)
known_tags =
preload_known_tags(
prepared_rows,
fetch_metadata,
metadata_cache,
existing_posts,
url_counts)
prepared_rows.map { |row|
preview_row(row,
fetch_metadata:,
metadata_cache:,
existing_posts:,
url_counts:,
known_tags:,
existing_parent_ids:)
}
end
def normalised_url value
PostUrlNormaliser.normalise(value)
end
private
def prepare_row row
source = row.symbolize_keys
url = source[:url].to_s.strip
normal_url = normalised_url(url)
source.merge(url_text: url, normal_url:, url_error: validate_url_safety(normal_url))
end
def preview_row row,
fetch_metadata:,
metadata_cache:,
existing_posts:,
url_counts:,
known_tags:,
existing_parent_ids:
attributes = initial_attributes(row)
provenance = initial_provenance(row)
tag_sources = initial_tag_sources(row, attributes, provenance)
field_warnings = initial_field_warnings(row)
base_warnings = initial_base_warnings(row)
url = row[:url_text]
provenance['url'] = 'manual'
normal_url = row[:normal_url]
url_for_metadata = normal_url || url
existing_post = normal_url.present? ? existing_posts[normal_url] : nil
validation_errors = {}
validation_errors[:url] = ['URL が不正です.'] if normal_url.blank?
if row[:url_error].present?
validation_errors[:url] = [row[:url_error]]
end
if normal_url.present? && url_counts[normal_url].to_i > 1
validation_errors[:url] = ['URL が重複しています.']
end
if row[:metadata_url].present? && row[:metadata_url] != url_for_metadata
clear_automatic_values!(attributes, provenance, tag_sources)
field_warnings = { }
base_warnings = [ ]
end
if validation_errors.blank? && normal_url.present? && existing_post
attributes['tags'] = merged_tags(tag_sources, provenance['tags'])
warnings_present = field_warnings.values.any?(&:present?) || base_warnings.present?
return { source_row: row[:source_row],
url: normal_url,
attributes:,
provenance:,
tag_sources:,
metadata_url: url_for_metadata,
skip_reason: 'existing',
existing_post_id: existing_post.id,
field_warnings:,
base_warnings:,
validation_errors:,
status: warnings_present ? 'warning' : 'ready' }
end
should_fetch = validation_errors.blank?
should_fetch &&= should_fetch_metadata?(fetch_metadata, row[:source_row].to_i)
if should_fetch
clear_fetch_warnings!(field_warnings)
metadata = metadata_for(url_for_metadata, metadata_cache)
if metadata[:validation_errors].present?
validation_errors.merge!(metadata[:validation_errors])
else
apply_metadata!(attributes, provenance, tag_sources, metadata[:data])
apply_fetch_warnings!(field_warnings, metadata[:warnings])
end
end
attributes['url'] = url
validate_basic_data(attributes, validation_errors)
validate_preview_tags(merged_tags(tag_sources, provenance['tags']),
validation_errors,
field_warnings,
known_tags)
validate_parents(attributes['parent_post_ids'], validation_errors, existing_parent_ids)
attributes.delete('url')
attributes['tags'] = merged_tags(tag_sources, provenance['tags'])
warnings_present = field_warnings.values.any?(&:present?) || base_warnings.present?
{ source_row: row[:source_row],
url: validation_errors[:url].present? ? url : (normal_url || url),
attributes:,
provenance:,
tag_sources:,
metadata_url: url_for_metadata,
skip_reason: nil,
existing_post_id: nil,
field_warnings:,
base_warnings:,
validation_errors:,
status: validation_errors.present? ? 'error' : (warnings_present ? 'warning' : 'ready') }
end
def should_fetch_metadata? fetch_metadata, source_row
case fetch_metadata
when true then true
when Integer then fetch_metadata == source_row
when false, nil then false
else raise ArgumentError, 'メタデータ取得対象が不正です.'
end
end
def initial_attributes row
attributes = row[:attributes]&.stringify_keys || { }
FIELDS.to_h { |field| [field, attributes[field].to_s] }
end
def initial_provenance row
provenance = row[:provenance]&.stringify_keys || { }
FIELDS.to_h { |field| [field, provenance[field].presence || 'automatic'] }
.merge('url' => 'manual')
end
def initial_tag_sources row, attributes, provenance
sources = row[:tag_sources]&.stringify_keys || { 'automatic' => '', 'manual' => '' }
sources['automatic'] = sources['automatic'].to_s
sources['manual'] =
provenance['tags'] == 'manual' ? attributes['tags'].to_s : sources['manual'].to_s
sources
end
def initial_field_warnings row
(row[:field_warnings] || { })
.stringify_keys
.transform_values { |value| Array(value).map(&:to_s) }
end
def initial_base_warnings row
Array(row[:base_warnings]).map(&:to_s)
end
def clear_automatic_values! attributes, provenance, tag_sources
['title', 'thumbnail_base', 'original_created_from',
'original_created_before', 'duration'].each do |field|
attributes[field] = '' if provenance[field] == 'automatic'
end
tag_sources['automatic'] = ''
attributes['tags'] = merged_tags(tag_sources, provenance['tags'])
end
def clear_fetch_warnings! field_warnings
FETCH_WARNING_FIELDS.each do |field|
field_warnings.delete(field)
end
end
def metadata_for url, cache
cache[url] ||= fetch_metadata(url)
end
def fetch_metadata url
return { data: { }, warnings: { 'url' => ['URL が空です.'] },
validation_errors: { } } if url.blank?
data = sanitise_metadata(PostMetadataFetcher.fetch(url).stringify_keys.compact)
warnings = { }
add_field_warning!(warnings, 'title', TITLE_FETCH_WARNING) if data['title'].blank?
if data['thumbnail_base'].blank?
add_field_warning!(warnings, 'thumbnail_base', THUMBNAIL_FETCH_WARNING)
end
{ data:, warnings:, validation_errors: { } }
rescue Preview::UrlSafety::UnsafeUrl => e
Rails.logger.info(
"post_import_metadata_fetch_unsafe_url "\
"#{ { error: e.class.name, message: e.message }.to_json }")
{ data: { }, warnings: { }, validation_errors: { url: [e.message] } }
rescue Preview::HttpFetcher::FetchFailed,
Preview::HttpFetcher::ResponseTooLarge => e
Rails.logger.info(
"post_import_metadata_fetch_failure "\
"#{ { error: e.class.name, message: e.message }.to_json }")
{ data: { }, warnings: { 'url' => [METADATA_FETCH_WARNING] }, validation_errors: { } }
end
def sanitise_metadata metadata
{
'title' => sanitise_metadata_title(metadata['title']),
'thumbnail_base' => sanitise_metadata_url(metadata['thumbnail_base']),
'original_created_from' => sanitise_metadata_time(metadata['original_created_from']),
'original_created_before' => sanitise_metadata_time(metadata['original_created_before']),
'duration' => sanitise_metadata_duration(metadata['duration']),
'tags' => metadata['tags'].to_s.presence }.compact
end
def sanitise_metadata_title value
value.is_a?(String) ? value.presence : nil
end
def sanitise_metadata_url value
return nil unless value.is_a?(String)
PostUrlNormaliser.normalise(value)
end
def sanitise_metadata_time value
return nil unless value.is_a?(String)
time = Time.iso8601(value)
time.nsec.zero? ? time.iso8601 : time.iso8601(9)
rescue ArgumentError, TypeError
nil
end
def sanitise_metadata_duration value
return nil unless value.is_a?(Numeric)
value.positive? ? value.to_i : nil
end
def preload_metadata! prepared_rows, fetch_metadata, metadata_cache, existing_posts, url_counts
urls = prepared_rows.filter_map { |row|
next unless row[:normal_url].present?
next if row[:url_error].present?
next unless should_fetch_metadata?(fetch_metadata, row[:source_row].to_i)
next if url_counts[row[:normal_url]].to_i > 1
next if existing_posts.key?(row[:normal_url])
row[:normal_url]
}.uniq
urls = urls.reject { metadata_cache.key?(_1) }
return if urls.empty?
url_queue = Queue.new
result_queue = Queue.new
urls.each { url_queue << _1 }
workers = [urls.length, 4].min.times.map {
Thread.new {
Rails.application.executor.wrap do
loop do
url = url_queue.pop(true)
result_queue << [url, safe_fetch_metadata(url)]
rescue ThreadError
break
end
end
}
}
Timeout.timeout(15) { workers.each(&:join) }
rescue Timeout::Error
workers&.each(&:kill)
ensure
workers&.each(&:join)
while result_queue&.size.to_i.positive?
url, result = result_queue.pop
metadata_cache[url] = result
end
urls&.each do |url|
metadata_cache[url] ||= { data: { }, warnings: { 'url' => [METADATA_FETCH_WARNING] } }
end
end
def safe_fetch_metadata url
fetch_metadata(url)
rescue Preview::UrlSafety::UnsafeUrl => e
Rails.logger.info(
"post_import_metadata_fetch_unsafe_url "\
"#{ { error: e.class.name, message: e.message }.to_json }")
{ data: { }, warnings: { }, validation_errors: { url: [e.message] } }
rescue StandardError => e
Rails.logger.error(
"post_import_metadata_fetch_unexpected_failure "\
"#{ { error: e.class.name, message: e.message }.to_json }")
{ data: { }, warnings: { 'url' => [METADATA_FETCH_WARNING] }, validation_errors: { } }
end
def preload_known_tags prepared_rows, fetch_metadata, metadata_cache, existing_posts, url_counts
names = prepared_rows.flat_map { |row|
attributes = initial_attributes(row)
provenance = initial_provenance(row)
tag_sources = initial_tag_sources(row, attributes, provenance)
if metadata_url_changed?(row)
clear_automatic_values!(attributes, provenance, tag_sources)
end
if should_apply_metadata_to_row?(row, fetch_metadata, existing_posts, url_counts)
metadata = metadata_for(row[:normal_url], metadata_cache)
apply_metadata!(attributes, provenance, tag_sources, metadata[:data])
end
preview_tag_names(merged_tags(tag_sources, provenance['tags']))
}.compact.uniq
return { } if names.empty?
Tag.joins(:tag_name)
.where(tag_names: { name: names })
.includes(:tag_name)
.to_a
.index_by(&:name)
end
def preload_parent_ids prepared_rows
ids = prepared_rows.flat_map { |row|
attributes = initial_attributes(row)
preview_parent_ids(attributes['parent_post_ids'])
}.uniq
return { } if ids.empty?
Post.where(id: ids).pluck(:id).to_h { [_1, true] }
end
def metadata_url_changed? row
row[:metadata_url].present? && row[:metadata_url] != (row[:normal_url] || row[:url_text])
end
def should_apply_metadata_to_row? row, fetch_metadata, existing_posts, url_counts
normal_url = row[:normal_url]
return false if normal_url.blank?
return false if row[:url_error].present?
return false if url_counts[normal_url].to_i > 1
return false if existing_posts.key?(normal_url)
should_fetch_metadata?(fetch_metadata, row[:source_row].to_i)
end
def validate_url_safety normal_url
return nil if normal_url.blank?
Preview::UrlSafety.validate(normal_url)
nil
rescue Preview::UrlSafety::UnsafeUrl => e
e.message
end
def apply_metadata! attributes, provenance, tag_sources, metadata
metadata.each do |field, value|
if field == 'tags'
next if provenance['tags'] == 'manual'
tag_sources['automatic'] = value.to_s
attributes['tags'] = merged_tags(tag_sources)
next
end
next unless provenance[field] == 'automatic'
attributes[field] = value
provenance[field] = 'automatic'
end
end
def apply_fetch_warnings! field_warnings, warnings
warnings.each do |field, values|
values.each do |value|
add_field_warning!(field_warnings, field, value)
end
end
end
def add_field_warning! field_warnings, field, message
field_warnings[field] ||= []
field_warnings[field] << message unless field_warnings[field].include?(message)
end
def merged_tags sources, origin = nil
return sources['manual'].to_s if origin == 'manual'
sources['automatic'].to_s
end
def preview_tag_names raw
names = raw.to_s.split
return [] if names.empty?
if names.any? { _1.downcase.start_with?('nico:') }
return []
end
names.map { |name| TagName.canonicalise(name.sub(/\[.*\]\z/, '')).first }
rescue Tag::SectionLiteralParseError
[]
end
def validate_preview_tags raw, errors, field_warnings, known_tags
names = raw.to_s.split
return if names.empty?
if names.any? { _1.downcase.start_with?('nico:') }
errors[:tags] = ['ニコニコ・タグは直接指定できません.']
return
end
parsed = names.map { |name| TagName.canonicalise(name.sub(/\[.*\]\z/, '')).first }
existing = parsed.filter_map { known_tags[_1] }
deprecated = existing.select(&:deprecated?).map(&:name)
errors[:tags] = ["廃止済みタグがあります: #{ deprecated.join(' ') }"] if deprecated.present?
known = existing.reject(&:deprecated?).map(&:name)
new_tags = parsed.uniq - known
if new_tags.present?
add_field_warning!(field_warnings, 'tags', "新規タグを作成します: #{ new_tags.join(' ') }")
end
rescue Tag::SectionLiteralParseError
errors[:tags] = ['タグ区間の記法が不正です.']
end
def validate_basic_data attributes, errors
post = Post.new(title: attributes['title'].presence,
url: attributes['url'],
thumbnail_base: attributes['thumbnail_base'].presence,
original_created_from: attributes['original_created_from'].presence,
original_created_before: attributes['original_created_before'].presence,
video_ms: parse_duration(attributes['duration'], errors))
post.valid?
post.errors.each do |error|
next if error.attribute == :url && error.type == :taken
(errors[error.attribute] ||= []) << error.message
end
end
def parse_duration value, errors
return nil if value.blank?
value.is_a?(Numeric) ? value.to_i : Tag.time_to_ms!(value.to_s, tag_name: '動画時間')
rescue Tag::SectionLiteralParseError
errors[:video_ms] = ['動画時間の記法が不正です.']
nil
end
def preview_parent_ids raw
raw.to_s.split.map { Integer(_1, exception: false) }.compact
end
def validate_parents raw, errors, existing_parent_ids
ids = raw.to_s.split.map { Integer(_1, exception: false) }
return if ids.compact.empty? && raw.to_s.blank?
if ids.any? { _1.nil? || _1 <= 0 }
errors[:parent_post_ids] = ['親投稿 Id. が不正です.']
return
end
if ids.uniq.any? { !existing_parent_ids[_1] }
errors[:parent_post_ids] = ['存在しない親投稿 Id. があります.']
end
end
end
+189
ファイルの表示
@@ -0,0 +1,189 @@
class PostImportRowNormaliser
ORIGINS = ['automatic', 'manual'].freeze
STRING_FIELDS = [
'title',
'thumbnail_base',
'original_created_from',
'original_created_before',
'tags',
'parent_post_ids'].freeze
FLEXIBLE_FIELDS = ['duration', 'video_ms'].freeze
ATTRIBUTE_FIELDS = (STRING_FIELDS + FLEXIBLE_FIELDS).freeze
def self.normalise! rows, allow_warning_fields: false
raise ArgumentError, '取込行の形式が不正です.' unless rows.is_a?(Array)
raise ArgumentError, '取込件数が多すぎます.' if rows.length > PostImportUrlListParser::MAX_ROWS
normalised_rows = rows.map { normalise_row!(_1, allow_warning_fields:) }
source_rows = normalised_rows.map { _1['source_row'] }
raise ArgumentError, '元行番号が重複しています.' if source_rows.uniq.length != source_rows.length
if normalised_rows.sum { row_bytesize(_1) } > PostImportUrlListParser::MAX_BYTES
raise ArgumentError, '取込データが大きすぎます.'
end
normalised_rows
end
def self.normalise_row! row, allow_warning_fields:
unless row.is_a?(Hash) || row.is_a?(ActionController::Parameters)
raise ArgumentError, '取込行の形式が不正です.'
end
parameters =
row.is_a?(ActionController::Parameters) ? row : ActionController::Parameters.new(row)
permitted = parameters.permit(*permitted_keys(allow_warning_fields))
normalised = permitted.to_h.deep_transform_keys { _1.to_s.underscore }
normalised['source_row'] = normalise_source_row!(normalised['source_row'])
normalise_url!(normalised['url'])
normalise_metadata_url!(normalised['metadata_url'])
normalise_attributes!(normalised.fetch('attributes', { }))
normalise_provenance!(normalised.fetch('provenance', { }))
normalise_tag_sources!(normalised['tag_sources'])
normalise_warning_values!(normalised, allow_warning_fields:)
if row_bytesize(normalised) > PostImportUrlListParser::MAX_BYTES
raise ArgumentError, '取込行が大きすぎます.'
end
normalised
end
def self.permitted_keys allow_warning_fields
keys = [
:source_row,
:sourceRow,
:url,
:metadata_url,
:metadataUrl,
{ attributes: {} },
{ provenance: {} },
{ tag_sources: {} },
{ tagSources: {} }]
return keys unless allow_warning_fields
keys + [
{ field_warnings: {} },
{ fieldWarnings: {} },
{ base_warnings: [] },
{ baseWarnings: [] }]
end
private_class_method :permitted_keys
def self.normalise_source_row! value
source_row = Integer(value, exception: false)
raise ArgumentError, '元行番号が不正です.' if source_row.nil? || source_row <= 0
source_row
end
private_class_method :normalise_source_row!
def self.normalise_url! value
unless value.is_a?(String)
raise ArgumentError, 'URL の形式が不正です.'
end
if value.bytesize > PostImportUrlListParser::MAX_URL_BYTES
raise ArgumentError, 'URL が長すぎます.'
end
end
private_class_method :normalise_url!
def self.normalise_metadata_url! value
raise ArgumentError, 'metadata_url の形式が不正です.' unless value.nil? || value.is_a?(String)
if value.to_s.bytesize > PostImportUrlListParser::MAX_URL_BYTES
raise ArgumentError, 'metadata_url が長すぎます.'
end
end
private_class_method :normalise_metadata_url!
def self.normalise_attributes! attributes
raise ArgumentError, 'attributes の形式が不正です.' unless attributes.is_a?(Hash)
raise ArgumentError, '取込項目が不正です.' unless (attributes.keys - ATTRIBUTE_FIELDS).empty?
attributes.each do |key, value|
case key
when *STRING_FIELDS
raise ArgumentError, '取込項目の型が不正です.' unless value.nil? || value.is_a?(String)
when *FLEXIBLE_FIELDS
unless value.nil? || value.is_a?(String) || value.is_a?(Numeric)
raise ArgumentError, '取込項目の型が不正です.'
end
end
if value.to_s.bytesize > PostImportUrlListParser::MAX_URL_BYTES
raise ArgumentError, '取込項目が大きすぎます.'
end
end
end
private_class_method :normalise_attributes!
def self.normalise_provenance! provenance
unless provenance.is_a?(Hash)
raise ArgumentError, 'provenance の形式が不正です.'
end
allowed = ATTRIBUTE_FIELDS + ['url']
unless (provenance.keys - allowed).empty?
raise ArgumentError, '値の由来が不正です.'
end
unless provenance.values.all? { ORIGINS.include?(_1) }
raise ArgumentError, '値の由来が不正です.'
end
end
private_class_method :normalise_provenance!
def self.normalise_tag_sources! tag_sources
return if tag_sources.nil?
unless tag_sources.is_a?(Hash)
raise ArgumentError, 'タグ由来の形式が不正です.'
end
unless (tag_sources.keys - ORIGINS).empty?
raise ArgumentError, 'タグ由来の形式が不正です.'
end
unless tag_sources.values.all? { _1.is_a?(String) }
raise ArgumentError, 'タグ由来の形式が不正です.'
end
if tag_sources.values.any? { _1.bytesize > PostImportUrlListParser::MAX_URL_BYTES }
raise ArgumentError, 'タグ由来が大きすぎます.'
end
if tag_sources.values.sum(&:bytesize) > PostImportUrlListParser::MAX_URL_BYTES
raise ArgumentError, 'タグ由来が大きすぎます.'
end
end
private_class_method :normalise_tag_sources!
def self.normalise_warning_values! normalised, allow_warning_fields:
return unless allow_warning_fields
field_warnings = normalised['field_warnings']
unless field_warnings.nil? || field_warnings.is_a?(Hash)
raise ArgumentError, '警告の形式が不正です.'
end
field_warnings&.each do |key, values|
unless ATTRIBUTE_FIELDS.include?(key) || key == 'url'
raise ArgumentError, '警告の形式が不正です.'
end
unless values.is_a?(Array) && values.all? { _1.is_a?(String) }
raise ArgumentError, '警告の形式が不正です.'
end
if values.any? { _1.bytesize > PostImportUrlListParser::MAX_URL_BYTES }
raise ArgumentError, '警告が大きすぎます.'
end
end
base_warnings = normalised['base_warnings']
return if base_warnings.nil?
unless base_warnings.is_a?(Array) && base_warnings.all? { _1.is_a?(String) }
raise ArgumentError, '警告の形式が不正です.'
end
if base_warnings.any? { _1.bytesize > PostImportUrlListParser::MAX_URL_BYTES }
raise ArgumentError, '警告が大きすぎます.'
end
end
private_class_method :normalise_warning_values!
def self.row_bytesize row
row.to_json.bytesize
end
private_class_method :row_bytesize
end
+104
ファイルの表示
@@ -0,0 +1,104 @@
class PostImportRunner
def initialize actor:, rows:
@actor = actor
@rows = rows
end
def run
normalised_rows = PostImportRowNormaliser.normalise!(@rows)
previews = PostImportPreviewer.new.preview_rows(rows: normalised_rows,
fetch_metadata: false)
preview_map = previews.index_by { _1[:source_row] }
results = normalised_rows.map do |row|
run_row(row, preview_map.fetch(row['source_row']))
end
{ created: results.count { _1[:status] == 'created' },
skipped: results.count { _1[:status] == 'skipped' },
failed: results.count { _1[:status] == 'failed' },
rows: results }
end
private
def run_row row, preview
attributes = row.fetch('attributes', { }).transform_keys { _1.to_s.underscore }
return { source_row: row['source_row'],
status: 'failed',
errors: preview[:validation_errors],
recoverable: true } if preview[:validation_errors].present?
if preview[:skip_reason] == 'existing'
return { source_row: row['source_row'],
status: 'skipped',
existing_post_id: preview[:existing_post_id] }
end
attributes['tags'] = preview[:attributes]['tags']
attributes['url'] = row['url']
post = PostCreator.new(actor: @actor, attributes:).create!
{ source_row: row['source_row'], status: 'created', post: PostRepr.base(post) }
rescue ActiveRecord::RecordInvalid => e
existing_post = existing_post_for_race(row, e.record)
if existing_post
return { source_row: row['source_row'],
status: 'skipped',
existing_post_id: existing_post.id }
end
{ source_row: row['source_row'],
status: 'failed',
errors: e.record.errors.to_hash,
recoverable: true }
rescue ActiveRecord::RecordNotUnique => e
raise unless url_record_not_unique?(e)
existing_post = existing_post_for_race(row)
raise unless existing_post
{ source_row: row['source_row'],
status: 'skipped',
existing_post_id: existing_post.id }
rescue Tag::NicoTagNormalisationError
{ source_row: row['source_row'],
status: 'failed',
errors: { tags: ['ニコニコ・タグは直接指定できません.'] },
recoverable: true }
rescue Tag::DeprecatedTagNormalisationError
{ source_row: row['source_row'],
status: 'failed',
errors: { tags: ['廃止済みタグは付与できません.'] },
recoverable: true }
rescue PostCreator::VideoMsParseError
{ source_row: row['source_row'],
status: 'failed',
errors: { duration: ['動画時間の記法が不正です.'] },
recoverable: true }
rescue ArgumentError
{ source_row: row['source_row'],
status: 'failed',
errors: { base: ['入力値が不正です.'] },
recoverable: true }
rescue StandardError => e
Rails.logger.error("post_import_runner_failure #{ { error: e.class.name,
message: e.message }.to_json }")
{ source_row: row['source_row'],
status: 'failed',
errors: { base: ['登録中にエラーが発生しました.'] } }
end
def existing_post_for_race row, record = nil
if record && !(record.errors.of_kind?(:url, :taken))
return nil
end
normal_url = PostUrlNormaliser.normalise(row['url'])
return nil if normal_url.blank?
Post.find_by(url: normal_url)
end
def url_record_not_unique? error
error.message.include?('index_posts_on_url')
end
end
+25
ファイルの表示
@@ -0,0 +1,25 @@
class PostImportUrlListParser
MAX_ROWS = 100
MAX_BYTES = 1.megabyte
MAX_URL_BYTES = 20.kilobytes
def self.parse source
raw = source.to_s
raise ArgumentError, '入力が大きすぎます.' if raw.bytesize > MAX_BYTES
rows = raw.split(/\r\n|\n|\r/).each_with_index.filter_map { |line, index|
url = line.strip
next if url.blank?
if url.bytesize > MAX_URL_BYTES
raise ArgumentError, "#{ index + 1 } 行目: URL が長すぎます."
end
{ source_row: index + 1, url: }
}
raise ArgumentError, 'URL を入力してください.' if rows.empty?
raise ArgumentError, "取込件数は #{ MAX_ROWS } 件までです." if rows.length > MAX_ROWS
rows
end
end
+131
ファイルの表示
@@ -0,0 +1,131 @@
require 'time'
class PostMetadataFetcher
TIMESTAMP_PATTERN =
Regexp.new(
'\A(\d{4})-(\d{2})-(\d{2})T(\d{2})' \
'(?::(\d{2})(?::(\d{2})(?:\.(\d+))?)?)?' \
'(Z|[+-]\d{2}:?\d{2})?\z')
def self.fetch raw_url
uri, = Preview::UrlSafety.validate(raw_url)
response = Preview::HttpFetcher.fetch(
uri.to_s,
max_bytes: Preview::ThumbnailFetcher::HTML_MAX_BYTES)
metadata = Preview::HtmlMetadataExtractor.extract(response)
document = Nokogiri::HTML.parse(response.body)
content = lambda { |name|
document
.at_css("meta[property='#{ name }'], meta[name='#{ name }']")
&.[]('content')
&.strip
&.presence
}
duration = content.call('og:video:duration') || content.call('video:duration')
published = content.call('article:published_time') || content.call('date')
created_range = original_created_range(published)
platform_tags = platform_tags(uri)
{ title: metadata[:title],
thumbnail_base:
Preview::KnownSiteExtractor.thumbnail_url(uri) || metadata[:image_url],
original_created_from: serialise_time(created_range&.first),
original_created_before: serialise_time(created_range&.last),
duration: duration&.to_f&.then { _1.positive? ? (_1 * 1_000).round : nil },
tags: platform_tags.join(' ') }
end
def self.platform_tags uri
return ['動画', 'YouTube'] if Preview::KnownSiteExtractor.youtube_video_id(uri)
return ['動画', 'ニコニコ'] if Preview::KnownSiteExtractor.niconico_video_id(uri)
[]
end
def self.original_created_range value
return nil if value.blank?
raw = value.to_s.strip
from, before =
case raw
when /\A(\d{4})\z/
year = Regexp.last_match(1).to_i
from = Time.zone.local(year, 1, 1)
[from, from + 1.year]
when /\A(\d{4})-(\d{2})\z/
year = Regexp.last_match(1).to_i
month = Regexp.last_match(2).to_i
from = Time.zone.local(year, month, 1)
[from, from + 1.month]
when /\A(\d{4})-(\d{2})-(\d{2})\z/
year = Regexp.last_match(1).to_i
month = Regexp.last_match(2).to_i
day = Regexp.last_match(3).to_i
from = Time.zone.local(year, month, day)
[from, from + 1.day]
else
parse_timestamp_range(raw)
end
return nil if from.nil? || before.nil?
[from, before]
rescue ArgumentError, TypeError
nil
end
def self.parse_timestamp_range raw
match = raw.match(TIMESTAMP_PATTERN)
return nil unless match
year = match[1].to_i
month = match[2].to_i
day = match[3].to_i
hour = match[4].to_i
minute = match[5]&.to_i || 0
second = match[6]&.to_i || 0
fraction = match[7]
offset = match[8]
whole_second = second + fractional_seconds(fraction)
from =
if offset.present?
Time.new(year, month, day, hour, minute, whole_second, parse_offset(offset)).in_time_zone
else
Time.zone.local(year, month, day, hour, minute, whole_second)
end
before =
if fraction.present?
from + (10**(-fraction.length))
elsif match[6].present?
from + 1.second
elsif match[5].present?
from + 1.minute
else
from + 1.hour
end
[from, before]
end
def self.parse_offset value
return '+00:00' if value == 'Z'
value.match?(/\A[+-]\d{2}:\d{2}\z/) ? value : "#{ value[0, 3] }:#{ value[3, 2] }"
end
def self.fractional_seconds value
return 0 if value.blank?
Rational(value.to_i, 10**value.length)
end
def self.serialise_time value
return nil if value.nil?
value.nsec.zero? ? value.iso8601 : value.iso8601(9)
end
private_class_method :platform_tags,
:original_created_range,
:parse_timestamp_range,
:parse_offset,
:fractional_seconds,
:serialise_time
end
+13
ファイルの表示
@@ -0,0 +1,13 @@
class PostUrlNormaliser
def self.normalise raw_url
value = raw_url.to_s.strip
uri = URI.parse(value)
return nil unless uri.is_a?(URI::HTTP) && uri.host.present?
uri.host = uri.host.downcase
uri.path = uri.path.sub(/\/\z/, '') if uri.path.present?
PostUrlSanitisationRule.sanitise(uri.to_s)
rescue URI::InvalidURIError
nil
end
end
+1
ファイルの表示
@@ -23,6 +23,7 @@ class PostVersionRecorder < VersionRecorder
{ title: @record.title, { title: @record.title,
url: @record.url, url: @record.url,
thumbnail_base: @record.thumbnail_base, thumbnail_base: @record.thumbnail_base,
video_ms: @record.video_ms,
tags: @record.snapshot_tag_names.join(' '), tags: @record.snapshot_tag_names.join(' '),
parent_post_ids: @record.snapshot_parent_post_ids.join(' '), parent_post_ids: @record.snapshot_parent_post_ids.join(' '),
original_created_from: @record.original_created_from, original_created_from: @record.original_created_from,
+21
ファイルの表示
@@ -0,0 +1,21 @@
module Preview
class HtmlMetadataExtractor
IMAGE_SELECTORS = [
'meta[property="og:image"]',
'meta[name="twitter:image"]',
'meta[name="thumbnail"]'
].freeze
def self.extract(response)
document = Nokogiri::HTML.parse(response.body)
image_url = IMAGE_SELECTORS.filter_map {
document.at_css(_1)&.[]('content')&.strip.presence
}.first
{ title: document.at_css('title')&.text&.strip,
image_url: image_url ? URI.join(response.url, image_url).to_s : nil }
rescue URI::InvalidURIError
{ title: document.at_css('title')&.text&.strip, image_url: nil }
end
end
end
+127
ファイルの表示
@@ -0,0 +1,127 @@
require 'net/http'
require 'json'
module Preview
class HttpFetcher
class FetchFailed < StandardError; end
class FetchTimeout < FetchFailed; end
class ResponseTooLarge < FetchFailed; end
MAX_REDIRECTS = 5
DEFAULT_MAX_BYTES = 5.megabytes
Response = Data.define(:body, :content_type, :url)
def self.fetch(raw_url,
max_bytes: DEFAULT_MAX_BYTES,
redirects: MAX_REDIRECTS)
uri, addresses = UrlSafety.validate(raw_url)
response = request(uri, addresses.first, max_bytes)
if response.is_a?(Net::HTTPRedirection)
location = response['location']
if redirects.zero?
log_failure(:redirect_limit,
url: uri.to_s,
redirects:,
location:,
content_type: response['content-type'],
content_length: response['content-length'])
raise FetchFailed, 'redirect が多すぎます.'
end
if location.blank?
log_failure(:blank_redirect_location,
url: uri.to_s,
redirects:,
content_type: response['content-type'],
content_length: response['content-length'])
raise FetchFailed, 'redirect 先が不正です.'
end
redirect_url =
begin
URI.join(uri, location).to_s
rescue URI::InvalidURIError => e
log_failure(:invalid_redirect_location,
url: uri.to_s,
redirects:,
location:,
error: e.class.name,
message: e.message)
raise FetchFailed, 'redirect 先が不正です.'
end
return fetch(redirect_url, max_bytes:, redirects: redirects - 1)
end
unless response.is_a?(Net::HTTPSuccess)
log_failure(:http_status,
url: uri.to_s,
code: response.code,
content_type: response['content-type'],
content_length: response['content-length'])
raise FetchFailed, "外部サーバーが HTTP #{ response.code } を返しました."
end
Response.new(response.body, response['content-type'].to_s, uri.to_s)
rescue Net::OpenTimeout, Net::ReadTimeout, Timeout::Error => e
log_failure(:timeout, url: uri&.to_s || raw_url, error: e.class.name, message: e.message)
raise FetchTimeout, e.message
rescue SocketError, SystemCallError, OpenSSL::SSL::SSLError, EOFError => e
log_failure(:network_error,
url: uri&.to_s || raw_url,
error: e.class.name,
message: e.message)
raise FetchFailed, e.message
end
def self.request(uri, ip_address, max_bytes)
http = Net::HTTP.new(uri.host, uri.port)
http.ipaddr = ip_address
http.use_ssl = uri.scheme == 'https'
http.open_timeout = 5
http.read_timeout = 8
http.write_timeout = 5
request = Net::HTTP::Get.new(uri)
request['User-Agent'] = 'BTRC-Hub thumbnail preview'
request['Accept'] = 'text/html,image/*;q=0.9,*/*;q=0.1'
http.request(request) do |response|
length = response['content-length'].to_i
if length > max_bytes
log_failure(:response_too_large,
url: uri.to_s,
content_type: response['content-type'],
content_length: response['content-length'],
max_bytes:)
raise ResponseTooLarge, '外部データが大きすぎます.'
end
body = +''
response.read_body do |chunk|
body << chunk
next unless body.bytesize > max_bytes
log_failure(:response_too_large,
url: uri.to_s,
content_type: response['content-type'],
content_length: response['content-length'],
bytes_read: body.bytesize,
max_bytes:)
raise ResponseTooLarge, '外部データが大きすぎます.'
end
response.instance_variable_set(:@body, body)
response.instance_variable_set(:@read, true)
return response
end
end
def self.log_failure(reason, **payload)
Rails.logger.warn("preview_http_fetcher_failure #{ { reason:, **payload }.to_json }")
end
private_class_method :request, :log_failure
end
end
+34
ファイルの表示
@@ -0,0 +1,34 @@
module Preview
class KnownSiteExtractor
def self.thumbnail_url(uri)
youtube_thumbnail(uri)
end
def self.youtube_video_id(uri)
case uri.host&.downcase
when 'youtu.be'
uri.path.split('/').reject(&:blank?).first
when 'www.youtube.com', 'youtube.com', 'm.youtube.com'
uri.path == '/watch' ? URI.decode_www_form(uri.query.to_s).to_h['v'] : nil
end&.then { _1 if _1.match?(/\A[A-Za-z0-9_-]{6,20}\z/) }
end
def self.niconico_video_id(uri)
case uri.host&.downcase
when 'www.nicovideo.jp', 'nicovideo.jp'
uri.path[%r{\A/watch/(sm\d+)\z}, 1]
when 'nico.ms'
uri.path[%r{\A/(sm\d+)\z}, 1]
end
end
def self.youtube_thumbnail(uri)
id = youtube_video_id(uri)
return unless id
"https://i.ytimg.com/vi/#{ id }/hqdefault.jpg"
end
private_class_method :youtube_thumbnail
end
end
+95
ファイルの表示
@@ -0,0 +1,95 @@
module Preview
class ThumbnailFetcher
class GenerationFailed < StandardError; end
ALLOWED_IMAGE_CONTENT_TYPES = [
'image/jpeg', 'image/png', 'image/gif', 'image/webp'
].freeze
HTML_MAX_BYTES = 1.megabyte
NICONICO_XML_MAX_BYTES = 256.kilobytes
def self.fetch(raw_url)
uri, = UrlSafety.validate(raw_url)
known_url = KnownSiteExtractor.thumbnail_url(uri)
image = fetch_image_or_nil(known_url) if known_url
return image if image
niconico_url = niconico_thumbnail_url(uri)
image = fetch_image_or_nil(niconico_url) if niconico_url
return image if image
page = HttpFetcher.fetch(uri.to_s, max_bytes: HTML_MAX_BYTES)
metadata = HtmlMetadataExtractor.extract(page)
raise GenerationFailed, 'サムネール画像が見つかりませんでした.' if metadata[:image_url].blank?
fetch_image!(metadata[:image_url])
end
def self.title(raw_url)
uri, = UrlSafety.validate(raw_url)
HtmlMetadataExtractor.extract(
HttpFetcher.fetch(uri.to_s, max_bytes: HTML_MAX_BYTES))[:title]
end
def self.fetch_image_or_nil(url)
return nil if url.blank?
response = HttpFetcher.fetch(url)
return nil unless allowed_image_content_type?(response.content_type)
response.body
rescue HttpFetcher::FetchTimeout
raise
rescue HttpFetcher::FetchFailed
nil
end
def self.fetch_image!(url)
response = HttpFetcher.fetch(url)
unless allowed_image_content_type?(response.content_type)
raise GenerationFailed, 'サムネール画像が見つかりませんでした.'
end
response.body
rescue HttpFetcher::FetchTimeout
raise
rescue HttpFetcher::ResponseTooLarge
raise
rescue HttpFetcher::FetchFailed
raise GenerationFailed, 'サムネール画像を取得できませんでした.'
end
def self.niconico_thumbnail_url(uri)
video_id = KnownSiteExtractor.niconico_video_id(uri)
return nil if video_id.blank?
response = HttpFetcher.fetch("https://ext.nicovideo.jp/api/getthumbinfo/#{ video_id }",
max_bytes: NICONICO_XML_MAX_BYTES)
xml = Nokogiri::XML(response.body)
return nil unless xml.at_xpath('/nicovideo_thumb_response/@status')&.value == 'ok'
xml.at_xpath('//thumbnail_url')&.text&.strip.presence
rescue HttpFetcher::FetchFailed, HttpFetcher::FetchTimeout => e
Rails.logger.info("preview_niconico_getthumbinfo_fallback #{ { url: uri.to_s,
video_id:,
error: e.class.name,
message: e.message }.to_json }")
nil
rescue Nokogiri::XML::SyntaxError => e
Rails.logger.info("preview_niconico_getthumbinfo_fallback #{ { url: uri.to_s,
video_id:,
error: e.class.name,
message: e.message }.to_json }")
nil
end
def self.allowed_image_content_type?(content_type)
mime_type = content_type.to_s.split(';', 2).first.downcase.strip
ALLOWED_IMAGE_CONTENT_TYPES.include?(mime_type)
end
private_class_method :fetch_image_or_nil, :fetch_image!,
:niconico_thumbnail_url,
:allowed_image_content_type?
end
end
+55
ファイルの表示
@@ -0,0 +1,55 @@
require 'resolv'
require 'ipaddr'
require 'uri'
module Preview
class UrlSafety
class UnsafeUrl < StandardError; end
FORBIDDEN_NETWORKS = [
'0.0.0.0/8', '10.0.0.0/8', '100.64.0.0/10', '127.0.0.0/8',
'169.254.0.0/16', '172.16.0.0/12', '192.0.0.0/24',
'192.0.2.0/24', '192.168.0.0/16', '198.18.0.0/15',
'198.51.100.0/24', '203.0.113.0/24', '224.0.0.0/4',
'240.0.0.0/4', '::/128', '::1/128', 'fc00::/7', 'fe80::/10',
'ff00::/8', '2001:db8::/32', '::ffff:0:0/96'
].map { IPAddr.new(_1) }.freeze
def self.validate(raw_url)
value = raw_url.to_s.strip
if value.match?(/\A[a-z][a-z0-9+\-.]*:/i)
unless value.match?(/\Ahttps?:\/\//i)
raise UnsafeUrl, 'http または https の URL を指定してください.'
end
else
value = "http://#{ value }"
end
uri = URI.parse(value)
unless ['http', 'https'].include?(uri.scheme&.downcase) && uri.host.present?
raise UnsafeUrl, 'http または https の URL を指定してください.'
end
raise UnsafeUrl, 'userinfo つき URL は使用できません.' if uri.userinfo.present?
addresses = Resolv.getaddresses(uri.host)
raise UnsafeUrl, 'URL のホストを解決できません.' if addresses.empty?
parsed_addresses = addresses.map { IPAddr.new(_1) }
if parsed_addresses.any? { |address| forbidden?(address) }
raise UnsafeUrl, '安全でない接続先は使用できません.'
end
[uri, parsed_addresses.map(&:to_s)]
rescue Resolv::ResolvError
raise UnsafeUrl, 'URL のホストを解決できません.'
rescue URI::InvalidURIError, IPAddr::InvalidAddressError
raise UnsafeUrl, 'URL が不正です.'
end
def self.forbidden?(address)
FORBIDDEN_NETWORKS.any? { _1.include?(address) }
end
private_class_method :forbidden?
end
end
+2 -1
ファイルの表示
@@ -73,7 +73,7 @@ class VersionRecorder
end end
def validate_event_type! def validate_event_type!
return if EVENT_TYPES.include?(@event_type) return if event_types.include?(@event_type)
raise ArgumentError, "Invalid event_type: #{ @event_type }" raise ArgumentError, "Invalid event_type: #{ @event_type }"
end end
@@ -84,4 +84,5 @@ class VersionRecorder
def snapshot_attributes = raise NotImplementedError def snapshot_attributes = raise NotImplementedError
def record_class = @record.class def record_class = @record.class
def event_types = self.class::EVENT_TYPES
end end
+14
ファイルの表示
@@ -33,6 +33,12 @@ Rails.application.routes.draw do
get :thumbnail get :thumbnail
end end
scope 'posts/import', controller: :post_imports do
post :preview
post :validate
post '', action: :create
end
resources :wiki_pages, path: 'wiki', only: [:index, :show, :create, :update] do resources :wiki_pages, path: 'wiki', only: [:index, :show, :create, :update] do
collection do collection do
get :search get :search
@@ -81,6 +87,11 @@ Rails.application.routes.draw do
end end
end end
get 'users/settings', to: 'user_settings#show'
patch 'users/settings', to: 'user_settings#update'
get 'users/theme_slots', to: 'user_theme_slots#index'
put 'users/theme_slots/:base_theme/:slot_no', to: 'user_theme_slots#update'
resources :users, only: [:create, :update] do resources :users, only: [:create, :update] do
collection do collection do
post :verify post :verify
@@ -113,5 +124,8 @@ Rails.application.routes.draw do
resources :skip_events, controller: :theatre_skip_events, only: [:index] resources :skip_events, controller: :theatre_skip_events, only: [:index]
end end
get 'materials/download.zip', to: 'materials#download'
get 'materials/versions', to: 'material_versions#index'
resources :material_sync_suppressions, path: 'materials/suppressions', only: [:index, :create]
resources :materials, only: [:index, :show, :create, :update, :destroy] resources :materials, only: [:index, :show, :create, :update, :destroy]
end end
+9
ファイルの表示
@@ -0,0 +1,9 @@
class AddVideoMsToPosts < ActiveRecord::Migration[8.0]
def change
add_column :posts, :video_ms, :integer
add_index :posts, [:video_ms, :id], name: 'idx_posts_video_ms_id'
add_check_constraint :posts, 'video_ms IS NULL OR video_ms > 0',
name: 'chk_posts_video_ms_positive'
end
end
+19
ファイルの表示
@@ -0,0 +1,19 @@
class CreatePostTagSections < ActiveRecord::Migration[8.0]
def change
create_table :post_tag_sections, primary_key: [:post_id, :tag_id, :begin_ms] do |t|
t.references :post, null: false, foreign_key: true, index: false
t.references :tag, null: false, foreign_key: true, index: false
t.integer :begin_ms, null: false
t.integer :end_ms, null: true
t.timestamps
t.index [:post_id, :begin_ms], name: 'idx_post_tag_sections_post_id_begin_ms'
t.check_constraint 'begin_ms >= 0',
name: 'chk_post_tag_sections_begin_ms_natural'
t.check_constraint 'end_ms IS NULL OR begin_ms < end_ms',
name: 'chk_post_tag_sections_end_ms_after_begin_ms'
end
end
end
+9
ファイルの表示
@@ -0,0 +1,9 @@
class AddVideoMsToPostVersions < ActiveRecord::Migration[8.0]
def change
add_column :post_versions, :video_ms, :integer
add_index :post_versions, [:video_ms, :post_id], name: 'idx_post_versions_video_ms_post_id'
add_check_constraint :post_versions, 'video_ms IS NULL OR video_ms > 0',
name: 'chk_post_versions_video_ms_positive'
end
end
+88
ファイルの表示
@@ -0,0 +1,88 @@
class EnhanceMaterialManagement < ActiveRecord::Migration[8.0]
def up
change_table :materials, bulk: true do |t|
t.integer :version_no, null: false, default: 1
end
change_table :material_versions, bulk: true do |t|
t.string :event_type
t.string :tag_name
t.string :tag_category
t.json :export_paths_json
t.bigint :file_blob_id
t.string :file_filename
t.string :file_content_type
t.bigint :file_byte_size
t.string :file_checksum
t.string :file_sha256
end
execute <<~SQL.squish
UPDATE material_versions
SET event_type = CASE
WHEN version_no = 1 THEN 'create'
ELSE 'update'
END
WHERE event_type IS NULL
SQL
change_column_null :material_versions, :event_type, false
add_index :material_versions, :file_blob_id
add_check_constraint :material_versions,
"event_type IN ('create', 'update', 'discard', 'restore')",
name: 'material_versions_event_type_valid'
create_table :material_export_items do |t|
t.references :material, null: false, foreign_key: true
t.string :profile, null: false, default: 'legacy_drive'
t.string :export_path, null: false
t.boolean :enabled, null: false, default: true
t.references :created_by_user, foreign_key: { to_table: :users }
t.timestamps
t.index [:profile, :export_path], unique: true
t.index [:material_id, :profile], unique: true
end
create_table :material_import_blocks do |t|
t.string :match_kind, null: false
t.string :sha256
t.string :external_path_pattern
t.string :reason, null: false
t.text :note
t.references :created_by_user, foreign_key: { to_table: :users }
t.timestamps
end
execute <<~SQL.squish
UPDATE materials
SET version_no = COALESCE(
(SELECT MAX(material_versions.version_no)
FROM material_versions
WHERE material_versions.material_id = materials.id),
1)
SQL
end
def down
drop_table :material_import_blocks
drop_table :material_export_items
remove_check_constraint :material_versions, name: 'material_versions_event_type_valid'
remove_index :material_versions, :file_blob_id
remove_column :material_versions, :event_type
remove_column :material_versions, :tag_name
remove_column :material_versions, :tag_category
remove_column :material_versions, :export_paths_json
remove_column :material_versions, :file_blob_id
remove_column :material_versions, :file_filename
remove_column :material_versions, :file_content_type
remove_column :material_versions, :file_byte_size
remove_column :material_versions, :file_checksum
remove_column :material_versions, :file_sha256
remove_column :materials, :version_no
end
end
+118
ファイルの表示
@@ -0,0 +1,118 @@
class ReconcileMaterialSyncSuppressions < ActiveRecord::Migration[8.0]
MATERIAL_VERSION_EVENT_CONSTRAINT = 'material_versions_event_type_valid'
def up
ensure_material_sync_suppressions!
ensure_material_source_columns!
ensure_material_active_source_key!
remove_material_file_suppression!
remove_material_version_file_suppression!
reconcile_material_version_event_constraint!
end
def down
raise ActiveRecord::IrreversibleMigration
end
private
def ensure_material_sync_suppressions!
return if table_exists?(:material_sync_suppressions)
create_table :material_sync_suppressions do |t|
t.string :source_kind, null: false
t.string :source_uri
t.string :drive_path
t.string :drive_file_id
t.string :normalized_source_key, null: false
t.string :reason, null: false
t.references :created_by_user, foreign_key: { to_table: :users }
t.timestamps
t.index :normalized_source_key, unique: true
t.index :source_kind
t.index :drive_file_id
end
end
def ensure_material_source_columns!
add_column :materials, :source_kind, :string unless column_exists?(:materials, :source_kind)
add_column :materials, :source_uri, :string unless column_exists?(:materials, :source_uri)
add_column :materials, :source_path, :string unless column_exists?(:materials, :source_path)
unless column_exists?(:materials, :source_file_id)
add_column :materials, :source_file_id, :string
end
unless column_exists?(:materials, :normalized_source_key)
add_column :materials, :normalized_source_key, :string
end
if index_exists?(:materials, :normalized_source_key)
remove_index :materials, column: :normalized_source_key
end
end
def ensure_material_active_source_key!
unless column_exists?(:materials, :active_normalized_source_key)
change_table :materials do |t|
t.virtual :active_normalized_source_key,
type: :string,
as: 'IF(discarded_at IS NULL, normalized_source_key, NULL)',
stored: false
end
end
return if index_exists?(:materials,
:active_normalized_source_key,
name: 'index_materials_on_active_normalized_source_key')
add_index :materials,
:active_normalized_source_key,
unique: true,
name: 'index_materials_on_active_normalized_source_key'
end
def remove_material_file_suppression!
if foreign_key_exists?(:materials, :users, column: :file_suppressed_by_user_id)
remove_foreign_key :materials, column: :file_suppressed_by_user_id
end
if index_exists?(:materials,
:file_suppressed_by_user_id,
name: 'index_materials_on_file_suppressed_by_user_id')
remove_index :materials, name: 'index_materials_on_file_suppressed_by_user_id'
end
remove_column :materials, :file_suppressed_at if column_exists?(
:materials,
:file_suppressed_at)
remove_column :materials, :file_suppressed_by_user_id if column_exists?(
:materials,
:file_suppressed_by_user_id)
remove_column :materials, :file_suppression_reason if column_exists?(
:materials,
:file_suppression_reason)
end
def remove_material_version_file_suppression!
remove_column :material_versions, :file_suppressed_at if column_exists?(
:material_versions,
:file_suppressed_at)
remove_column :material_versions, :file_suppression_reason if column_exists?(
:material_versions,
:file_suppression_reason)
end
def reconcile_material_version_event_constraint!
constraint = check_constraints(:material_versions).find do |candidate|
candidate.name == MATERIAL_VERSION_EVENT_CONSTRAINT
end
return if constraint && !constraint.expression.include?('suppress')
if constraint
remove_check_constraint :material_versions, name: MATERIAL_VERSION_EVENT_CONSTRAINT
end
add_check_constraint :material_versions,
"event_type IN ('create', 'update', 'discard', 'restore')",
name: MATERIAL_VERSION_EVENT_CONSTRAINT
end
end
+22
ファイルの表示
@@ -0,0 +1,22 @@
class CreateMaterialSyncSources < ActiveRecord::Migration[8.0]
def change
create_table :material_sync_sources do |t|
t.string :name, null: false
t.string :source_kind, null: false
t.string :source_uri
t.string :source_path
t.string :source_file_id
t.string :profile, null: false, default: 'legacy_drive'
t.boolean :enabled, null: false, default: true
t.string :default_tag_name
t.string :export_path_prefix
t.datetime :last_synced_at
t.references :created_by_user, foreign_key: { to_table: :users }
t.references :updated_by_user, foreign_key: { to_table: :users }
t.timestamps
t.index :enabled
t.index :source_kind
end
end
end
+19
ファイルの表示
@@ -0,0 +1,19 @@
class AddSourceSnapshotToMaterialVersions < ActiveRecord::Migration[8.0]
def change
unless column_exists?(:material_versions, :source_kind)
add_column :material_versions, :source_kind, :string
end
unless column_exists?(:material_versions, :source_uri)
add_column :material_versions, :source_uri, :string
end
unless column_exists?(:material_versions, :source_path)
add_column :material_versions, :source_path, :string
end
unless column_exists?(:material_versions, :source_file_id)
add_column :material_versions, :source_file_id, :string
end
unless column_exists?(:material_versions, :normalized_source_key)
add_column :material_versions, :normalized_source_key, :string
end
end
end
+171
ファイルの表示
@@ -0,0 +1,171 @@
require 'json'
class BackfillInitialMaterialVersions < ActiveRecord::Migration[8.0]
class MigrationMaterial < ActiveRecord::Base
self.table_name = 'materials'
end
class MigrationMaterialExportItem < ActiveRecord::Base
self.table_name = 'material_export_items'
end
class MigrationMaterialVersion < ActiveRecord::Base
self.table_name = 'material_versions'
end
class MigrationStorageAttachment < ActiveRecord::Base
self.table_name = 'active_storage_attachments'
end
class MigrationStorageBlob < ActiveRecord::Base
self.table_name = 'active_storage_blobs'
end
class MigrationTag < ActiveRecord::Base
self.table_name = 'tags'
end
class MigrationTagName < ActiveRecord::Base
self.table_name = 'tag_names'
end
def up
say_with_time 'Backfilling initial material versions' do
materials_without_versions.find_in_batches(batch_size: 500) do |materials|
material_ids = materials.map(&:id)
tags_by_id = load_tags(materials.map(&:tag_id).compact.uniq)
export_paths_by_material_id = load_export_paths(material_ids)
file_snapshots_by_material_id = load_file_snapshots(material_ids)
rows = materials.map do |material|
tag = tags_by_id[material.tag_id]
file_snapshot = file_snapshots_by_material_id[material.id] || empty_file_snapshot
{ material_id: material.id,
version_no: 1,
event_type: 'create',
url: material.url,
parent_id: material.parent_id,
tag_id: material.tag_id,
tag_name: tag&.fetch(:name, nil),
tag_category: tag&.fetch(:category, nil),
source_kind: material.source_kind,
source_uri: material.source_uri,
source_path: material.source_path,
source_file_id: material.source_file_id,
normalized_source_key: material.normalized_source_key,
export_paths_json: export_paths_by_material_id[material.id] || { },
discarded_at: material.discarded_at,
file_blob_id: file_snapshot[:file_blob_id],
file_filename: file_snapshot[:file_filename],
file_content_type: file_snapshot[:file_content_type],
file_byte_size: file_snapshot[:file_byte_size],
file_checksum: file_snapshot[:file_checksum],
file_sha256: file_snapshot[:file_sha256],
created_by_user_id: material.created_by_user_id,
updated_by_user_id: material.updated_by_user_id,
created_at: material.created_at,
updated_at: material.created_at }
end
MigrationMaterialVersion.insert_all!(rows) if rows.present?
end
end
execute <<~SQL.squish
UPDATE materials
SET version_no = COALESCE(
(
SELECT MAX(material_versions.version_no)
FROM material_versions
WHERE material_versions.material_id = materials.id
),
1
)
SQL
end
def down
raise ActiveRecord::IrreversibleMigration
end
private
def materials_without_versions
MigrationMaterial.where.not(id: MigrationMaterialVersion.select(:material_id))
end
def load_tags tag_ids
return { } if tag_ids.empty?
tag_name_ids_by_tag_id = MigrationTag.where(id: tag_ids).pluck(:id, :tag_name_id).to_h
tag_names_by_id =
MigrationTagName
.where(id: tag_name_ids_by_tag_id.values.uniq)
.pluck(:id, :name)
.to_h
categories_by_tag_id = MigrationTag.where(id: tag_ids).pluck(:id, :category).to_h
tag_ids.each_with_object({ }) do |tag_id, hash|
hash[tag_id] = { name: tag_names_by_id[tag_name_ids_by_tag_id[tag_id]],
category: categories_by_tag_id[tag_id] }
end
end
def load_export_paths material_ids
MigrationMaterialExportItem
.where(material_id: material_ids)
.order(:material_id, :profile)
.pluck(:material_id, :profile, :export_path)
.each_with_object({ }) do |(material_id, profile, export_path), hash|
hash[material_id] ||= { }
hash[material_id][profile] = export_path
end
end
def load_file_snapshots material_ids
attachments_by_material_id =
MigrationStorageAttachment
.where(record_type: 'Material', name: 'file', record_id: material_ids)
.pluck(:record_id, :blob_id)
.to_h
blobs_by_id =
MigrationStorageBlob
.where(id: attachments_by_material_id.values.uniq)
.index_by(&:id)
attachments_by_material_id.each_with_object({ }) do |(material_id, blob_id), hash|
blob = blobs_by_id[blob_id]
next unless blob
hash[material_id] = {
file_blob_id: blob.id,
file_filename: blob.filename.to_s,
file_content_type: blob.content_type,
file_byte_size: blob.byte_size,
file_checksum: blob.checksum,
file_sha256: blob_metadata(blob)['sha256']
}
end
end
def empty_file_snapshot
{ file_blob_id: nil,
file_filename: nil,
file_content_type: nil,
file_byte_size: nil,
file_checksum: nil,
file_sha256: nil }
end
def blob_metadata blob
metadata = blob.metadata
return metadata if metadata.is_a?(Hash)
return { } if metadata.blank?
JSON.parse(metadata)
rescue JSON::ParserError
{ }
end
end
+31
ファイルの表示
@@ -0,0 +1,31 @@
class RebuildSettingsAsTypedUserSettings < ActiveRecord::Migration[8.0]
def change
remove_foreign_key :settings, :users if foreign_key_exists?(:settings, :users)
remove_index :settings, :user_id if index_exists?(:settings, :user_id)
remove_column :settings, :key, :string if column_exists?(:settings, :key)
remove_column :settings, :value, :json if column_exists?(:settings, :value)
change_column_null :settings, :user_id, false
add_column :settings, :theme, :string, null: false, default: 'system'
add_column :settings,
:auto_fetch_title,
:string,
null: false,
default: 'manual'
add_column :settings,
:auto_fetch_thumbnail,
:string,
null: false,
default: 'manual'
add_column :settings,
:wiki_editor_mode,
:string,
null: false,
default: 'split'
add_index :settings, :user_id, unique: true
add_foreign_key :settings, :users
end
end
+22
ファイルの表示
@@ -0,0 +1,22 @@
class CreateUserThemeSlots < ActiveRecord::Migration[8.0]
def change
create_table :user_theme_slots do |t|
t.references :user, null: false, foreign_key: true
t.string :base_theme, null: false
t.integer :slot_no, null: false
t.json :tokens, null: false
t.timestamps
end
add_index :user_theme_slots,
[:user_id, :base_theme, :slot_no],
unique: true,
name: 'index_user_theme_slots_on_user_theme_and_slot'
add_check_constraint :user_theme_slots,
"base_theme IN ('light', 'dark')",
name: 'user_theme_slots_base_theme_valid'
add_check_constraint :user_theme_slots,
'slot_no BETWEEN 1 AND 3',
name: 'user_theme_slots_slot_no_valid'
end
end
+61
ファイルの表示
@@ -0,0 +1,61 @@
class CreatePostUrlSanitisationRules < ActiveRecord::Migration[8.0]
class PostUrlSanitisationRule < ActiveRecord::Base
self.table_name = 'post_url_sanitisation_rules'
end
def up
create_table :post_url_sanitisation_rules, id: :integer, primary_key: :priority do |t|
t.string :source_pattern, null: false
t.string :replacement, null: false
t.timestamps
t.datetime :discarded_at
t.index :source_pattern, unique: true
t.index :discarded_at
end
now = Time.current
PostUrlSanitisationRule.insert_all!([
{ priority: 10,
source_pattern: '\Ahttps?://youtu\.be/([^/?#]+)(?:[?#].*)?\z',
replacement: 'https://www.youtube.com/watch?v=\1',
created_at: now,
updated_at: now },
{ priority: 20,
source_pattern: '\Ahttps?://(?:www\.|m\.)?youtube\.com/live/([^/?#]+)(?:[?#].*)?\z',
replacement: 'https://www.youtube.com/watch?v=\1',
created_at: now,
updated_at: now },
{ priority: 30,
source_pattern: '\Ahttps?://(?:www\.|m\.)?youtube\.com/shorts/([^/?#]+)(?:[?#].*)?\z',
replacement: 'https://www.youtube.com/watch?v=\1',
created_at: now,
updated_at: now },
{ priority: 40,
source_pattern: '\Ahttps?://(?:www\.|m\.)?youtube\.com/embed/([^/?#]+)(?:[?#].*)?\z',
replacement: 'https://www.youtube.com/watch?v=\1',
created_at: now,
updated_at: now },
{ priority: 50,
source_pattern:
'\Ahttps?://(?:www\.|m\.)?youtube\.com/watch\?(?:[^#&]+&)*v=([^&#]+)(?:[&#].*)?\z',
replacement: 'https://www.youtube.com/watch?v=\1',
created_at: now,
updated_at: now },
{ priority: 60,
source_pattern: '\Ahttps?://nico\.ms/([^/?#]+)(?:[?#].*)?\z',
replacement: 'https://www.nicovideo.jp/watch/\1',
created_at: now,
updated_at: now },
{ priority: 70,
source_pattern: '\Ahttps?://(?:www\.)?nicovideo\.jp/watch/([^?#/]+)(?:[?#].*)?\z',
replacement: 'https://www.nicovideo.jp/watch/\1',
created_at: now,
updated_at: now }])
end
def down
drop_table :post_url_sanitisation_rules
end
end
生成ファイル
+160 -19
ファイルの表示
@@ -10,7 +10,7 @@
# #
# It's strongly recommended that you check this file into your version control system. # It's strongly recommended that you check this file into your version control system.
ActiveRecord::Schema[8.0].define(version: 2026_06_21_000000) do ActiveRecord::Schema[8.0].define(version: 2026_07_13_000000) do
create_table "active_storage_attachments", charset: "utf8mb4", collation: "utf8mb4_0900_ai_ci", force: :cascade do |t| create_table "active_storage_attachments", charset: "utf8mb4", collation: "utf8mb4_0900_ai_ci", force: :cascade do |t|
t.string "name", null: false t.string "name", null: false
t.string "record_type", null: false t.string "record_type", null: false
@@ -130,6 +130,69 @@ ActiveRecord::Schema[8.0].define(version: 2026_06_21_000000) do
t.index ["ip_address"], name: "index_ip_addresses_on_ip_address", unique: true t.index ["ip_address"], name: "index_ip_addresses_on_ip_address", unique: true
end end
create_table "material_export_items", charset: "utf8mb4", collation: "utf8mb4_0900_ai_ci", force: :cascade do |t|
t.bigint "material_id", null: false
t.string "profile", default: "legacy_drive", null: false
t.string "export_path", null: false
t.boolean "enabled", default: true, null: false
t.bigint "created_by_user_id"
t.datetime "created_at", null: false
t.datetime "updated_at", null: false
t.index ["created_by_user_id"], name: "index_material_export_items_on_created_by_user_id"
t.index ["material_id", "profile"], name: "index_material_export_items_on_material_id_and_profile", unique: true
t.index ["material_id"], name: "index_material_export_items_on_material_id"
t.index ["profile", "export_path"], name: "index_material_export_items_on_profile_and_export_path", unique: true
end
create_table "material_import_blocks", charset: "utf8mb4", collation: "utf8mb4_0900_ai_ci", force: :cascade do |t|
t.string "match_kind", null: false
t.string "sha256"
t.string "external_path_pattern"
t.string "reason", null: false
t.text "note"
t.bigint "created_by_user_id"
t.datetime "created_at", null: false
t.datetime "updated_at", null: false
t.index ["created_by_user_id"], name: "index_material_import_blocks_on_created_by_user_id"
end
create_table "material_sync_sources", charset: "utf8mb4", collation: "utf8mb4_0900_ai_ci", force: :cascade do |t|
t.string "name", null: false
t.string "source_kind", null: false
t.string "source_uri"
t.string "source_path"
t.string "source_file_id"
t.string "profile", default: "legacy_drive", null: false
t.boolean "enabled", default: true, null: false
t.string "default_tag_name"
t.string "export_path_prefix"
t.datetime "last_synced_at"
t.bigint "created_by_user_id"
t.bigint "updated_by_user_id"
t.datetime "created_at", null: false
t.datetime "updated_at", null: false
t.index ["created_by_user_id"], name: "index_material_sync_sources_on_created_by_user_id"
t.index ["enabled"], name: "index_material_sync_sources_on_enabled"
t.index ["source_kind"], name: "index_material_sync_sources_on_source_kind"
t.index ["updated_by_user_id"], name: "index_material_sync_sources_on_updated_by_user_id"
end
create_table "material_sync_suppressions", charset: "utf8mb4", collation: "utf8mb4_0900_ai_ci", force: :cascade do |t|
t.string "source_kind", null: false
t.string "source_uri"
t.string "drive_path"
t.string "drive_file_id"
t.string "normalized_source_key", null: false
t.string "reason", null: false
t.bigint "created_by_user_id"
t.datetime "created_at", null: false
t.datetime "updated_at", null: false
t.index ["created_by_user_id"], name: "index_material_sync_suppressions_on_created_by_user_id"
t.index ["drive_file_id"], name: "index_material_sync_suppressions_on_drive_file_id"
t.index ["normalized_source_key"], name: "index_material_sync_suppressions_on_normalized_source_key", unique: true
t.index ["source_kind"], name: "index_material_sync_suppressions_on_source_kind"
end
create_table "material_versions", charset: "utf8mb4", collation: "utf8mb4_0900_ai_ci", force: :cascade do |t| create_table "material_versions", charset: "utf8mb4", collation: "utf8mb4_0900_ai_ci", force: :cascade do |t|
t.bigint "material_id", null: false t.bigint "material_id", null: false
t.integer "version_no", null: false t.integer "version_no", null: false
@@ -141,14 +204,31 @@ ActiveRecord::Schema[8.0].define(version: 2026_06_21_000000) do
t.datetime "created_at", null: false t.datetime "created_at", null: false
t.datetime "updated_at", null: false t.datetime "updated_at", null: false
t.datetime "discarded_at" t.datetime "discarded_at"
t.string "event_type", null: false
t.string "tag_name"
t.string "tag_category"
t.json "export_paths_json"
t.bigint "file_blob_id"
t.string "file_filename"
t.string "file_content_type"
t.bigint "file_byte_size"
t.string "file_checksum"
t.string "file_sha256"
t.string "source_kind"
t.string "source_uri"
t.string "source_path"
t.string "source_file_id"
t.string "normalized_source_key"
t.index ["created_by_user_id"], name: "index_material_versions_on_created_by_user_id" t.index ["created_by_user_id"], name: "index_material_versions_on_created_by_user_id"
t.index ["discarded_at"], name: "index_material_versions_on_discarded_at" t.index ["discarded_at"], name: "index_material_versions_on_discarded_at"
t.index ["file_blob_id"], name: "index_material_versions_on_file_blob_id"
t.index ["material_id", "version_no"], name: "index_material_versions_on_material_id_and_version_no", unique: true t.index ["material_id", "version_no"], name: "index_material_versions_on_material_id_and_version_no", unique: true
t.index ["material_id"], name: "index_material_versions_on_material_id" t.index ["material_id"], name: "index_material_versions_on_material_id"
t.index ["parent_id"], name: "index_material_versions_on_parent_id" t.index ["parent_id"], name: "index_material_versions_on_parent_id"
t.index ["tag_id"], name: "index_material_versions_on_tag_id" t.index ["tag_id"], name: "index_material_versions_on_tag_id"
t.index ["updated_by_user_id"], name: "index_material_versions_on_updated_by_user_id" t.index ["updated_by_user_id"], name: "index_material_versions_on_updated_by_user_id"
t.index ["url"], name: "index_material_versions_on_url" t.index ["url"], name: "index_material_versions_on_url"
t.check_constraint "`event_type` in (_utf8mb4'create',_utf8mb4'update',_utf8mb4'discard',_utf8mb4'restore')", name: "material_versions_event_type_valid"
end end
create_table "materials", charset: "utf8mb4", collation: "utf8mb4_0900_ai_ci", force: :cascade do |t| create_table "materials", charset: "utf8mb4", collation: "utf8mb4_0900_ai_ci", force: :cascade do |t|
@@ -161,6 +241,14 @@ ActiveRecord::Schema[8.0].define(version: 2026_06_21_000000) do
t.datetime "updated_at", null: false t.datetime "updated_at", null: false
t.datetime "discarded_at" t.datetime "discarded_at"
t.virtual "active_url", type: :string, as: "if((`discarded_at` is null),`url`,NULL)" t.virtual "active_url", type: :string, as: "if((`discarded_at` is null),`url`,NULL)"
t.integer "version_no", default: 1, null: false
t.string "source_kind"
t.string "source_uri"
t.string "source_path"
t.string "source_file_id"
t.string "normalized_source_key"
t.virtual "active_normalized_source_key", type: :string, as: "if((`discarded_at` is null),`normalized_source_key`,NULL)"
t.index ["active_normalized_source_key"], name: "index_materials_on_active_normalized_source_key", unique: true
t.index ["active_url"], name: "index_materials_on_active_url", unique: true t.index ["active_url"], name: "index_materials_on_active_url", unique: true
t.index ["created_by_user_id"], name: "index_materials_on_created_by_user_id" t.index ["created_by_user_id"], name: "index_materials_on_created_by_user_id"
t.index ["discarded_at"], name: "index_materials_on_discarded_at" t.index ["discarded_at"], name: "index_materials_on_discarded_at"
@@ -210,6 +298,19 @@ ActiveRecord::Schema[8.0].define(version: 2026_06_21_000000) do
t.index ["target_post_id"], name: "index_post_similarities_on_target_post_id" t.index ["target_post_id"], name: "index_post_similarities_on_target_post_id"
end end
create_table "post_tag_sections", primary_key: ["post_id", "tag_id", "begin_ms"], charset: "utf8mb4", collation: "utf8mb4_0900_ai_ci", force: :cascade do |t|
t.bigint "post_id", null: false
t.bigint "tag_id", null: false
t.integer "begin_ms", null: false
t.integer "end_ms"
t.datetime "created_at", null: false
t.datetime "updated_at", null: false
t.index ["post_id", "begin_ms"], name: "idx_post_tag_sections_post_id_begin_ms"
t.index ["tag_id"], name: "fk_rails_8be3847903"
t.check_constraint "(`end_ms` is null) or (`begin_ms` < `end_ms`)", name: "chk_post_tag_sections_end_ms_after_begin_ms"
t.check_constraint "`begin_ms` >= 0", name: "chk_post_tag_sections_begin_ms_natural"
end
create_table "post_tags", charset: "utf8mb4", collation: "utf8mb4_0900_ai_ci", force: :cascade do |t| create_table "post_tags", charset: "utf8mb4", collation: "utf8mb4_0900_ai_ci", force: :cascade do |t|
t.bigint "post_id", null: false t.bigint "post_id", null: false
t.bigint "tag_id", null: false t.bigint "tag_id", null: false
@@ -230,6 +331,16 @@ ActiveRecord::Schema[8.0].define(version: 2026_06_21_000000) do
t.index ["tag_id"], name: "index_post_tags_on_tag_id" t.index ["tag_id"], name: "index_post_tags_on_tag_id"
end end
create_table "post_url_sanitisation_rules", primary_key: "priority", id: :integer, charset: "utf8mb4", collation: "utf8mb4_0900_ai_ci", force: :cascade do |t|
t.string "source_pattern", null: false
t.string "replacement", null: false
t.datetime "created_at", null: false
t.datetime "updated_at", null: false
t.datetime "discarded_at"
t.index ["discarded_at"], name: "index_post_url_sanitisation_rules_on_discarded_at"
t.index ["source_pattern"], name: "index_post_url_sanitisation_rules_on_source_pattern", unique: true
end
create_table "post_versions", charset: "utf8mb4", collation: "utf8mb4_0900_ai_ci", force: :cascade do |t| create_table "post_versions", charset: "utf8mb4", collation: "utf8mb4_0900_ai_ci", force: :cascade do |t|
t.bigint "post_id", null: false t.bigint "post_id", null: false
t.integer "version_no", null: false t.integer "version_no", null: false
@@ -243,9 +354,12 @@ ActiveRecord::Schema[8.0].define(version: 2026_06_21_000000) do
t.datetime "original_created_before" t.datetime "original_created_before"
t.datetime "created_at", null: false t.datetime "created_at", null: false
t.bigint "created_by_user_id" t.bigint "created_by_user_id"
t.integer "video_ms"
t.index ["created_by_user_id"], name: "index_post_versions_on_created_by_user_id" t.index ["created_by_user_id"], name: "index_post_versions_on_created_by_user_id"
t.index ["post_id", "version_no"], name: "index_post_versions_on_post_id_and_version_no", unique: true t.index ["post_id", "version_no"], name: "index_post_versions_on_post_id_and_version_no", unique: true
t.index ["post_id"], name: "index_post_versions_on_post_id" t.index ["post_id"], name: "index_post_versions_on_post_id"
t.index ["video_ms", "post_id"], name: "idx_post_versions_video_ms_post_id"
t.check_constraint "(`video_ms` is null) or (`video_ms` > 0)", name: "chk_post_versions_video_ms_positive"
t.check_constraint "`event_type` in (_utf8mb4'create',_utf8mb4'update',_utf8mb4'discard',_utf8mb4'restore')", name: "post_versions_event_type_valid" t.check_constraint "`event_type` in (_utf8mb4'create',_utf8mb4'update',_utf8mb4'discard',_utf8mb4'restore')", name: "post_versions_event_type_valid"
t.check_constraint "`version_no` > 0", name: "post_versions_version_no_positive" t.check_constraint "`version_no` > 0", name: "post_versions_version_no_positive"
end end
@@ -260,18 +374,36 @@ ActiveRecord::Schema[8.0].define(version: 2026_06_21_000000) do
t.datetime "original_created_before" t.datetime "original_created_before"
t.datetime "updated_at", null: false t.datetime "updated_at", null: false
t.integer "version_no", null: false t.integer "version_no", null: false
t.integer "video_ms"
t.index ["uploaded_user_id"], name: "index_posts_on_uploaded_user_id" t.index ["uploaded_user_id"], name: "index_posts_on_uploaded_user_id"
t.index ["url"], name: "index_posts_on_url", unique: true t.index ["url"], name: "index_posts_on_url", unique: true
t.index ["video_ms", "id"], name: "idx_posts_video_ms_id"
t.check_constraint "(`video_ms` is null) or (`video_ms` > 0)", name: "chk_posts_video_ms_positive"
t.check_constraint "`version_no` > 0", name: "chk_posts_version_no_positive" t.check_constraint "`version_no` > 0", name: "chk_posts_version_no_positive"
end end
create_table "settings", charset: "utf8mb4", collation: "utf8mb4_0900_ai_ci", force: :cascade do |t| create_table "settings", charset: "utf8mb4", collation: "utf8mb4_0900_ai_ci", force: :cascade do |t|
t.bigint "user_id", null: false t.bigint "user_id", null: false
t.string "key", null: false
t.json "value", null: false
t.datetime "created_at", null: false t.datetime "created_at", null: false
t.datetime "updated_at", null: false t.datetime "updated_at", null: false
t.index ["user_id"], name: "index_settings_on_user_id" t.string "theme", default: "system", null: false
t.string "auto_fetch_title", default: "manual", null: false
t.string "auto_fetch_thumbnail", default: "manual", null: false
t.string "wiki_editor_mode", default: "split", null: false
t.index ["user_id"], name: "index_settings_on_user_id", unique: true
end
create_table "wiki_assets", charset: "utf8mb4", collation: "utf8mb4_0900_ai_ci", force: :cascade do |t|
t.bigint "wiki_page_id", null: false
t.integer "no", null: false
t.string "alt_text"
t.binary "sha256", limit: 32, null: false
t.bigint "created_by_user_id", null: false
t.datetime "created_at", null: false
t.datetime "updated_at", null: false
t.index ["created_by_user_id"], name: "index_wiki_assets_on_created_by_user_id"
t.index ["wiki_page_id", "no"], name: "index_wiki_assets_on_wiki_page_id_and_no", unique: true
t.index ["wiki_page_id", "sha256"], name: "index_wiki_assets_on_wiki_page_id_and_sha256", unique: true
end end
create_table "tag_implications", charset: "utf8mb4", collation: "utf8mb4_0900_ai_ci", force: :cascade do |t| create_table "tag_implications", charset: "utf8mb4", collation: "utf8mb4_0900_ai_ci", force: :cascade do |t|
@@ -319,9 +451,9 @@ ActiveRecord::Schema[8.0].define(version: 2026_06_21_000000) do
t.string "event_type", null: false t.string "event_type", null: false
t.string "name", null: false t.string "name", null: false
t.string "category", null: false t.string "category", null: false
t.datetime "deprecated_at"
t.text "aliases", null: false t.text "aliases", null: false
t.text "parent_tag_ids", null: false t.text "parent_tag_ids", null: false
t.datetime "deprecated_at"
t.datetime "created_at", null: false t.datetime "created_at", null: false
t.bigint "created_by_user_id" t.bigint "created_by_user_id"
t.index ["created_at"], name: "index_tag_versions_on_created_at" t.index ["created_at"], name: "index_tag_versions_on_created_at"
@@ -334,10 +466,10 @@ ActiveRecord::Schema[8.0].define(version: 2026_06_21_000000) do
create_table "tags", charset: "utf8mb4", collation: "utf8mb4_0900_ai_ci", force: :cascade do |t| create_table "tags", charset: "utf8mb4", collation: "utf8mb4_0900_ai_ci", force: :cascade do |t|
t.bigint "tag_name_id", null: false t.bigint "tag_name_id", null: false
t.string "category", default: "general", null: false t.string "category", default: "general", null: false
t.datetime "deprecated_at"
t.datetime "created_at", null: false t.datetime "created_at", null: false
t.datetime "updated_at", null: false t.datetime "updated_at", null: false
t.integer "post_count", default: 0, null: false t.integer "post_count", default: 0, null: false
t.datetime "deprecated_at"
t.datetime "discarded_at" t.datetime "discarded_at"
t.integer "version_no", null: false t.integer "version_no", null: false
t.index ["deprecated_at"], name: "index_tags_on_deprecated_at" t.index ["deprecated_at"], name: "index_tags_on_deprecated_at"
@@ -459,6 +591,19 @@ ActiveRecord::Schema[8.0].define(version: 2026_06_21_000000) do
t.index ["post_id"], name: "index_user_post_views_on_post_id" t.index ["post_id"], name: "index_user_post_views_on_post_id"
end end
create_table "user_theme_slots", charset: "utf8mb4", collation: "utf8mb4_0900_ai_ci", force: :cascade do |t|
t.bigint "user_id", null: false
t.string "base_theme", null: false
t.integer "slot_no", null: false
t.json "tokens", null: false
t.datetime "created_at", null: false
t.datetime "updated_at", null: false
t.index ["user_id", "base_theme", "slot_no"], name: "index_user_theme_slots_on_user_theme_and_slot", unique: true
t.index ["user_id"], name: "index_user_theme_slots_on_user_id"
t.check_constraint "`base_theme` in (_utf8mb4'light',_utf8mb4'dark')", name: "user_theme_slots_base_theme_valid"
t.check_constraint "`slot_no` between 1 and 3", name: "user_theme_slots_slot_no_valid"
end
create_table "users", charset: "utf8mb4", collation: "utf8mb4_0900_ai_ci", force: :cascade do |t| create_table "users", charset: "utf8mb4", collation: "utf8mb4_0900_ai_ci", force: :cascade do |t|
t.string "name" t.string "name"
t.string "inheritance_code", limit: 64, null: false t.string "inheritance_code", limit: 64, null: false
@@ -469,19 +614,6 @@ ActiveRecord::Schema[8.0].define(version: 2026_06_21_000000) do
t.index ["banned_at"], name: "index_users_on_banned_at" t.index ["banned_at"], name: "index_users_on_banned_at"
end end
create_table "wiki_assets", charset: "utf8mb4", collation: "utf8mb4_0900_ai_ci", force: :cascade do |t|
t.bigint "wiki_page_id", null: false
t.integer "no", null: false
t.string "alt_text"
t.binary "sha256", limit: 32, null: false
t.bigint "created_by_user_id", null: false
t.datetime "created_at", null: false
t.datetime "updated_at", null: false
t.index ["created_by_user_id"], name: "index_wiki_assets_on_created_by_user_id"
t.index ["wiki_page_id", "no"], name: "index_wiki_assets_on_wiki_page_id_and_no", unique: true
t.index ["wiki_page_id", "sha256"], name: "index_wiki_assets_on_wiki_page_id_and_sha256", unique: true
end
create_table "wiki_lines", charset: "utf8mb4", collation: "utf8mb4_0900_ai_ci", force: :cascade do |t| create_table "wiki_lines", charset: "utf8mb4", collation: "utf8mb4_0900_ai_ci", force: :cascade do |t|
t.string "sha256", limit: 64, null: false t.string "sha256", limit: 64, null: false
t.text "body", null: false t.text "body", null: false
@@ -567,6 +699,12 @@ ActiveRecord::Schema[8.0].define(version: 2026_06_21_000000) do
add_foreign_key "gekanator_question_suggestions", "users" add_foreign_key "gekanator_question_suggestions", "users"
add_foreign_key "gekanator_questions", "gekanator_question_suggestions" add_foreign_key "gekanator_questions", "gekanator_question_suggestions"
add_foreign_key "gekanator_questions", "users", column: "created_by_id" add_foreign_key "gekanator_questions", "users", column: "created_by_id"
add_foreign_key "material_export_items", "materials"
add_foreign_key "material_export_items", "users", column: "created_by_user_id"
add_foreign_key "material_import_blocks", "users", column: "created_by_user_id"
add_foreign_key "material_sync_sources", "users", column: "created_by_user_id"
add_foreign_key "material_sync_sources", "users", column: "updated_by_user_id"
add_foreign_key "material_sync_suppressions", "users", column: "created_by_user_id"
add_foreign_key "material_versions", "materials" add_foreign_key "material_versions", "materials"
add_foreign_key "material_versions", "materials", column: "parent_id" add_foreign_key "material_versions", "materials", column: "parent_id"
add_foreign_key "material_versions", "tags" add_foreign_key "material_versions", "tags"
@@ -584,6 +722,8 @@ ActiveRecord::Schema[8.0].define(version: 2026_06_21_000000) do
add_foreign_key "post_implications", "posts", column: "parent_post_id" add_foreign_key "post_implications", "posts", column: "parent_post_id"
add_foreign_key "post_similarities", "posts" add_foreign_key "post_similarities", "posts"
add_foreign_key "post_similarities", "posts", column: "target_post_id" add_foreign_key "post_similarities", "posts", column: "target_post_id"
add_foreign_key "post_tag_sections", "posts"
add_foreign_key "post_tag_sections", "tags"
add_foreign_key "post_tags", "posts" add_foreign_key "post_tags", "posts"
add_foreign_key "post_tags", "tags" add_foreign_key "post_tags", "tags"
add_foreign_key "post_tags", "users", column: "created_user_id" add_foreign_key "post_tags", "users", column: "created_user_id"
@@ -623,6 +763,7 @@ ActiveRecord::Schema[8.0].define(version: 2026_06_21_000000) do
add_foreign_key "user_ips", "users" add_foreign_key "user_ips", "users"
add_foreign_key "user_post_views", "posts" add_foreign_key "user_post_views", "posts"
add_foreign_key "user_post_views", "users" add_foreign_key "user_post_views", "users"
add_foreign_key "user_theme_slots", "users"
add_foreign_key "wiki_assets", "users", column: "created_by_user_id" add_foreign_key "wiki_assets", "users", column: "created_by_user_id"
add_foreign_key "wiki_assets", "wiki_pages" add_foreign_key "wiki_assets", "wiki_pages"
add_foreign_key "wiki_pages", "tag_names" add_foreign_key "wiki_pages", "tag_names"
+55
ファイルの表示
@@ -7,3 +7,58 @@
# ["Action", "Comedy", "Drama", "Horror"].each do |genre_name| # ["Action", "Comedy", "Drama", "Horror"].each do |genre_name|
# MovieGenre.find_or_create_by!(name: genre_name) # MovieGenre.find_or_create_by!(name: genre_name)
# end # end
post_url_sanitisation_rules = [
{ priority: 10,
source_pattern: '\Ahttps?://youtu\.be/([^/?#]+)(?:[?#].*)?\z',
replacement: 'https://www.youtube.com/watch?v=\1' },
{ priority: 20,
source_pattern: '\Ahttps?://(?:www\.|m\.)?youtube\.com/live/([^/?#]+)(?:[?#].*)?\z',
replacement: 'https://www.youtube.com/watch?v=\1' },
{ priority: 30,
source_pattern: '\Ahttps?://(?:www\.|m\.)?youtube\.com/shorts/([^/?#]+)(?:[?#].*)?\z',
replacement: 'https://www.youtube.com/watch?v=\1' },
{ priority: 40,
source_pattern: '\Ahttps?://(?:www\.|m\.)?youtube\.com/embed/([^/?#]+)(?:[?#].*)?\z',
replacement: 'https://www.youtube.com/watch?v=\1' },
{ priority: 50,
source_pattern:
'\Ahttps?://(?:www\.|m\.)?youtube\.com/watch\?(?:[^#&]+&)*v=([^&#]+)(?:[&#].*)?\z',
replacement: 'https://www.youtube.com/watch?v=\1' },
{ priority: 60,
source_pattern: '\Ahttps?://nico\.ms/([^/?#]+)(?:[?#].*)?\z',
replacement: 'https://www.nicovideo.jp/watch/\1' },
{ priority: 70,
source_pattern: '\Ahttps?://(?:www\.)?nicovideo\.jp/watch/([^?#/]+)(?:[?#].*)?\z',
replacement: 'https://www.nicovideo.jp/watch/\1' }
]
post_url_sanitisation_rule_scope = PostUrlSanitisationRule.unscoped
post_url_sanitisation_rules.each do |attributes|
priority = attributes.fetch(:priority)
source_pattern = attributes.fetch(:source_pattern)
next if post_url_sanitisation_rule_scope.exists?(priority:)
next if post_url_sanitisation_rule_scope.exists?(source_pattern:)
post_url_sanitisation_rule_scope.create!(attributes)
end
material_sync_source_uri = ENV['MATERIAL_SYNC_SOURCE_URI']
material_sync_source_file_id = ENV['MATERIAL_SYNC_SOURCE_FILE_ID']
if material_sync_source_uri.present? || material_sync_source_file_id.present?
source = MaterialSyncSource.find_or_initialize_by(
name: ENV.fetch('MATERIAL_SYNC_SOURCE_NAME', 'Legacy URI material sync'))
source.assign_attributes(
source_kind: ENV.fetch('MATERIAL_SYNC_SOURCE_KIND', 'uri'),
source_uri: material_sync_source_uri,
source_path: ENV['MATERIAL_SYNC_SOURCE_PATH'],
source_file_id: material_sync_source_file_id,
profile: ENV.fetch('MATERIAL_SYNC_SOURCE_PROFILE', 'legacy_drive'),
enabled: ENV.fetch('MATERIAL_SYNC_SOURCE_ENABLED', 'true') != 'false',
default_tag_name: ENV['MATERIAL_SYNC_SOURCE_TAG_NAME'],
export_path_prefix: ENV['MATERIAL_SYNC_EXPORT_PATH_PREFIX'])
source.save!
end
-1102
ファイルの表示
ファイル差分が大きすぎるため省略します 差分を読込み
-15
ファイルの表示
@@ -1,15 +0,0 @@
{
"name": "lib",
"version": "1.0.0",
"main": "screenshot.js",
"scripts": {
"test": "echo \"Error: no test specified\" && exit 1"
},
"keywords": [],
"author": "",
"license": "ISC",
"description": "",
"dependencies": {
"puppeteer": "^24.10.0"
}
}
-18
ファイルの表示
@@ -1,18 +0,0 @@
const puppeteer = require ('puppeteer')
const fs = require ('fs')
void (async () => {
const url = process.argv[2]
const output = process.argv[3]
const browser = await puppeteer.launch ({
args: ['--no-sandbox', '--disable-setuid-sandbox'] })
const page = await browser.newPage ()
await page.setViewport ({ width: 960, height: 960 })
await page.goto (url, { waitUntil: 'networkidle2', timeout: 15000 })
await page.screenshot ({ path: output })
await browser.close ()
}) ()
+51
ファイルの表示
@@ -0,0 +1,51 @@
namespace :materials do
desc '素材同期'
task sync: :environment do
result = MaterialSyncRunner.sync_enabled!
message = [
"materials:sync imported=#{ result.imported }",
"updated=#{ result.updated }",
"unchanged=#{ result.unchanged }",
"suppressed=#{ result.suppressed }",
"failed=#{ result.failed }"
].join(' ')
Rails.logger.info(message)
puts message
result.errors.each do |error|
Rails.logger.warn(error.to_json)
warn error.to_json
end
end
namespace :thumbnails do
desc '素材サムネールを補完'
task backfill: :environment do
counts = Hash.new(0)
Material
.with_attached_file
.with_attached_thumbnail
.find_each do |material|
next unless material.file.attached?
next if material.thumbnail.attached?
result = MaterialThumbnailGenerator.generate!(material)
counts[result] += 1
end
message =
"materials:thumbnails:backfill " \
"attached=#{ counts[:attached] } " \
"no_file=#{ counts[:no_file] } " \
"unsupported_content_type=#{ counts[:unsupported_content_type] } " \
"generation_failed=#{ counts[:generation_failed] } " \
"attach_failed=#{ counts[:attach_failed] } " \
"file_not_found=#{ counts[:file_not_found] } " \
"mini_magick_error=#{ counts[:mini_magick_error] }"
Rails.logger.info(message)
puts message
end
end
end
+65
ファイルの表示
@@ -0,0 +1,65 @@
require 'rails_helper'
RSpec.describe 'database seeds' do
before do
PostUrlSanitisationRule.unscoped.delete_all
end
it 'registers the initial post URL sanitisation rules' do
load_seeds
urls = {
'https://youtu.be/abc123?si=share' => youtube_url('abc123'),
'https://www.youtube.com/live/abc123?t=10' => youtube_url('abc123'),
'https://youtube.com/shorts/abc123?feature=share' => youtube_url('abc123'),
'https://m.youtube.com/embed/abc123' => youtube_url('abc123'),
'https://youtube.com/watch?feature=share&v=abc123&t=10' => youtube_url('abc123'),
'https://nico.ms/sm123?from=share#fragment' => nico_url('sm123'),
'https://www.nicovideo.jp/watch/sm123?ref=share#fragment' => nico_url('sm123')
}
expect(PostUrlSanitisationRule.count).to eq(7)
urls.each do |url, canonical_url|
expect(PostUrlSanitisationRule.sanitise(url)).to eq(canonical_url)
end
end
it 'does not overwrite or restore an existing rule' do
rule = PostUrlSanitisationRule.create!(
priority: 10,
source_pattern: '\Ahttps://example\.com/custom\z',
replacement: 'https://example.com/replacement'
)
rule.discard!
original_attributes = rule.reload.attributes
2.times { load_seeds }
persisted_rule = PostUrlSanitisationRule.unscoped.find(10)
expect(persisted_rule.attributes).to eq(original_attributes)
expect(PostUrlSanitisationRule.unscoped.count).to eq(7)
end
it 'does not duplicate a rule moved to another priority' do
source_pattern = '\Ahttps?://youtu\.be/([^/?#]+)(?:[?#].*)?\z'
PostUrlSanitisationRule.create!(
priority: 80,
source_pattern:,
replacement: 'https://example.com/custom/\1'
)
load_seeds
rules = PostUrlSanitisationRule.unscoped
expect(rules.where(source_pattern:).count).to eq(1)
expect(rules.find(80).replacement).to eq('https://example.com/custom/\1')
end
def load_seeds
load Rails.root.join('db/seeds.rb')
end
def youtube_url(video_id) = "https://www.youtube.com/watch?v=#{ video_id }"
def nico_url(video_id) = "https://www.nicovideo.jp/watch/#{ video_id }"
end
+8
ファイルの表示
@@ -0,0 +1,8 @@
FactoryBot.define do
factory :post_tag_section do
association :post
association :tag
begin_ms { 1_000 }
end_ms { 2_000 }
end
end
+6
ファイルの表示
@@ -0,0 +1,6 @@
FactoryBot.define do
factory :post_tag do
association :post
association :tag
end
end
+8
ファイルの表示
@@ -0,0 +1,8 @@
FactoryBot.define do
factory :post do
sequence(:url) { |n| "https://example.com/factory-post-#{ n }" }
title { 'factory post' }
thumbnail_base { nil }
uploaded_user { nil }
end
end
+57
ファイルの表示
@@ -0,0 +1,57 @@
require 'rails_helper'
RSpec.describe MaterialExportItem, type: :model do
let(:user) { create(:user, :member) }
let(:tag) { Tag.create!(tag_name: TagName.create!(name: 'export_item'), category: :material) }
let(:material) do
Material.create!(tag:, url: 'https://example.com/material',
created_by_user: user, updated_by_user: user)
end
it 'rejects blank export_path' do
item = described_class.new(material:, profile: 'legacy_drive', export_path: '')
expect(item).not_to be_valid
expect(item.errors[:export_path]).to be_present
end
it 'rejects absolute export_path' do
item = described_class.new(material:, profile: 'legacy_drive',
export_path: '/素材/a.png')
expect(item).not_to be_valid
expect(item.errors[:export_path]).to be_present
end
it 'rejects parent traversal export_path' do
item = described_class.new(material:, profile: 'legacy_drive',
export_path: '素材/../a.png')
expect(item).not_to be_valid
expect(item.errors[:export_path]).to be_present
end
it 'rejects double slash export_path' do
item = described_class.new(material:, profile: 'legacy_drive',
export_path: '素材//a.png')
expect(item).not_to be_valid
expect(item.errors[:export_path]).to be_present
end
it 'rejects dot segment export_path' do
item = described_class.new(material:, profile: 'legacy_drive',
export_path: './素材/a.png')
expect(item).not_to be_valid
expect(item.errors[:export_path]).to be_present
end
it 'rejects trailing slash export_path' do
item = described_class.new(material:, profile: 'legacy_drive',
export_path: '素材/a/')
expect(item).not_to be_valid
expect(item.errors[:export_path]).to be_present
end
end
+19
ファイルの表示
@@ -0,0 +1,19 @@
require 'rails_helper'
RSpec.describe Material, type: :model do
let(:user) { create(:user, :member) }
it 'allows only material source kinds, not suppression prefix kinds' do
material = described_class.new(url: 'https://example.com/material',
source_kind: 'google_drive_path_prefix',
source_path: '素材/危険',
created_by_user: user,
updated_by_user: user)
expect(material).not_to be_valid
expect(material.errors[:source_kind]).to be_present
material.source_kind = 'google_drive_path'
expect(material).to be_valid
end
end
+68
ファイルの表示
@@ -0,0 +1,68 @@
require 'rails_helper'
RSpec.describe Post, type: :model do
before do
PostUrlSanitisationRule.unscoped.delete_all
end
describe 'URL normalisation' do
it 'normalises the HTTP URL before applying sanitisation rules' do
PostUrlSanitisationRule.create!(
priority: 10,
source_pattern: '\\Ahttps://example\\.com/videos/([^/]+)\\z',
replacement: 'https://example.com/watch/\\1'
)
post = described_class.create!(
title: 'normalised URL',
url: ' https://EXAMPLE.com/videos/123/ '
)
expect(post.url).to eq('https://example.com/watch/123')
end
it 'does not normalise an unchanged URL when another attribute changes' do
post = create(:post)
post.update_column(:url, 'https://EXAMPLE.com/unchanged/')
post.update!(title: 'updated title')
expect(post.reload.url).to eq('https://EXAMPLE.com/unchanged/')
end
it 'validates the sanitised URL length' do
path = 'a' * 375
PostUrlSanitisationRule.create!(
priority: 10,
source_pattern: '\\Ahttps://example\\.com/(a+)\\z',
replacement: 'https://example.com/\\1\\1'
)
post = described_class.new(
title: 'long URL',
url: "https://example.com/#{ path }"
)
expect(post).to be_invalid
expect(post.errors.details.fetch(:url)).to include(
error: :too_long,
count: 768
)
end
it 'validates uniqueness after sanitisation' do
PostUrlSanitisationRule.create!(
priority: 10,
source_pattern: '\\Ahttps://example\\.com/alias\\z',
replacement: 'https://example.com/canonical'
)
create(:post, url: 'https://example.com/canonical')
post = described_class.new(title: 'duplicate URL', url: 'https://example.com/alias')
expect(post).to be_invalid
expect(post.errors.details.fetch(:url)).to include(error: :taken, value: post.url)
end
end
end
+41
ファイルの表示
@@ -0,0 +1,41 @@
RSpec.describe PostTag, type: :model do
describe '#sections' do
it 'loads sections by post_id and tag_id' do
post_tag = create(:post_tag)
section = create(:post_tag_section,
post: post_tag.post,
tag: post_tag.tag,
begin_ms: 1000,
end_ms: 2000)
expect(post_tag.sections).to contain_exactly(section)
end
it 'does not load sections for another tag on the same post' do
post = create(:post)
tag = create(:tag)
other_tag = create(:tag)
post_tag = create(:post_tag, post:, tag:)
create(:post_tag_section,
post:,
tag: other_tag,
begin_ms: 1000,
end_ms: 2000)
expect(post_tag.sections).to be_empty
end
it 'allows open-ended sections' do
post_tag = create(:post_tag)
section = create(:post_tag_section,
post: post_tag.post,
tag: post_tag.tag,
begin_ms: 1000,
end_ms: nil)
expect(section).to be_valid
expect(post_tag.sections).to contain_exactly(section)
end
end
end
+265
ファイルの表示
@@ -0,0 +1,265 @@
require 'rails_helper'
RSpec.describe PostUrlSanitisationRule, type: :model do
before do
described_class.unscoped.delete_all
end
describe 'validations' do
it 'requires a source pattern' do
rule = described_class.new(priority: 10, source_pattern: nil, replacement: '')
expect(rule).to be_invalid
expect(rule.errors.details.fetch(:source_pattern)).to eq([{ error: :blank }])
end
it 'requires a unique source pattern' do
described_class.create!(priority: 10, source_pattern: 'source', replacement: 'first')
rule = described_class.new(
priority: 20,
source_pattern: 'source',
replacement: 'second'
)
expect(rule).to be_invalid
expect(rule.errors.details.fetch(:source_pattern)).to include(
error: :taken,
value: 'source'
)
end
it 'rejects an invalid regexp' do
rule = described_class.new(priority: 10, source_pattern: '[', replacement: '')
expect(rule).to be_invalid
expect(rule.errors[:source_pattern]).to include('変な正規表現だね〜(笑)')
end
end
describe '.sanitise' do
it 'applies each active rule once in priority order' do
described_class.create!(priority: 30, source_pattern: 'c', replacement: 'd')
described_class.create!(priority: 10, source_pattern: 'a', replacement: 'aa')
described_class.create!(priority: 20, source_pattern: 'aa', replacement: 'c')
discarded = described_class.create!(
priority: 5,
source_pattern: '.',
replacement: 'discarded'
)
discarded.discard!
expect(described_class.sanitise('a')).to eq('d')
end
it 'canonicalises the initial YouTube and Nico URL forms' do
create_initial_rules
urls = {
'https://youtu.be/abc123?si=share' => youtube_url('abc123'),
'https://www.youtube.com/live/abc123?t=10' => youtube_url('abc123'),
'https://youtube.com/shorts/abc123?feature=share' => youtube_url('abc123'),
'https://m.youtube.com/embed/abc123' => youtube_url('abc123'),
'https://youtube.com/watch?feature=share&v=abc123&t=10' => youtube_url('abc123'),
'https://nico.ms/sm123?from=share#fragment' => nico_url('sm123'),
'https://www.nicovideo.jp/watch/sm123?ref=share#fragment' => nico_url('sm123')
}
urls.each do |url, canonical_url|
expect(described_class.sanitise(url)).to eq(canonical_url)
end
end
end
describe '.apply!' do
it 'locks posts in ID order and updates through temporary URLs' do
first = create(:post, url: 'https://example.com/source/1')
second = create(:post, url: 'https://example.com/source/2')
create_source_rule
sql = capture_sql { described_class.apply! }
lock_sql = sql.find { _1.match?(/SELECT .*posts.*FOR UPDATE/i) }
expect(lock_sql).to match(/ORDER BY .*posts.*id.* ASC/i)
expect(sql.grep(/post-url-sanitising\.invalid/).size).to eq(2)
expect(first.reload.url).to eq('https://example.com/canonical/1')
expect(second.reload.url).to eq('https://example.com/canonical/2')
end
it 'loads and compiles rules only once' do
create(:post, url: 'https://example.com/source/1')
create(:post, url: 'https://example.com/source/2')
create_source_rule
expect(described_class).to receive(:rules).once.and_call_original
described_class.apply!
end
it 'does not change post versions, version_no, or updated_at' do
post = create(:post, url: 'https://example.com/source/1')
post.update_columns(version_no: 7, updated_at: 1.day.ago)
original_updated_at = post.reload.updated_at
create_source_rule
expect { described_class.apply! }.not_to change(PostVersion, :count)
post.reload
expect(post.url).to eq('https://example.com/canonical/1')
expect(post.version_no).to eq(7)
expect(post.updated_at).to eq(original_updated_at)
end
invalid_urls = {
'a blank URL' => '',
'an unparseable URL' => 'https://[',
'a non-HTTP URL' => 'ftp://example.com/file',
'an HTTP URL without a host' => 'https:/path'
}
invalid_urls.each do |description, sanitised_url|
it "rolls back every update when sanitisation produces #{ description }" do
valid_post = create(:post, url: 'https://example.com/source/1')
invalid_post = create(:post, url: 'https://example.com/invalid')
create_source_rule
described_class.create!(
priority: 20,
source_pattern: '\\Ahttps://example\\.com/invalid\\z',
replacement: sanitised_url
)
expect { described_class.apply! }
.to raise_error(described_class::InvalidUrlError) { |error|
expect(error.invalid_rows).to eq([
{ post_id: invalid_post.id,
original_url: 'https://example.com/invalid',
sanitised_url: }
])
}
expect(valid_post.reload.url).to eq('https://example.com/source/1')
expect(invalid_post.reload.url).to eq('https://example.com/invalid')
end
end
it 'rolls back every update when sanitisation produces a URL longer than 768 characters' do
path = 'a' * 375
original_url = "https://example.com/#{ path }"
sanitised_url = "https://example.com/#{ path }#{ path }"
valid_post = create(:post, url: 'https://example.com/source/1')
invalid_post = create(:post, url: original_url)
create_source_rule
described_class.create!(
priority: 20,
source_pattern: '\\Ahttps://example\\.com/(a+)\\z',
replacement: 'https://example.com/\\1\\1'
)
expect { described_class.apply! }
.to raise_error(described_class::InvalidUrlError) { |error|
expect(error.invalid_rows).to eq([
{ post_id: invalid_post.id,
original_url:,
sanitised_url: }
])
}
expect(valid_post.reload.url).to eq('https://example.com/source/1')
expect(invalid_post.reload.url).to eq(original_url)
end
it 'rejects sanitised URL collisions case-insensitively without changing posts' do
source = create(:post, url: 'https://example.com/source/Foo')
canonical = create(:post, url: 'https://example.com/canonical/foo')
create_source_rule
expect { described_class.apply! }
.to raise_error(described_class::UrlConflictError) { |error|
expect(error.conflicts).to contain_exactly(
{ url: 'https://example.com/canonical/Foo',
post_id: source.id,
original_url: 'https://example.com/source/Foo' },
{ url: 'https://example.com/canonical/foo',
post_id: canonical.id,
original_url: 'https://example.com/canonical/foo' }
)
}
expect(source.reload.url).to eq('https://example.com/source/Foo')
expect(canonical.reload.url).to eq('https://example.com/canonical/foo')
expect(Post.count).to eq(2)
end
it 'converts a persisted URL constraint race into UrlConflictError' do
post = create(:post, url: 'https://example.com/source/1')
create_source_rule
conflict = { url: 'https://example.com/canonical/1',
post_id: post.id,
original_url: post.url }
database_error = ActiveRecord::RecordNotUnique.new('duplicate URL')
allow_any_instance_of(ActiveRecord::Relation)
.to receive(:update_all).and_raise(database_error)
allow(described_class).to receive(:build_persisted_conflicts).and_return([conflict])
expect { described_class.apply! }
.to raise_error(described_class::UrlConflictError) { |error|
expect(error.conflicts).to eq([conflict])
expect(error.cause).to equal(database_error)
}
expect(post.reload.url).to eq('https://example.com/source/1')
end
it 're-raises an unidentified RecordNotUnique error' do
post = create(:post, url: 'https://example.com/source/1')
create_source_rule
database_error = ActiveRecord::RecordNotUnique.new('another unique constraint')
allow_any_instance_of(ActiveRecord::Relation)
.to receive(:update_all).and_raise(database_error)
allow(described_class).to receive(:build_persisted_conflicts).and_return([])
expect { described_class.apply! }.to raise_error(database_error)
expect(post.reload.url).to eq('https://example.com/source/1')
end
end
def capture_sql
statements = []
subscriber = lambda do |_name, _start, _finish, _id, payload|
binds = payload.fetch(:binds, []).map { _1.value_for_database.to_s }
statements << ([payload.fetch(:sql)] + binds).join(' ')
end
ActiveSupport::Notifications.subscribed(subscriber, 'sql.active_record') { yield }
statements
end
def create_source_rule
described_class.create!(
priority: 10,
source_pattern: '\\Ahttps://example\\.com/source/(.+)\\z',
replacement: 'https://example.com/canonical/\\1'
)
end
def create_initial_rules
rules = [
['\\Ahttps?://youtu\\.be/([^/?#]+)(?:[?#].*)?\\z', youtube_url('\\1')],
['\\Ahttps?://(?:www\\.|m\\.)?youtube\\.com/live/([^/?#]+)(?:[?#].*)?\\z',
youtube_url('\\1')],
['\\Ahttps?://(?:www\\.|m\\.)?youtube\\.com/shorts/([^/?#]+)(?:[?#].*)?\\z',
youtube_url('\\1')],
['\\Ahttps?://(?:www\\.|m\\.)?youtube\\.com/embed/([^/?#]+)(?:[?#].*)?\\z',
youtube_url('\\1')],
['\\Ahttps?://(?:www\\.|m\\.)?youtube\\.com/watch\\?(?:[^#&]+&)*' \
'v=([^&#]+)(?:[&#].*)?\\z', youtube_url('\\1')],
['\\Ahttps?://nico\\.ms/([^/?#]+)(?:[?#].*)?\\z', nico_url('\\1')],
['\\Ahttps?://(?:www\\.)?nicovideo\\.jp/watch/([^?#/]+)(?:[?#].*)?\\z',
nico_url('\\1')]
]
rules.each_with_index do |(source_pattern, replacement), index|
described_class.create!(
priority: (index + 1) * 10,
source_pattern:,
replacement:
)
end
end
def youtube_url(video_id) = "https://www.youtube.com/watch?v=#{ video_id }"
def nico_url(video_id) = "https://www.nicovideo.jp/watch/#{ video_id }"
end
+34
ファイルの表示
@@ -0,0 +1,34 @@
require 'rails_helper'
RSpec.describe Setting, type: :model do
it 'accepts the default typed user settings' do
setting = described_class.new({ user: create(:user) }.merge(described_class.defaults))
expect(setting).to be_valid
end
it 'requires one settings row per user' do
user = create(:user)
described_class.create!({ user: }.merge(described_class.defaults))
duplicate = described_class.new({ user: }.merge(described_class.defaults))
expect(duplicate).not_to be_valid
expect(duplicate.errors[:user_id]).to be_present
end
it 'validates enum-like settings columns' do
setting = described_class.new(
{ user: create(:user),
theme: 'neon',
auto_fetch_title: 'sometimes',
auto_fetch_thumbnail: 'sometimes',
wiki_editor_mode: 'sideways' })
expect(setting).not_to be_valid
expect(setting.errors[:theme]).to be_present
expect(setting.errors[:auto_fetch_title]).to be_present
expect(setting.errors[:auto_fetch_thumbnail]).to be_present
expect(setting.errors[:wiki_editor_mode]).to be_present
end
end
+36
ファイルの表示
@@ -0,0 +1,36 @@
require 'rails_helper'
RSpec.describe TagImplication, type: :model do
it 'rejects a parent tag that would create a cycle' do
child = create(:tag, name: 'tag_implication_cycle_child')
parent = create(:tag, name: 'tag_implication_cycle_parent')
described_class.create!(tag: child, parent_tag: parent)
implication = described_class.new(tag: parent, parent_tag: child)
expect(implication).not_to be_valid
expect(implication.errors[:parent_tag_id]).to include(
'親タグに子孫タグを指定すると循環します.'
)
expect(implication.errors[:base]).to be_present
end
it 'terminates even when existing data already contains a cycle' do
child = create(:tag, name: 'tag_implication_existing_cycle_child')
parent = create(:tag, name: 'tag_implication_existing_cycle_parent')
ancestor = create(:tag, name: 'tag_implication_existing_cycle_ancestor')
described_class.create!(tag: parent, parent_tag: ancestor)
described_class.insert_all!(
[
{ tag_id: ancestor.id, parent_tag_id: parent.id,
created_at: Time.current, updated_at: Time.current }
]
)
implication = described_class.new(tag: child, parent_tag: parent)
expect(implication).to be_valid
end
end
+100 -1
ファイルの表示
@@ -19,6 +19,95 @@ RSpec.describe Tag, type: :model do
expect(error.tag_names).to eq([deprecated_tag.name]) expect(error.tag_names).to eq([deprecated_tag.name])
} }
end end
it 'rejects invalid section literals instead of treating them as zero' do
expect {
described_class.normalise_tags!(
['normalise_invalid_section[1:aa-2:00]'],
with_sections: true
)
}.to raise_error(Tag::SectionLiteralParseError)
end
it 'parses open-ended section literals' do
result = described_class.normalise_tags!(
['伊地知ニジカ[1:00-]'],
with_sections: true
)
tag = result.fetch(:tags).find { _1.name == '伊地知ニジカ' }
expect(result.fetch(:sections).fetch(tag.id)).to eq([[60_000, nil]])
end
it 'parses omitted begin as zero' do
result = described_class.normalise_tags!(
['伊地知ニジカ[-1:00]'],
with_sections: true
)
tag = result.fetch(:tags).find { _1.name == '伊地知ニジカ' }
expect(result.fetch(:sections).fetch(tag.id)).to eq([[0, 60_000]])
end
it 'treats fully open section literals as plain tags' do
result = described_class.normalise_tags!(
['伊地知ニジカ[-]'],
with_sections: true
)
tag = result.fetch(:tags).find { _1.name == '伊地知ニジカ' }
expect(result.fetch(:sections)[tag.id]).to be_nil
end
it 'treats [0:00-] as a plain tag' do
result = described_class.normalise_tags!(
['伊地知ニジカ[0:00-]'],
with_sections: true
)
tag = result.fetch(:tags).find { _1.name == '伊地知ニジカ' }
expect(result.fetch(:sections)[tag.id]).to be_nil
end
it 'expands zero-width sections to one millisecond' do
result = described_class.normalise_tags!(
['伊地知ニジカ[1:00-1:00]'],
with_sections: true
)
tag = result.fetch(:tags).find { _1.name == '伊地知ニジカ' }
expect(result.fetch(:sections).fetch(tag.id)).to eq([[60_000, 60_001]])
end
it 'swaps reversed section boundaries' do
result = described_class.normalise_tags!(
['伊地知ニジカ[2:00-1:00]'],
with_sections: true
)
tag = result.fetch(:tags).find { _1.name == '伊地知ニジカ' }
expect(result.fetch(:sections).fetch(tag.id)).to eq([[60_000, 120_000]])
end
it 'merges open-ended sections over later bounded sections' do
result = described_class.normalise_tags!(
['伊地知ニジカ[1:00-][2:00-3:00]'],
with_sections: true
)
tag = result.fetch(:tags).find { _1.name == '伊地知ニジカ' }
expect(result.fetch(:sections).fetch(tag.id)).to eq([[60_000, nil]])
end
it 'merges adjacent bounded and open-ended sections' do
result = described_class.normalise_tags!(
['伊地知ニジカ[1:00-3:00][3:00-]'],
with_sections: true
)
tag = result.fetch(:tags).find { _1.name == '伊地知ニジカ' }
expect(result.fetch(:sections).fetch(tag.id)).to eq([[60_000, nil]])
end
end end
describe '.expand_parent_tags' do describe '.expand_parent_tags' do
@@ -54,7 +143,17 @@ RSpec.describe Tag, type: :model do
first = create(:tag, name: 'expand_cycle_first') first = create(:tag, name: 'expand_cycle_first')
second = create(:tag, name: 'expand_cycle_second') second = create(:tag, name: 'expand_cycle_second')
TagImplication.create!(tag: first, parent_tag: second) TagImplication.create!(tag: first, parent_tag: second)
TagImplication.create!(tag: second, parent_tag: first) now = Time.current
TagImplication.insert_all!(
[
{
tag_id: second.id,
parent_tag_id: first.id,
created_at: now,
updated_at: now
}
]
)
expect(described_class.expand_parent_tags([first])).to contain_exactly(first, second) expect(described_class.expand_parent_tags([first])).to contain_exactly(first, second)
end end
+40
ファイルの表示
@@ -0,0 +1,40 @@
require 'rails_helper'
RSpec.describe UserThemeSlot, type: :model do
it 'accepts a theme slot for one user, base theme, and slot number' do
slot = described_class.new(user: create(:user),
base_theme: 'light',
slot_no: 1,
tokens: { 'background' => '0 0% 100%' })
expect(slot).to be_valid
end
it 'requires unique slot numbers per user and base theme' do
user = create(:user)
described_class.create!(user:,
base_theme: 'dark',
slot_no: 2,
tokens: { 'background' => '222.2 84% 4.9%' })
duplicate = described_class.new(user:,
base_theme: 'dark',
slot_no: 2,
tokens: { 'background' => '0 0% 100%' })
expect(duplicate).not_to be_valid
expect(duplicate.errors[:user_id]).to be_present
end
it 'validates base theme, slot number, and token object shape' do
slot = described_class.new(user: create(:user),
base_theme: 'system',
slot_no: 4,
tokens: 'not-object')
expect(slot).not_to be_valid
expect(slot.errors[:base_theme]).to be_present
expect(slot.errors[:slot_no]).to be_present
expect(slot.errors[:tokens]).to be_present
end
end
+466 -34
ファイルの表示
@@ -1,7 +1,10 @@
require 'rails_helper' require 'rails_helper'
RSpec.describe 'Materials API', type: :request do RSpec.describe 'Materials API', type: :request do
include ActiveJob::TestHelper
let!(:member_user) { create(:user, :member) } let!(:member_user) { create(:user, :member) }
let!(:admin_user) { create(:user, :admin) }
let!(:guest_user) { create(:user) } let!(:guest_user) { create(:user) }
def dummy_upload(filename: 'dummy.png', type: 'image/png', body: 'dummy') def dummy_upload(filename: 'dummy.png', type: 'image/png', body: 'dummy')
@@ -13,22 +16,29 @@ RSpec.describe 'Materials API', type: :request do
end end
def build_material(tag:, user:, parent: nil, file: dummy_upload, url: nil) def build_material(tag:, user:, parent: nil, file: dummy_upload, url: nil)
Material.new(tag:, parent:, url:, created_by_user: user, updated_by_user: user).tap do |material| Material.new(tag:, parent:, url:,
created_by_user: user,
updated_by_user: user).tap do |material|
material.file.attach(file) if file material.file.attach(file) if file
material.save! material.save!
end end
end end
describe 'GET /materials' do describe 'GET /materials' do
let!(:tag_a) { Tag.create!(tag_name: TagName.create!(name: 'material_index_a'), category: :material) } let!(:tag_a) do
let!(:tag_b) { Tag.create!(tag_name: TagName.create!(name: 'material_index_b'), category: :material) } Tag.create!(tag_name: TagName.create!(name: 'material_index_a'), category: :material)
end
let!(:tag_b) do
Tag.create!(tag_name: TagName.create!(name: 'material_index_b'), category: :material)
end
let!(:material_a) do let!(:material_a) do
build_material(tag: tag_a, user: member_user, file: dummy_upload(filename: 'a.png')) build_material(tag: tag_a, user: member_user, file: dummy_upload(filename: 'a.png'))
end end
let!(:material_b) do let!(:material_b) do
build_material(tag: tag_b, user: member_user, parent: material_a, file: dummy_upload(filename: 'b.png')) build_material(tag: tag_b, user: member_user, parent: material_a,
file: dummy_upload(filename: 'b.png'))
end end
before do before do
@@ -54,27 +64,36 @@ RSpec.describe 'Materials API', type: :request do
'name' => 'material_index_b', 'name' => 'material_index_b',
'category' => 'material' 'category' => 'material'
) )
expect(row['created_by_user']).to include(
'id' => member_user.id,
'name' => member_user.name
)
expect(row['content_type']).to eq('image/png') expect(row['content_type']).to eq('image/png')
expect(row['media_kind']).to eq('image')
end end
it 'filters materials by tag_id' do it 'filters materials by q' do
get '/materials', params: { tag_id: material_a.tag_id } get '/materials', params: { q: 'material_index_a' }
expect(response).to have_http_status(:ok) expect(response).to have_http_status(:ok)
expect(json['count']).to eq(1) expect(json['count']).to eq(1)
expect(response_materials.map { |m| m['id'] }).to eq([material_a.id]) expect(response_materials.map { |m| m['id'] }).to eq([material_a.id])
end end
it 'filters materials by parent_id' do it 'filters materials by tag_state' do
get '/materials', params: { parent_id: material_a.id } untagged =
build_material(tag: nil, user: member_user,
file: dummy_upload(filename: 'untagged.png'))
get '/materials', params: { tag_state: 'untagged' }
expect(response).to have_http_status(:ok) expect(response).to have_http_status(:ok)
expect(json['count']).to eq(1) expect(json['count']).to eq(1)
expect(response_materials.map { |m| m['id'] }).to eq([material_b.id]) expect(response_materials.map { |m| m['id'] }).to eq([untagged.id])
end
it 'filters materials by media_kind' do
get '/materials', params: { media_kind: 'image' }
expect(response).to have_http_status(:ok)
expect(json['count']).to eq(2)
expect(response_materials.map { |m| m['media_kind'] }).to all(eq('image'))
end end
it 'paginates and keeps total count' do it 'paginates and keeps total count' do
@@ -94,10 +113,74 @@ RSpec.describe 'Materials API', type: :request do
expect(response_materials.size).to eq(1) expect(response_materials.size).to eq(1)
expect(response_materials.first['id']).to eq(material_b.id) expect(response_materials.first['id']).to eq(material_b.id)
end end
it 'filters by descendant tags and returns stable parent tag groups' do
root =
Tag.create!(tag_name: TagName.create!(name: 'material_scope_root'),
category: :material)
child_b =
Tag.create!(tag_name: TagName.create!(name: 'material_scope_b'),
category: :material)
child_a =
Tag.create!(tag_name: TagName.create!(name: 'material_scope_a'),
category: :material)
deprecated =
Tag.create!(tag_name: TagName.create!(name: 'material_scope_old'),
category: :material,
deprecated_at: Time.current)
grandchild =
Tag.create!(tag_name: TagName.create!(name: 'material_scope_grandchild'),
category: :material)
root_material =
build_material(tag: root, user: member_user,
file: dummy_upload(filename: 'root.png'))
child_a_material =
build_material(tag: child_a, user: member_user,
file: dummy_upload(filename: 'child_a.png'))
grandchild_material =
build_material(tag: grandchild, user: member_user,
file: dummy_upload(filename: 'grandchild.png'))
TagImplication.create!(parent_tag: root, tag: child_b)
TagImplication.create!(parent_tag: root, tag: child_a)
TagImplication.create!(parent_tag: child_b, tag: deprecated)
TagImplication.create!(parent_tag: deprecated, tag: grandchild)
get '/materials',
params: { tag_id: root.id, include_descendants: '1',
group_by: 'parent_tag', sort: 'id', direction: 'asc' }
expect(response).to have_http_status(:ok)
expect(response_materials.map { |m| m['id'] })
.to include(root_material.id, child_a_material.id, grandchild_material.id)
expect(json['tag_scope']).to eq(
'tag' => {
'id' => root.id,
'name' => 'material_scope_root',
'category' => 'material'
},
'include_descendants' => true
)
expect(json['groups'].map { |group| group.dig('tag', 'name') })
.to eq(['material_scope_a', 'material_scope_b', 'material_scope_root'])
expect(json['groups'].find { |group| group.dig('tag', 'id') == child_b.id }
.fetch('material_ids')).to eq([grandchild_material.id])
end
it 'returns nil tag_scope when tag_id is unknown' do
get '/materials', params: { tag_id: 999_999, group_by: 'parent_tag' }
expect(response).to have_http_status(:ok)
expect(json['tag_scope']).to be_nil
expect(json['groups']).to eq([])
expect(response_materials).to eq([])
end
end end
describe 'GET /materials/:id' do describe 'GET /materials/:id' do
let!(:tag) { Tag.create!(tag_name: TagName.create!(name: 'material_show'), category: :material) } let!(:tag) do
Tag.create!(tag_name: TagName.create!(name: 'material_show'), category: :material)
end
let!(:material) do let!(:material) do
build_material(tag:, user: member_user, file: dummy_upload(filename: 'show.png')) build_material(tag:, user: member_user, file: dummy_upload(filename: 'show.png'))
end end
@@ -138,9 +221,22 @@ RSpec.describe 'Materials API', type: :request do
end end
end end
context 'when logged in' do context 'when logged in but not member' do
before { sign_in_as(guest_user) } before { sign_in_as(guest_user) }
it 'returns 403' do
post '/materials', params: {
tag: 'material_create_guest_forbidden',
file: dummy_upload
}
expect(response).to have_http_status(:forbidden)
end
end
context 'when member' do
before { sign_in_as(member_user) }
it 'returns 422 when tag is blank' do it 'returns 422 when tag is blank' do
post '/materials', params: { tag: ' ', file: dummy_upload } post '/materials', params: { tag: ' ', file: dummy_upload }
@@ -162,24 +258,68 @@ RSpec.describe 'Materials API', type: :request do
expect do expect do
post '/materials', params: { post '/materials', params: {
tag: 'material_create_new', tag: 'material_create_new',
file: dummy_upload(filename: 'created.png') file: dummy_upload(filename: 'created.png'),
export_paths: { legacy_drive: '伊地知ニジカ/created.png' }
} }
end.to change(Material, :count).by(1) end.to change(Material, :count).by(1)
.and change(Tag, :count).by(1) .and change(Tag, :count).by(1)
.and change(TagName, :count).by(1) .and change(TagName, :count).by(1)
.and change(MaterialVersion, :count).by(1)
expect(response).to have_http_status(:created) expect(response).to have_http_status(:created)
material = Material.order(:id).last material = Material.order(:id).last
expect(material.tag.name).to eq('material_create_new') expect(material.tag.name).to eq('material_create_new')
expect(material.tag.category).to eq('material') expect(material.tag.category).to eq('material')
expect(material.created_by_user).to eq(guest_user) expect(material.created_by_user).to eq(member_user)
expect(material.updated_by_user).to eq(guest_user) expect(material.updated_by_user).to eq(member_user)
expect(material.file.attached?).to be(true) expect(material.file.attached?).to be(true)
expect(material.version_no).to eq(1)
expect(material.material_versions.first.event_type).to eq('create')
expect(material.material_export_items.first.export_path).to eq('伊地知ニジカ/created.png')
expect(material.material_versions.first.export_paths_json).to eq(
'legacy_drive' => '伊地知ニジカ/created.png'
)
expect(json['id']).to eq(material.id) expect(json['id']).to eq(material.id)
expect(json.dig('tag', 'name')).to eq('material_create_new') expect(json.dig('tag', 'name')).to eq('material_create_new')
expect(json['content_type']).to eq('image/png') expect(json['content_type']).to eq('image/png')
expect(json.dig('export_paths', 'legacy_drive')).to eq('伊地知ニジカ/created.png')
end
it 'creates a create tag_version for a newly created material tag' do
expect do
post '/materials', params: {
tag: 'material_create_versioned_tag',
file: dummy_upload(filename: 'created.png')
}
end.to change(TagVersion, :count).by(1)
expect(response).to have_http_status(:created)
tag = Tag.joins(:tag_name).find_by!(tag_names: { name: 'material_create_versioned_tag' })
version = tag.tag_versions.order(:version_no).last
expect(version.event_type).to eq('create')
expect(version.name).to eq('material_create_versioned_tag')
expect(version.category).to eq('material')
expect(version.created_by_user).to eq(member_user)
end
it 'snapshots attached file metadata and sha256' do
post '/materials', params: {
tag: 'material_create_file_version',
file: dummy_upload(filename: 'created.png', body: 'sha-body')
}
expect(response).to have_http_status(:created)
version = Material.order(:id).last.material_versions.first
expect(version.file_blob_id).to be_present
expect(version.file_filename).to eq('created.png')
expect(version.file_content_type).to eq('image/png')
expect(version.file_byte_size).to eq('sha-body'.bytesize)
expect(version.file_sha256).to eq(Digest::SHA256.hexdigest('sha-body'))
end end
it 'returns 422 when the existing tag is not material/character' do it 'returns 422 when the existing tag is not material/character' do
@@ -219,11 +359,33 @@ RSpec.describe 'Materials API', type: :request do
expect(response).to have_http_status(:created) expect(response).to have_http_status(:created)
expect(json['url']).to eq('https://example.com/material-source') expect(json['url']).to eq('https://example.com/material-source')
end end
it 'rejects sha256-blocked file upload' do
sha256 = Digest::SHA256.hexdigest('blocked-body')
MaterialImportBlock.create!(match_kind: 'sha256',
sha256:,
reason: 'copyright_high_risk',
created_by_user: admin_user)
expect do
post '/materials', params: {
tag: 'material_blocked_create',
file: dummy_upload(filename: 'blocked.png', body: 'blocked-body')
}
end.not_to change(Material, :count)
expect(response).to have_http_status(:unprocessable_entity)
expect(json.fetch('errors')).to include(
'file' => ['抑止された素材です: copyright_high_risk']
)
end
end end
end end
describe 'PUT /materials/:id' do describe 'PUT /materials/:id' do
let!(:tag) { Tag.create!(tag_name: TagName.create!(name: 'material_update_old'), category: :material) } let!(:tag) do
Tag.create!(tag_name: TagName.create!(name: 'material_update_old'), category: :material)
end
let!(:material) do let!(:material) do
build_material(tag:, user: member_user, file: dummy_upload(filename: 'old.png')) build_material(tag:, user: member_user, file: dummy_upload(filename: 'old.png'))
end end
@@ -277,25 +439,26 @@ RSpec.describe 'Materials API', type: :request do
'tag' => ['タグは必須です.']) 'tag' => ['タグは必須です.'])
end end
it 'returns 422 when both file and url are blank' do it 'keeps the existing file when file and url are omitted' do
put "/materials/#{ material.id }", params: { put "/materials/#{ material.id }", params: {
tag: 'material_update_no_payload' tag: 'material_update_no_payload'
} }
expect(response).to have_http_status(:unprocessable_entity) expect(response).to have_http_status(:ok)
expect(json.fetch('errors')).to include( expect(material.reload.file.attached?).to be(true)
'file' => ['ファイルまたは URL は必須です.'],
'url' => ['ファイルまたは URL は必須です.'])
end end
it 'updates tag, url, file, and updated_by_user' do it 'updates tag, url, file, and updated_by_user' do
old_blob_id = material.file.blob.id old_blob_id = material.file.blob.id
put "/materials/#{ material.id }", params: { expect do
tag: 'material_update_new', put "/materials/#{ material.id }", params: {
url: 'https://example.com/updated-source', tag: 'material_update_new',
file: dummy_upload(filename: 'updated.jpg', type: 'image/jpeg') url: 'https://example.com/updated-source',
} file: dummy_upload(filename: 'updated.jpg', type: 'image/jpeg'),
export_paths: { legacy_drive: '伊地知ニジカ/updated.jpg' }
}
end.to change(MaterialVersion, :count).by(2)
expect(response).to have_http_status(:ok) expect(response).to have_http_status(:ok)
@@ -306,8 +469,15 @@ RSpec.describe 'Materials API', type: :request do
expect(material.updated_by_user).to eq(member_user) expect(material.updated_by_user).to eq(member_user)
expect(material.file.attached?).to be(true) expect(material.file.attached?).to be(true)
expect(material.file.blob.id).not_to eq(old_blob_id) expect(material.file.blob.id).not_to eq(old_blob_id)
expect(ActiveStorage::Blob.where(id: old_blob_id).exists?).to be(true)
expect(material.file.blob.filename.to_s).to eq('updated.jpg') expect(material.file.blob.filename.to_s).to eq('updated.jpg')
expect(material.file.blob.content_type).to eq('image/jpeg') expect(material.file.blob.content_type).to eq('image/jpeg')
expect(material.version_no).to eq(2)
expect(material.material_versions.order(:version_no).last.event_type).to eq('update')
expect(material.material_export_items.first.export_path).to eq('伊地知ニジカ/updated.jpg')
expect(material.material_versions.order(:version_no).last.export_paths_json).to eq(
'legacy_drive' => '伊地知ニジカ/updated.jpg'
)
expect(json['id']).to eq(material.id) expect(json['id']).to eq(material.id)
expect(json['file']).to be_present expect(json['file']).to be_present
@@ -315,7 +485,74 @@ RSpec.describe 'Materials API', type: :request do
expect(json.dig('tag', 'name')).to eq('material_update_new') expect(json.dig('tag', 'name')).to eq('material_update_new')
end end
it 'purges the existing file when file is omitted and url is provided' do it 'creates a create tag_version when update creates a new tag' do
expect do
put "/materials/#{ material.id }", params: {
tag: 'material_update_versioned_tag',
file: dummy_upload(filename: 'updated.png')
}
end.to change(Tag, :count).by(1)
.and change(TagName, :count).by(1)
.and change(TagVersion, :count).by(1)
expect(response).to have_http_status(:ok)
tag = Tag.joins(:tag_name).find_by!(tag_names: { name: 'material_update_versioned_tag' })
version = tag.tag_versions.order(:version_no).last
expect(version.event_type).to eq('create')
expect(version.name).to eq('material_update_versioned_tag')
expect(version.category).to eq('material')
expect(version.created_by_user).to eq(member_user)
end
it 'backfills a create tag_version for an existing material tag without history' do
existing_tag =
Tag.create!(tag_name: TagName.create!(name: 'material_update_existing_no_history'),
category: :material)
expect(existing_tag.tag_versions).to be_empty
expect do
put "/materials/#{ material.id }", params: {
tag: 'material_update_existing_no_history',
file: dummy_upload(filename: 'updated.png')
}
end.to change(TagVersion, :count).by(1)
expect(response).to have_http_status(:ok)
version = existing_tag.reload.tag_versions.order(:version_no).last
expect(version.event_type).to eq('create')
expect(version.name).to eq('material_update_existing_no_history')
expect(version.category).to eq('material')
expect(version.created_by_user).to eq(member_user)
end
it 'backfills a create tag_version for an existing character tag without history' do
existing_tag =
Tag.create!(tag_name: TagName.create!(name: 'material_update_character_no_history'),
category: :character)
expect(existing_tag.tag_versions).to be_empty
expect do
put "/materials/#{ material.id }", params: {
tag: 'material_update_character_no_history',
file: dummy_upload(filename: 'updated.png')
}
end.to change(TagVersion, :count).by(1)
expect(response).to have_http_status(:ok)
version = existing_tag.reload.tag_versions.order(:version_no).last
expect(version.event_type).to eq('create')
expect(version.name).to eq('material_update_character_no_history')
expect(version.category).to eq('character')
expect(version.created_by_user).to eq(member_user)
end
it 'detaches the existing file without purging blob when url replaces file' do
old_blob_id = material.file.blob.id old_blob_id = material.file.blob.id
put "/materials/#{ material.id }", params: { put "/materials/#{ material.id }", params: {
@@ -331,9 +568,7 @@ RSpec.describe 'Materials API', type: :request do
expect(material.updated_by_user).to eq(member_user) expect(material.updated_by_user).to eq(member_user)
expect(material.file.attached?).to be(false) expect(material.file.attached?).to be(false)
expect( expect(ActiveStorage::Blob.where(id: old_blob_id).exists?).to be(true)
ActiveStorage::Blob.where(id: old_blob_id).exists?
).to be(false)
expect(json['id']).to eq(material.id) expect(json['id']).to eq(material.id)
expect(json['file']).to be_nil expect(json['file']).to be_nil
@@ -341,11 +576,208 @@ RSpec.describe 'Materials API', type: :request do
expect(json.dig('tag', 'name')).to eq('material_update_remove_file') expect(json.dig('tag', 'name')).to eq('material_update_remove_file')
expect(json['url']).to eq('https://example.com/updated-source') expect(json['url']).to eq('https://example.com/updated-source')
end end
it 'does not increase version for the same snapshot update' do
MaterialVersionRecorder.record!(material:, event_type: :create,
created_by_user: member_user)
TagVersioning.ensure_snapshot!(tag, created_by_user: member_user)
expect do
put "/materials/#{ material.id }", params: {
tag: 'material_update_old'
}
end.not_to change(MaterialVersion, :count)
expect(tag.reload.tag_versions.count).to eq(1)
expect(response).to have_http_status(:ok)
expect(material.reload.version_no).to eq(1)
end
it 'records update version when only export_path changes' do
MaterialVersionRecorder.record!(material:, event_type: :create,
created_by_user: member_user)
expect do
put "/materials/#{ material.id }", params: {
tag: 'material_update_old',
export_paths: { legacy_drive: '素材/only-path.png' }
}
end.to change(MaterialVersion, :count).by(1)
expect(response).to have_http_status(:ok)
expect(material.reload.material_export_items.first.export_path).to eq('素材/only-path.png')
expect(material.material_versions.order(:version_no).last.export_paths_json).to eq(
'legacy_drive' => '素材/only-path.png'
)
end
it 'removes export_path item when blank is submitted' do
MaterialExportItem.create!(material:, profile: 'legacy_drive',
export_path: '素材/remove.png',
created_by_user: member_user)
MaterialVersionRecorder.record!(material:, event_type: :create,
created_by_user: member_user)
expect do
put "/materials/#{ material.id }", params: {
tag: 'material_update_old',
export_paths: { legacy_drive: '' }
}
end.to change(MaterialExportItem, :count).by(-1)
.and change(MaterialVersion, :count).by(1)
expect(response).to have_http_status(:ok)
expect(material.reload.material_export_items).to be_empty
expect(material.material_versions.order(:version_no).last.export_paths_json).to eq({})
end
it 'rejects sha256-blocked replacement file' do
sha256 = Digest::SHA256.hexdigest('blocked-update')
MaterialImportBlock.create!(match_kind: 'sha256',
sha256:,
reason: 'source_owner_request',
created_by_user: admin_user)
put "/materials/#{ material.id }", params: {
tag: 'material_update_old',
file: dummy_upload(filename: 'blocked.png', body: 'blocked-update')
}
expect(response).to have_http_status(:unprocessable_entity)
expect(material.reload.file.blob.filename.to_s).to eq('old.png')
end
end
end
describe 'GET /materials/download.zip' do
let!(:tag_a) { Tag.create!(tag_name: TagName.create!(name: 'zip_a'), category: :material) }
let!(:tag_b) { Tag.create!(tag_name: TagName.create!(name: 'zip_b'), category: :material) }
let!(:material_a) do
build_material(tag: tag_a, user: member_user,
file: dummy_upload(filename: 'a.png', body: 'zip-a'))
end
let!(:material_b) do
build_material(tag: tag_b, user: member_user,
file: dummy_upload(filename: 'b.png', body: 'zip-b'))
end
before do
MaterialExportItem.create!(material: material_a, profile: 'legacy_drive',
export_path: '素材/a.png',
created_by_user: member_user)
MaterialExportItem.create!(material: material_b, profile: 'legacy_drive',
export_path: '素材/b.png',
created_by_user: member_user)
end
it 'uses material_export_items.export_path as ZIP entry paths' do
get '/materials/download.zip', params: { profile: 'legacy_drive' }
expect(response).to have_http_status(:ok)
expect(response.media_type).to eq('application/zip')
expect(response.body.b).to include('素材/a.png'.b)
expect(response.body.b).to include('素材/b.png'.b)
end
it 'filters by tag_id' do
get '/materials/download.zip', params: { profile: 'legacy_drive', tag_id: tag_a.id }
expect(response).to have_http_status(:ok)
expect(response.body.b).to include('素材/a.png'.b)
expect(response.body.b).not_to include('素材/b.png'.b)
end
it 'does not include disabled export items' do
material_b.material_export_items.first.update!(enabled: false)
get '/materials/download.zip', params: { profile: 'legacy_drive' }
expect(response).to have_http_status(:ok)
expect(response.body.b).to include('素材/a.png'.b)
expect(response.body.b).not_to include('素材/b.png'.b)
end
end
describe 'GET /materials/versions' do
let!(:tag) do
Tag.create!(tag_name: TagName.create!(name: 'material_history'), category: :material)
end
let!(:material) do
build_material(tag:, user: member_user, file: dummy_upload(filename: 'history.png'))
end
it 'returns material versions in reverse chronological order' do
MaterialVersionRecorder.record!(material:, event_type: :create,
created_by_user: member_user)
material.update!(url: 'https://example.com/history')
MaterialVersionRecorder.record!(material:, event_type: :update,
created_by_user: admin_user)
get '/materials/versions', params: { material_id: material.id }
expect(response).to have_http_status(:ok)
expect(json['count']).to eq(2)
versions = json.fetch('versions')
expect(versions.map { |v| v['version_no'] }).to eq([2, 1])
expect(versions.first).to include(
'material_id' => material.id,
'event_type' => 'update',
'tag_name' => 'material_history',
'file_filename' => 'history.png',
'url' => 'https://example.com/history'
)
expect(versions.first['created_by_user']).to include('id' => admin_user.id)
end
it 'filters material versions by tag and event_type' do
MaterialVersionRecorder.record!(material:, event_type: :create,
created_by_user: member_user)
get '/materials/versions', params: {
tag: 'material_history',
event_type: 'create'
}
expect(response).to have_http_status(:ok)
expect(json.fetch('versions').map { |v| v['material_id'] }).to include(material.id)
expect(json.fetch('versions').map { |v| v['event_type'] }).to all(eq('create'))
end
end
describe 'material sync suppressions' do
it 'requires member permission for index' do
sign_out
get '/materials/suppressions'
expect(response).to have_http_status(:unauthorized)
end
it 'creates source suppression as member' do
sign_in_as(member_user)
expect do
post '/materials/suppressions', params: {
source_kind: 'google_drive_path_prefix',
drive_path: '伊地知ニジカ/危険素材',
reason: 'copyright_high_risk'
}
end.to change(MaterialSyncSuppression, :count).by(1)
expect(response).to have_http_status(:created)
expect(json).to include(
'source_kind' => 'google_drive_path_prefix',
'drive_path' => '伊地知ニジカ/危険素材',
'normalized_source_key' => 'google_drive_path_prefix:伊地知ニジカ/危険素材'
)
end end
end end
describe 'DELETE /materials/:id' do describe 'DELETE /materials/:id' do
let!(:tag) { Tag.create!(tag_name: TagName.create!(name: 'material_destroy'), category: :material) } let!(:tag) do
Tag.create!(tag_name: TagName.create!(name: 'material_destroy'), category: :material)
end
let!(:material) do let!(:material) do
build_material(tag:, user: member_user, file: dummy_upload(filename: 'destroy.png')) build_material(tag:, user: member_user, file: dummy_upload(filename: 'destroy.png'))
end end
+112
ファイルの表示
@@ -0,0 +1,112 @@
require 'rails_helper'
RSpec.describe 'Post imports API', type: :request do
let(:member) { create(:user, :member) }
before do
allow(Preview::UrlSafety).to receive(:validate) do |url|
[URI.parse(url), ['8.8.8.8']]
end
allow(PostMetadataFetcher).to receive(:fetch).and_return(
title: 'fetched title',
thumbnail_base: nil,
tags: ''
)
end
describe 'POST /posts/import/preview' do
it 'requires a member' do
sign_out
post '/posts/import/preview', params: { source: 'https://example.com/post' }
expect(response).to have_http_status(:unauthorized)
sign_in_as(create(:user, :guest))
post '/posts/import/preview', params: { source: 'https://example.com/post' }
expect(response).to have_http_status(:forbidden)
end
it 'parses a URL list and returns preview rows' do
sign_in_as(member)
post '/posts/import/preview', params: {
source: " https://example.com/one \r\n\r\nhttps://example.com/two"
}
expect(response).to have_http_status(:ok)
expect(json.fetch('rows').map { _1.fetch('source_row') }).to eq([1, 3])
expect(json.fetch('rows').map { _1.fetch('url') }).to eq([
'https://example.com/one',
'https://example.com/two'
])
end
it 'returns a safe 400 response for an invalid source' do
sign_in_as(member)
post '/posts/import/preview', params: { source: '' }
expect(response).to have_http_status(:bad_request)
expect(json.fetch('message')).to eq('URL を入力してください.')
end
end
describe 'POST /posts/import/validate' do
it 'accepts camel-case row properties and returns their warnings' do
sign_in_as(member)
post '/posts/import/validate', params: {
rows: [{
sourceRow: '1',
url: 'https://example.com/post',
metadataUrl: 'https://example.com/post',
attributes: { title: 'manual title' },
provenance: { url: 'manual', title: 'manual' },
tagSources: { automatic: '', manual: '' },
fieldWarnings: { title: ['old warning'] },
baseWarnings: ['base warning']
}],
changed_row: -1
}
expect(response).to have_http_status(:ok)
result = json.fetch('rows').first
expect(result.fetch('source_row')).to eq(1)
expect(result.fetch('tag_sources')).to eq('automatic' => '', 'manual' => '')
expect(result.fetch('field_warnings')).to eq('title' => ['old warning'])
expect(result.fetch('base_warnings')).to eq(['base warning'])
end
it 'rejects a non-array rows value with 400' do
sign_in_as(member)
post '/posts/import/validate', params: { rows: { sourceRow: 1 }, changed_row: -1 }
expect(response).to have_http_status(:bad_request)
expect(json.fetch('message')).to eq('取込行の形式が不正です.')
end
end
describe 'POST /posts/import' do
it 'returns a formal skipped result for an existing post' do
existing = create(:post, url: 'https://example.com/existing')
sign_in_as(member)
post '/posts/import', params: {
rows: [{
sourceRow: 1,
url: existing.url,
attributes: { title: 'ignored' },
provenance: { url: 'manual', title: 'manual' },
tagSources: { automatic: '', manual: '' }
}]
}
expect(response).to have_http_status(:ok)
expect(json).to include('created' => 0, 'skipped' => 1, 'failed' => 0)
expect(json.fetch('rows').first).to include(
'status' => 'skipped',
'existing_post_id' => existing.id
)
end
end
end
+287
ファイルの表示
@@ -1,4 +1,5 @@
require 'rails_helper' require 'rails_helper'
require 'base64'
require 'set' require 'set'
include ActiveSupport::Testing::TimeHelpers include ActiveSupport::Testing::TimeHelpers
@@ -8,6 +9,11 @@ RSpec.describe 'Posts API', type: :request do
# resized_thumbnail! が MiniMagick 依存でコケやすいので request spec ではスタブしとくのが無難。 # resized_thumbnail! が MiniMagick 依存でコケやすいので request spec ではスタブしとくのが無難。
before do before do
allow_any_instance_of(Post).to receive(:resized_thumbnail!).and_return(true) allow_any_instance_of(Post).to receive(:resized_thumbnail!).and_return(true)
allow(Post).to receive(:resized_thumbnail_attachment).and_return(
io: StringIO.new('dummy'),
filename: 'resized_thumbnail.jpg',
content_type: 'image/jpeg'
)
end end
def create_nico_tag!(name) def create_nico_tag!(name)
@@ -19,6 +25,18 @@ RSpec.describe 'Posts API', type: :request do
Rack::Test::UploadedFile.new(StringIO.new('dummy'), 'image/jpeg', original_filename: 'dummy.jpg') Rack::Test::UploadedFile.new(StringIO.new('dummy'), 'image/jpeg', original_filename: 'dummy.jpg')
end end
def real_thumbnail_upload
gif =
Base64.decode64(
'R0lGODdhAQABAIABAP///wAAACwAAAAAAQABAAACAkQBADs=')
Rack::Test::UploadedFile.new(
StringIO.new(gif),
'image/gif',
original_filename: 'thumbnail.gif'
)
end
def post_write_params params = { } def post_write_params params = { }
{ parent_post_ids: '' }.merge(params) { parent_post_ids: '' }.merge(params)
end end
@@ -36,6 +54,7 @@ RSpec.describe 'Posts API', type: :request do
title: post.title, title: post.title,
url: post.url, url: post.url,
thumbnail_base: post.thumbnail_base, thumbnail_base: post.thumbnail_base,
video_ms: post.video_ms,
tags: post.snapshot_tag_names.join(' '), tags: post.snapshot_tag_names.join(' '),
parent_post_ids: post.snapshot_parent_post_ids.join(' '), parent_post_ids: post.snapshot_parent_post_ids.join(' '),
original_created_from: post.original_created_from, original_created_from: post.original_created_from,
@@ -127,6 +146,22 @@ RSpec.describe 'Posts API', type: :request do
expect(all_tag_names).to include("spec_tag") expect(all_tag_names).to include("spec_tag")
end end
it 'keeps children and sections keys in non-detail tag responses' do
PostTagSection.create!(post: hit_post, tag:, begin_ms: 1_000, end_ms: nil)
get '/posts'
expect(response).to have_http_status(:ok)
hit_json = json.fetch('posts').find { |post| post['id'] == hit_post.id }
tag_json = hit_json.fetch('tags').find { |item| item['name'] == 'spec_tag' }
expect(tag_json.fetch('children')).to eq([])
expect(tag_json.fetch('sections')).to eq([
{ 'begin_ms' => 1_000, 'end_ms' => nil }
])
end
context "when q is provided" do context "when q is provided" do
it "filters posts by q (hit case)" do it "filters posts by q (hit case)" do
get "/posts", params: { tags: "spec_tag" } get "/posts", params: { tags: "spec_tag" }
@@ -696,6 +731,66 @@ RSpec.describe 'Posts API', type: :request do
expect(json['tags'][0]).to have_key('name') expect(json['tags'][0]).to have_key('name')
end end
it '201 when posting manually with a thumbnail' do
sign_in_as(member)
allow(Post).to receive(:resized_thumbnail_attachment).and_call_original
post '/posts', params: post_write_params(
title: 'thumbnail post',
url: 'https://example.com/thumbnail-post',
tags: 'spec_tag',
thumbnail: real_thumbnail_upload
)
expect(response).to have_http_status(:created)
post_record = Post.find(json.fetch('id'))
expect(post_record.thumbnail).to be_attached
expect(post_record.thumbnail.blob.content_type).to eq('image/jpeg')
expect { post_record.thumbnail.download }.not_to raise_error
end
it 'resizes the thumbnail before the create transaction begins' do
sign_in_as(member)
open_transactions = []
baseline_open_transactions = Post.connection.open_transactions
allow(Post).to receive(:resized_thumbnail_attachment) do |_upload|
open_transactions << Post.connection.open_transactions
{ io: StringIO.new('dummy'),
filename: 'resized_thumbnail.jpg',
content_type: 'image/jpeg' }
end
post '/posts', params: post_write_params(
title: 'transaction post',
url: 'https://example.com/transaction-post',
tags: 'spec_tag',
thumbnail: dummy_upload
)
expect(response).to have_http_status(:created)
expect(open_transactions).to eq([baseline_open_transactions])
end
it 'returns 422 and does not create a post when thumbnail resize fails' do
sign_in_as(member)
allow(Post).to receive(:resized_thumbnail_attachment).and_raise(MiniMagick::Error)
expect {
post '/posts', params: post_write_params(
title: 'broken thumbnail post',
url: 'https://example.com/broken-thumbnail-post',
tags: 'spec_tag',
thumbnail: dummy_upload
)
}.not_to change(Post, :count)
expect(response).to have_http_status(:unprocessable_entity)
expect(json.fetch('errors')).to include(
'thumbnail' => ['サムネイル画像の変換に失敗しました.']
)
end
it '201 and creates post + tags when member and tags have aliases' do it '201 and creates post + tags when member and tags have aliases' do
sign_in_as(member) sign_in_as(member)
@@ -767,6 +862,198 @@ RSpec.describe 'Posts API', type: :request do
expect(saved_names).not_to include('deprecated_parent', 'deprecated_grandparent') expect(saved_names).not_to include('deprecated_parent', 'deprecated_grandparent')
end end
it 'returns validation error for an invalid section literal' do
sign_in_as(member)
post '/posts', params: post_write_params(
title: 'invalid section literal',
url: 'https://example.com/invalid-section-literal',
tags: 'spec_tag[1:aa-2:00]',
thumbnail: dummy_upload
)
expect(response).to have_http_status(:unprocessable_entity)
expect(json).to include(
'type' => 'validation_error',
'message' => '入力内容を確認してください.'
)
expect(json.fetch('errors')).to include(
'tags' => ['タグ区間の記法が不正です.']
)
end
it 'creates a video post with duration' do
sign_in_as(member)
post '/posts', params: post_write_params(
title: 'video post',
url: 'https://example.com/video-post',
tags: '動画 spec_tag',
duration: '3:00.500',
thumbnail: dummy_upload
)
expect(response).to have_http_status(:created)
expect(Post.find(json.fetch('id')).video_ms).to eq(180_500)
expect(json.fetch('video_ms')).to eq(180_500)
end
it 'creates a video post with number input duration seconds' do
sign_in_as(member)
post '/posts', params: post_write_params(
title: 'video post seconds',
url: 'https://example.com/video-post-seconds',
tags: '動画 spec_tag',
duration: '180.5',
thumbnail: dummy_upload
)
expect(response).to have_http_status(:created)
expect(Post.find(json.fetch('id')).video_ms).to eq(180_500)
expect(json.fetch('video_ms')).to eq(180_500)
end
it 'clears video_ms when the saved tags do not include 動画' do
sign_in_as(member)
post '/posts', params: post_write_params(
title: 'non video post',
url: 'https://example.com/non-video-post',
tags: 'spec_tag',
duration: '3:00',
thumbnail: dummy_upload
)
expect(response).to have_http_status(:created)
expect(Post.find(json.fetch('id')).video_ms).to be_nil
end
it 'returns validation error when a bounded section exceeds duration' do
sign_in_as(member)
post '/posts', params: post_write_params(
title: 'too long section',
url: 'https://example.com/too-long-section',
tags: '動画 伊地知ニジカ[2:50-3:10]',
duration: '3:00',
thumbnail: dummy_upload
)
expect(response).to have_http_status(:unprocessable_entity)
expect(json.fetch('errors')).to include(
'video_ms' => ['タグ区間の終端が動画時間を超えてゐます.']
)
end
it 'saves open-ended sections with end_ms NULL' do
sign_in_as(member)
post '/posts', params: post_write_params(
title: 'open ended section literal',
url: 'https://example.com/open-ended-section-literal',
tags: '動画 伊地知ニジカ[1:00-]',
duration: '3:00',
thumbnail: dummy_upload
)
expect(response).to have_http_status(:created)
created_post = Post.find(json.fetch('id'))
tag = Tag.joins(:tag_name).find_by!(tag_names: { name: '伊地知ニジカ' })
section = PostTagSection.find_by!(post: created_post, tag:)
expect(section.begin_ms).to eq(60_000)
expect(section.end_ms).to be_nil
end
it 'does not save sections for [-]' do
sign_in_as(member)
post '/posts', params: post_write_params(
title: 'fully open section literal',
url: 'https://example.com/fully-open-section-literal',
tags: '伊地知ニジカ[-]',
thumbnail: dummy_upload
)
expect(response).to have_http_status(:created)
created_post = Post.find(json.fetch('id'))
tag = Tag.joins(:tag_name).find_by!(tag_names: { name: '伊地知ニジカ' })
expect(PostTagSection.find_by(post: created_post, tag:)).to be_nil
end
it 'does not save sections for [0:00-]' do
sign_in_as(member)
post '/posts', params: post_write_params(
title: 'zero open ended section literal',
url: 'https://example.com/zero-open-ended-section-literal',
tags: '伊地知ニジカ[0:00-]',
thumbnail: dummy_upload
)
expect(response).to have_http_status(:created)
created_post = Post.find(json.fetch('id'))
tag = Tag.joins(:tag_name).find_by!(tag_names: { name: '伊地知ニジカ' })
expect(PostTagSection.find_by(post: created_post, tag:)).to be_nil
end
it 'returns end_ms null for open-ended sections in show response' do
sign_in_as(member)
post '/posts', params: post_write_params(
title: 'show open ended section literal',
url: 'https://example.com/show-open-ended-section-literal',
tags: '動画 伊地知ニジカ[1:00-]',
duration: '3:00',
thumbnail: dummy_upload
)
expect(response).to have_http_status(:created)
get "/posts/#{ json.fetch('id') }"
expect(response).to have_http_status(:ok)
tag_json = json.fetch('tags').find { |item| item['name'] == '伊地知ニジカ' }
expect(tag_json.fetch('sections')).to eq([
{ 'begin_ms' => 60_000, 'end_ms' => nil }
])
end
it 'allows open-ended sections when begin is within duration' do
sign_in_as(member)
post '/posts', params: post_write_params(
title: 'valid open ended section literal',
url: 'https://example.com/valid-open-ended-section-literal',
tags: '動画 伊地知ニジカ[1:00-]',
duration: '3:00',
thumbnail: dummy_upload
)
expect(response).to have_http_status(:created)
end
it 'rejects open-ended sections when begin equals duration' do
sign_in_as(member)
post '/posts', params: post_write_params(
title: 'invalid open ended section literal',
url: 'https://example.com/invalid-open-ended-section-literal',
tags: '動画 伊地知ニジカ[3:00-]',
duration: '3:00',
thumbnail: dummy_upload
)
expect(response).to have_http_status(:unprocessable_entity)
expect(json.fetch('errors')).to include(
'video_ms' => ['タグ区間の開始が動画時間以上です.']
)
end
context "when nico tag already exists in tags" do context "when nico tag already exists in tags" do
before do before do
Tag.find_undiscard_or_create_by!( Tag.find_undiscard_or_create_by!(
+32 -14
ファイルの表示
@@ -1,28 +1,46 @@
require "rails_helper" require 'rails_helper'
RSpec.describe "Preview", type: :request do RSpec.describe 'Preview', type: :request do
describe "GET /preview/title" do describe 'GET /preview/title' do
it "401 unless logged in" do it '401 unless logged in' do
sign_out sign_out
get "/preview/title", params: { url: "example.com" } get '/preview/title', params: { url: 'example.com' }
expect(response).to have_http_status(:unauthorized) expect(response).to have_http_status(:unauthorized)
end end
it "400 when url blank" do it '403 when logged in as guest' do
sign_in_as(create(:user)) sign_in_as(create(:user, :guest))
get "/preview/title", params: { url: "" } get '/preview/title', params: { url: 'example.com' }
expect(response).to have_http_status(:forbidden)
end
it '400 when url blank' do
sign_in_as(create(:user, :member))
get '/preview/title', params: { url: '' }
expect(response).to have_http_status(:bad_request) expect(response).to have_http_status(:bad_request)
end end
it "returns parsed title (stubbing URI.open)" do it 'returns parsed title' do
sign_in_as(create(:user)) sign_in_as(create(:user, :member))
fake_html = "<html><head><title> Hello </title></head></html>" allow(Preview::ThumbnailFetcher)
allow(URI).to receive(:open).and_return(StringIO.new(fake_html)) .to receive(:title)
.with('example.com')
.and_return('Hello')
get "/preview/title", params: { url: "example.com" } get '/preview/title', params: { url: 'example.com' }
expect(response).to have_http_status(:ok) expect(response).to have_http_status(:ok)
expect(json["title"]).to eq("Hello") expect(json['title']).to eq('Hello')
end
it '413 when fetched response is too large' do
sign_in_as(create(:user, :member))
allow(Preview::ThumbnailFetcher)
.to receive(:title)
.and_raise(Preview::HttpFetcher::ResponseTooLarge, '外部データが大きすぎます.')
get '/preview/title', params: { url: 'example.com' }
expect(response).to have_http_status(:payload_too_large)
end end
end end
end end
+11
ファイルの表示
@@ -56,6 +56,17 @@ RSpec.describe "TagChildren", type: :request do
expect(response).to have_http_status(:no_content) expect(response).to have_http_status(:no_content)
end end
it 'returns 422 and does not create relation when the new link makes a cycle' do
TagImplication.create!(tag: parent, parent_tag: child)
expect {
do_request
}.not_to change(TagImplication, :count)
expect(response).to have_http_status(:unprocessable_entity)
expect(TagImplication.where(tag: child, parent_tag: parent)).not_to exist
end
end end
context "when Tag.find raises (invalid ids)" do context "when Tag.find raises (invalid ids)" do
+183 -3
ファイルの表示
@@ -581,6 +581,58 @@ RSpec.describe 'Tags API', type: :request do
expect(response).to have_http_status(:ok) expect(response).to have_http_status(:ok)
expect(wiki_page.reload.wiki_versions.count).to eq(before_wiki_version_count) expect(wiki_page.reload.wiki_versions.count).to eq(before_wiki_version_count)
end end
it 'full update で旧名 alias が残った tag を PATCH で旧名へ戻せる' do
put "/tags/#{ tag.id }", params: {
name: 'patch_roundtrip_target',
category: 'general',
aliases: 'unko',
parent_tags: '',
deprecated: '0',
}
expect(response).to have_http_status(:ok)
expect(TagName.find_by!(name: 'spec_tag').canonical).to eq(tag.reload.tag_name)
patch "/tags/#{ tag.id }", params: { name: 'spec_tag' }
expect(response).to have_http_status(:ok)
tag.reload
expect(tag.name).to eq('spec_tag')
expect(tag.tag_name.canonical_id).to be_nil
expect(TagName.find_by!(name: 'patch_roundtrip_target').canonical).to eq(tag.tag_name)
end
it '別 tag の正規名には変更できない' do
wiki_page =
Wiki::Commit.create_content!(
tag_name: tag.tag_name,
body: 'patch collision wiki',
created_by_user: member_user,
message: 'init')
patch "/tags/#{ tag.id }", params: { name: 'unknown' }
expect(response).to have_http_status(:unprocessable_entity)
expect(json.fetch('errors')).to include(
'name' => ['その名前は既に使はれてゐます.']
)
expect(tag.reload.name).to eq('spec_tag')
expect(tag.tag_name.aliases.map(&:name)).to contain_exactly('unko')
expect(wiki_page.reload.tag_name).to eq(tag.tag_name)
end
it 'system tag の name は変更できない' do
system_tag = Tag.bot
patch "/tags/#{ system_tag.id }", params: { name: 'patch_system_tag_renamed' }
expect(response).to have_http_status(:unprocessable_entity)
expect(system_tag.reload.name).to eq('bot操作')
end
end end
end end
@@ -745,7 +797,17 @@ RSpec.describe 'Tags API', type: :request do
) )
TagImplication.create!(tag: first, parent_tag: root_material) TagImplication.create!(tag: first, parent_tag: root_material)
TagImplication.create!(tag: second, parent_tag: first) TagImplication.create!(tag: second, parent_tag: first)
TagImplication.create!(tag: first, parent_tag: second) now = Time.current
TagImplication.insert_all!(
[
{
tag_id: first.id,
parent_tag_id: second.id,
created_at: now,
updated_at: now
}
]
)
get '/tags/with-depth', params: { parent: root_material.id } get '/tags/with-depth', params: { parent: root_material.id }
@@ -1029,6 +1091,42 @@ RSpec.describe 'Tags API', type: :request do
expect(versions.second.created_by_user_id).to eq(member_user.id) expect(versions.second.created_by_user_id).to eq(member_user.id)
end end
it '同じ tag の旧 alias へ戻しても rename できる' do
put "/tags/#{ tag.id }", params: {
name: 'put_roundtrip_b',
category: 'general',
aliases: 'unko',
parent_tags: '',
deprecated: '0',
}
expect(response).to have_http_status(:ok)
put "/tags/#{ tag.id }", params: {
name: 'spec_tag',
category: 'general',
aliases: 'unko',
parent_tags: '',
deprecated: '0',
}
expect(response).to have_http_status(:ok)
tag.reload
expect(tag.name).to eq('spec_tag')
expect(TagName.find_by!(name: 'put_roundtrip_b').canonical).to eq(tag.tag_name)
expect(tag.tag_name.aliases.map(&:name)).to contain_exactly('put_roundtrip_b', 'unko')
expect(tag.tag_name.aliases.map(&:name)).not_to include('spec_tag')
expect(alias_tn.reload.canonical).to eq(tag.tag_name)
version = tag.tag_versions.order(:version_no).last
expect(version.event_type).to eq('update')
expect(version.name).to eq('spec_tag')
expect(version.aliases.split).to contain_exactly('put_roundtrip_b', 'unko')
end
it 'parent tag の snapshot も作成する' do it 'parent tag の snapshot も作成する' do
old_parent = Tag.create!( old_parent = Tag.create!(
tag_name: TagName.create!(name: 'put_snapshot_old_parent'), tag_name: TagName.create!(name: 'put_snapshot_old_parent'),
@@ -1153,6 +1251,48 @@ RSpec.describe 'Tags API', type: :request do
) )
end end
it 'wiki を持つ tag を旧 alias へ戻しても wiki を新 canonical へ移す' do
wiki_page =
Wiki::Commit.create_content!(
tag_name: tag.tag_name,
body: 'wiki body before',
created_by_user: member_user,
message: 'init')
expect {
put "/tags/#{ tag.id }", params: {
name: 'put_wiki_roundtrip_b',
category: 'general',
aliases: 'unko',
parent_tags: '',
deprecated: '0',
}
put "/tags/#{ tag.id }", params: {
name: 'spec_tag',
category: 'general',
aliases: 'unko',
parent_tags: '',
deprecated: '0',
}
}
.to change(TagVersion, :count).by(3)
.and change(WikiVersion, :count).by(2)
expect(response).to have_http_status(:ok)
tag.reload
expect(wiki_page.reload.tag_name).to eq(tag.tag_name)
expect(TagName.find_by!(name: 'put_wiki_roundtrip_b').wiki_page).to be_nil
expect(TagName.find_by!(name: 'put_wiki_roundtrip_b').canonical).to eq(tag.tag_name)
versions = wiki_page.wiki_versions.order(:version_no).last(2)
expect(versions.map(&:event_type)).to eq(['update', 'update'])
expect(versions.map(&:title)).to eq(['put_wiki_roundtrip_b', 'spec_tag'])
end
it '別名を他 tag から奪った場合、奪はれた側の tag version も作成する' do it '別名を他 tag から奪った場合、奪はれた側の tag version も作成する' do
old_owner = Tag.create!( old_owner = Tag.create!(
tag_name: TagName.create!(name: 'put_alias_old_owner'), tag_name: TagName.create!(name: 'put_alias_old_owner'),
@@ -1191,9 +1331,49 @@ RSpec.describe 'Tags API', type: :request do
expect(old_owner_versions.second.aliases.split).not_to include('put_stolen_alias') expect(old_owner_versions.second.aliases.split).not_to include('put_stolen_alias')
end end
it 'parent_tags に指定すると循環する tag は 422 にする' do it '別 tag の alias 名を rename で奪へる' do
pending '#332 で対応予定' old_owner = Tag.create!(
tag_name: TagName.create!(name: 'put_alias_collision_owner'),
category: :general
)
stolen_alias = TagName.create!(
name: 'put_alias_collision_name',
canonical: old_owner.tag_name
)
wiki_page =
Wiki::Commit.create_content!(
tag_name: tag.tag_name,
body: 'put collision wiki',
created_by_user: member_user,
message: 'init')
put "/tags/#{ tag.id }", params: {
name: 'put_alias_collision_name',
category: 'general',
aliases: 'unko',
parent_tags: '',
deprecated: '0',
}
expect(response).to have_http_status(:ok)
tag.reload
old_owner.reload
stolen_alias.reload
expect(tag.name).to eq('put_alias_collision_name')
expect(stolen_alias.canonical_id).to be_nil
expect(TagName.find_by!(name: 'spec_tag').canonical).to eq(tag.tag_name)
expect(old_owner.tag_name.aliases.map(&:name)).not_to include('put_alias_collision_name')
old_owner_versions = old_owner.tag_versions.order(:version_no)
expect(old_owner_versions.last.event_type).to eq('update')
expect(old_owner_versions.last.aliases.split).not_to include('put_alias_collision_name')
expect(wiki_page.reload.tag_name).to eq(tag.tag_name)
end
it 'parent_tags に指定すると循環する tag は 422 にする' do
child = Tag.create!( child = Tag.create!(
tag_name: TagName.create!(name: 'put_cycle_child'), tag_name: TagName.create!(name: 'put_cycle_child'),
category: :general category: :general
+98
ファイルの表示
@@ -0,0 +1,98 @@
require 'rails_helper'
RSpec.describe 'user settings', type: :request do
describe 'GET /users/settings' do
it 'requires a current user' do
sign_out
get '/users/settings'
expect(response).to have_http_status(:unauthorized)
end
it 'returns defaults and creates a missing settings row' do
user = create(:user)
sign_in_as(user)
expect {
get '/users/settings'
}.to change(Setting, :count).by(1)
expect(response).to have_http_status(:ok)
expect(json).to eq(
'theme' => 'system',
'auto_fetch_title' => 'manual',
'auto_fetch_thumbnail' => 'manual',
'wiki_editor_mode' => 'split')
expect(user.reload.setting).to be_present
end
end
describe 'PATCH /users/settings' do
it 'requires a current user' do
sign_out
patch '/users/settings', params: { theme: 'dark' }, as: :json
expect(response).to have_http_status(:unauthorized)
end
it 'updates multiple typed settings at once' do
user = create(:user)
sign_in_as(user)
patch '/users/settings',
params: {
theme: 'dark',
auto_fetch_title: 'auto',
auto_fetch_thumbnail: 'off',
wiki_editor_mode: 'preview' },
as: :json
expect(response).to have_http_status(:ok)
expect(json).to include(
'theme' => 'dark',
'auto_fetch_title' => 'auto',
'auto_fetch_thumbnail' => 'off',
'wiki_editor_mode' => 'preview')
expect(user.reload.setting).to have_attributes(
theme: 'dark',
auto_fetch_title: 'auto',
auto_fetch_thumbnail: 'off',
wiki_editor_mode: 'preview')
end
it 'does not treat Rails parameter wrapping as an editable setting' do
user = create(:user)
sign_in_as(user)
patch '/users/settings',
params: { theme: 'light', user_setting: { theme: 'dark' } },
as: :json
expect(response).to have_http_status(:ok)
expect(user.reload.setting.theme).to eq('light')
end
it 'returns validation_error for type mismatches' do
sign_in_as(create(:user))
patch '/users/settings', params: { theme: 1 }, as: :json
expect(response).to have_http_status(:unprocessable_entity)
expect(json).to include('type' => 'validation_error')
expect(json.fetch('errors')).to include(
'theme' => ['値の型が不正です.'])
end
it 'returns validation_error for invalid enum values' do
sign_in_as(create(:user))
patch '/users/settings', params: { theme: 'neon' }, as: :json
expect(response).to have_http_status(:unprocessable_entity)
expect(json).to include('type' => 'validation_error')
expect(json.fetch('errors').fetch('theme')).to be_present
end
end
end
+104
ファイルの表示
@@ -0,0 +1,104 @@
require 'rails_helper'
RSpec.describe 'user theme slots', type: :request do
describe 'GET /users/theme_slots' do
it 'requires a current user' do
sign_out
get '/users/theme_slots'
expect(response).to have_http_status(:unauthorized)
end
it 'returns only the current user slots in stable order' do
user = create(:user)
other_user = create(:user)
sign_in_as(user)
UserThemeSlot.create!(user:, base_theme: 'light', slot_no: 2, tokens: { 'a' => 1 })
UserThemeSlot.create!(user:, base_theme: 'dark', slot_no: 1, tokens: { 'b' => 2 })
UserThemeSlot.create!(user: other_user,
base_theme: 'dark',
slot_no: 3,
tokens: { 'c' => 3 })
get '/users/theme_slots'
expect(response).to have_http_status(:ok)
expect(json.map { |slot| [slot.fetch('base_theme'), slot.fetch('slot_no')] }).to eq(
[['dark', 1], ['light', 2]])
expect(json.map { |slot| slot.fetch('tokens') }).to eq(
[{ 'b' => 2 }, { 'a' => 1 }])
end
end
describe 'PUT /users/theme_slots/:base_theme/:slot_no' do
it 'requires a current user' do
sign_out
put '/users/theme_slots/light/1',
params: { tokens: { background: '0 0% 100%' } },
as: :json
expect(response).to have_http_status(:unauthorized)
end
it 'creates a theme slot' do
user = create(:user)
sign_in_as(user)
expect {
put '/users/theme_slots/light/1',
params: { tokens: { background: '0 0% 100%', tagColours: { nico: '#ffffff' } } },
as: :json
}.to change(UserThemeSlot, :count).by(1)
expect(response).to have_http_status(:ok)
expect(json).to include('base_theme' => 'light', 'slot_no' => 1)
expect(json.fetch('tokens')).to include(
'background' => '0 0% 100%',
'tagColours' => { 'nico' => '#ffffff' })
expect(user.theme_slots.find_by!(base_theme: 'light', slot_no: 1).tokens).to include(
'background' => '0 0% 100%')
end
it 'updates an existing theme slot instead of creating a duplicate' do
user = create(:user)
sign_in_as(user)
UserThemeSlot.create!(user:, base_theme: 'dark', slot_no: 2, tokens: { 'old' => true })
expect {
put '/users/theme_slots/dark/2',
params: { tokens: { background: '222.2 84% 4.9%' } },
as: :json
}.not_to change(UserThemeSlot, :count)
expect(response).to have_http_status(:ok)
expect(user.theme_slots.find_by!(base_theme: 'dark', slot_no: 2).tokens).to eq(
'background' => '222.2 84% 4.9%')
end
it 'rejects invalid path parameters and token shapes' do
sign_in_as(create(:user))
put '/users/theme_slots/system/4',
params: { tokens: 'not-object' },
as: :json
expect(response).to have_http_status(:unprocessable_entity)
expect(json.fetch('errors')).to include(
'base_theme' => ['値が不正です.'])
end
it 'rejects non-object tokens for valid slots' do
sign_in_as(create(:user))
put '/users/theme_slots/light/1',
params: { tokens: 'not-object' },
as: :json
expect(response).to have_http_status(:unprocessable_entity)
expect(json.fetch('errors')).to include(
'tokens' => ['JSON object で指定してください.'])
end
end
end
+23
ファイルの表示
@@ -0,0 +1,23 @@
require 'rails_helper'
RSpec.describe MaterialFileSha256 do
describe '.metadata_sha256' do
it 'reads sha256 from Hash metadata' do
blob = instance_double(ActiveStorage::Blob, metadata: { 'sha256' => 'abc123' })
expect(described_class.metadata_sha256(blob)).to eq('abc123')
end
it 'reads sha256 from JSON string metadata' do
blob = instance_double(ActiveStorage::Blob, metadata: '{"sha256":"abc123"}')
expect(described_class.metadata_sha256(blob)).to eq('abc123')
end
it 'returns nil for invalid metadata' do
blob = instance_double(ActiveStorage::Blob, metadata: '{')
expect(described_class.metadata_sha256(blob)).to be_nil
end
end
end
+128
ファイルの表示
@@ -0,0 +1,128 @@
require 'rails_helper'
RSpec.describe MaterialSyncImporter do
let(:user) { create(:user, :member) }
let(:tag) { Tag.create!(tag_name: TagName.create!(name: 'sync_tag'), category: :material) }
def tempfile_for body
Tempfile.new(['material-sync-importer', '.png']).tap do |file|
file.binmode
file.write(body)
file.rewind
end
end
it 'creates an untagged material from a sync candidate with file_downloader' do
result =
described_class.import!(
source_kind: 'google_drive_file',
source_path: '素材/a.png',
source_file_id: 'drive-file-a',
normalized_source_key: 'google_drive_file:drive-file-a',
filename: 'a.png',
content_type: 'image/png',
file_sha256: Digest::SHA256.hexdigest('sync-body'),
file_downloader: -> { tempfile_for('sync-body') },
export_path: '素材/a.png',
tag: nil,
url: nil,
created_by_user: user,
updated_by_user: user)
material = result.material.reload
expect(result.action).to eq(:imported)
expect(material.tag).to be_nil
expect(material.url).to be_nil
expect(material.file).to be_attached
expect(material.file.blob.filename.to_s).to eq('a.png')
expect(material.material_export_items.first.export_path).to eq('素材/a.png')
end
it 'does not clear a human-assigned tag or URL on existing Drive sync' do
material =
Material.create!(tag:, url: 'https://example.com/human',
source_kind: 'google_drive_file',
source_path: '素材/a.png',
source_file_id: 'drive-file-a',
created_by_user: user,
updated_by_user: user)
material.file.attach(
io: StringIO.new('same-body'),
filename: 'a.png',
content_type: 'image/png')
material.file.blob.metadata['sha256'] = Digest::SHA256.hexdigest('same-body')
material.file.blob.save!
MaterialExportItem.create!(material:,
profile: 'legacy_drive',
export_path: '素材/a.png',
created_by_user: user)
result =
described_class.import!(
source_kind: 'google_drive_file',
source_path: '素材/a.png',
source_file_id: 'drive-file-a',
filename: 'a.png',
content_type: 'image/png',
file_sha256: Digest::SHA256.hexdigest('same-body'),
export_path: '素材/a.png',
tag: nil,
url: nil,
updated_by_user: user)
expect(result.action).to eq(:unchanged)
expect(material.reload.tag).to eq(tag)
expect(material.url).to eq('https://example.com/human')
end
it 'suppresses a Google Drive candidate by relative path prefix' do
MaterialSyncSuppression.create!(source_kind: 'google_drive_path_prefix',
drive_path: '素材/危険',
reason: 'copyright_high_risk',
created_by_user: user)
result =
described_class.import!(
source_kind: 'google_drive_file',
source_path: '素材/危険/a.png',
source_file_id: 'drive-file-b',
filename: 'a.png',
content_type: 'image/png',
file_sha256: Digest::SHA256.hexdigest('blocked-body'),
file_downloader: -> { tempfile_for('blocked-body') },
export_path: '素材/危険/a.png',
updated_by_user: user)
expect(result.action).to eq(:suppressed)
expect(result.suppression.normalized_source_key)
.to eq('google_drive_path_prefix:素材/危険')
expect(Material.find_by(source_file_id: 'drive-file-b')).to be_nil
end
it 'rechecks import blocks after downloaded file sha256 is known' do
MaterialImportBlock.create!(match_kind: 'sha256',
sha256: Digest::SHA256.hexdigest('blocked-body'),
reason: 'copyright_high_risk',
created_by_user: user)
blob_count = ActiveStorage::Blob.count
expect do
result =
described_class.import!(
source_kind: 'google_drive_file',
source_path: '素材/blocked.png',
source_file_id: 'drive-file-blocked',
filename: 'blocked.png',
content_type: 'image/png',
file_downloader: -> { tempfile_for('blocked-body') },
export_path: '素材/blocked.png',
updated_by_user: user)
expect(result.action).to eq(:suppressed)
expect(result.suppression.reason).to eq('copyright_high_risk')
end.not_to change(Material, :count)
expect(ActiveStorage::Blob.count).to eq(blob_count)
expect(Material.find_by(source_file_id: 'drive-file-blocked')).to be_nil
end
end
+89
ファイルの表示
@@ -0,0 +1,89 @@
require 'rails_helper'
RSpec.describe MaterialSyncRunner do
let(:user) { create(:user, :member) }
let(:source) do
MaterialSyncSource.create!(
name: 'Drive source',
source_kind: 'google_drive_path',
source_file_id: 'folder-123',
profile: 'legacy_drive',
created_by_user: user,
updated_by_user: user)
end
let(:drive_client) { instance_double(GoogleDrive::ApiClient) }
describe '#sync!' do
it 'imports google drive path candidates as they are yielded' do
first_entry = { id: 'file-1',
name: 'a.png',
mime_type: 'image/png',
relative_path: '素材/a.png',
sha256_checksum: 'sha-a',
web_view_link: 'https://drive.google.com/file/d/file-1/view',
web_content_link: nil }
second_entry = { id: 'file-2',
name: 'b.png',
mime_type: 'image/png',
relative_path: '素材/b.png',
sha256_checksum: 'sha-b',
web_view_link: 'https://drive.google.com/file/d/file-2/view',
web_content_link: nil }
yielded = []
allow(GoogleDrive::ApiClient).to receive(:new).and_return(drive_client)
allow(drive_client).to receive(:each_material_file_under_folder) do |folder_id, &block|
expect(folder_id).to eq('folder-123')
block.call(first_entry)
expect(yielded).to eq(['素材/a.png'])
block.call(second_entry)
end
allow(MaterialSyncImporter).to receive(:import!) do |candidate|
yielded << candidate.fetch(:source_path)
instance_double(MaterialSyncImporter::Result,
action: yielded.last == '素材/a.png' ? :imported : :updated)
end
result = described_class.new(source).sync!
expect(yielded).to eq(['素材/a.png', '素材/b.png'])
expect(result.imported).to eq(1)
expect(result.updated).to eq(1)
expect(result.unchanged).to eq(0)
expect(source.reload.last_synced_at).to be_present
end
it 'logs google drive path progress at first item and summary' do
entry = { id: 'file-1',
name: 'a.png',
mime_type: 'image/png',
relative_path: '素材/a.png',
sha256_checksum: 'sha-a',
web_view_link: 'https://drive.google.com/file/d/file-1/view',
web_content_link: nil }
logged = []
allow(GoogleDrive::ApiClient).to receive(:new).and_return(drive_client)
allow(drive_client).to receive(:each_material_file_under_folder) do |_folder_id, &block|
block.call(entry)
end
allow(MaterialSyncImporter).to receive(:import!)
.and_return(instance_double(MaterialSyncImporter::Result, action: :imported))
allow(Rails.logger).to receive(:info) { |message| logged << JSON.parse(message) }
described_class.new(source).sync!
progress_logs = logged.select { |row| row['folder_id'] == 'folder-123' }
expect(progress_logs.map { |row| row['progress'] }).to eq(['scan', 'summary'])
expect(progress_logs.last).to include(
'material_sync_source_id' => source.id,
'material_sync_source_name' => 'Drive source',
'scanned_count' => 1,
'imported' => 1,
'updated' => 0,
'unchanged' => 0,
'suppressed' => 0,
'failed' => 0)
end
end
end
+187
ファイルの表示
@@ -0,0 +1,187 @@
require 'rails_helper'
RSpec.describe PostImportPreviewer do
before do
allow(Preview::UrlSafety).to receive(:validate) do |url|
[URI.parse(url), ['8.8.8.8']]
end
end
def row(source_row:, url:, attributes: { }, provenance: { }, **values)
{
source_row:,
url:,
attributes:,
provenance:
}.merge(values)
end
describe '#preview_rows' do
it 'marks an existing post for skipping without fetching metadata' do
existing = create(:post, url: 'https://example.com/existing')
expect(PostMetadataFetcher).not_to receive(:fetch)
result = described_class.new.preview_rows(rows: [
row(source_row: 1, url: 'https://EXAMPLE.com/existing/')
]).first
expect(result).to include(
url: 'https://example.com/existing',
skip_reason: 'existing',
existing_post_id: existing.id,
validation_errors: {}
)
expect(result.fetch(:field_warnings)).to eq({})
end
it 'reports list duplicates as URL errors instead of existing skips' do
create(:post, url: 'https://example.com/duplicate')
expect(PostMetadataFetcher).not_to receive(:fetch)
results = described_class.new.preview_rows(rows: [
row(source_row: 1, url: 'https://example.com/duplicate'),
row(source_row: 2, url: 'https://EXAMPLE.com/duplicate/')
])
expect(results.map { _1[:validation_errors] }).to all(
include(url: ['URL が重複しています.'])
)
expect(results).to all(include(skip_reason: nil, existing_post_id: nil))
end
it 'does not fetch metadata for an unsafe URL' do
allow(Preview::UrlSafety).to receive(:validate)
.and_raise(Preview::UrlSafety::UnsafeUrl, '安全でない接続先は使用できません.')
expect(PostMetadataFetcher).not_to receive(:fetch)
result = described_class.new.preview_rows(rows: [
row(source_row: 1, url: 'https://unsafe.example/post')
]).first
expect(result.fetch(:validation_errors)).to include(
url: ['安全でない接続先は使用できません.']
)
end
it 'applies metadata to automatic fields and recognises metadata tags' do
Tag.create!(name: 'known-tag', category: :general)
allow(PostMetadataFetcher).to receive(:fetch).and_return(
title: 'metadata title',
thumbnail_base: 'https://example.com/thumb.jpg',
duration: 2_000,
tags: 'known-tag'
)
result = described_class.new.preview_rows(rows: [
row(source_row: 1, url: 'https://example.com/new')
]).first
expect(result.fetch(:attributes)).to include(
'title' => 'metadata title',
'thumbnail_base' => 'https://example.com/thumb.jpg',
'duration' => 2_000,
'tags' => 'known-tag'
)
expect(result.fetch(:field_warnings)).not_to have_key('tags')
expect(PostMetadataFetcher).to have_received(:fetch).once
end
it 'preserves manual empty values when metadata is available' do
allow(PostMetadataFetcher).to receive(:fetch).and_return(
title: 'metadata title',
tags: 'metadata-tag'
)
result = described_class.new.preview_rows(rows: [
row(
source_row: 1,
url: 'https://example.com/manual',
attributes: { title: '', tags: '' },
provenance: { title: 'manual', tags: 'manual' },
tag_sources: { automatic: 'old-tag', manual: '' }
)
]).first
expect(result.fetch(:attributes)).to include('title' => '', 'tags' => '')
expect(result.fetch(:tag_sources)).to include('manual' => '')
end
it 'clears warnings derived from an old metadata URL' do
allow(PostMetadataFetcher).to receive(:fetch).and_return(
title: 'new title',
thumbnail_base: 'https://example.com/new-thumb.jpg',
tags: ''
)
result = described_class.new.preview_rows(rows: [
row(
source_row: 1,
url: 'https://example.com/new-url',
attributes: { title: 'old title', tags: 'old-tag' },
provenance: { title: 'automatic', tags: 'automatic' },
tag_sources: { automatic: 'old-tag', manual: '' },
metadata_url: 'https://example.com/old-url',
field_warnings: { tags: ['old tag warning'], url: ['old fetch warning'] },
base_warnings: ['old base warning']
)
]).first
expect(result.fetch(:attributes)).to include('title' => 'new title', 'tags' => '')
expect(result.fetch(:base_warnings)).to eq([])
expect(result.fetch(:field_warnings).values.flatten)
.not_to include('old tag warning', 'old fetch warning')
end
it 'isolates an unexpected metadata failure to the affected URL' do
allow(PostMetadataFetcher).to receive(:fetch) do |url|
raise 'fetch failure' if url.include?('failure')
{ title: 'successful title', tags: '' }
end
allow(Rails.logger).to receive(:error)
results = described_class.new.preview_rows(rows: [
row(source_row: 1, url: 'https://example.com/failure'),
row(source_row: 2, url: 'https://example.com/success')
])
expect(results[0].fetch(:field_warnings)).to include(
'url' => [described_class::METADATA_FETCH_WARNING]
)
expect(results[1].fetch(:attributes)).to include('title' => 'successful title')
end
it 'turns an unsafe URL detected during metadata fetching into a URL error' do
allow(PostMetadataFetcher).to receive(:fetch)
.and_raise(Preview::UrlSafety::UnsafeUrl, '安全でない接続先です.')
allow(Rails.logger).to receive(:info)
result = described_class.new.preview_rows(rows: [
row(source_row: 1, url: 'https://example.com/redirects-to-private')
]).first
expect(result.fetch(:validation_errors)).to include(
url: ['安全でない接続先です.']
)
end
it 'validates parent IDs from the preloaded set' do
parent = create(:post)
results = described_class.new.preview_rows(
rows: [
row(source_row: 1, url: 'https://example.com/valid-parent',
attributes: { parent_post_ids: parent.id.to_s }),
row(source_row: 2, url: 'https://example.com/missing-parent',
attributes: { parent_post_ids: '999999' })
],
fetch_metadata: false
)
expect(results[0].fetch(:validation_errors)).not_to have_key(:parent_post_ids)
expect(results[1].fetch(:validation_errors)).to include(
parent_post_ids: ['存在しない親投稿 Id. があります.']
)
end
end
end
+77
ファイルの表示
@@ -0,0 +1,77 @@
require 'rails_helper'
RSpec.describe PostImportRowNormaliser do
def valid_row(overrides = { })
{
sourceRow: '1',
url: 'https://example.com/post',
metadataUrl: 'https://example.com/post',
attributes: { title: 'title', duration: 1_000 },
provenance: { url: 'manual', title: 'automatic' },
tagSources: { automatic: 'tag', manual: '' }
}.deep_merge(overrides)
end
describe '.normalise!' do
it 'normalises aliases and source rows into a permitted plain hash' do
row = ActionController::Parameters.new(valid_row)
expect(described_class.normalise!([row])).to eq([
{
'source_row' => 1,
'url' => 'https://example.com/post',
'metadata_url' => 'https://example.com/post',
'attributes' => { 'title' => 'title', 'duration' => 1_000 },
'provenance' => { 'url' => 'manual', 'title' => 'automatic' },
'tag_sources' => { 'automatic' => 'tag', 'manual' => '' }
}
])
end
it 'normalises source rows before checking duplicates' do
rows = [valid_row, valid_row(sourceRow: 1, url: 'https://example.com/other')]
expect { described_class.normalise!(rows) }
.to raise_error(ArgumentError, '元行番号が重複しています.')
end
it 'rejects non-array batches and non-hash rows' do
expect { described_class.normalise!({}) }
.to raise_error(ArgumentError, '取込行の形式が不正です.')
expect { described_class.normalise!(['row']) }
.to raise_error(ArgumentError, '取込行の形式が不正です.')
end
it 'rejects unknown attributes and invalid field types' do
expect { described_class.normalise!([valid_row(attributes: { unknown: 'x' })]) }
.to raise_error(ArgumentError, '取込項目が不正です.')
expect { described_class.normalise!([valid_row(attributes: { title: [] })]) }
.to raise_error(ArgumentError, '取込項目の型が不正です.')
expect { described_class.normalise!([valid_row(attributes: { duration: false })]) }
.to raise_error(ArgumentError, '取込項目の型が不正です.')
end
it 'rejects unknown provenance and tag-source values' do
expect { described_class.normalise!([valid_row(provenance: { title: 'mapped' })]) }
.to raise_error(ArgumentError, '値の由来が不正です.')
expect { described_class.normalise!([valid_row(tagSources: { mapped: 'tag' })]) }
.to raise_error(ArgumentError, 'タグ由来の形式が不正です.')
end
it 'accepts warning fields only at the validation boundary' do
row = valid_row.merge(
fieldWarnings: { title: ['取得できませんでした.'] },
baseWarnings: ['確認してください.']
)
without_warnings = described_class.normalise!([row]).first
with_warnings = described_class.normalise!([row], allow_warning_fields: true).first
expect(without_warnings).not_to include('field_warnings', 'base_warnings')
expect(with_warnings).to include(
'field_warnings' => { 'title' => ['取得できませんでした.'] },
'base_warnings' => ['確認してください.']
)
end
end
end

変更されたファイルが多すぎるため,一部のファイルは表示されません さらに表示