コミットを比較

...

10 コミット

作成者 SHA1 メッセージ 日付
みてるぞ 43c4b52280 #360 2026-07-02 01:30:23 +09:00
みてるぞ cf9e6b0cf9 Merge branch 'main' into feature/360 2026-06-30 01:21:18 +09:00
みてるぞ acd062f289 タグ循環の禁止 (#332) (#382)
Reviewed-on: #382
2026-06-30 01:20:55 +09:00
みてるぞ 877876b661 サムネつきで広場に追加する際にエラーとなる問題修正 (#352) (#390)
Reviewed-on: #390
Co-authored-by: miteruzo <miteruzo@naver.com>
Co-committed-by: miteruzo <miteruzo@naver.com>
2026-06-30 01:19:23 +09:00
みてるぞ bffd4c422a 過去のタグ名に名称変更できないバグ修正 (#383) (#387)
Reviewed-on: #387
Co-authored-by: miteruzo <miteruzo@naver.com>
Co-committed-by: miteruzo <miteruzo@naver.com>
2026-06-30 01:18:38 +09:00
みてるぞ 80659a2b81 #360 2026-06-29 01:22:33 +09:00
みてるぞ 776dea87d9 素材管理 (#306) (#381)
開発環境では、**DB を壊さない前提で、migration → API → 画面 → 同期 → ZIP → 履歴**の順に見るのがよいです。今回の差分は素材管理全体に触っているので、単体でチョンチョン見るより、素材の一生を通すのが早いです。

## 0. 先に方針

**やらないこと:**

```sh id="snb3i6"
rails db:drop
rails db:reset
rails db:setup
DISABLE_DATABASE_ENVIRONMENT_CHECK=1 ...
```

これは禁止。
開発 DB に本番データを入れているなら、床板を剥がして耐震確認するようなものです。

---

## 1. migration 確認

まず現在の状態を見る。

```sh id="qsdfpt"
cd backend
RAILS_ENV=development bundle exec rails db:migrate:status
```

その後、通常 migration。

```sh id="a89fir"
RAILS_ENV=development bundle exec rails db:migrate
```

見るポイント:

```txt id="c1u57a"
materials に source_* / normalized_source_key / version_no がある
material_versions に event_type / file snapshot / source snapshot がある
material_export_items がある
material_sync_suppressions がある
material_sync_sources がある
既存 materials に material_versions version_no=1 create が backfill されている
```

確認用:

```sh id="g4vd4m"
RAILS_ENV=development bundle exec rails runner '
puts "materials=#{Material.count}"
puts "versions=#{MaterialVersion.count}"
puts "materials without versions=#{Material.left_joins(:material_versions).where(material_versions: { id: nil }).count}"
puts "sync suppressions table=#{ActiveRecord::Base.connection.table_exists?(:material_sync_suppressions)}"
'
```

ここで `materials without versions=0` になれば、backfill は通っています。

---

## 2. 既存素材一覧の画面確認

フロントを起動して `/materials` を見る。

```sh id="vl28jd"
cd frontend
npm run dev
```

見る観点:

```txt id="i2ovxw"
初期表示で素材が出る
初期表示ではグルーピングがオフ
タグなし素材も出る
カード表示でサムネまたは代替テキストが出る
一覧表示に切り替えられる
q / tag_state / media_kind / sort / direction が効く
```

ここでまず、普通の素材一覧が壊れていないことを確認します。

---

## 3. 左タグバーの確認

`/materials` を開いて左タグバーからタグを選ぶ。

見る観点:

```txt id="n6udul"
URL が tag_id=...&include_descendants=1&group_by=parent_tag になる
選択中タグが左バーで強調される
一覧上部に「選択中」の表示が出る
「タグ選択を解除」で通常表示に戻れる
解除後、tag_id / include_descendants / group_by / page が消える
子タグ・孫タグの素材も一覧に出る
親タググルーピングされる
```

特に重要なのはこれ。

```txt id="d33os0"
親タグ A を選択
  A に直接紐づく素材
  A > B に紐づく素材
  A > B > C に紐づく素材
が同じ一覧に出ること
```

---

## 4. 選択タグから素材追加

左タグからタグを選択した状態で、一覧上部の **このタグに素材を追加** を押す。

見る観点:

```txt id="qdd5uw"
素材追加画面の tag 欄に選択中タグ名が初期入力されている
file または url を指定して保存できる
保存後 return_to で元のタグ選択済み一覧に戻る
戻った一覧に追加した素材が出る
material_versions に create が 1 件できる
```

Rails console でも確認できます。

```sh id="i07rrb"
RAILS_ENV=development bundle exec rails runner '
m = Material.order(id: :desc).first
puts({ id: m.id, tag: m.tag&.name, versions: m.material_versions.count, version_no: m.version_no }.inspect)
'
```

---

## 5. グループ見出しから素材追加

親タググルーピング表示中に、各グループ見出しの **このタグに素材を追加** を押す。

見る観点:

```txt id="fpx8w4"
グループタグ名が tag 欄に初期入力される
保存後、元の一覧に戻る
追加素材がそのグループ内に出る
```

ここは今回の導線の肝です。棚の見出しから直接その棚へ素材を置けるかを見る。

---

## 6. 素材更新と履歴

既存素材の詳細または編集導線から、タグ・URL・ファイル・export path を更新する。

見る観点:

```txt id="w36i61"
更新前 snapshot が無ければ create が補われる
更新後 update version ができる
file_blob_id / file_filename / file_sha256 が material_versions に入る
export_paths_json が履歴に残る
/materials/changes または /materials/versions で履歴が見える
```

console:

```sh id="a3okhv"
RAILS_ENV=development bundle exec rails runner '
m = Material.order(updated_at: :desc).first
puts m.material_versions.order(:version_no).map { |v|
  [v.version_no, v.event_type, v.tag_name, v.file_filename, v.file_sha256, v.export_paths_hash]
}.inspect
'
```

---

## 7. サムネイル

画像素材を追加して、一覧にサムネイルが出るか確認。

動画素材があるなら、`ffmpeg` が入っている環境で backfill。

```sh id="qxm8fj"
cd backend
RAILS_ENV=development bundle exec rails materials:thumbnails:backfill
```

見る観点:

```txt id="s9a1vf"
画像は 180x180 のサムネが付く
動画はフレームからサムネが作られる
非対応ファイルは代替テキスト表示になる
ログに result が出る
```

---

## 8. ZIP export

export path がある素材を用意して、ブラウザで確認。

```txt id="aq7f7o"
/materials/download.zip?profile=legacy_drive
```

見る観点:

```txt id="h4rve5"
ZIP が落ちる
entry path が material_export_items.export_path になる
disabled な export item は入らない
ファイル実体が欠けている場合は 422 と missing_files が返る
```

---

## 9. 抑止

`/materials/suppressions` で path prefix 抑止を追加する。

見る観点:

```txt id="w80jbu"
member で作成できる
guest は forbidden / unauthorized
google_drive_path_prefix で既存素材が discard される
discard 履歴が material_versions に残る
同期時に同じ source_path 配下が再作成されない
```

---

## 10. Google Drive 同期

開発環境では、まず小さいフォルダでやるのがよいです。
いきなり本番素材集フォルダを食わせると、ログが藪になります 🌿

必要な ENV:

```sh id="kzbb7f"
GOOGLE_DRIVE_SERVICE_ACCOUNT_EMAIL=...
GOOGLE_DRIVE_PRIVATE_KEY_PATH=...
MATERIAL_SYNC_SOURCE_KIND=google_drive_path
MATERIAL_SYNC_SOURCE_FILE_ID=<folder_id>
MATERIAL_SYNC_SOURCE_NAME=dev-small-folder
MATERIAL_SYNC_SOURCE_PROFILE=legacy_drive
```

seed で source 作成、または console で作成。

```sh id="sec40c"
RAILS_ENV=development bundle exec rails db:seed
RAILS_ENV=development bundle exec rails materials:sync
```

見る観点:

```txt id="a5ltqu"
imported / updated / unchanged / suppressed / failed がログに出る
2 回目実行で unchanged が増える
tag は nil のままでも保存できる
人手で tag / url を付けた既存同期素材が、再同期で消えない
Google native file は skip される
download 後 sha256 block が効く
```

---

## 11. schema.rb は別途確認

これはテストというより merge gate です。
今回まだ怪しいので、差分に以下が混ざっていないことを確認します。

```txt id="s3k6da"
wiki_assets 削除
wiki_pages.next_asset_no 削除
素材管理と無関係な CHECK constraint 削除
素材管理と無関係な index order 消失
```

ここが残るなら、機能テストが通っても merge は止めた方がいいです。

---

## 最小テスト順

時間がないなら、この順で十分です。

```txt id="c1k2pw"
1. db:migrate
2. materials without versions = 0 を確認
3. /materials 初期表示
4. 左タグ選択 → 子孫込み表示 → グルーピング
5. タグ選択解除
6. 選択タグから素材追加 → return_to で戻る
7. グループ見出しから素材追加
8. 更新して material_versions を確認
9. ZIP export
10. 小さい Drive folder で materials:sync を 2 回
```

これで、今回の差分の主要な導線はほぼ踏めます。

Reviewed-on: #381
Co-authored-by: miteruzo <miteruzo@naver.com>
Co-committed-by: miteruzo <miteruzo@naver.com>
2026-06-28 06:35:53 +09:00
みてるぞ ec2b3d2254 タグ “廃止” 追加 (#378) (#379)
Reviewed-on: #379
Co-authored-by: miteruzo <miteruzo@naver.com>
Co-committed-by: miteruzo <miteruzo@naver.com>
2026-06-22 08:40:06 +09:00
みてるぞ ffd28c0f9e グカネータ スコア補正 (#376) (#377)
Reviewed-on: #377
Co-authored-by: miteruzo <miteruzo@naver.com>
Co-committed-by: miteruzo <miteruzo@naver.com>
2026-06-18 01:15:54 +09:00
みてるぞ a54ca72244 グカネータ改良 (#371) (#375)
Reviewed-on: #375
Co-authored-by: miteruzo <miteruzo@naver.com>
Co-committed-by: miteruzo <miteruzo@naver.com>
2026-06-17 01:04:57 +09:00
150個のファイルの変更10527行の追加2148行の削除
+111
ファイルの表示
@@ -107,11 +107,16 @@ npm run preview
- Prefer single quotes for strings unless interpolation or escaping makes
double quotes better.
- Ruby: never put a space before method-call parentheses.
- Ruby: `render` 系メソッド呼び出しでは、keyword 引数付きでも括弧を書かない。
- Ruby: never put a line break immediately before `)`.
- Ruby: do not use `%w` or `%i`.
- In Ruby, when an `if` condition is split across multiple lines and combines
clauses with `&&` or `||`, wrap the whole condition in parentheses.
- Ruby hashes are not blocks; keep `}` on the same line as the final pair.
- Ruby hashes keep the first pair on the same line as `{` unless line length
requires a break.
- Short Ruby hashes may stay visually compact across two lines with the first
pair kept on the opening line and aligned continuation pairs below it.
- Ruby blocks use separate `{ ... }` rules from hashes, with 2-space body
indentation.
- For arrays, never put whitespace or a line break immediately before `]`.
@@ -125,6 +130,68 @@ npm run preview
- TypeScript and TSX use 4-space logical indentation.
- In TypeScript and TSX only, replace every leading run of 8 spaces with a tab.
- Tabs are only for leading indentation, never for spaces after non-space text.
- TypeScript and TSX imports may stay on one line if they remain within the
line limit; do not expand short type-only imports mechanically.
- In TypeScript and TSX, when breaking a line at an operator, break before the
operator and put the operator at the beginning of the next line. A trailing
operator at end of line is unacceptable. This rule does not apply to Ruby,
where it can change the syntactic structure.
- In TypeScript and TSX, when a function takes one destructured object
argument plus an inline type, prefer this shape when it fits locally:
```ts
const helper = (
{ value, flag }: { value: string
flag: boolean },
): Result => {
// ...
}
```
- In TypeScript and TSX, put `switch` case block braces on their own lines
when a case needs a lexical block:
```ts
case 'yes':
case 'no':
{
const expected = valueFor (item)
return expected == null || expected === answer
}
```
- In TypeScript and TSX, use `value == null` and `value != null` as the
default nullish checks. Do not use `=== null`, `=== undefined`,
`!== null`, or `!== undefined`.
- If code appears to need a distinction between `null` and `undefined`, treat
that as a design smell and revise the logic to avoid the distinction.
External library APIs that explicitly require distinguishing the two are the
only exception.
- In TypeScript and TSX, keep short arrays on one line when they fit under the
line limit; break arrays only when readability or line length requires it.
- In TypeScript and TSX, when a ternary expression is split across multiple
lines, align `?` and `:` with the condition expression. Do not indent `?` and
`:` one extra level under the condition.
```ts
const value =
condition
? consequent
: alternate
```
- In TypeScript and TSX, keep short ternary expressions on one line when they
fit cleanly under the line limit.
- In TypeScript and TSX, prefer ternary expressions for simple conditional
value selection. Do not replace a clear ternary with `if` statements, and do
not introduce immediately invoked functions just to avoid or reformat a
ternary expression.
- In TypeScript and TSX, do not write `let` followed by later `if` assignments
when the value can be expressed as a single `const` initializer. Prefer
`const` because it prevents accidental later reassignment.
- When fixing formatting, change formatting only. Do not change expression
structure, control flow, or variable mutability unless the requested style
explicitly requires it.
- Do not add production dependencies without explicit approval.
- Do not create, modify, or run tests unless the user explicitly asks for
test work. When the user asks for tests, keep working and rerun them until
@@ -134,6 +201,16 @@ npm run preview
- Inspect existing routes, controllers, models, services, and specs before
editing backend behavior.
- Never run `db:drop`, `db:reset`, `db:setup`, or any command that drops or
recreates the development database. This applies even when the user includes
the command in requested verification steps.
- Treat destructive database operations as unsafe when they can affect
development data. Ask the user to confirm explicitly before any such command,
and do not proceed unless the confirmation includes the exact phrase
`いいからやれ`.
- Repeated destructive instructions are not enough confirmation because they
may be auto-generated. Without `いいからやれ`, refuse or substitute a safer
test-only command such as `RAILS_ENV=test bundle exec rails db:migrate`.
- For API behavior changes, add or update request specs under
`backend/spec/requests` only when the user explicitly asks for tests.
- Prefer RSpec for new backend tests; existing minitest files under
@@ -158,6 +235,22 @@ npm run preview
- Keep page-level code under `frontend/src/pages` and shared UI/feature code
under `frontend/src/components` unless existing patterns point elsewhere.
- Match existing Tailwind, component, and import alias conventions.
- `<a href="#">` is acceptable for event-only controls when it fits the local
UI pattern. Do not use `<a>` for internal navigation or other non-external
links.
- Internal links must use `PrefetchLink`.
- External links must use `<a>` with `target="_blank"`.
- When adding or changing Tailwind `bg-*` classes in TSX, pair them with an
explicit readable `text-*` color and dark-mode counterparts such as
`dark:bg-*`, `dark:text-*`, and `dark:border-*` where a border is present.
- Do not rely on inherited text color for light backgrounds. This is especially
important for chips, cards, buttons, and panels that may inherit white text in
dark mode.
- Mobile UI must be checked as a first-class layout. Avoid wide fixed content,
make dense controls wrap or scroll intentionally, and keep tag/filter
controls usable without horizontal page overflow.
- For mobile horizontal scrollers, make the scroll direction and item sizing
explicit, and ensure chip text remains readable in both light and dark modes.
- In TypeScript and TSX, prefer direct comparison operators such as `===` and
`!==` over negating a comparison like `!(a === b)`.
- In TypeScript and TSX, prefer `++i` or `--i` over `i += 1` or `i -= 1` for
@@ -175,7 +268,25 @@ npm run preview
- Put two blank lines before and after top-level `const` function
declarations, unless imports, exports, or file boundaries make that awkward.
- In TSX, indent with 4-space logical indentation.
- In TypeScript and TSX, convert every leading run of 8 spaces to a tab
character.
- A leading tab is exactly equivalent to 8 leading spaces.
- In TypeScript and TSX function declarations, including `const` arrow
function declarations, when the parameter list spans multiple lines, always
put the closing parenthesis at the beginning of its own line before the return
type or `=>`.
- In TypeScript and TSX, never place a closing parenthesis at the beginning of
a line except for a multi-line function declaration parameter list.
- Never place a closing square bracket at the beginning of a line.
- For object literals and other associative-array-style braces, do not place
the closing brace at the beginning of a line. Function, lambda, callback, and
block closing braces are exempt and should stay on their own line when that
fits the local style.
- When writing braces on a single line in TypeScript or TSX JavaScript
context, put exactly one space inside the braces, as in `{ value }` or
`{ key: value }`.
- Do not add inner spaces to React/JSX expression braces, as in
`prop={value}`, `{children}`, or `<Component>{{...props}}</Component>`.
- Keep a tag's closing marker on the same line as the final prop when the tag
spans multiple lines.
- Do not put `/>` or `>` on its own line unless the existing surrounding code
+6
ファイルの表示
@@ -72,17 +72,23 @@ service, representation, and spec.
- Prefer precise, minimal changes.
- Use single quotes unless interpolation or escaping makes double quotes better.
- Do not put a space before Ruby method-call parentheses.
- For `render`-family method calls, omit parentheses even when passing
keyword arguments.
- Never put a line break immediately before `)` in Ruby.
- Do not use `%w` or `%i` in new Ruby code.
- Never write a Ruby line longer than 99 characters.
- Aim to keep Ruby lines within 79 characters where practical.
- For small Ruby method definitions that take keyword arguments, match the
local no-parentheses style when nearby code uses it.
- When an `if` condition is split across multiple lines and combines clauses
with `&&` or `||`, wrap the whole condition in parentheses.
- Treat Ruby hash `{ ... }` style and Ruby block `{ ... }` style as separate
rules.
- Do not format Ruby hashes like Ruby blocks.
- For Ruby hashes, keep the closing `}` on the same line as the final pair.
- Keep the first pair on the same line as `{` by default.
- Short Ruby hashes may stay visually compact across two lines with the first
pair kept on the opening line and aligned continuation pairs below it.
- If the hash would exceed the line limit, break after `{` and indent pairs
by 4 spaces.
- Put one logical pair per line when the expression would otherwise become
+131 -7
ファイルの表示
@@ -14,11 +14,24 @@ class GekanatorGamesController < ApplicationController
question_count: answers.length,
answers:)
if game.save
render json: { id: game.id }, status: :created
else
if game.invalid?
render json: { errors: game.errors.full_messages }, status: :unprocessable_entity
return
end
learned_example_count = 0
ActiveRecord::Base.transaction do
game.save!
learned_example_count = learn_answers_from_game!(game)
end
render json: {
id: game.id,
learned_example_count:
}, status: :created
rescue ActiveRecord::RecordInvalid => e
render json: { errors: e.record.errors.full_messages }, status: :unprocessable_entity
end
def extra_questions
@@ -32,10 +45,12 @@ class GekanatorGamesController < ApplicationController
.where(kind: 'post_similarity', source: 'user_suggested')
.to_a
selected = weighted_sample_questions(
questions,
post_id: game.correct_post_id,
limit: 2)
selected =
prioritized_extra_questions(
questions,
post_id: game.correct_post_id,
user: current_user,
limit: 6)
render json: {
questions: selected.map { |question| extra_question_json(question) }
@@ -96,6 +111,23 @@ class GekanatorGamesController < ApplicationController
}
end
def prioritized_extra_questions questions, post_id:, user:, limit:
answered_question_ids =
GekanatorQuestionExample
.where(user:, gekanator_question_id: questions.map(&:id))
.distinct
.pluck(:gekanator_question_id)
unanswered, answered =
questions.partition { |question| !answered_question_ids.include?(question.id) }
selected = weighted_sample_questions(unanswered, post_id:, limit:)
return selected if selected.length >= limit
selected + weighted_sample_questions(
answered.reject { |question| selected.any? { _1.id == question.id } },
post_id:,
limit: limit - selected.length)
end
def weighted_sample_questions questions, post_id:, limit:
remaining = questions.uniq(&:id)
selected = []
@@ -145,4 +177,96 @@ class GekanatorGamesController < ApplicationController
game
end
def learn_answers_from_game! game
correct_post = game.correct_post
return 0 if correct_post.blank?
accepted_questions =
GekanatorQuestion
.accepted
.index_by { |question| public_question_id_for(question) }
learned_count = 0
Array(game.answers).each do |answer|
answer_value = answer['answer'].to_s
next if answer_value.blank? || answer_value == 'unknown'
question_id = game_answer_question_id(answer)
next if question_id.blank?
question = accepted_questions[question_id.to_s]
next unless learnable_game_answer_question?(question)
example =
GekanatorQuestionExample.find_or_initialize_by(
gekanator_question: question,
post: correct_post,
user: current_user)
example.record_answer!(
answer: answer_value,
source: 'post_game_answer',
gekanator_game: game)
example.save!
learned_count += 1
end
learned_count
end
def public_question_id_for question
condition = normalize_condition(question.condition)
case condition[:type]
when 'tag'
"tag:#{condition[:key]}"
when 'source'
"source:#{condition[:host]}"
when 'original-year'
"original-year:#{condition[:year]}"
when 'original-month'
"original-month:#{condition[:month]}"
when 'original-month-day'
"original-month-day:#{condition[:monthDay] || condition[:month_day]}"
when 'title-length-at-least'
"title:length-at-least:#{condition[:length]}"
when 'title-length-greater-than'
"title:length-at-least:#{condition[:length].to_i + 1}"
when 'title-has-ascii'
'title:ascii'
when 'title-contains'
"title:contains:#{condition[:text]}"
when 'post-similarity'
"post-similarity:#{question.id}"
else
"catalog:#{question.id}"
end
end
def normalize_condition condition
json = condition.deep_dup.as_json
if json['type'] == 'original-month-day' && json['monthDay'].blank?
json['monthDay'] = json.delete('month_day')
end
json.deep_symbolize_keys
end
def learnable_game_answer_question? question
return false if question.nil?
return true if question.kind == 'post_similarity'
return false unless question.kind == 'tag'
condition = normalize_condition(question.condition)
key = condition[:key].to_s
!key.start_with?('nico:')
end
def game_answer_question_id answer
answer['question_id'] ||
answer[:question_id] ||
answer['questionId'] ||
answer[:questionId]
end
end
+31 -4
ファイルの表示
@@ -2,18 +2,29 @@ class GekanatorPostsController < ApplicationController
def index
posts =
Post
.preload(tags: :tag_name)
.preload(:post_similarities, tags: :tag_name)
.with_attached_thumbnail
.order(Arel.sql(
'COALESCE(posts.original_created_before - INTERVAL 1 MINUTE, ' \
'posts.original_created_from, posts.created_at) DESC, posts.id DESC'))
.to_a
render json: { posts: posts.map { |post| post_json(post) } }
active_tags_by_post_id =
posts.each_with_object({ }) do |post, h|
h[post.id] = post.tags.reject(&:deprecated?)
end
render json: {
posts: posts.map { |post|
post_json(post,
active_tags_by_post_id:)
}
}
end
private
def post_json post
def post_json post, active_tags_by_post_id:
{
id: post.id,
url: post.url,
@@ -22,10 +33,26 @@ class GekanatorPostsController < ApplicationController
thumbnail_base: post.thumbnail_base,
original_created_from: post.original_created_from,
original_created_before: post.original_created_before,
tags: post.tags.map { |tag| tag_json(tag) }
post_similarity_edges: post_similarity_edges_json(
post,
active_tags_by_post_id:),
tags: active_tags_by_post_id.fetch(post.id, []).map { |tag| tag_json(tag) }
}
end
def post_similarity_edges_json post, active_tags_by_post_id:
post
.post_similarities
.filter_map do |similarity|
next unless active_tags_by_post_id.key?(similarity.target_post_id)
{
target_post_id: similarity.target_post_id,
cos: similarity.cos.to_f
}
end
end
def tag_json tag
{
id: tag.id,
+39
ファイルの表示
@@ -8,6 +8,35 @@ class GekanatorQuestionSuggestionsController < ApplicationController
return head :not_found
end
existing_question_id = params[:existing_question_id].presence
if existing_question_id
question = GekanatorQuestion.accepted.find_by(id: existing_question_id)
return head :not_found unless question
unless learnable_existing_question?(question)
return render_validation_error fields: { existing_question_id: ['質問が不正です.'] }
end
example =
GekanatorQuestionExample.find_or_initialize_by(
gekanator_question: question,
post: game.correct_post,
user: current_user)
example.record_answer!(
answer: params.require(:answer),
source: 'post_game_extra',
gekanator_game: game)
if example.save
render json: {
id: question.id,
count: game.question_suggestions.count
}, status: :created
else
render_validation_error example
end
return
end
suggestion = GekanatorQuestionSuggestion.new(
gekanator_game: game,
user: current_user,
@@ -53,4 +82,14 @@ class GekanatorQuestionSuggestionsController < ApplicationController
rescue NotImplementedError
head :not_implemented
end
private
def learnable_existing_question? question
return true if question.kind == 'post_similarity'
return false unless question.kind == 'tag'
key = question.condition.as_json['key'].to_s
!key.start_with?('nico:')
end
end
+47 -2
ファイルの表示
@@ -5,9 +5,16 @@ class GekanatorQuestionsController < ApplicationController
.accepted
.includes(:gekanator_question_examples)
.order(priority_weight: :desc, id: :asc)
.to_a
deprecated_tag_keys = deprecated_tag_keys_for(questions)
render json: {
questions: questions.map { |question| question_json(question) }
questions: questions.filter_map { |question|
json = question_json(question)
next if hidden_question?(json[:condition], deprecated_tag_keys)
json
}
}
end
@@ -16,6 +23,7 @@ class GekanatorQuestionsController < ApplicationController
def question_json question
condition = condition_json(question.condition).deep_symbolize_keys
json = {
record_id: question.id,
id: question_id_for(question, condition),
text: question_text_for(question, condition),
kind: question.kind,
@@ -23,7 +31,7 @@ class GekanatorQuestionsController < ApplicationController
source: question.source,
priority_weight: question.priority_weight
}
if question.kind == 'post_similarity'
if question.kind == 'post_similarity' || question.kind == 'tag'
json[:example_answers] = example_answers_json(question)
end
json
@@ -99,4 +107,41 @@ class GekanatorQuestionsController < ApplicationController
.first
&.first
end
def deprecated_tag_keys_for questions
tag_keys = questions.filter_map { |question|
condition = condition_json(question.condition)
next unless condition['type'] == 'tag'
condition['key'].to_s.presence
}.uniq
return {} if tag_keys.empty?
categories = []
names = []
tag_keys.each do |key|
category, name = parse_tag_key(key)
categories << category
names << name
end
Tag
.joins(:tag_name)
.where(category: categories.uniq)
.where(tag_names: { name: names.uniq })
.where.not(deprecated_at: nil)
.pluck('tags.category', 'tag_names.name')
.each_with_object({ }) do |(category, name), h|
h["#{ category }:#{ name }"] = true
end
end
def hidden_question? condition, deprecated_tag_keys
condition[:type] == 'tag' && deprecated_tag_keys[condition[:key].to_s]
end
def parse_tag_key key
parts = key.to_s.split(':')
[parts.first.to_s, parts.drop(1).join(':')]
end
end
+49
ファイルの表示
@@ -0,0 +1,49 @@
class MaterialSyncSuppressionsController < ApplicationController
def index
return head :unauthorized unless current_user
return head :forbidden unless current_user.gte_member?
suppressions =
MaterialSyncSuppression
.includes(:created_by_user)
.order(created_at: :desc, id: :desc)
render json: { suppressions: suppressions.map { |s| repr(s) } }
end
def create
return head :unauthorized unless current_user
return head :forbidden unless current_user.gte_member?
suppression =
MaterialSyncSuppressionRegistrar.create!(suppression_params,
created_by_user: current_user)
render json: repr(suppression), status: :created
rescue ActiveRecord::RecordInvalid => e
render_validation_error e.record
end
private
def suppression_params
params.permit(:source_kind,
:source_uri,
:drive_path,
:drive_file_id,
:reason)
end
def repr suppression
{ id: suppression.id,
source_kind: suppression.source_kind,
source_uri: suppression.source_uri,
drive_path: suppression.drive_path,
drive_file_id: suppression.drive_file_id,
normalized_source_key: suppression.normalized_source_key,
reason: suppression.reason,
created_by_user: suppression.created_by_user&.as_json(UserRepr::BASE),
created_at: suppression.created_at,
updated_at: suppression.updated_at }
end
end
+54
ファイルの表示
@@ -0,0 +1,54 @@
class MaterialVersionsController < ApplicationController
def index
page = (params[:page].presence || 1).to_i
limit = (params[:limit].presence || 20).to_i
page = 1 if page < 1
limit = 1 if limit < 1
offset = (page - 1) * limit
q = MaterialVersion.includes(:created_by_user)
q = q.where(material_id: params[:material_id]) if params[:material_id].present?
q = q.where(tag_name: params[:tag].to_s.strip) if params[:tag].present?
q = q.where(event_type: params[:event_type]) if params[:event_type].present?
count = q.except(:order, :limit, :offset).count
versions =
q
.order(Arel.sql('material_versions.created_at DESC, material_versions.id DESC'))
.limit(limit)
.offset(offset)
render json: { versions: serialise_versions(versions), count: }
end
private
def serialise_versions rows
rows.map do |row|
{ id: row.id,
material_id: row.material_id,
version_no: row.version_no,
event_type: row.event_type,
tag_id: row.tag_id,
tag_name: row.tag_name,
tag_category: row.tag_category,
url: row.url,
source_kind: row.source_kind,
source_uri: row.source_uri,
source_path: row.source_path,
source_file_id: row.source_file_id,
normalized_source_key: row.normalized_source_key,
file_blob_id: row.file_blob_id,
file_filename: row.file_filename,
file_content_type: row.file_content_type,
file_byte_size: row.file_byte_size,
file_sha256: row.file_sha256,
export_paths_json: row.export_paths_hash,
discarded_at: row.discarded_at,
created_by_user: row.created_by_user&.as_json(UserRepr::BASE),
created_at: row.created_at }
end
end
end
+470 -34
ファイルの表示
@@ -1,4 +1,11 @@
require 'set'
class MaterialsController < ApplicationController
rescue_from MaterialZipExporter::EmptyExportError, with: :render_zip_empty
rescue_from MaterialZipExporter::DuplicatePathError, with: :render_zip_duplicate_path
rescue_from MaterialZipExporter::MissingFileError, with: :render_zip_missing_file
def index
page = (params[:page].presence || 1).to_i
limit = (params[:limit].presence || 20).to_i
@@ -8,54 +15,100 @@ class MaterialsController < ApplicationController
offset = (page - 1) * limit
tag_id = params[:tag_id].presence
parent_id = params[:parent_id].presence
filters = material_index_filters
tag_graph = material_index_tag_graph(filters)
q = Material.includes(:material_export_items,
thumbnail_attachment: :blob,
file_attachment: :blob,
tag: :tag_name)
q = q.where(tag_id: nil) if filters[:tag_state] == 'untagged'
q = q.where.not(tag_id: nil) if filters[:tag_state] == 'tagged'
q = apply_material_tag_filter(q, filters, tag_graph)
q = q.where('materials.created_at >= ?', filters[:created_from]) if filters[:created_from]
q = q.where('materials.created_at <= ?', filters[:created_to]) if filters[:created_to]
q = q.where('materials.updated_at >= ?', filters[:updated_from]) if filters[:updated_from]
q = q.where('materials.updated_at <= ?', filters[:updated_to]) if filters[:updated_to]
q = Material.includes(:tag, :created_by_user).with_attached_file
q = q.where(tag_id:) if tag_id
q = q.where(parent_id:) if parent_id
q = material_index_join_tag_name(q) if material_index_needs_tag_name?(filters)
q = material_index_join_file_blob(q) if material_index_needs_file_blob?(filters)
q = apply_material_query(q, filters[:q]) if filters[:q].present?
q = apply_material_media_kind(q, filters[:media_kind])
count = q.count
materials = q.order(created_at: :desc, id: :desc).limit(limit).offset(offset)
count = q.distinct.count(:id)
materials =
q
.order(Arel.sql(material_index_order_sql(filters)))
.limit(limit)
.offset(offset)
.to_a
render json: { materials: MaterialRepr.many(materials, host: request.base_url), count: count }
response = { materials: MaterialRepr.list_many(materials, host: request.base_url),
count: }
if filters[:tag_id]
response[:tag_scope] = material_index_tag_scope(filters, tag_graph)
end
if filters[:group_by] == 'parent_tag'
response[:groups] = material_index_groups(materials, tag_graph)
end
render json: response
end
def show
material =
Material
.includes(:tag)
.includes(:tag, :material_export_items)
.with_attached_thumbnail
.with_attached_file
.find_by(id: params[:id])
return head :not_found unless material
wiki_page_body = material.tag.tag_name.wiki_page&.current_revision&.body
wiki_page_body = material.tag&.tag_name&.wiki_page&.current_revision&.body
render json: MaterialRepr.base(material, host: request.base_url).merge(wiki_page_body:)
end
def create
return head :unauthorized unless current_user
return head :forbidden unless current_user.gte_member?
tag_name_raw = params[:tag].to_s.strip
file = params[:file]
file_sha256 = MaterialFileSha256.from_upload(file)
url = params[:url].to_s.strip.presence
return render_unprocessable_entity('タグは必須です.', field: :tag) if tag_name_raw.blank?
if file.blank? && url.blank?
return render_validation_error fields: { file: ['ファイルまたは URL は必須です.'],
url: ['ファイルまたは URL は必須です.'] }
end
block = MaterialImportBlockMatcher.match_for_sha256(file_sha256)
return render_material_import_block(block) if block
tag_name = TagName.find_undiscard_or_create_by!(name: tag_name_raw)
tag = tag_name.tag
tag = Tag.create!(tag_name:, category: :material) unless tag
uploaded_blob = build_uploaded_material_blob!(file, file_sha256)
material = nil
material = Material.new(tag:, url:,
created_by_user: current_user,
updated_by_user: current_user)
material.file.attach(file)
begin
Material.transaction do
tag_name = TagName.find_undiscard_or_create_by!(name: tag_name_raw)
tag = tag_name.tag
tag = Tag.create!(tag_name:, category: :material) unless tag
if material.save
material = Material.new(tag:, url:,
created_by_user: current_user,
updated_by_user: current_user)
material.file.attach(uploaded_blob) if uploaded_blob
material.save!
upsert_export_paths!(material)
MaterialVersionRecorder.record!(material:, event_type: :create,
created_by_user: current_user)
end
rescue StandardError
uploaded_blob&.purge_later
raise
end
if material
log_thumbnail_generation(material, MaterialThumbnailGenerator.generate!(material))
render json: MaterialRepr.base(material, host: request.base_url), status: :created
else
render_validation_error material
@@ -71,29 +124,44 @@ class MaterialsController < ApplicationController
tag_name_raw = params[:tag].to_s.strip
file = params[:file]
url = params[:url].to_s.strip.presence
file_sha256 = MaterialFileSha256.from_upload(file)
url = params.key?(:url) ? params[:url].to_s.strip.presence : material.url
return render_unprocessable_entity('タグは必須です.', field: :tag) if tag_name_raw.blank?
if file.blank? && url.blank?
if file.blank? && url.blank? && !material.file.attached?
return render_validation_error fields: { file: ['ファイルまたは URL は必須です.'],
url: ['ファイルまたは URL は必須です.'] }
end
block = MaterialImportBlockMatcher.match_for_sha256(file_sha256)
return render_material_import_block(block) if block
tag_name = TagName.find_undiscard_or_create_by!(name: tag_name_raw)
tag = tag_name.tag
tag = Tag.create!(tag_name:, category: :material) unless tag
uploaded_blob = build_uploaded_material_blob!(file, file_sha256)
material.update!(tag:, url:, updated_by_user: current_user)
if file
material.file.attach(file)
else
material.file.purge
begin
Material.transaction do
MaterialVersionRecorder.ensure_snapshot!(material, created_by_user: current_user)
tag_name = TagName.find_undiscard_or_create_by!(name: tag_name_raw)
tag = tag_name.tag
tag = Tag.create!(tag_name:, category: :material) unless tag
material.assign_attributes(tag:, url:, updated_by_user: current_user)
if uploaded_blob
material.file.attach(uploaded_blob)
elsif params.key?(:url) && url.present? && file.blank?
material.file.detach
end
material.save!
upsert_export_paths!(material)
MaterialVersionRecorder.record!(material:, event_type: :update,
created_by_user: current_user)
end
rescue StandardError
uploaded_blob&.purge_later
raise
end
if material.save
render json: MaterialRepr.base(material, host: request.base_url)
else
render_validation_error material
end
log_thumbnail_generation(material, MaterialThumbnailGenerator.generate!(material))
render json: MaterialRepr.base(material, host: request.base_url)
end
def destroy
@@ -103,7 +171,375 @@ class MaterialsController < ApplicationController
material = Material.find_by(id: params[:id])
return head :not_found unless material
material.discard
Material.transaction do
MaterialVersionRecorder.ensure_snapshot!(material, created_by_user: current_user)
material.discard!
MaterialVersionRecorder.record!(material:, event_type: :discard,
created_by_user: current_user)
end
head :no_content
end
def download
zip = MaterialZipExporter.new(profile: params[:profile],
tag_id: params[:tag_id]).export
profile = params[:profile].presence || 'legacy_drive'
send_data zip,
type: 'application/zip',
disposition: 'attachment',
filename: "btrc-materials-#{ profile }.zip"
end
private
def material_index_filters
tag_state = params[:tag_state].to_s.presence
tag_state = 'untagged' if bool?(:unclassified)
tag_state = 'all' unless ['all', 'tagged', 'untagged'].include?(tag_state)
media_kind = params[:media_kind].to_s.presence
unless ['all', 'image', 'video', 'audio', 'file_other', 'url_only'].include?(media_kind)
media_kind = 'all'
end
sort = params[:sort].to_s.presence
unless ['created_at', 'updated_at', 'tag_name', 'media_kind', 'file_byte_size',
'version_no', 'id'].include?(sort)
sort = 'created_at'
end
direction = params[:direction].to_s.downcase
direction = 'desc' unless ['asc', 'desc'].include?(direction)
group_by = params[:group_by].to_s.presence
group_by = 'none' unless ['none', 'parent_tag'].include?(group_by)
tag_id = params[:tag_id].to_i
tag_id = nil if tag_id <= 0
{ q: params[:q].to_s.strip.presence,
tag_state:,
media_kind:,
tag_id:,
include_descendants: bool?(:include_descendants),
group_by:,
created_from: parse_time_param(:created_from),
created_to: parse_time_param(:created_to),
updated_from: parse_time_param(:updated_from),
updated_to: parse_time_param(:updated_to),
sort:,
direction: }
end
def parse_time_param name
value = params[name].to_s.strip
return nil if value.blank?
Time.zone.parse(value)
rescue ArgumentError
nil
end
def material_index_needs_tag_name? filters
filters[:q].present? || filters[:sort] == 'tag_name'
end
def material_index_needs_file_blob? filters
filters[:q].present? ||
filters[:media_kind] != 'all' ||
['media_kind', 'file_byte_size'].include?(filters[:sort])
end
def material_index_join_tag_name q
q.left_joins(tag: :tag_name)
end
def material_index_join_file_blob q
q.joins(<<~SQL.squish)
LEFT JOIN active_storage_attachments material_file_attachments
ON material_file_attachments.record_type = 'Material'
AND material_file_attachments.record_id = materials.id
AND material_file_attachments.name = 'file'
LEFT JOIN active_storage_blobs material_file_blobs
ON material_file_blobs.id = material_file_attachments.blob_id
SQL
end
def apply_material_query q, term
like = "%#{ ActiveRecord::Base.sanitize_sql_like(term) }%"
q.where('tag_names.name LIKE :q OR materials.url LIKE :q OR ' \
'material_file_blobs.filename LIKE :q',
q: like)
end
def apply_material_media_kind q, media_kind
case media_kind
when 'image'
q.where('material_file_blobs.content_type LIKE ?', 'image/%')
when 'video'
q.where('material_file_blobs.content_type LIKE ?', 'video/%')
when 'audio'
q.where('material_file_blobs.content_type LIKE ?', 'audio/%')
when 'file_other'
q.where('material_file_attachments.id IS NOT NULL')
.where.not('material_file_blobs.content_type LIKE ?', 'image/%')
.where.not('material_file_blobs.content_type LIKE ?', 'video/%')
.where.not('material_file_blobs.content_type LIKE ?', 'audio/%')
when 'url_only'
q.where('material_file_attachments.id IS NULL').where.not(url: [nil, ''])
else
q
end
end
def apply_material_tag_filter q, filters, tag_graph
return q unless filters[:tag_id]
tag_ids =
if tag_graph
tag_graph[:scope_tag_ids]
else
[filters[:tag_id]]
end
q.where(tag_id: tag_ids)
end
def material_index_tag_graph filters
return nil unless filters[:tag_id]
children_by_parent_id = Hash.new { |h, k| h[k] = [] }
TagImplication.pluck(:parent_tag_id, :tag_id).each do |parent_id, child_id|
children_by_parent_id[parent_id] << child_id
end
scope_tag_ids =
if filters[:include_descendants]
collect_material_descendant_ids(filters[:tag_id], children_by_parent_id)
else
Set.new([filters[:tag_id]])
end
tags_by_id =
Tag
.joins(:tag_name)
.where(id: scope_tag_ids)
.pluck('tags.id', 'tag_names.name', 'tags.category', 'tags.deprecated_at')
.each_with_object({ }) do |(id, name, category, deprecated_at), hash|
hash[id] = { id:, name:, category:, deprecated: deprecated_at.present? }
end
selected_tag = tags_by_id[filters[:tag_id]]
return nil unless selected_tag
group_tag_id_by_tag_id = { }
children_by_parent_id[filters[:tag_id]].each do |child_tag_id|
assign_material_group_branch(child_tag_id,
children_by_parent_id:,
tags_by_id:,
group_tag_id_by_tag_id:,
current_group_tag_id: nil,
seen: Set.new([filters[:tag_id]]))
end
{ selected_tag_id: filters[:tag_id],
selected_tag:,
scope_tag_ids: scope_tag_ids.to_a,
tags_by_id:,
group_tag_id_by_tag_id: }
end
def collect_material_descendant_ids selected_tag_id, children_by_parent_id
ids = Set.new([selected_tag_id])
stack = [selected_tag_id]
until stack.empty?
tag_id = stack.pop
children_by_parent_id[tag_id].each do |child_tag_id|
next if ids.include?(child_tag_id)
ids << child_tag_id
stack << child_tag_id
end
end
ids
end
def assign_material_group_branch tag_id, children_by_parent_id:, tags_by_id:,
group_tag_id_by_tag_id:, current_group_tag_id:, seen:
return if seen.include?(tag_id)
tag = tags_by_id[tag_id]
return unless tag
seen = seen.dup << tag_id
next_group_tag_id = current_group_tag_id
next_group_tag_id = tag_id if next_group_tag_id.nil? && !tag[:deprecated]
group_tag_id_by_tag_id[tag_id] = next_group_tag_id if next_group_tag_id
children_by_parent_id[tag_id].each do |child_tag_id|
assign_material_group_branch(child_tag_id,
children_by_parent_id:,
tags_by_id:,
group_tag_id_by_tag_id:,
current_group_tag_id: next_group_tag_id,
seen:)
end
end
def material_index_groups materials, tag_graph
return [] unless tag_graph
groups_by_key = { }
materials.each do |material|
next unless material.tag_id
group_tag_id =
if material.tag_id == tag_graph[:selected_tag_id]
tag_graph[:selected_tag_id]
else
tag_graph[:group_tag_id_by_tag_id][material.tag_id] || tag_graph[:selected_tag_id]
end
group_tag = tag_graph[:tags_by_id][group_tag_id]
next unless group_tag
key = "tag:#{ group_tag_id }"
group =
groups_by_key[key] ||= { key:,
tag: group_tag_repr(group_tag),
material_ids: [],
count: 0 }
group[:material_ids] << material.id
group[:count] += 1
end
groups_by_key.values.sort_by { |group| group[:tag][:name] }
end
def material_index_tag_scope filters, tag_graph
return nil unless tag_graph
{
tag: group_tag_repr(tag_graph[:selected_tag]),
include_descendants: filters[:include_descendants]
}
end
def group_tag_repr group_tag
{ id: group_tag[:id],
name: group_tag[:name],
category: group_tag[:category] }
end
def material_index_order_sql filters
direction = filters[:direction] == 'asc' ? 'ASC' : 'DESC'
sort_sql =
case filters[:sort]
when 'tag_name'
'tag_names.name'
when 'media_kind'
material_media_kind_sql
when 'file_byte_size'
'material_file_blobs.byte_size'
when 'updated_at'
'materials.updated_at'
when 'version_no'
'materials.version_no'
when 'id'
'materials.id'
else
'materials.created_at'
end
"#{ sort_sql } #{ direction }, materials.id #{ direction }"
end
def material_media_kind_sql
"CASE " \
"WHEN material_file_attachments.id IS NULL AND materials.url IS NOT NULL THEN 5 " \
"WHEN material_file_blobs.content_type LIKE 'image/%' THEN 1 " \
"WHEN material_file_blobs.content_type LIKE 'video/%' THEN 2 " \
"WHEN material_file_blobs.content_type LIKE 'audio/%' THEN 3 " \
"WHEN material_file_attachments.id IS NOT NULL THEN 4 " \
"ELSE 6 END"
end
def upsert_export_paths! material
raw = params[:export_paths]
return if raw.blank?
export_paths = raw.respond_to?(:to_unsafe_h) ? raw.to_unsafe_h : raw.to_h
export_paths.each do |profile, export_path|
profile = profile.to_s
export_path = export_path.to_s.strip
item = material.material_export_items.find_or_initialize_by(profile:)
if export_path.blank?
item.destroy! if item.persisted?
next
end
item.export_path = export_path
item.enabled = true
item.created_by_user ||= current_user
item.save!
end
end
def render_zip_empty
render_unprocessable_entity('ZIP export 対象の素材がありません.')
end
def render_zip_duplicate_path error
render_unprocessable_entity("ZIP export path が重複してゐます: #{ error.message }")
end
def render_zip_missing_file error
missing_files = error.missing_files.map do |missing_file|
{ material_id: missing_file.material_id,
export_path: missing_file.export_path,
blob_id: missing_file.blob_id,
filename: missing_file.filename }
end
render json: { type: 'validation_error',
message: 'ZIP export に必要な素材ファイルが欠損しています.',
errors: { },
base_errors: ['ZIP export に必要な素材ファイルが欠損しています.'],
missing_files: },
status: :unprocessable_entity
end
def build_uploaded_material_blob! file, file_sha256
return nil unless file
file.tempfile.rewind
blob = ActiveStorage::Blob.create_and_upload!(
io: file.tempfile,
filename: file.original_filename,
content_type: file.content_type,
)
MaterialFileSha256.assign_metadata_sha256!(blob, file_sha256)
blob
ensure
file.tempfile.rewind if file&.tempfile
end
def render_material_import_block block
render_validation_error fields: { file: ["抑止された素材です: #{ block.reason }"] }
end
def log_thumbnail_generation material, result
material.reload
Rails.logger.info(
"Material thumbnail generation: material_id=#{ material.id } " \
"result=#{ result } " \
"thumbnail_attached=#{ material.thumbnail.attached? } " \
"content_type=#{ material.content_type }")
end
end
+19 -11
ファイルの表示
@@ -140,24 +140,23 @@ class PostsController < ApplicationController
original_created_from = params[:original_created_from]
original_created_before = params[:original_created_before]
parent_post_ids = parse_parent_post_ids
resized_thumbnail = thumbnail.present? ? Post.resized_thumbnail_attachment(thumbnail) : nil
post = Post.new(title:, url:, thumbnail_base: nil, uploaded_user: current_user,
original_created_from:, original_created_before:)
post.thumbnail.attach(thumbnail) if thumbnail.present?
post.thumbnail.attach(resized_thumbnail) if resized_thumbnail
ApplicationRecord.transaction do
post.save!
tags = Tag.normalise_tags!(tag_names)
tags = Tag.normalise_tags!(tag_names, deny_deprecated: true)
TagVersioning.record_tag_snapshots!(tags, created_by_user: current_user)
tags = Tag.expand_parent_tags(tags)
tags = Tag.expand_parent_tags(tags).reject(&:deprecated?)
sync_post_tags!(post, tags)
sync_parent_posts!(post, parent_post_ids)
post.resized_thumbnail!
PostVersionRecorder.record!(post:, event_type: :create, created_by_user: current_user)
end
@@ -165,6 +164,10 @@ class PostsController < ApplicationController
render json: PostRepr.base(post), status: :created
rescue Tag::NicoTagNormalisationError
render_validation_error fields: { tags: 'ニコニコ・タグは直接指定できません.' }
rescue Tag::DeprecatedTagNormalisationError
render_unprocessable_entity '廃止済みタグは付与できません.', field: :tags
rescue MiniMagick::Error
render_validation_error fields: { thumbnail: ['サムネイル画像の変換に失敗しました.'] }
rescue ArgumentError => e
render_validation_error fields: { parent_post_ids: [e.message] }
rescue ActiveRecord::RecordInvalid => e
@@ -255,6 +258,8 @@ class PostsController < ApplicationController
render json:, status: :ok
rescue Tag::NicoTagNormalisationError
render_validation_error fields: { tags: ['ニコニコ・タグは直接指定できません.'] }
rescue Tag::DeprecatedTagNormalisationError
render_unprocessable_entity '廃止済みタグは付与できません.', field: :tags
rescue ArgumentError => e
render_validation_error fields: { parent_post_ids: [e.message] }
rescue ActiveRecord::RecordInvalid => e
@@ -378,7 +383,7 @@ class PostsController < ApplicationController
end
def build_tag_tree_for tags
tags = tags.to_a
tags = tags.reject(&:deprecated?).to_a
tag_ids = tags.map(&:id)
implications = TagImplication.where(parent_tag_id: tag_ids, tag_id: tag_ids)
@@ -501,7 +506,8 @@ class PostsController < ApplicationController
end
def editable_tag_names_from_post post
post.tags.not_nico.joins(:tag_name).order('tag_names.name').pluck('tag_names.name')
post.tags.not_nico.where(deprecated_at: nil)
.joins(:tag_name).order('tag_names.name').pluck('tag_names.name')
end
def post_incoming_snapshot title:, original_created_from:, original_created_before:,
@@ -533,9 +539,10 @@ class PostsController < ApplicationController
end
def incoming_tag_names_for_snapshot raw_tag_names
tags = Tag.normalise_tags!(raw_tag_names, with_tagme: false)
tags = Tag.normalise_tags!(raw_tag_names, with_tagme: false,
deny_deprecated: true)
Tag.expand_parent_tags(tags).map(&:name).uniq.sort
Tag.expand_parent_tags(tags).reject(&:deprecated?).map(&:name).uniq.sort
end
def post_conflict_json post:, base_version_no:, base_snapshot:,
@@ -622,13 +629,14 @@ class PostsController < ApplicationController
original_created_from: snapshot[:original_created_from],
original_created_before: snapshot[:original_created_before])
editable_tags = Tag.normalise_tags!(snapshot[:tag_names], with_tagme: false)
editable_tags = Tag.normalise_tags!(snapshot[:tag_names], with_tagme: false,
deny_deprecated: true)
TagVersioning.record_tag_snapshots!(editable_tags, created_by_user: current_user)
readonly_tags = post.tags.nico.to_a
tags = readonly_tags + editable_tags
tags = Tag.expand_parent_tags(tags)
tags = Tag.expand_parent_tags(tags).reject(&:deprecated?)
sync_post_tags!(post, tags)
sync_parent_posts!(post, snapshot[:parent_post_ids])
+3
ファイルの表示
@@ -17,6 +17,7 @@ class TagVersionsController < ApplicationController
AND prev.version_no = tag_versions.version_no - 1
SQL
.select('tag_versions.*', 'prev.name AS prev_name', 'prev.category AS prev_category',
'prev.deprecated_at AS prev_deprecated_at',
'prev.aliases AS prev_aliases', 'prev.parent_tag_ids AS prev_parent_tag_ids')
q = q.where('tag_versions.tag_id = ?', tag_id) if tag_id
@@ -62,6 +63,8 @@ class TagVersionsController < ApplicationController
event_type: row.event_type,
name: { current: row.name, prev: row.attributes['prev_name'] },
category: { current: row.category, prev: row.attributes['prev_category'] },
deprecated_at: { current: row.deprecated_at&.iso8601,
prev: row.attributes['prev_deprecated_at']&.iso8601 },
aliases: build_version_values(cur_aliases, prev_aliases, key: :name),
parent_tags:,
created_at: row.created_at.iso8601,
+278 -48
ファイルの表示
@@ -1,5 +1,6 @@
require 'net/http'
require 'uri'
require 'set'
class TagsController < ApplicationController
@@ -14,6 +15,8 @@ class TagsController < ApplicationController
post_count_between[1] = nil if post_count_between[1] < 0
created_between = params[:created_from].presence, params[:created_to].presence
updated_between = params[:updated_from].presence, params[:updated_to].presence
deprecated_given = params.key?(:deprecated)
deprecated = bool?(:deprecated)
order = params[:order].to_s.split(':', 2).map(&:strip)
unless order[0].in?(['name', 'category', 'post_count', 'created_at', 'updated_at'])
@@ -48,6 +51,9 @@ class TagsController < ApplicationController
q = q.where('tags.created_at <= ?', created_between[1]) if created_between[1]
q = q.where('tags.updated_at >= ?', updated_between[0]) if updated_between[0]
q = q.where('tags.updated_at <= ?', updated_between[1]) if updated_between[1]
if deprecated_given
q = deprecated ? q.where.not(deprecated_at: nil) : q.where(deprecated_at: nil)
end
sort_sql =
case order[0]
@@ -76,39 +82,20 @@ class TagsController < ApplicationController
def with_depth
parent_tag_id = params[:parent].to_i
parent_tag_id = nil if parent_tag_id <= 0
material_filter = material_filter_param(default: 'any')
graph = build_with_depth_graph(material_filter)
tag_ids =
if parent_tag_id
TagImplication.where(parent_tag_id:).select(:tag_id)
visible_child_tag_ids(parent_tag_id, graph)
else
Tag.where.not(id: TagImplication.select(:tag_id)).select(:id)
visible_root_tag_ids(graph)
end
tags =
Tag
.joins(:tag_name)
.includes(:tag_name, :materials, tag_name: :wiki_page)
.where(category: [:meme, :character, :material])
.where(id: tag_ids)
.order('tag_names.name')
.distinct
.to_a
has_children_tag_ids =
if tags.empty?
[]
else
TagImplication
.joins(:tag)
.where(parent_tag_id: tags.map(&:id),
tags: { category: [:meme, :character, :material] })
.distinct
.pluck(:parent_tag_id)
end
render json: tags.map { |tag|
TagRepr.base(tag).merge(has_children: has_children_tag_ids.include?(tag.id), children: [])
}
render json: tag_ids
.sort_by { |tag_id| graph[:tags_by_id][tag_id][:name] }
.map { |tag_id| with_depth_lightweight_repr(tag_id, graph) }
end
def autocomplete
@@ -133,6 +120,7 @@ class TagsController < ApplicationController
base = Tag.joins(:tag_name)
.includes(:tag_name, :materials, tag_name: :wiki_page)
.where(deprecated_at: nil)
base = base.where('tags.post_count > 0') if present_only
canonical_hit =
@@ -232,13 +220,16 @@ class TagsController < ApplicationController
def materials_by_name
name = params[:name].to_s.strip
return render_bad_request('name は必須です.') if name.blank?
material_filter = material_filter_param(default: 'any')
tag = Tag.joins(:tag_name)
.includes(:tag_name, :materials, tag_name: :wiki_page)
.find_by(tag_names: { name: })
return head :not_found unless tag
render json: build_tag_children(tag)
graph = build_with_depth_graph(material_filter)
render json: build_tag_children(tag, graph:)
end
def update_all
@@ -252,18 +243,21 @@ class TagsController < ApplicationController
category = params[:category].to_s.strip
return render_unprocessable_entity('名前は必須です.', field: :name) if name.blank?
return render_unprocessable_entity('カテゴリは必須です.', field: :category) if category.blank?
return render_unprocessable_entity '廃止状態は必須です.', field: :deprecated unless params.key?(:deprecated)
if name != tag.name &&
tag.in?([Tag.tagme, Tag.bot, Tag.no_deerjikist, Tag.video, Tag.niconico])
return render_unprocessable_entity('システム・タグの名称は変更できません.', field: :name)
end
if tag.nico? || category == 'nico'
return render_unprocessable_entity('ニコタグは変更できません.', field: :category)
end
return unless validate_tag_rename(tag, name)
alias_names = params[:aliases].to_s.split.uniq
parent_names = params[:parent_tags].to_s.split.uniq
deprecated = bool?(:deprecated)
if tag.nico? && deprecated
return render_unprocessable_entity 'ニコタグは廃止できません.', field: :deprecated
end
if tag.nico? || category == 'nico'
return render_unprocessable_entity 'ニコタグは変更できません.', field: :category
end
ApplicationRecord.transaction do
TagVersioning.ensure_snapshot!(tag, created_by_user: current_user)
@@ -272,8 +266,12 @@ class TagsController < ApplicationController
name_changed = name != old_name
wiki_page = tag.tag_name.wiki_page if name_changed
tag.update!(category:)
tag.tag_name.update!(name:)
if tag.deprecated? == deprecated
tag.update!(category:)
else
tag.update!(category:, deprecated_at: deprecated ? Time.current : nil)
end
rename_tag_name!(tag, name) if name_changed
alias_names << old_name if name_changed
alias_names.delete(name)
@@ -300,11 +298,19 @@ class TagsController < ApplicationController
name = params[:name].presence
category = params[:category].presence
deprecated_given = params.key?(:deprecated)
deprecated = bool?(:deprecated)
tag = Tag.find(params[:id])
if tag.nico? && deprecated_given && deprecated
return render_unprocessable_entity 'ニコタグは廃止できません.', field: :deprecated
end
return unless validate_tag_rename(tag, name)
if tag.nico? || (category.present? && category == 'nico')
return render_unprocessable_entity('ニコタグは変更できません.', field: :category)
return render_unprocessable_entity 'ニコタグは変更できません.', field: :category
end
ApplicationRecord.transaction do
@@ -314,8 +320,11 @@ class TagsController < ApplicationController
name_changed = name.present? && name != old_name
wiki_page = tag.tag_name.wiki_page if name_changed
tag.tag_name.update!(name:) if name.present?
rename_tag_name!(tag, name) if name_changed
tag.update!(category:) if category.present?
if deprecated_given && tag.deprecated? != deprecated
tag.update!(deprecated_at: deprecated ? Time.current : nil)
end
tag.reload
@@ -332,18 +341,178 @@ class TagsController < ApplicationController
private
def build_tag_children tag
def material_filter_param default:
value = params[:material_filter].to_s.presence
return default unless ['present', 'missing', 'any'].include?(value)
value
end
def build_with_depth_graph material_filter
children_by_parent_id = Hash.new { |h, k| h[k] = [] }
parent_ids_by_child_id = Hash.new { |h, k| h[k] = [] }
TagImplication.pluck(:parent_tag_id, :tag_id).each do |parent_id, child_id|
children_by_parent_id[parent_id] << child_id
parent_ids_by_child_id[child_id] << parent_id
end
tag_ids = (children_by_parent_id.keys +
parent_ids_by_child_id.keys +
Tag.where(category: ['meme', 'character', 'material']).pluck(:id)).uniq
material_tag_ids = Material.unscoped.kept.where.not(tag_id: nil).distinct.pluck(:tag_id).to_set
tags_by_id = Tag.joins(:tag_name)
.where(id: tag_ids)
.pluck('tags.id', 'tag_names.name', 'tags.category', 'tags.deprecated_at')
.each_with_object({ }) do |(id, name, category, deprecated_at), h|
h[id] = { name:, category:, deprecated: deprecated_at.present?,
has_material: material_tag_ids.include?(id) }
end
{ children_by_parent_id:, parent_ids_by_child_id:, tags_by_id:,
visible_child_tag_ids_by_parent_id: { },
visible_subtree_by_tag_id: { },
material_filter: }
end
def visible_root_tag_ids graph
root_ids = Set.new
graph[:tags_by_id].each do |tag_id, attrs|
next unless visible_root_tag?(tag_id, graph)
next unless visible_subtree?(tag_id, graph)
if attrs[:deprecated]
collect_visible_child_tag_ids(tag_id, graph, root_ids, Set.new)
else
root_ids << tag_id
end
end
root_ids.to_a
end
def visible_root_tag? tag_id, graph
seen = Set.new([tag_id])
stack = graph[:parent_ids_by_child_id][tag_id].dup
until stack.empty?
parent_id = stack.pop
next if seen.include?(parent_id)
seen << parent_id
parent = graph[:tags_by_id][parent_id]
next unless parent
return false unless parent[:deprecated]
stack.concat(graph[:parent_ids_by_child_id][parent_id])
end
true
end
def visible_child_tag_ids parent_tag_id, graph
cache = graph[:visible_child_tag_ids_by_parent_id]
return cache[parent_tag_id] if cache.key?(parent_tag_id)
visible_ids = Set.new
graph[:children_by_parent_id][parent_tag_id].each do |child_tag_id|
collect_visible_child_tag_ids(child_tag_id, graph, visible_ids, Set.new([parent_tag_id]))
end
cache[parent_tag_id] = visible_ids.to_a
end
def collect_visible_child_tag_ids tag_id, graph, visible_ids, seen
return if seen.include?(tag_id)
seen = seen.dup << tag_id
tag = graph[:tags_by_id][tag_id]
return unless tag
if tag[:deprecated]
graph[:children_by_parent_id][tag_id].each do |child_tag_id|
collect_visible_child_tag_ids(child_tag_id, graph, visible_ids, seen)
end
return
end
if with_depth_visible_tag?(tag, graph[:material_filter])
visible_ids << tag_id
return
end
visible_ids << tag_id if graph[:children_by_parent_id][tag_id].any? { |child_tag_id|
visible_subtree?(child_tag_id, graph)
}
end
def with_depth_visible_tag? tag, material_filter
return false unless tag[:category].in?(['meme', 'character', 'material']) && !tag[:deprecated]
case material_filter
when 'present'
tag[:has_material]
when 'missing'
!tag[:has_material]
else
true
end
end
def visible_subtree? tag_id, graph
cache = graph[:visible_subtree_by_tag_id]
return cache[tag_id] if cache.key?(tag_id)
tag = graph[:tags_by_id][tag_id]
return cache[tag_id] = false unless tag
if tag[:deprecated]
return cache[tag_id] =
graph[:children_by_parent_id][tag_id].any? { |child_tag_id|
visible_subtree?(child_tag_id, graph)
}
end
cache[tag_id] =
with_depth_visible_tag?(tag, graph[:material_filter]) ||
graph[:children_by_parent_id][tag_id].any? { |child_tag_id|
visible_subtree?(child_tag_id, graph)
}
end
def with_depth_lightweight_repr tag_id, graph
tag = graph[:tags_by_id][tag_id]
{ id: tag_id,
name: tag[:name],
category: tag[:category],
deprecated: tag[:deprecated],
has_material: tag[:has_material],
has_children: visible_child_tag_ids(tag_id, graph).present?,
children: [] }
end
def build_tag_children tag, graph: nil
material = tag.materials.first
file = nil
content_type = nil
if material&.file&.attached?
file = rails_storage_proxy_url(material.file, only_path: false)
content_type = material.file.blob.content_type
tag_graph = graph && graph[:tags_by_id][tag.id]
material =
nil if tag_graph && !with_depth_visible_tag?(tag_graph, graph[:material_filter])
children = tag.children.sort_by(&:name)
if graph
children = children.filter { |child_tag| visible_subtree?(child_tag.id, graph) }
end
TagRepr.base(tag).merge(
children: tag.children.sort_by { _1.name }.map { build_tag_children(_1) },
material: material.as_json&.merge(file:, content_type:))
children: children.map { build_tag_children(_1, graph:) },
has_material: tag_graph ? tag_graph[:has_material] : material.present?,
material: material && MaterialRepr.base(material, host: request.base_url))
end
def record_tag_version! tag, event_type:, created_by_user:, name_changed: false, wiki_page: nil
@@ -365,6 +534,67 @@ class TagsController < ApplicationController
created_by_user:)
end
def validate_tag_rename tag, name
return true if name.blank? || name == tag.name
if tag.in?([Tag.tagme, Tag.bot, Tag.no_deerjikist, Tag.video, Tag.niconico])
render_unprocessable_entity 'システム・タグの名称は変更できません.', field: :name
return false
end
target_tag_name = TagName.with_discarded.find_by(name:)
return true if target_tag_name.nil?
return true if target_tag_name.canonical_id?
render_unprocessable_entity 'その名前は既に使はれてゐます.', field: :name
false
end
def rename_tag_name! tag, name
return if name == tag.name
current_tag_name = tag.tag_name
target_tag_name = TagName.with_discarded.find_by(name:)
if target_tag_name.nil?
current_tag_name.update!(name:)
return
end
promote_tag_alias!(
tag,
current_tag_name:,
promoted_tag_name: target_tag_name)
end
def promote_tag_alias! tag, current_tag_name:, promoted_tag_name:
old_owner_tag = promoted_tag_name.canonical&.tag
if old_owner_tag && old_owner_tag != tag
TagVersioning.ensure_snapshot!(old_owner_tag, created_by_user: current_user)
end
promoted_tag_name.undiscard! if promoted_tag_name.discarded?
promoted_tag_name.update!(canonical: nil)
TagName.with_discarded
.where(canonical_id: current_tag_name.id)
.where.not(id: promoted_tag_name.id)
.find_each do |alias_tag_name|
alias_tag_name.update!(canonical: promoted_tag_name)
end
current_tag_name.wiki_page&.update!(tag_name: promoted_tag_name)
tag.update!(tag_name: promoted_tag_name)
current_tag_name.association(:wiki_page).reset
current_tag_name.association(:tag).reset
current_tag_name.reload.update!(canonical: promoted_tag_name)
return unless old_owner_tag && old_owner_tag != tag
record_tag_version!(old_owner_tag.reload, event_type: :update, created_by_user: current_user)
end
def update_aliases! tag, alias_names
alias_names = alias_names.uniq
+11 -7
ファイルの表示
@@ -4,17 +4,18 @@ class WikiPagesController < ApplicationController
def index
title = params[:title].to_s.strip
if title.blank?
return render json: WikiPageRepr.base(WikiPage.joins(:tag_name).includes(:tag_name))
return render json: WikiPageRepr.base(
WikiPage.joins(:tag_name).includes(tag_name: :tag))
end
q = WikiPage.joins(:tag_name).includes(:tag_name)
q = WikiPage.joins(:tag_name).includes(tag_name: :tag)
.where('tag_names.name LIKE ?', "%#{ WikiPage.sanitize_sql_like(title) }%")
render json: WikiPageRepr.base(q.limit(20))
end
def show
page = WikiPage.joins(:tag_name)
.includes(:tag_name)
.includes(tag_name: :tag)
.find_by(id: params[:id])
render_wiki_page_or_404 page
end
@@ -22,7 +23,7 @@ class WikiPagesController < ApplicationController
def show_by_title
title = params[:title].to_s.strip
page = WikiPage.joins(:tag_name)
.includes(:tag_name)
.includes(tag_name: :tag)
.find_by(tag_name: { name: title })
render_wiki_page_or_404 page
end
@@ -51,7 +52,7 @@ class WikiPagesController < ApplicationController
from = params[:from].presence
to = params[:to].presence
page = WikiPage.joins(:tag_name).includes(:tag_name).find(id)
page = WikiPage.joins(:tag_name).includes(tag_name: :tag).find(id)
from_rev = from && page.wiki_revisions.find(from)
to_rev = to ? page.wiki_revisions.find(to) : page.current_revision
@@ -76,6 +77,7 @@ class WikiPagesController < ApplicationController
render json: { wiki_page_id: page.id,
title: page.title,
deprecated_at: page.deprecated_at,
older_revision_id: from_rev&.id,
newer_revision_id: to_rev.id,
diff: diff_json }
@@ -157,7 +159,7 @@ class WikiPagesController < ApplicationController
def changes
id = params[:id].presence
q = WikiRevision.joins(wiki_page: :tag_name)
.includes(:created_user, wiki_page: :tag_name)
.includes(:created_user, wiki_page: { tag_name: :tag })
.order(id: :desc)
q = q.where(wiki_page_id: id) if id
@@ -165,7 +167,9 @@ class WikiPagesController < ApplicationController
{ revision_id: rev.id,
pred: rev.base_revision_id,
succ: nil,
wiki_page: { id: rev.wiki_page_id, title: rev.wiki_page.title },
wiki_page: { id: rev.wiki_page_id,
title: rev.wiki_page.title,
deprecated_at: rev.wiki_page.deprecated_at },
user: rev.created_user && { id: rev.created_user.id, name: rev.created_user.name },
kind: rev.kind,
message: rev.message,
+2 -2
ファイルの表示
@@ -1,7 +1,7 @@
class GekanatorQuestionExample < ApplicationRecord
ANSWERS = GekanatorQuestionSuggestion::ANSWERS
NON_UNKNOWN_ANSWERS = ANSWERS - ['unknown']
SOURCES = ['initial_suggestion', 'post_game_extra'].freeze
SOURCES = ['initial_suggestion', 'post_game_answer', 'post_game_extra'].freeze
belongs_to :gekanator_question
belongs_to :post
@@ -35,7 +35,7 @@ class GekanatorQuestionExample < ApplicationRecord
self.answer_counts = counts
self.sample_count = counts.values.sum
self.gekanator_game = gekanator_game if gekanator_game.present?
self.source = source if new_record?
self.source = source
apply_aggregated_answer!(preferred_answer: answer)
self
-13
ファイルの表示
@@ -1,5 +1,4 @@
class GekanatorQuestionSuggestion < ApplicationRecord
MAX_QUESTIONS_PER_GAME = 3
ANSWERS = ['yes', 'no', 'partial', 'probably_no', 'unknown'].freeze
belongs_to :gekanator_game
@@ -10,16 +9,4 @@ class GekanatorQuestionSuggestion < ApplicationRecord
validates :question_text, presence: true, length: { maximum: 1000 }
validates :answer, presence: true, inclusion: { in: ANSWERS }
validates :processed, inclusion: { in: [true, false] }
validate :question_suggestion_limit_per_game, on: :create
private
def question_suggestion_limit_per_game
return if gekanator_game_id.blank?
count = GekanatorQuestionSuggestion.where(gekanator_game_id:).count
if count >= MAX_QUESTIONS_PER_GAME
errors.add(:base, '質問追加数を超えてゐます.')
end
end
end
+43 -2
ファイルの表示
@@ -1,6 +1,9 @@
class Material < ApplicationRecord
include MyDiscard
SOURCE_KINDS = ['uri', 'google_drive_path', 'google_drive_file',
'legacy_drive_path'].freeze
default_scope -> { kept }
belongs_to :parent, class_name: 'Material', optional: true
@@ -10,9 +13,18 @@ class Material < ApplicationRecord
belongs_to :created_by_user, class_name: 'User', optional: true
belongs_to :updated_by_user, class_name: 'User', optional: true
has_one_attached :file, dependent: :purge
has_many :material_versions, dependent: :destroy
has_many :material_export_items, dependent: :destroy
validates :tag_id, presence: true, uniqueness: true
has_one_attached :file, dependent: :purge
has_one_attached :thumbnail, dependent: :purge
before_validation :assign_normalized_source_key
validates :tag_id, uniqueness: true, allow_nil: true
validates :source_kind,
inclusion: { in: SOURCE_KINDS },
allow_blank: true
validate :file_must_be_attached
validate :tag_must_be_material_category
@@ -23,8 +35,37 @@ class Material < ApplicationRecord
file.blob.content_type
end
def file_byte_size
return nil unless file&.attached?
file.blob.byte_size
end
def file_filename
return nil unless file&.attached?
file.blob.filename.to_s
end
def snapshot_export_paths
material_export_items.order(:profile).pluck(:profile, :export_path).to_h
end
private
def assign_normalized_source_key
if source_kind.blank?
self.normalized_source_key = nil
return
end
self.normalized_source_key =
MaterialSyncSuppression.normalize_source_key(source_kind:,
source_uri:,
drive_path: source_path,
drive_file_id: source_file_id)
end
def file_must_be_attached
return if url.present? || file.attached?
+48
ファイルの表示
@@ -0,0 +1,48 @@
class MaterialExportItem < ApplicationRecord
VALID_PROFILES = ['legacy_drive'].freeze
belongs_to :material
belongs_to :created_by_user, class_name: 'User', optional: true
validates :profile, presence: true, inclusion: { in: VALID_PROFILES }
validates :export_path, presence: true, uniqueness: { scope: :profile }
validates :material_id, uniqueness: { scope: :profile }
validate :export_path_must_be_relative_safe_path
scope :enabled, -> { where(enabled: true) }
private
def export_path_must_be_relative_safe_path
return if export_path.blank?
if export_path.start_with?('/')
errors.add(:export_path, '絶対パスは使へません.')
end
if export_path.match?(/\A[A-Za-z]:\//)
errors.add(:export_path, '絶対パスは使へません.')
end
if export_path.include?('\\')
errors.add(:export_path, '/ 区切りで指定してください.')
end
if export_path.include?("\0")
errors.add(:export_path, 'NUL は使へません.')
end
parts = export_path.split('/')
if export_path.include?('//')
errors.add(:export_path, '空の path segment は使へません.')
end
if parts.any? { |part| part.in?(['.', '..']) }
errors.add(:export_path, '.. は使へません.')
end
if export_path.end_with?('/')
errors.add(:export_path, 'directory path は使へません.')
end
end
end
+29
ファイルの表示
@@ -0,0 +1,29 @@
class MaterialImportBlock < ApplicationRecord
MATCH_KINDS = ['sha256', 'exact_path', 'path_prefix', 'manual'].freeze
REASONS = [
'copyright_high_risk',
'copyright_takedown',
'adult_or_sensitive',
'personal_information',
'malware_or_dangerous_file',
'duplicate_or_low_quality',
'source_owner_request',
'other'
].freeze
belongs_to :created_by_user, class_name: 'User', optional: true
validates :match_kind, presence: true, inclusion: { in: MATCH_KINDS }
validates :reason, presence: true, inclusion: { in: REASONS }
validates :sha256, length: { is: 64 }, allow_blank: true
validate :match_value_must_be_present
private
def match_value_must_be_present
return if match_kind == 'manual'
return if sha256.present? || external_path_pattern.present?
errors.add(:base, 'sha256 または external_path_pattern は必須です.')
end
end
+45
ファイルの表示
@@ -0,0 +1,45 @@
class MaterialSyncSource < ApplicationRecord
SOURCE_KINDS = ['uri', 'google_drive_path', 'google_drive_file', 'legacy_drive_path'].freeze
belongs_to :created_by_user, class_name: 'User', optional: true
belongs_to :updated_by_user, class_name: 'User', optional: true
validates :name, presence: true
validates :source_kind,
presence: true,
inclusion: { in: SOURCE_KINDS }
validates :profile, presence: true, inclusion: { in: MaterialExportItem::VALID_PROFILES }
validate :source_value_must_be_present
scope :enabled, -> { where(enabled: true) }
def normalized_source_key
MaterialSyncSuppression.normalize_source_key(source_kind:,
source_uri:,
source_path:,
source_file_id:)
end
private
def source_value_must_be_present
return if source_value_present?
errors.add(:base, '同期元を指定してください.')
end
def source_value_present?
case source_kind
when 'uri'
source_uri.present?
when 'google_drive_file'
source_file_id.present? || source_uri.present?
when 'google_drive_path'
source_file_id.present? || source_uri.present?
when 'legacy_drive_path'
source_path.present?
else
normalized_source_key.present?
end
end
end
+105
ファイルの表示
@@ -0,0 +1,105 @@
class MaterialSyncSuppression < ApplicationRecord
SOURCE_KINDS = [
'uri',
'google_drive_path',
'google_drive_path_prefix',
'google_drive_file',
'legacy_drive_path',
'legacy_drive_path_prefix'
].freeze
REASONS = [
'copyright_high_risk',
'copyright_takedown',
'adult_or_sensitive',
'personal_information',
'malware_or_dangerous_file',
'duplicate_or_low_quality',
'source_owner_request',
'other'
].freeze
belongs_to :created_by_user, class_name: 'User', optional: true
before_validation :assign_normalized_source_key
validates :source_kind, presence: true, inclusion: { in: SOURCE_KINDS }
validates :reason, presence: true, inclusion: { in: REASONS }
validates :normalized_source_key, presence: true, uniqueness: true
validate :source_value_must_be_present
validate :drive_path_must_be_safe_relative_path
def self.normalize_source_key source_kind:,
source_uri: nil,
drive_path: nil,
drive_file_id: nil,
source_path: nil,
source_file_id: nil
kind = source_kind.to_s
value =
case kind
when 'uri'
source_uri.to_s.strip
when 'google_drive_path', 'google_drive_path_prefix',
'legacy_drive_path', 'legacy_drive_path_prefix'
(source_path || drive_path).to_s.strip.gsub(%r{/+}, '/')
when 'google_drive_file'
(source_file_id || drive_file_id).to_s.strip
else
''
end
return nil if kind.blank? || value.blank?
"#{ kind }:#{ value }"
end
private
def assign_normalized_source_key
normalize_path_fields!
self.normalized_source_key =
self.class.normalize_source_key(source_kind:,
source_uri:,
drive_path:,
drive_file_id:)
end
def source_value_must_be_present
return if normalized_source_key.present?
errors.add(:base, '同期元を指定してください.')
end
def drive_path_must_be_safe_relative_path
return unless source_kind.in?(
['google_drive_path', 'google_drive_path_prefix',
'legacy_drive_path', 'legacy_drive_path_prefix'])
value = drive_path.to_s
return if value.blank?
if value.start_with?('/') || value.match?(/\A[A-Za-z]:\//)
errors.add(:drive_path, '相対 path を指定してください.')
end
if value.start_with?('My Drive/', 'マイドライブ/')
errors.add(:drive_path, '同期元フォルダからの相対 path を指定してください.')
end
errors.add(:drive_path, 'NUL は使へません.') if value.include?("\0")
errors.add(:drive_path, '連続スラッシュは使へません.') if value.include?('//')
errors.add(:drive_path, '末尾スラッシュは使へません.') if value.end_with?('/')
parts = value.split('/')
if parts.any? { |part| part.in?(['.', '..']) }
errors.add(:drive_path, '. や .. は使へません.')
end
end
def normalize_path_fields!
return unless source_kind.in?(
['google_drive_path', 'google_drive_path_prefix',
'legacy_drive_path', 'legacy_drive_path_prefix'])
self.drive_path = drive_path.to_s.tr('\\', '/').strip.presence
end
end
+17
ファイルの表示
@@ -0,0 +1,17 @@
class MaterialVersion < ApplicationRecord
EVENT_TYPE_MAP = { create: 'create',
update: 'update',
discard: 'discard',
restore: 'restore' }.freeze
include VersionRecord
belongs_to :material
belongs_to :tag, optional: true
belongs_to :parent, class_name: 'Material', optional: true
belongs_to :updated_by_user, class_name: 'User', optional: true
def export_paths_hash
export_paths_json || {}
end
end
+17 -6
ファイルの表示
@@ -1,5 +1,19 @@
class Post < ApplicationRecord
require 'mini_magick'
require 'stringio'
def self.resized_thumbnail_attachment(upload)
upload.rewind
image = MiniMagick::Image.read(upload.read)
image.resize '180x180'
image.format 'jpg'
{ io: StringIO.new(image.to_blob),
filename: 'resized_thumbnail.jpg',
content_type: 'image/jpeg' }
ensure
upload.rewind
end
belongs_to :uploaded_user, class_name: 'User', optional: true
@@ -7,6 +21,8 @@ class Post < ApplicationRecord
has_many :active_post_tags, -> { kept }, class_name: 'PostTag', inverse_of: :post
has_many :post_tags_with_discarded, -> { with_discarded }, class_name: 'PostTag'
has_many :tags, through: :active_post_tags
has_many :active_tags, -> { where(tags: { deprecated_at: nil }) },
through: :active_post_tags, source: :tag
has_many :user_post_views, dependent: :delete_all
has_many :post_similarities, dependent: :delete_all
@@ -85,12 +101,7 @@ class Post < ApplicationRecord
def resized_thumbnail!
return unless thumbnail.attached?
image = MiniMagick::Image.read(thumbnail.download)
image.resize '180x180'
thumbnail.purge
thumbnail.attach(io: File.open(image.path),
filename: 'resized_thumbnail.jpg',
content_type: 'image/jpeg')
thumbnail.attach(self.class.resized_thumbnail_attachment(StringIO.new(thumbnail.download)))
end
private
+24 -1
ファイルの表示
@@ -8,6 +8,15 @@ class Tag < ApplicationRecord
;
end
class DeprecatedTagNormalisationError < ArgumentError
attr_reader :tag_names
def initialize tag_names
@tag_names = Array(tag_names)
super('deprecated tags are not allowed')
end
end
has_many :post_tags, inverse_of: :tag
has_many :active_post_tags, -> { kept }, class_name: 'PostTag', inverse_of: :tag
has_many :post_tags_with_discarded, -> { with_discarded }, class_name: 'PostTag'
@@ -58,6 +67,7 @@ class Tag < ApplicationRecord
validate :nico_tag_name_must_start_with_nico
validate :tag_name_must_be_canonical
validate :category_must_be_deerjikist_with_deerjikists
validate :nico_tags_cannot_be_deprecated
scope :nico_tags, -> { nico }
@@ -77,6 +87,8 @@ class Tag < ApplicationRecord
(self.tag_name ||= build_tag_name).name = val
end
def deprecated? = deprecated_at?
def has_wiki = wiki_page.present?
def material_id = materials.first&.id
@@ -92,7 +104,8 @@ class Tag < ApplicationRecord
def self.normalise_tags! tag_names, with_tagme: true,
with_no_deerjikist: true,
deny_nico: true
deny_nico: true,
deny_deprecated: false
if deny_nico && tag_names.any? { |n| n.downcase.start_with?('nico:') }
raise NicoTagNormalisationError
end
@@ -101,6 +114,10 @@ class Tag < ApplicationRecord
pf, cat = CATEGORY_PREFIXES.find { |p, _| name.downcase.start_with?(p) } || ['', nil]
name = TagName.canonicalise(name.sub(/\A#{ pf }/i, '')).first
find_or_create_by_tag_name!(name, category: (cat || :general)).tap do |tag|
if deny_deprecated && tag.deprecated?
raise DeprecatedTagNormalisationError, [tag.name]
end
tag.update!(category: cat) if cat && tag.category != cat
end
end
@@ -228,4 +245,10 @@ class Tag < ApplicationRecord
errors.add :category, 'ニジラーと紐づいてゐるタグはニジラー・カテゴリである必要があります.'
end
end
def nico_tags_cannot_be_deprecated
if nico? && deprecated_at.present?
errors.add :deprecated_at, 'ニコタグは廃止できません.'
end
end
end
+24
ファイルの表示
@@ -6,6 +6,7 @@ class TagImplication < ApplicationRecord
validates :parent_tag_id, presence: true
validate :parent_tag_mustnt_be_itself
validate :parent_tag_mustnt_create_cycle
private
@@ -14,4 +15,27 @@ class TagImplication < ApplicationRecord
errors.add :parent_tag_id, '親タグは子タグと同一であってはなりません.'
end
end
def parent_tag_mustnt_create_cycle
return if tag_id.blank? || parent_tag_id.blank?
return if errors[:parent_tag_id].present?
seen = { }
stack = [parent_tag_id]
until stack.empty?
current_id = stack.pop
next if seen[current_id]
seen[current_id] = true
if current_id == tag_id
errors.add :parent_tag_id, '親タグに子孫タグを指定すると循環します.'
errors.add :base, 'タグの親子関係が循環します.'
return
end
stack.concat(TagImplication.where(tag_id: current_id).pluck(:parent_tag_id))
end
end
end
+12 -4
ファイルの表示
@@ -1,15 +1,23 @@
module VersionRecord
extend ActiveSupport::Concern
DEFAULT_EVENT_TYPE_MAP = { create: 'create',
update: 'update',
discard: 'discard',
restore: 'restore' }.freeze
def readonly? = persisted?
included do
event_type_map = if const_defined?(:EVENT_TYPE_MAP, false)
const_get(:EVENT_TYPE_MAP)
else
DEFAULT_EVENT_TYPE_MAP
end
belongs_to :created_by_user, class_name: 'User', optional: true
enum :event_type, { create: 'create',
update: 'update',
discard: 'discard',
restore: 'restore' }, prefix: true, validate: true
enum :event_type, event_type_map, prefix: true, validate: true
validates :version_no, presence: true, numericality: { only_integer: true, greater_than: 0 }
validates :event_type, presence: true
+1
ファイルの表示
@@ -22,6 +22,7 @@ class WikiPage < ApplicationRecord
validates :body, presence: true
def title = tag_name.name
def deprecated_at = tag_name.tag&.deprecated_at
def title= val
(self.tag_name ||= build_tag_name).name = val
+87 -2
ファイルの表示
@@ -2,7 +2,9 @@
module MaterialRepr
BASE = { only: [:id, :url, :created_at, :updated_at],
BASE = { only: [:id, :url, :version_no, :source_kind, :source_uri,
:source_path, :source_file_id, :normalized_source_key,
:created_at, :updated_at],
methods: [:content_type],
include: { tag: TagRepr::BASE,
created_by_user: UserRepr::BASE,
@@ -15,10 +17,93 @@ module MaterialRepr
file: if material.file.attached?
Rails.application.routes.url_helpers.rails_storage_proxy_url(
material.file, host:)
end)
end,
thumbnail: thumbnail_url(material, host:),
thumbnail_fallback_text: thumbnail_fallback_text(material),
thumbnail_fallback_kind: thumbnail_fallback_kind(material),
media_kind: media_kind(material),
file_byte_size: material.file_byte_size,
export_paths: export_paths(material),
export_items: export_items(material))
end
def many materials, host:
materials.map { |m| base(m, host:) }
end
def list material, host:
{ id: material.id,
version_no: material.version_no,
url: material.url,
source_kind: material.source_kind,
source_uri: material.source_uri,
source_path: material.source_path,
source_file_id: material.source_file_id,
normalized_source_key: material.normalized_source_key,
tag: compact_tag(material.tag),
thumbnail: thumbnail_url(material, host:),
thumbnail_fallback_text: thumbnail_fallback_text(material),
thumbnail_fallback_kind: thumbnail_fallback_kind(material),
media_kind: media_kind(material),
content_type: material.content_type,
file_byte_size: material.file_byte_size,
created_at: material.created_at,
updated_at: material.updated_at,
export_paths: export_paths(material),
export_items: export_items(material) }
end
def list_many materials, host:
materials.map { |m| list(m, host:) }
end
def export_paths material
material.material_export_items.each_with_object({ }) do |item, hash|
hash[item.profile] = item.enabled ? item.export_path : ''
end
end
def export_items material
material.material_export_items.map do |item|
{ id: item.id,
profile: item.profile,
export_path: item.export_path,
enabled: item.enabled }
end
end
def thumbnail_url material, host:
return nil unless material.thumbnail.attached?
Rails.application.routes.url_helpers.rails_storage_proxy_url(
material.thumbnail, host:)
end
def thumbnail_fallback_text material
material.tag&.name || material.created_at&.strftime('%Y-%m-%d')
end
def thumbnail_fallback_kind material
material.tag.present? ? 'tag_name' : 'created_at'
end
def media_kind material
return 'url_only' unless material.file.attached?
content_type = material.file.blob.content_type.to_s
return 'image' if content_type.start_with?('image/')
return 'video' if content_type.start_with?('video/')
return 'audio' if content_type.start_with?('audio/')
'file_other'
end
def compact_tag tag
return nil unless tag
{ id: tag.id,
name: tag.name,
category: tag.category,
deprecated_at: tag.deprecated_at }
end
end
+1 -1
ファイルの表示
@@ -53,7 +53,7 @@ module PostRepr
end
def tag_json tags
tags.map { |tag| TagRepr.inline(tag) }
tags.reject(&:deprecated?).map { |tag| TagRepr.inline(tag) }
end
def thumbnail_url post
+1 -1
ファイルの表示
@@ -2,7 +2,7 @@
module TagRepr
BASE = { only: [:id, :category, :post_count, :created_at, :updated_at],
BASE = { only: [:id, :category, :post_count, :created_at, :updated_at, :deprecated_at],
methods: [:name, :has_wiki, :material_id, :has_deerjikists] }.freeze
module_function
+1 -1
ファイルの表示
@@ -2,7 +2,7 @@
module WikiPageRepr
BASE = { methods: [:title] }.freeze
BASE = { methods: [:title, :deprecated_at] }.freeze
module_function
+233
ファイルの表示
@@ -0,0 +1,233 @@
require 'json'
require 'jwt'
require 'net/http'
require 'openssl'
require 'tempfile'
require 'uri'
module GoogleDrive
class ApiClient
DRIVE_ENDPOINT = 'https://www.googleapis.com/drive/v3'
TOKEN_ENDPOINT = 'https://oauth2.googleapis.com/token'
FOLDER_MIME_TYPE = 'application/vnd.google-apps.folder'
NATIVE_FILE_MIME_TYPE_PREFIX = 'application/vnd.google-apps.'
DRIVE_SCOPE = 'https://www.googleapis.com/auth/drive.readonly'
def initialize service_account_email: ENV['GOOGLE_DRIVE_SERVICE_ACCOUNT_EMAIL'],
private_key: ENV['GOOGLE_DRIVE_PRIVATE_KEY'],
private_key_path: ENV['GOOGLE_DRIVE_PRIVATE_KEY_PATH'],
subject: ENV['GOOGLE_DRIVE_SUBJECT']
@service_account_email = service_account_email.to_s
@private_key = load_private_key(private_key, private_key_path)
@subject = subject.to_s.presence
@access_token = nil
end
def list_material_files_under_folder folder_id
files = []
each_material_file_under_folder(folder_id) { |entry| files << entry }
files
end
def each_material_file_under_folder folder_id
return enum_for(__method__, folder_id) unless block_given?
walk_folder(folder_id, nil) do |entry, relative_path|
next if entry['mimeType'] == FOLDER_MIME_TYPE
next if native_file?(entry['mimeType'])
yield build_file_entry(entry, relative_path)
end
end
def fetch_material_file file_id
metadata = get_file(file_id)
return nil if native_file?(metadata['mimeType'])
build_file_entry(metadata, metadata['name'])
end
def download_to_tempfile file_id, filename:
tempfile = Tempfile.new(['material-sync', File.extname(filename.to_s)])
tempfile.binmode
request_binary("/files/#{ file_id }",
{ alt: 'media', supportsAllDrives: true }) do |chunk|
tempfile.write(chunk.b)
end
tempfile.rewind
tempfile
rescue StandardError
tempfile&.close!
raise
end
def extract_file_id value
raw = value.to_s.strip
return nil if raw.blank?
return raw unless raw.include?('/')
uri = URI.parse(raw)
return uri.query.to_s[%r{(?:^|&)id=([^&]+)}, 1] if uri.query.present?
return uri.path[%r{/folders/([^/]+)}, 1] if uri.path.include?('/folders/')
uri.path[%r{/d/([^/]+)}, 1]
rescue URI::InvalidURIError
nil
end
private
def walk_folder folder_id, prefix, &block
list_children(folder_id).each do |entry|
relative_path = MaterialSyncExportPath.build(prefix:,
relative_path: entry['name'])
if entry['mimeType'] == FOLDER_MIME_TYPE
walk_folder(entry['id'], relative_path, &block)
next
end
block.call(entry, relative_path)
end
end
def build_file_entry entry, relative_path
{ id: entry['id'],
name: entry['name'],
mime_type: entry['mimeType'],
relative_path: MaterialSyncExportPath.normalize_path(relative_path),
sha256_checksum: entry['sha256Checksum'],
web_view_link: entry['webViewLink'],
web_content_link: entry['webContentLink'] }
end
def list_children folder_id
files = []
page_token = nil
loop do
response =
request_json('/files', {
q: "'#{ folder_id }' in parents and trashed = false",
fields: 'nextPageToken,files(id,name,mimeType,sha256Checksum,' \
'webViewLink,webContentLink)',
orderBy: 'folder,name',
pageSize: 1000,
supportsAllDrives: true,
includeItemsFromAllDrives: true,
pageToken: page_token }.compact)
files.concat(response.fetch('files'))
page_token = response['nextPageToken']
break if page_token.blank?
end
files
end
def get_file file_id
request_json("/files/#{ file_id }",
fields: 'id,name,mimeType,sha256Checksum,webViewLink,webContentLink',
supportsAllDrives: true)
end
def request_json path, params = {}
response = request(:get, path, params:)
unless response.is_a?(Net::HTTPSuccess)
raise "Google Drive API error: #{ response.code } #{ response.body }"
end
JSON.parse(response.body)
end
def request_binary path, params = {}
request(:get, path, params:) do |response|
unless response.is_a?(Net::HTTPSuccess)
raise "Google Drive download error: #{ response.code } #{ response.body }"
end
response.read_body do |chunk|
yield chunk.b
end
end
end
def request method, path, params: nil
uri = URI(DRIVE_ENDPOINT + path)
uri.query = URI.encode_www_form(params) if params.present?
klass =
case method
when :get
Net::HTTP::Get
else
raise ArgumentError, "Unsupported Google Drive request method: #{ method }"
end
request = klass.new(uri)
request['Authorization'] = "Bearer #{ access_token }"
Net::HTTP.start(uri.host,
uri.port,
use_ssl: true,
open_timeout: 10,
read_timeout: 60) do |http|
if block_given?
http.request(request) do |response|
return yield response
end
else
http.request(request)
end
end
end
def access_token
@access_token ||= fetch_access_token
end
def fetch_access_token
raise 'GOOGLE_DRIVE_SERVICE_ACCOUNT_EMAIL is required' if @service_account_email.blank?
raise 'Google Drive private key is required' if @private_key.blank?
payload = { iss: @service_account_email,
scope: DRIVE_SCOPE,
aud: TOKEN_ENDPOINT,
exp: 1.hour.from_now.to_i,
iat: Time.current.to_i }
payload[:sub] = @subject if @subject.present?
assertion = JWT.encode(payload, @private_key, 'RS256')
uri = URI(TOKEN_ENDPOINT)
request = Net::HTTP::Post.new(uri)
request.set_form_data(
grant_type: 'urn:ietf:params:oauth:grant-type:jwt-bearer',
assertion:)
response =
Net::HTTP.start(uri.host,
uri.port,
use_ssl: true,
open_timeout: 10,
read_timeout: 30) do |http|
http.request(request)
end
unless response.is_a?(Net::HTTPSuccess)
raise "Google OAuth error: #{ response.code } #{ response.body }"
end
JSON.parse(response.body).fetch('access_token')
end
def load_private_key private_key, private_key_path
raw = private_key.to_s
raw = raw.gsub('\n', "\n") if raw.present?
raw = File.read(private_key_path) if raw.blank? && private_key_path.present?
return nil if raw.blank?
OpenSSL::PKey::RSA.new(raw)
end
def native_file? mime_type
mime_type.to_s.start_with?(NATIVE_FILE_MIME_TYPE_PREFIX)
end
end
end
+57
ファイルの表示
@@ -0,0 +1,57 @@
require 'digest'
require 'json'
class MaterialFileSha256
def self.blob_metadata blob
metadata = blob.metadata
return metadata if metadata.is_a?(Hash)
return { } if metadata.blank?
JSON.parse(metadata)
rescue JSON::ParserError
{ }
end
def self.metadata_sha256 blob
blob_metadata(blob)['sha256'].presence
end
def self.assign_metadata_sha256! blob, sha256
return if sha256.blank?
metadata = blob_metadata(blob)
metadata['sha256'] = sha256
blob.metadata = metadata
blob.save! if blob.changed?
end
def self.from_blob blob, allow_download: false
sha256 = metadata_sha256(blob)
return sha256 if sha256.present?
return nil unless allow_download
begin
blob.open do |file|
sha256 = Digest::SHA256.file(file.path).hexdigest
assign_metadata_sha256!(blob, sha256)
sha256
end
rescue ActiveStorage::FileNotFoundError, ArgumentError => error
Rails.logger.warn(
"MaterialFileSha256.from_blob failed for blob_id=#{blob.id}: " \
"#{error.class}: #{error.message}",
)
nil
end
end
def self.from_upload upload
tempfile = upload&.tempfile
return nil unless tempfile
tempfile.rewind
Digest::SHA256.file(tempfile.path).hexdigest.tap do
tempfile.rewind
end
end
end
+7
ファイルの表示
@@ -0,0 +1,7 @@
class MaterialImportBlockMatcher
def self.match_for_sha256 sha256
return nil if sha256.blank?
MaterialImportBlock.find_by(match_kind: 'sha256', sha256:)
end
end
+56
ファイルの表示
@@ -0,0 +1,56 @@
class MaterialSyncExportPath
class << self
def build prefix: nil, relative_path: nil, filename: nil, source_file_id: nil
path = normalize_path(relative_path)
path = fallback_filename(filename, source_file_id) if path.blank?
normalized_prefix = normalize_path(prefix)
return path if normalized_prefix.blank?
return normalized_prefix if path.blank?
[normalized_prefix, path].join('/')
end
def normalize_path path
raw = path.to_s.delete("\0").tr('\\', '/').strip
raw = raw.sub(%r{\A[A-Za-z]:/+}, '')
raw = raw.sub(%r{\A/+}, '')
return nil if raw.blank?
segments = raw.split('/').filter_map do |segment|
normalized = normalize_segment(segment)
normalized.presence
end
return nil if segments.empty?
segments.join('/')
end
def uniquify path, source_file_id
normalized = normalize_path(path)
return normalized if normalized.blank? || source_file_id.blank?
ext = File.extname(normalized)
base = ext.present? ? normalized.delete_suffix(ext) : normalized
"#{ base }--#{ source_file_id }#{ ext }"
end
private
def fallback_filename filename, source_file_id
name = normalize_path(filename)
return name if name.present?
return nil if source_file_id.blank?
source_file_id.to_s
end
def normalize_segment segment
cleaned = segment.to_s.delete("\0").tr('\\/', '_').strip
return nil if cleaned.blank? || cleaned == '.'
return '__' if cleaned == '..'
cleaned
end
end
end
+259
ファイルの表示
@@ -0,0 +1,259 @@
require 'digest'
class MaterialSyncImporter
Result = Struct.new(:material, :action, :suppressed, :suppression, keyword_init: true)
def self.import! attributes
new(attributes).import!
end
def initialize attributes
@attributes = attributes
end
def import!
source = source_attributes
key = normalized_source_key(source)
raise ArgumentError, 'normalized_source_key is required for material sync' if key.blank?
suppression = suppression_for(source)
if suppression
return Result.new(material: nil,
action: :suppressed,
suppressed: true,
suppression:)
end
material =
Material
.unscoped
.find_or_initialize_by(normalized_source_key: key)
if unchanged?(material, source)
return Result.new(material:,
action: :unchanged,
suppressed: false,
suppression: nil)
end
event_type =
if material.new_record?
:create
elsif material.discarded?
:restore
else
:update
end
uploaded_blob = uploaded_blob!
if @import_block
return Result.new(material: nil,
action: :suppressed,
suppressed: true,
suppression: @import_block)
end
Material.transaction do
if material.persisted?
MaterialVersionRecorder.ensure_snapshot!(material,
created_by_user: @attributes[:updated_by_user])
end
material.assign_attributes(material_attributes_for(material).merge(source))
material.discarded_at = nil if material.respond_to?(:discarded_at=)
material.file.attach(uploaded_blob) if uploaded_blob
material.save!
upsert_export_item!(material)
MaterialVersionRecorder.record!(material:,
event_type:,
created_by_user: @attributes[:updated_by_user])
end
MaterialThumbnailGenerator.generate!(material)
Result.new(material:,
action: event_type == :create ? :imported : :updated,
suppressed: false,
suppression: nil)
rescue StandardError
uploaded_blob&.purge_later
raise
ensure
close_tempfile!
end
private
def source_attributes
{ source_kind: @attributes[:source_kind],
source_uri: @attributes[:source_uri],
source_path: @attributes[:source_path],
source_file_id: @attributes[:source_file_id] }
end
def normalized_source_key source
MaterialSyncSuppression.normalize_source_key(source_kind: source[:source_kind],
source_uri: source[:source_uri],
source_path: source[:source_path],
source_file_id: source[:source_file_id])
end
def material_attributes_for material
attrs =
@attributes
.except(:source_kind, :source_uri, :source_path, :source_file_id,
:file_blob, :file_tempfile, :file_downloader,
:filename, :content_type,
:file_sha256, :export_path, :profile, :tag, :url)
if material.new_record?
attrs[:tag] = @attributes[:tag] if @attributes.key?(:tag)
attrs[:url] = @attributes[:url] if @attributes.key?(:url)
return attrs
end
if @attributes.key?(:tag) && @attributes[:tag].present?
attrs[:tag] = @attributes[:tag]
end
if @attributes.key?(:url) && @attributes[:url].present?
attrs[:url] = @attributes[:url]
end
attrs
end
def suppression_for source
if source[:source_kind] == 'google_drive_file' && source[:source_path].present?
return MaterialSyncSuppressionMatcher.match_google_drive_candidate(
drive_file_id: source[:source_file_id],
relative_path: source[:source_path])
end
MaterialSyncSuppressionMatcher.match(**source)
end
def uploaded_blob!
return @attributes[:file_blob] if @attributes[:file_blob]
tempfile = upload_tempfile
return nil unless tempfile
tempfile.rewind
file_sha256 = @attributes[:file_sha256] || Digest::SHA256.file(tempfile.path).hexdigest
@attributes[:file_sha256] = file_sha256
@import_block = MaterialImportBlockMatcher.match_for_sha256(file_sha256)
if @import_block
tempfile.rewind
return nil
end
blob = ActiveStorage::Blob.create_and_upload!(
io: tempfile,
filename: @attributes[:filename],
content_type: @attributes[:content_type])
MaterialFileSha256.assign_metadata_sha256!(blob, file_sha256)
tempfile.rewind
blob
end
def upload_tempfile
return @upload_tempfile if defined?(@upload_tempfile)
@upload_tempfile = @attributes[:file_tempfile]
if @upload_tempfile.blank? && @attributes[:file_downloader]
@upload_tempfile = @attributes[:file_downloader].call
end
@upload_tempfile
end
def upsert_export_item! material
export_path = resolved_export_path(material)
return if export_path.blank?
item = material.material_export_items.find_or_initialize_by(profile: effective_profile)
item.export_path = export_path
item.enabled = true
item.created_by_user ||= @attributes[:created_by_user]
item.save!
end
def export_path_taken_by_other? material, profile, export_path
MaterialExportItem
.where(profile:, export_path:)
.where.not(material_id: material.id)
.exists?
end
def close_tempfile!
tempfile = upload_tempfile
return unless tempfile
tempfile.close! unless tempfile.closed?
rescue StandardError
nil
end
def unchanged? material, source
return false if material.new_record? || material.discarded?
return false unless same_source_attributes?(material, source)
return false unless same_export_path?(material)
return false unless same_material_attributes?(material)
same_file_snapshot?(material)
end
def same_source_attributes? material, source
material.source_kind == source[:source_kind] \
&& material.source_uri == source[:source_uri] \
&& material.source_path == source[:source_path] \
&& material.source_file_id == source[:source_file_id]
end
def same_export_path? material
expected = resolved_export_path(material)
current = material.material_export_items.find { |item| item.profile == effective_profile }
return current.blank? if expected.blank?
current&.enabled && current.export_path == expected
end
def same_material_attributes? material
same_tag_attribute?(material) && same_url_attribute?(material)
end
def same_file_snapshot? material
expected_sha256 = @attributes[:file_sha256].to_s.presence
if expected_sha256.present?
return false unless material.file.attached?
return MaterialFileSha256.metadata_sha256(material.file.blob) == expected_sha256
end
!material.file.attached?
end
def effective_profile
@attributes[:profile].presence || 'legacy_drive'
end
def resolved_export_path material
export_path = @attributes[:export_path].to_s.strip
return export_path if export_path.blank?
return export_path unless export_path_taken_by_other?(material, effective_profile, export_path)
MaterialSyncExportPath.uniquify(export_path, @attributes[:source_file_id])
end
def same_tag_attribute? material
return true unless @attributes.key?(:tag)
return true if @attributes[:tag].blank?
material.tag_id == @attributes[:tag].id
end
def same_url_attribute? material
return true unless @attributes.key?(:url)
return true if @attributes[:url].blank?
material.url == @attributes[:url]
end
end
+251
ファイルの表示
@@ -0,0 +1,251 @@
class MaterialSyncRunner
Result = Struct.new(:imported, :updated, :unchanged, :suppressed, :failed,
:errors, keyword_init: true)
def self.sync_enabled!
results = MaterialSyncSource.enabled.order(:id).map { |source| new(source).sync! }
Result.new(imported: results.sum(&:imported),
updated: results.sum(&:updated),
unchanged: results.sum(&:unchanged),
suppressed: results.sum(&:suppressed),
failed: results.sum(&:failed),
errors: results.flat_map(&:errors))
end
def initialize source
@source = source
end
def sync!
result = Result.new(imported: 0, updated: 0, unchanged: 0,
suppressed: 0, failed: 0, errors: [])
if @source.source_kind == 'google_drive_path'
sync_google_drive_path!(result)
else
candidates.each do |candidate|
next if candidate.blank?
sync_candidate!(candidate, result)
end
end
@source.update!(last_synced_at: Time.current)
log_result(result)
result
rescue NotImplementedError, StandardError => e
result.failed += 1
result.errors << { source_id: @source.id, error: e.message }
log_result(result)
result
end
private
def candidates
case @source.source_kind
when 'uri'
[uri_candidate]
when 'google_drive_file'
[google_drive_file_candidate]
when 'legacy_drive_path'
raise NotImplementedError, 'legacy_drive_path material sync is not implemented'
else
raise NotImplementedError, "Unsupported material sync source_kind: #{ @source.source_kind }"
end
end
def uri_candidate
{ source_kind: @source.source_kind,
source_uri: @source.source_uri,
source_path: @source.source_path,
source_file_id: @source.source_file_id,
url: @source.source_uri,
tag: nil,
created_by_user: @source.created_by_user,
updated_by_user: @source.updated_by_user || @source.created_by_user }
end
def sync_candidate! candidate, result
block = MaterialImportBlockMatcher.match_for_sha256(candidate[:file_sha256])
if block
result.suppressed += 1
Rails.logger.info(
material_sync_log(action: 'suppressed',
reason: block.reason,
normalized_source_key: candidate_normalized_source_key(candidate)))
return
end
suppression = suppression_for(candidate)
if suppression
result.suppressed += 1
Rails.logger.info(
material_sync_log(action: 'suppressed',
normalized_source_key: suppression.normalized_source_key))
return
end
import = MaterialSyncImporter.import!(candidate)
result.public_send("#{ import.action }=", result.public_send(import.action) + 1)
rescue StandardError => e
result.failed += 1
result.errors << { source_id: @source.id,
normalized_source_key: candidate_normalized_source_key(candidate),
error: e.message }
Rails.logger.warn(
material_sync_log(action: 'failed',
error: e.message,
normalized_source_key: candidate_normalized_source_key(candidate)))
ensure
close_candidate_tempfile(candidate)
end
def google_drive_path_candidates
folder_id = google_drive_folder_id
Enumerator.new do |entries|
drive_client.each_material_file_under_folder(folder_id).each do |entry|
entries << build_google_drive_candidate(entry)
end
end
end
def sync_google_drive_path! result
folder_id = google_drive_folder_id
scanned_count = 0
drive_client.each_material_file_under_folder(folder_id) do |entry|
scanned_count += 1
sync_candidate!(build_google_drive_candidate(entry), result)
log_google_drive_progress(folder_id, scanned_count, result) if progress_log_scan_count?(scanned_count)
end
log_google_drive_progress(folder_id, scanned_count, result, summary: true)
end
def google_drive_file_candidate
entry = drive_client.fetch_material_file(google_drive_file_id)
return nil unless entry
build_google_drive_candidate(entry, single_file: true)
end
def build_google_drive_candidate entry, single_file: false
relative_path =
if single_file
nil
else
entry[:relative_path]
end
export_path =
MaterialSyncExportPath.build(prefix: @source.export_path_prefix,
relative_path:,
filename: entry[:name],
source_file_id: entry[:id])
{ source_kind: 'google_drive_file',
source_uri: entry[:web_view_link] || google_drive_file_url(entry[:id]),
source_path: relative_path.presence || MaterialSyncExportPath.normalize_path(entry[:name]),
source_file_id: entry[:id],
filename: entry[:name],
content_type: entry[:mime_type],
file_downloader: lambda {
drive_client.download_to_tempfile(entry[:id], filename: entry[:name])
},
file_sha256: entry[:sha256_checksum],
export_path:,
profile: @source.profile,
tag: nil,
url: nil,
created_by_user: @source.created_by_user,
updated_by_user: @source.updated_by_user || @source.created_by_user }
end
def candidate_normalized_source_key candidate
MaterialSyncSuppression.normalize_source_key(source_kind: candidate[:source_kind],
source_uri: candidate[:source_uri],
source_path: candidate[:source_path],
source_file_id: candidate[:source_file_id])
end
def suppression_for candidate
if candidate[:source_kind] == 'google_drive_file' && candidate[:source_path].present?
return MaterialSyncSuppressionMatcher.match_google_drive_candidate(
drive_file_id: candidate[:source_file_id],
relative_path: candidate[:source_path])
end
MaterialSyncSuppressionMatcher.match(source_kind: candidate[:source_kind],
source_uri: candidate[:source_uri],
source_path: candidate[:source_path],
source_file_id: candidate[:source_file_id])
end
def drive_client
@drive_client ||= GoogleDrive::ApiClient.new
end
def google_drive_folder_id
google_drive_target_id.tap do |id|
if id.blank?
raise ArgumentError, 'google_drive_path source_file_id or source_uri is required'
end
end
end
def google_drive_file_id
google_drive_target_id.tap do |id|
if id.blank?
raise ArgumentError, 'google_drive_file source_file_id or source_uri is required'
end
end
end
def google_drive_target_id
@source.source_file_id.presence || drive_client.extract_file_id(@source.source_uri)
end
def google_drive_file_url file_id
"https://drive.google.com/file/d/#{ file_id }/view"
end
def close_candidate_tempfile candidate
tempfile = candidate[:file_tempfile]
return unless tempfile
tempfile.close! unless tempfile.closed?
rescue StandardError
nil
end
def log_result result
Rails.logger.info(material_sync_log(imported: result.imported,
updated: result.updated,
unchanged: result.unchanged,
suppressed: result.suppressed,
failed: result.failed))
end
def progress_log_scan_count? scanned_count
scanned_count == 1 || (scanned_count % 50).zero?
end
def log_google_drive_progress folder_id, scanned_count, result, summary: false
Rails.logger.info(
material_sync_log(folder_id:,
scanned_count:,
imported: result.imported,
updated: result.updated,
unchanged: result.unchanged,
suppressed: result.suppressed,
failed: result.failed,
progress: summary ? 'summary' : 'scan'))
end
def material_sync_log fields
{ material_sync_source_id: @source.id,
material_sync_source_name: @source.name }.merge(fields).to_json
end
end
+68
ファイルの表示
@@ -0,0 +1,68 @@
class MaterialSyncSuppressionMatcher
def self.match source_kind:,
source_uri: nil,
drive_path: nil,
drive_file_id: nil,
source_path: nil,
source_file_id: nil
key = MaterialSyncSuppression.normalize_source_key(source_kind:,
source_uri:,
drive_path:,
drive_file_id:,
source_path:,
source_file_id:)
return nil if key.blank?
MaterialSyncSuppression.find_by(normalized_source_key: key)
end
def self.match_google_drive_candidate drive_file_id:, relative_path:
normalized_path = validate_relative_path!(relative_path)
exact = [
normalize_key('google_drive_file', drive_file_id),
normalize_key('google_drive_path', normalized_path),
normalize_key('legacy_drive_path', normalized_path),
].compact
suppression =
MaterialSyncSuppression.find_by(normalized_source_key: exact)
return suppression if suppression
MaterialSyncSuppression
.where(source_kind: ['google_drive_path_prefix', 'legacy_drive_path_prefix'])
.where('drive_path = :path OR :path LIKE CONCAT(drive_path, "/%")', path: normalized_path)
.order(:id)
.first
end
def self.suppressed?(...)
match(...).present?
end
def self.validate_relative_path! relative_path
raw = relative_path.to_s.delete("\0").tr('\\', '/').strip
raise ArgumentError, 'relative_path is required' if raw.blank?
raise ArgumentError, 'relative_path must be relative' if raw.start_with?('/')
raise ArgumentError, 'relative_path must be relative' if raw.match?(/\A[A-Za-z]:\//)
if raw.start_with?('My Drive/', 'マイドライブ/')
raise ArgumentError, 'relative_path must be relative to source folder'
end
raise ArgumentError, 'relative_path must not contain //' if raw.include?('//')
raise ArgumentError, 'relative_path must not end with /' if raw.end_with?('/')
parts = raw.split('/')
if parts.any? { |part| part.in?(['.', '..']) }
raise ArgumentError, 'relative_path must not contain dot segments'
end
raw
end
def self.normalize_key source_kind, value
return nil if value.blank?
MaterialSyncSuppression.normalize_source_key(source_kind:, source_path: value,
source_file_id: value)
end
end
+51
ファイルの表示
@@ -0,0 +1,51 @@
class MaterialSyncSuppressionRegistrar
def self.create! attributes, created_by_user:
new(attributes, created_by_user:).create!
end
def initialize attributes, created_by_user:
@attributes = attributes
@created_by_user = created_by_user
end
def create!
suppression = nil
MaterialSyncSuppression.transaction do
suppression = MaterialSyncSuppression.create!(
@attributes.merge(created_by_user: @created_by_user))
discard_existing_materials!(suppression)
end
suppression
end
private
def discard_existing_materials! suppression
matching_materials(suppression).find_each do |material|
MaterialVersionRecorder.ensure_snapshot!(material, created_by_user: @created_by_user)
material.discard!
MaterialVersionRecorder.record!(material:,
event_type: :discard,
created_by_user: @created_by_user)
end
end
def matching_materials suppression
materials = Material.unscoped.kept
case suppression.source_kind
when 'google_drive_path', 'legacy_drive_path'
materials.where(source_path: suppression.drive_path)
when 'google_drive_path_prefix', 'legacy_drive_path_prefix'
path = suppression.drive_path.to_s
materials.where(
'source_path = :path OR source_path LIKE :prefix',
path:,
prefix: "#{ path }/%")
else
materials.where(normalized_source_key: suppression.normalized_source_key)
end
end
end
+126
ファイルの表示
@@ -0,0 +1,126 @@
# frozen_string_literal: true
require 'mini_magick'
require 'open3'
require 'tempfile'
class MaterialThumbnailGenerator
SIZE = '180x180'
class << self
def generate! material
new(material).generate!
end
end
def initialize material
@material = material
@ffmpeg_stderr = []
end
def generate!
return log_result(:no_file) unless @material.file.attached?
return log_result(:unsupported_content_type) unless image? || video?
@material.file.blob.open do |file|
thumbnail = image? ? image_thumbnail(file.path) : video_thumbnail(file.path)
return log_result(:generation_failed) unless thumbnail
return attach_thumbnail(thumbnail)
end
rescue ActiveStorage::FileNotFoundError => e
log_result(:file_not_found, error: e)
rescue MiniMagick::Error => e
log_result(:mini_magick_error, error: e)
rescue ArgumentError, StandardError => e
log_result(:generation_failed, error: e)
end
private
def image? = content_type.start_with?('image/')
def video? = content_type.start_with?('video/')
def content_type
return nil unless @material.file.attached?
@material.file.blob.content_type.to_s
end
def image_thumbnail path
image = MiniMagick::Image.open(path)
image.resize(SIZE)
image.format('jpg')
image
end
def video_thumbnail path
[1, 0].each do |seconds|
tempfile = Tempfile.new(['material-thumbnail', '.jpg'])
tempfile.close
ok = extract_video_frame(path, tempfile.path, seconds)
next unless ok && File.size?(tempfile.path)
return image_thumbnail(tempfile.path)
ensure
tempfile&.unlink
end
nil
end
def extract_video_frame input_path, output_path, seconds
_stdout, stderr, status =
Open3.capture3('ffmpeg',
'-y',
'-ss', seconds.to_s,
'-i', input_path,
'-frames:v', '1',
'-f', 'image2',
output_path)
@ffmpeg_stderr << stderr if stderr.present?
status.success?
rescue Errno::ENOENT => e
@ffmpeg_stderr << "ffmpeg unavailable: #{ e.message }"
false
end
def attach_thumbnail image
blob = nil
File.open(image.path) do |io|
blob = ActiveStorage::Blob.create_and_upload!(
io:,
filename: 'material-thumbnail.jpg',
content_type: 'image/jpeg')
end
@material.thumbnail.attach(blob)
log_result(:attached)
rescue StandardError => e
blob&.purge_later
log_result(:attach_failed, error: e)
end
def log_result result, error: nil
log_payload = { material_id: @material.id,
file_blob_id: file_blob_id,
content_type:,
result:,
error_class: error&.class&.name,
error_message: error&.message,
ffmpeg_stderr: @ffmpeg_stderr.join("\n").presence }
if [:attached, :no_file, :unsupported_content_type].include?(result)
Rails.logger.info("Material thumbnail generation: #{ log_payload.to_json }")
else
Rails.logger.warn("Material thumbnail generation: #{ log_payload.to_json }")
end
result
end
def file_blob_id
return nil unless @material.file.attached?
@material.file.blob.id
end
end
+73
ファイルの表示
@@ -0,0 +1,73 @@
class MaterialVersionRecorder < VersionRecorder
EVENT_TYPES = ['create', 'update', 'discard', 'restore'].freeze
def self.record! material:, event_type:, created_by_user:, file_snapshot: nil
new(material:, event_type:, created_by_user:, file_snapshot:).record!
end
def initialize material:, event_type:, created_by_user:, file_snapshot: nil
@file_snapshot = file_snapshot
super(record: material, event_type:, created_by_user:)
end
def self.ensure_snapshot! material, created_by_user:
return if material.material_versions.exists?
record!(material:, event_type: :create,
created_by_user: material.created_by_user || created_by_user)
end
private
def version_class = MaterialVersion
def version_association = :material_versions
def record_key = :material
def snapshot_attributes
blob = @record.file.attached? ? @record.file.blob : nil
file_snapshot = build_file_snapshot(blob)
{ url: @record.url,
parent: @record.parent,
tag: @record.tag,
tag_name: @record.tag&.name,
tag_category: @record.tag&.category,
source_kind: @record.source_kind,
source_uri: @record.source_uri,
source_path: @record.source_path,
source_file_id: @record.source_file_id,
normalized_source_key: @record.normalized_source_key,
export_paths_json: @record.snapshot_export_paths,
discarded_at: @record.discarded_at,
file_blob_id: file_snapshot[:file_blob_id],
file_filename: file_snapshot[:file_filename],
file_content_type: file_snapshot[:file_content_type],
file_byte_size: file_snapshot[:file_byte_size],
file_checksum: file_snapshot[:file_checksum],
file_sha256: file_snapshot[:file_sha256] }
end
def build_file_snapshot blob
return @file_snapshot if @file_snapshot
return empty_file_snapshot unless blob
{ file_blob_id: blob.id,
file_filename: blob.filename.to_s,
file_content_type: blob.content_type,
file_byte_size: blob.byte_size,
file_checksum: blob.checksum,
file_sha256: MaterialFileSha256.metadata_sha256(blob) }
end
def empty_file_snapshot
{ file_blob_id: nil,
file_filename: nil,
file_content_type: nil,
file_byte_size: nil,
file_checksum: nil,
file_sha256: nil }
end
def event_types = self.class::EVENT_TYPES
end
+148
ファイルの表示
@@ -0,0 +1,148 @@
require 'stringio'
require 'zlib'
# Initial implementation keeps every file payload and the final ZIP in memory.
# Keep this service boundary stable so job/cached export paths can replace it later.
class MaterialZipExporter
Entry = Struct.new(:path, :data, :mtime, keyword_init: true)
MissingFile = Struct.new(:material_id, :export_path, :blob_id, :filename, keyword_init: true)
class EmptyExportError < StandardError; end
class DuplicatePathError < StandardError; end
class MissingFileError < StandardError
attr_reader :missing_files
def initialize missing_files
@missing_files = missing_files
super("Missing files: #{missing_files.map(&:export_path).join(', ')}")
end
end
def initialize profile: 'legacy_drive', tag_id: nil
@profile = profile.presence || 'legacy_drive'
@tag_id = tag_id.presence
end
def export
entries = build_entries
raise EmptyExportError if entries.empty?
ZipWriter.write(entries)
end
private
def build_entries
rows = MaterialExportItem
.enabled
.includes(material: { file_attachment: :blob })
.joins(:material)
.merge(Material.kept)
.where(profile: @profile)
.order(:export_path)
rows = rows.where(materials: { tag_id: @tag_id }) if @tag_id
missing_files = []
entries = rows.filter_map do |item|
material = item.material
next unless material.file.attached?
data = download_blob(item, missing_files)
next unless data
Entry.new(path: item.export_path,
data:,
mtime: material.updated_at || Time.current)
end
raise MissingFileError.new(missing_files) if missing_files.any?
paths = entries.map(&:path)
duplicated = paths.find { |path| paths.count(path) > 1 }
raise DuplicatePathError, duplicated if duplicated
entries
end
def download_blob item, missing_files
blob = item.material.file.blob
blob.download
rescue ActiveStorage::FileNotFoundError
missing_files << MissingFile.new(
material_id: item.material_id,
export_path: item.export_path,
blob_id: blob.id,
filename: blob.filename.to_s,
)
nil
end
class ZipWriter
VERSION_NEEDED = 20
GP_FLAG = 0x0800
COMPRESSION_STORE = 0
def self.write entries
new(entries).write
end
def initialize entries
@entries = entries
@central_directory = []
end
def write
io = StringIO.new(''.b)
@entries.each do |entry|
write_entry(io, entry)
end
central_start = io.pos
@central_directory.each { |header| io.write(header) }
central_size = io.pos - central_start
io.write([0x06054b50, 0, 0, @entries.size, @entries.size,
central_size, central_start, 0].pack('VvvvvVVv'))
io.string
end
private
def write_entry io, entry
path = entry.path.b
data = entry.data.b
crc32 = Zlib.crc32(data)
dos_time, dos_date = dos_timestamp(entry.mtime)
offset = io.pos
local_header = [0x04034b50, VERSION_NEEDED, GP_FLAG, COMPRESSION_STORE,
dos_time, dos_date, crc32, data.bytesize, data.bytesize,
path.bytesize, 0].pack('VvvvvvVVVvv')
io.write(local_header)
io.write(path)
io.write(data)
@central_directory << central_header(path:, crc32:, size: data.bytesize,
dos_time:, dos_date:, offset:)
end
def central_header path:, crc32:, size:, dos_time:, dos_date:, offset:
[0x02014b50, VERSION_NEEDED, VERSION_NEEDED, GP_FLAG, COMPRESSION_STORE,
dos_time, dos_date, crc32, size, size, path.bytesize, 0, 0, 0, 0, 0,
offset].pack('VvvvvvvVVVvvvvvVV') + path
end
def dos_timestamp time
local = time.to_time
dos_time = (local.hour << 11) | (local.min << 5) | (local.sec / 2)
dos_date = ((local.year - 1980) << 9) | (local.month << 5) | local.day
[dos_time, dos_date]
end
end
end
+3 -2
ファイルの表示
@@ -1,6 +1,6 @@
module Similarity
class Calc
def self.call model, tgt
def self.call model, tgt, scope: nil
similarity_model = "#{ model.name }Similarity".constantize
# 最大保存件数
@@ -8,7 +8,8 @@ module Similarity
similarity_model.delete_all
posts = model.includes(tgt).select(:id).to_a
scope ||= model.all
posts = scope.includes(tgt).select(:id).to_a
tag_ids = { }
tag_cnts = { }
+1
ファイルの表示
@@ -16,6 +16,7 @@ class TagVersionRecorder < VersionRecorder
def snapshot_attributes
{ name: @record.name,
category: @record.category,
deprecated_at: @record.deprecated_at,
aliases: @record.snapshot_aliases.join(' '),
parent_tag_ids: @record.snapshot_parent_tag_ids.join(' ') }
end
+2 -1
ファイルの表示
@@ -73,7 +73,7 @@ class VersionRecorder
end
def validate_event_type!
return if EVENT_TYPES.include?(@event_type)
return if event_types.include?(@event_type)
raise ArgumentError, "Invalid event_type: #{ @event_type }"
end
@@ -84,4 +84,5 @@ class VersionRecorder
def snapshot_attributes = raise NotImplementedError
def record_class = @record.class
def event_types = self.class::EVENT_TYPES
end
+3
ファイルの表示
@@ -113,5 +113,8 @@ Rails.application.routes.draw do
resources :skip_events, controller: :theatre_skip_events, only: [:index]
end
get 'materials/download.zip', to: 'materials#download'
get 'materials/versions', to: 'material_versions#index'
resources :material_sync_suppressions, path: 'materials/suppressions', only: [:index, :create]
resources :materials, only: [:index, :show, :create, :update, :destroy]
end
+20
ファイルの表示
@@ -0,0 +1,20 @@
class AddDeprecatedAtToTags < ActiveRecord::Migration[8.0]
def up
add_column :tags, :deprecated_at, :datetime, after: :category
add_column :tag_versions, :deprecated_at, :datetime, after: :parent_tag_ids
add_index :tags, :deprecated_at
add_check_constraint :tags, "deprecated_at IS NULL OR category <> 'nico'",
name: 'chk_tags_deprecated_at_not_nico'
end
def down
remove_check_constraint :tags, name: 'chk_tags_deprecated_at_not_nico'
remove_index :tags, :deprecated_at
remove_column :tag_versions, :deprecated_at, :datetime
remove_column :tags, :deprecated_at
end
end
+88
ファイルの表示
@@ -0,0 +1,88 @@
class EnhanceMaterialManagement < ActiveRecord::Migration[8.0]
def up
change_table :materials, bulk: true do |t|
t.integer :version_no, null: false, default: 1
end
change_table :material_versions, bulk: true do |t|
t.string :event_type
t.string :tag_name
t.string :tag_category
t.json :export_paths_json
t.bigint :file_blob_id
t.string :file_filename
t.string :file_content_type
t.bigint :file_byte_size
t.string :file_checksum
t.string :file_sha256
end
execute <<~SQL.squish
UPDATE material_versions
SET event_type = CASE
WHEN version_no = 1 THEN 'create'
ELSE 'update'
END
WHERE event_type IS NULL
SQL
change_column_null :material_versions, :event_type, false
add_index :material_versions, :file_blob_id
add_check_constraint :material_versions,
"event_type IN ('create', 'update', 'discard', 'restore')",
name: 'material_versions_event_type_valid'
create_table :material_export_items do |t|
t.references :material, null: false, foreign_key: true
t.string :profile, null: false, default: 'legacy_drive'
t.string :export_path, null: false
t.boolean :enabled, null: false, default: true
t.references :created_by_user, foreign_key: { to_table: :users }
t.timestamps
t.index [:profile, :export_path], unique: true
t.index [:material_id, :profile], unique: true
end
create_table :material_import_blocks do |t|
t.string :match_kind, null: false
t.string :sha256
t.string :external_path_pattern
t.string :reason, null: false
t.text :note
t.references :created_by_user, foreign_key: { to_table: :users }
t.timestamps
end
execute <<~SQL.squish
UPDATE materials
SET version_no = COALESCE(
(SELECT MAX(material_versions.version_no)
FROM material_versions
WHERE material_versions.material_id = materials.id),
1)
SQL
end
def down
drop_table :material_import_blocks
drop_table :material_export_items
remove_check_constraint :material_versions, name: 'material_versions_event_type_valid'
remove_index :material_versions, :file_blob_id
remove_column :material_versions, :event_type
remove_column :material_versions, :tag_name
remove_column :material_versions, :tag_category
remove_column :material_versions, :export_paths_json
remove_column :material_versions, :file_blob_id
remove_column :material_versions, :file_filename
remove_column :material_versions, :file_content_type
remove_column :material_versions, :file_byte_size
remove_column :material_versions, :file_checksum
remove_column :material_versions, :file_sha256
remove_column :materials, :version_no
end
end
+118
ファイルの表示
@@ -0,0 +1,118 @@
class ReconcileMaterialSyncSuppressions < ActiveRecord::Migration[8.0]
MATERIAL_VERSION_EVENT_CONSTRAINT = 'material_versions_event_type_valid'
def up
ensure_material_sync_suppressions!
ensure_material_source_columns!
ensure_material_active_source_key!
remove_material_file_suppression!
remove_material_version_file_suppression!
reconcile_material_version_event_constraint!
end
def down
raise ActiveRecord::IrreversibleMigration
end
private
def ensure_material_sync_suppressions!
return if table_exists?(:material_sync_suppressions)
create_table :material_sync_suppressions do |t|
t.string :source_kind, null: false
t.string :source_uri
t.string :drive_path
t.string :drive_file_id
t.string :normalized_source_key, null: false
t.string :reason, null: false
t.references :created_by_user, foreign_key: { to_table: :users }
t.timestamps
t.index :normalized_source_key, unique: true
t.index :source_kind
t.index :drive_file_id
end
end
def ensure_material_source_columns!
add_column :materials, :source_kind, :string unless column_exists?(:materials, :source_kind)
add_column :materials, :source_uri, :string unless column_exists?(:materials, :source_uri)
add_column :materials, :source_path, :string unless column_exists?(:materials, :source_path)
unless column_exists?(:materials, :source_file_id)
add_column :materials, :source_file_id, :string
end
unless column_exists?(:materials, :normalized_source_key)
add_column :materials, :normalized_source_key, :string
end
if index_exists?(:materials, :normalized_source_key)
remove_index :materials, column: :normalized_source_key
end
end
def ensure_material_active_source_key!
unless column_exists?(:materials, :active_normalized_source_key)
change_table :materials do |t|
t.virtual :active_normalized_source_key,
type: :string,
as: 'IF(discarded_at IS NULL, normalized_source_key, NULL)',
stored: false
end
end
return if index_exists?(:materials,
:active_normalized_source_key,
name: 'index_materials_on_active_normalized_source_key')
add_index :materials,
:active_normalized_source_key,
unique: true,
name: 'index_materials_on_active_normalized_source_key'
end
def remove_material_file_suppression!
if foreign_key_exists?(:materials, :users, column: :file_suppressed_by_user_id)
remove_foreign_key :materials, column: :file_suppressed_by_user_id
end
if index_exists?(:materials,
:file_suppressed_by_user_id,
name: 'index_materials_on_file_suppressed_by_user_id')
remove_index :materials, name: 'index_materials_on_file_suppressed_by_user_id'
end
remove_column :materials, :file_suppressed_at if column_exists?(
:materials,
:file_suppressed_at)
remove_column :materials, :file_suppressed_by_user_id if column_exists?(
:materials,
:file_suppressed_by_user_id)
remove_column :materials, :file_suppression_reason if column_exists?(
:materials,
:file_suppression_reason)
end
def remove_material_version_file_suppression!
remove_column :material_versions, :file_suppressed_at if column_exists?(
:material_versions,
:file_suppressed_at)
remove_column :material_versions, :file_suppression_reason if column_exists?(
:material_versions,
:file_suppression_reason)
end
def reconcile_material_version_event_constraint!
constraint = check_constraints(:material_versions).find do |candidate|
candidate.name == MATERIAL_VERSION_EVENT_CONSTRAINT
end
return if constraint && !constraint.expression.include?('suppress')
if constraint
remove_check_constraint :material_versions, name: MATERIAL_VERSION_EVENT_CONSTRAINT
end
add_check_constraint :material_versions,
"event_type IN ('create', 'update', 'discard', 'restore')",
name: MATERIAL_VERSION_EVENT_CONSTRAINT
end
end
+22
ファイルの表示
@@ -0,0 +1,22 @@
class CreateMaterialSyncSources < ActiveRecord::Migration[8.0]
def change
create_table :material_sync_sources do |t|
t.string :name, null: false
t.string :source_kind, null: false
t.string :source_uri
t.string :source_path
t.string :source_file_id
t.string :profile, null: false, default: 'legacy_drive'
t.boolean :enabled, null: false, default: true
t.string :default_tag_name
t.string :export_path_prefix
t.datetime :last_synced_at
t.references :created_by_user, foreign_key: { to_table: :users }
t.references :updated_by_user, foreign_key: { to_table: :users }
t.timestamps
t.index :enabled
t.index :source_kind
end
end
end
+19
ファイルの表示
@@ -0,0 +1,19 @@
class AddSourceSnapshotToMaterialVersions < ActiveRecord::Migration[8.0]
def change
unless column_exists?(:material_versions, :source_kind)
add_column :material_versions, :source_kind, :string
end
unless column_exists?(:material_versions, :source_uri)
add_column :material_versions, :source_uri, :string
end
unless column_exists?(:material_versions, :source_path)
add_column :material_versions, :source_path, :string
end
unless column_exists?(:material_versions, :source_file_id)
add_column :material_versions, :source_file_id, :string
end
unless column_exists?(:material_versions, :normalized_source_key)
add_column :material_versions, :normalized_source_key, :string
end
end
end
+171
ファイルの表示
@@ -0,0 +1,171 @@
require 'json'
class BackfillInitialMaterialVersions < ActiveRecord::Migration[8.0]
class MigrationMaterial < ActiveRecord::Base
self.table_name = 'materials'
end
class MigrationMaterialExportItem < ActiveRecord::Base
self.table_name = 'material_export_items'
end
class MigrationMaterialVersion < ActiveRecord::Base
self.table_name = 'material_versions'
end
class MigrationStorageAttachment < ActiveRecord::Base
self.table_name = 'active_storage_attachments'
end
class MigrationStorageBlob < ActiveRecord::Base
self.table_name = 'active_storage_blobs'
end
class MigrationTag < ActiveRecord::Base
self.table_name = 'tags'
end
class MigrationTagName < ActiveRecord::Base
self.table_name = 'tag_names'
end
def up
say_with_time 'Backfilling initial material versions' do
materials_without_versions.find_in_batches(batch_size: 500) do |materials|
material_ids = materials.map(&:id)
tags_by_id = load_tags(materials.map(&:tag_id).compact.uniq)
export_paths_by_material_id = load_export_paths(material_ids)
file_snapshots_by_material_id = load_file_snapshots(material_ids)
rows = materials.map do |material|
tag = tags_by_id[material.tag_id]
file_snapshot = file_snapshots_by_material_id[material.id] || empty_file_snapshot
{ material_id: material.id,
version_no: 1,
event_type: 'create',
url: material.url,
parent_id: material.parent_id,
tag_id: material.tag_id,
tag_name: tag&.fetch(:name, nil),
tag_category: tag&.fetch(:category, nil),
source_kind: material.source_kind,
source_uri: material.source_uri,
source_path: material.source_path,
source_file_id: material.source_file_id,
normalized_source_key: material.normalized_source_key,
export_paths_json: export_paths_by_material_id[material.id] || { },
discarded_at: material.discarded_at,
file_blob_id: file_snapshot[:file_blob_id],
file_filename: file_snapshot[:file_filename],
file_content_type: file_snapshot[:file_content_type],
file_byte_size: file_snapshot[:file_byte_size],
file_checksum: file_snapshot[:file_checksum],
file_sha256: file_snapshot[:file_sha256],
created_by_user_id: material.created_by_user_id,
updated_by_user_id: material.updated_by_user_id,
created_at: material.created_at,
updated_at: material.created_at }
end
MigrationMaterialVersion.insert_all!(rows) if rows.present?
end
end
execute <<~SQL.squish
UPDATE materials
SET version_no = COALESCE(
(
SELECT MAX(material_versions.version_no)
FROM material_versions
WHERE material_versions.material_id = materials.id
),
1
)
SQL
end
def down
raise ActiveRecord::IrreversibleMigration
end
private
def materials_without_versions
MigrationMaterial.where.not(id: MigrationMaterialVersion.select(:material_id))
end
def load_tags tag_ids
return { } if tag_ids.empty?
tag_name_ids_by_tag_id = MigrationTag.where(id: tag_ids).pluck(:id, :tag_name_id).to_h
tag_names_by_id =
MigrationTagName
.where(id: tag_name_ids_by_tag_id.values.uniq)
.pluck(:id, :name)
.to_h
categories_by_tag_id = MigrationTag.where(id: tag_ids).pluck(:id, :category).to_h
tag_ids.each_with_object({ }) do |tag_id, hash|
hash[tag_id] = { name: tag_names_by_id[tag_name_ids_by_tag_id[tag_id]],
category: categories_by_tag_id[tag_id] }
end
end
def load_export_paths material_ids
MigrationMaterialExportItem
.where(material_id: material_ids)
.order(:material_id, :profile)
.pluck(:material_id, :profile, :export_path)
.each_with_object({ }) do |(material_id, profile, export_path), hash|
hash[material_id] ||= { }
hash[material_id][profile] = export_path
end
end
def load_file_snapshots material_ids
attachments_by_material_id =
MigrationStorageAttachment
.where(record_type: 'Material', name: 'file', record_id: material_ids)
.pluck(:record_id, :blob_id)
.to_h
blobs_by_id =
MigrationStorageBlob
.where(id: attachments_by_material_id.values.uniq)
.index_by(&:id)
attachments_by_material_id.each_with_object({ }) do |(material_id, blob_id), hash|
blob = blobs_by_id[blob_id]
next unless blob
hash[material_id] = {
file_blob_id: blob.id,
file_filename: blob.filename.to_s,
file_content_type: blob.content_type,
file_byte_size: blob.byte_size,
file_checksum: blob.checksum,
file_sha256: blob_metadata(blob)['sha256']
}
end
end
def empty_file_snapshot
{ file_blob_id: nil,
file_filename: nil,
file_content_type: nil,
file_byte_size: nil,
file_checksum: nil,
file_sha256: nil }
end
def blob_metadata blob
metadata = blob.metadata
return metadata if metadata.is_a?(Hash)
return { } if metadata.blank?
JSON.parse(metadata)
rescue JSON::ParserError
{ }
end
end
生成ファイル
+99 -1
ファイルの表示
@@ -10,7 +10,7 @@
#
# It's strongly recommended that you check this file into your version control system.
ActiveRecord::Schema[8.0].define(version: 2026_06_12_000000) do
ActiveRecord::Schema[8.0].define(version: 2026_06_26_010000) do
create_table "active_storage_attachments", charset: "utf8mb4", collation: "utf8mb4_0900_ai_ci", force: :cascade do |t|
t.string "name", null: false
t.string "record_type", null: false
@@ -130,6 +130,69 @@ ActiveRecord::Schema[8.0].define(version: 2026_06_12_000000) do
t.index ["ip_address"], name: "index_ip_addresses_on_ip_address", unique: true
end
create_table "material_export_items", charset: "utf8mb4", collation: "utf8mb4_0900_ai_ci", force: :cascade do |t|
t.bigint "material_id", null: false
t.string "profile", default: "legacy_drive", null: false
t.string "export_path", null: false
t.boolean "enabled", default: true, null: false
t.bigint "created_by_user_id"
t.datetime "created_at", null: false
t.datetime "updated_at", null: false
t.index ["created_by_user_id"], name: "index_material_export_items_on_created_by_user_id"
t.index ["material_id", "profile"], name: "index_material_export_items_on_material_id_and_profile", unique: true
t.index ["material_id"], name: "index_material_export_items_on_material_id"
t.index ["profile", "export_path"], name: "index_material_export_items_on_profile_and_export_path", unique: true
end
create_table "material_import_blocks", charset: "utf8mb4", collation: "utf8mb4_0900_ai_ci", force: :cascade do |t|
t.string "match_kind", null: false
t.string "sha256"
t.string "external_path_pattern"
t.string "reason", null: false
t.text "note"
t.bigint "created_by_user_id"
t.datetime "created_at", null: false
t.datetime "updated_at", null: false
t.index ["created_by_user_id"], name: "index_material_import_blocks_on_created_by_user_id"
end
create_table "material_sync_sources", charset: "utf8mb4", collation: "utf8mb4_0900_ai_ci", force: :cascade do |t|
t.string "name", null: false
t.string "source_kind", null: false
t.string "source_uri"
t.string "source_path"
t.string "source_file_id"
t.string "profile", default: "legacy_drive", null: false
t.boolean "enabled", default: true, null: false
t.string "default_tag_name"
t.string "export_path_prefix"
t.datetime "last_synced_at"
t.bigint "created_by_user_id"
t.bigint "updated_by_user_id"
t.datetime "created_at", null: false
t.datetime "updated_at", null: false
t.index ["created_by_user_id"], name: "index_material_sync_sources_on_created_by_user_id"
t.index ["enabled"], name: "index_material_sync_sources_on_enabled"
t.index ["source_kind"], name: "index_material_sync_sources_on_source_kind"
t.index ["updated_by_user_id"], name: "index_material_sync_sources_on_updated_by_user_id"
end
create_table "material_sync_suppressions", charset: "utf8mb4", collation: "utf8mb4_0900_ai_ci", force: :cascade do |t|
t.string "source_kind", null: false
t.string "source_uri"
t.string "drive_path"
t.string "drive_file_id"
t.string "normalized_source_key", null: false
t.string "reason", null: false
t.bigint "created_by_user_id"
t.datetime "created_at", null: false
t.datetime "updated_at", null: false
t.index ["created_by_user_id"], name: "index_material_sync_suppressions_on_created_by_user_id"
t.index ["drive_file_id"], name: "index_material_sync_suppressions_on_drive_file_id"
t.index ["normalized_source_key"], name: "index_material_sync_suppressions_on_normalized_source_key", unique: true
t.index ["source_kind"], name: "index_material_sync_suppressions_on_source_kind"
end
create_table "material_versions", charset: "utf8mb4", collation: "utf8mb4_0900_ai_ci", force: :cascade do |t|
t.bigint "material_id", null: false
t.integer "version_no", null: false
@@ -141,14 +204,31 @@ ActiveRecord::Schema[8.0].define(version: 2026_06_12_000000) do
t.datetime "created_at", null: false
t.datetime "updated_at", null: false
t.datetime "discarded_at"
t.string "event_type", null: false
t.string "tag_name"
t.string "tag_category"
t.json "export_paths_json"
t.bigint "file_blob_id"
t.string "file_filename"
t.string "file_content_type"
t.bigint "file_byte_size"
t.string "file_checksum"
t.string "file_sha256"
t.string "source_kind"
t.string "source_uri"
t.string "source_path"
t.string "source_file_id"
t.string "normalized_source_key"
t.index ["created_by_user_id"], name: "index_material_versions_on_created_by_user_id"
t.index ["discarded_at"], name: "index_material_versions_on_discarded_at"
t.index ["file_blob_id"], name: "index_material_versions_on_file_blob_id"
t.index ["material_id", "version_no"], name: "index_material_versions_on_material_id_and_version_no", unique: true
t.index ["material_id"], name: "index_material_versions_on_material_id"
t.index ["parent_id"], name: "index_material_versions_on_parent_id"
t.index ["tag_id"], name: "index_material_versions_on_tag_id"
t.index ["updated_by_user_id"], name: "index_material_versions_on_updated_by_user_id"
t.index ["url"], name: "index_material_versions_on_url"
t.check_constraint "`event_type` in (_utf8mb4'create',_utf8mb4'update',_utf8mb4'discard',_utf8mb4'restore')", name: "material_versions_event_type_valid"
end
create_table "materials", charset: "utf8mb4", collation: "utf8mb4_0900_ai_ci", force: :cascade do |t|
@@ -161,6 +241,14 @@ ActiveRecord::Schema[8.0].define(version: 2026_06_12_000000) do
t.datetime "updated_at", null: false
t.datetime "discarded_at"
t.virtual "active_url", type: :string, as: "if((`discarded_at` is null),`url`,NULL)"
t.integer "version_no", default: 1, null: false
t.string "source_kind"
t.string "source_uri"
t.string "source_path"
t.string "source_file_id"
t.string "normalized_source_key"
t.virtual "active_normalized_source_key", type: :string, as: "if((`discarded_at` is null),`normalized_source_key`,NULL)"
t.index ["active_normalized_source_key"], name: "index_materials_on_active_normalized_source_key", unique: true
t.index ["active_url"], name: "index_materials_on_active_url", unique: true
t.index ["created_by_user_id"], name: "index_materials_on_created_by_user_id"
t.index ["discarded_at"], name: "index_materials_on_discarded_at"
@@ -319,6 +407,7 @@ ActiveRecord::Schema[8.0].define(version: 2026_06_12_000000) do
t.string "event_type", null: false
t.string "name", null: false
t.string "category", null: false
t.datetime "deprecated_at"
t.text "aliases", null: false
t.text "parent_tag_ids", null: false
t.datetime "created_at", null: false
@@ -336,10 +425,13 @@ ActiveRecord::Schema[8.0].define(version: 2026_06_12_000000) do
t.datetime "created_at", null: false
t.datetime "updated_at", null: false
t.integer "post_count", default: 0, null: false
t.datetime "deprecated_at"
t.datetime "discarded_at"
t.integer "version_no", null: false
t.index ["deprecated_at"], name: "index_tags_on_deprecated_at"
t.index ["discarded_at"], name: "index_tags_on_discarded_at"
t.index ["tag_name_id"], name: "index_tags_on_tag_name_id", unique: true
t.check_constraint "(`deprecated_at` is null) or (`category` <> _utf8mb4'nico')", name: "chk_tags_deprecated_at_not_nico"
t.check_constraint "`version_no` > 0", name: "chk_tags_version_no_positive"
end
@@ -563,6 +655,12 @@ ActiveRecord::Schema[8.0].define(version: 2026_06_12_000000) do
add_foreign_key "gekanator_question_suggestions", "users"
add_foreign_key "gekanator_questions", "gekanator_question_suggestions"
add_foreign_key "gekanator_questions", "users", column: "created_by_id"
add_foreign_key "material_export_items", "materials"
add_foreign_key "material_export_items", "users", column: "created_by_user_id"
add_foreign_key "material_import_blocks", "users", column: "created_by_user_id"
add_foreign_key "material_sync_sources", "users", column: "created_by_user_id"
add_foreign_key "material_sync_sources", "users", column: "updated_by_user_id"
add_foreign_key "material_sync_suppressions", "users", column: "created_by_user_id"
add_foreign_key "material_versions", "materials"
add_foreign_key "material_versions", "materials", column: "parent_id"
add_foreign_key "material_versions", "tags"
+18
ファイルの表示
@@ -7,3 +7,21 @@
# ["Action", "Comedy", "Drama", "Horror"].each do |genre_name|
# MovieGenre.find_or_create_by!(name: genre_name)
# end
material_sync_source_uri = ENV['MATERIAL_SYNC_SOURCE_URI']
material_sync_source_file_id = ENV['MATERIAL_SYNC_SOURCE_FILE_ID']
if material_sync_source_uri.present? || material_sync_source_file_id.present?
source = MaterialSyncSource.find_or_initialize_by(
name: ENV.fetch('MATERIAL_SYNC_SOURCE_NAME', 'Legacy URI material sync'))
source.assign_attributes(
source_kind: ENV.fetch('MATERIAL_SYNC_SOURCE_KIND', 'uri'),
source_uri: material_sync_source_uri,
source_path: ENV['MATERIAL_SYNC_SOURCE_PATH'],
source_file_id: material_sync_source_file_id,
profile: ENV.fetch('MATERIAL_SYNC_SOURCE_PROFILE', 'legacy_drive'),
enabled: ENV.fetch('MATERIAL_SYNC_SOURCE_ENABLED', 'true') != 'false',
default_tag_name: ENV['MATERIAL_SYNC_SOURCE_TAG_NAME'],
export_path_prefix: ENV['MATERIAL_SYNC_EXPORT_PATH_PREFIX'])
source.save!
end
+1 -1
ファイルの表示
@@ -1,6 +1,6 @@
namespace :post_similarity do
desc '関聯投稿テーブル作成'
task calc: :environment do
Similarity::Calc.call(Post, :tags)
Similarity::Calc.call(Post, :active_tags)
end
end
+1 -1
ファイルの表示
@@ -1,6 +1,6 @@
namespace :tag_similarity do
desc '関聯タグ・テーブル作成'
task calc: :environment do
Similarity::Calc.call(Tag, :posts)
Similarity::Calc.call(Tag, :posts, scope: Tag.where(deprecated_at: nil))
end
end
+51
ファイルの表示
@@ -0,0 +1,51 @@
namespace :materials do
desc '素材同期'
task sync: :environment do
result = MaterialSyncRunner.sync_enabled!
message = [
"materials:sync imported=#{ result.imported }",
"updated=#{ result.updated }",
"unchanged=#{ result.unchanged }",
"suppressed=#{ result.suppressed }",
"failed=#{ result.failed }"
].join(' ')
Rails.logger.info(message)
puts message
result.errors.each do |error|
Rails.logger.warn(error.to_json)
warn error.to_json
end
end
namespace :thumbnails do
desc '素材サムネールを補完'
task backfill: :environment do
counts = Hash.new(0)
Material
.with_attached_file
.with_attached_thumbnail
.find_each do |material|
next unless material.file.attached?
next if material.thumbnail.attached?
result = MaterialThumbnailGenerator.generate!(material)
counts[result] += 1
end
message =
"materials:thumbnails:backfill " \
"attached=#{ counts[:attached] } " \
"no_file=#{ counts[:no_file] } " \
"unsupported_content_type=#{ counts[:unsupported_content_type] } " \
"generation_failed=#{ counts[:generation_failed] } " \
"attach_failed=#{ counts[:attach_failed] } " \
"file_not_found=#{ counts[:file_not_found] } " \
"mini_magick_error=#{ counts[:mini_magick_error] }"
Rails.logger.info(message)
puts message
end
end
end
+57
ファイルの表示
@@ -0,0 +1,57 @@
require 'rails_helper'
RSpec.describe MaterialExportItem, type: :model do
let(:user) { create(:user, :member) }
let(:tag) { Tag.create!(tag_name: TagName.create!(name: 'export_item'), category: :material) }
let(:material) do
Material.create!(tag:, url: 'https://example.com/material',
created_by_user: user, updated_by_user: user)
end
it 'rejects blank export_path' do
item = described_class.new(material:, profile: 'legacy_drive', export_path: '')
expect(item).not_to be_valid
expect(item.errors[:export_path]).to be_present
end
it 'rejects absolute export_path' do
item = described_class.new(material:, profile: 'legacy_drive',
export_path: '/素材/a.png')
expect(item).not_to be_valid
expect(item.errors[:export_path]).to be_present
end
it 'rejects parent traversal export_path' do
item = described_class.new(material:, profile: 'legacy_drive',
export_path: '素材/../a.png')
expect(item).not_to be_valid
expect(item.errors[:export_path]).to be_present
end
it 'rejects double slash export_path' do
item = described_class.new(material:, profile: 'legacy_drive',
export_path: '素材//a.png')
expect(item).not_to be_valid
expect(item.errors[:export_path]).to be_present
end
it 'rejects dot segment export_path' do
item = described_class.new(material:, profile: 'legacy_drive',
export_path: './素材/a.png')
expect(item).not_to be_valid
expect(item.errors[:export_path]).to be_present
end
it 'rejects trailing slash export_path' do
item = described_class.new(material:, profile: 'legacy_drive',
export_path: '素材/a/')
expect(item).not_to be_valid
expect(item.errors[:export_path]).to be_present
end
end
+19
ファイルの表示
@@ -0,0 +1,19 @@
require 'rails_helper'
RSpec.describe Material, type: :model do
let(:user) { create(:user, :member) }
it 'allows only material source kinds, not suppression prefix kinds' do
material = described_class.new(url: 'https://example.com/material',
source_kind: 'google_drive_path_prefix',
source_path: '素材/危険',
created_by_user: user,
updated_by_user: user)
expect(material).not_to be_valid
expect(material.errors[:source_kind]).to be_present
material.source_kind = 'google_drive_path'
expect(material).to be_valid
end
end
+36
ファイルの表示
@@ -0,0 +1,36 @@
require 'rails_helper'
RSpec.describe TagImplication, type: :model do
it 'rejects a parent tag that would create a cycle' do
child = create(:tag, name: 'tag_implication_cycle_child')
parent = create(:tag, name: 'tag_implication_cycle_parent')
described_class.create!(tag: child, parent_tag: parent)
implication = described_class.new(tag: parent, parent_tag: child)
expect(implication).not_to be_valid
expect(implication.errors[:parent_tag_id]).to include(
'親タグに子孫タグを指定すると循環します.'
)
expect(implication.errors[:base]).to be_present
end
it 'terminates even when existing data already contains a cycle' do
child = create(:tag, name: 'tag_implication_existing_cycle_child')
parent = create(:tag, name: 'tag_implication_existing_cycle_parent')
ancestor = create(:tag, name: 'tag_implication_existing_cycle_ancestor')
described_class.create!(tag: parent, parent_tag: ancestor)
described_class.insert_all!(
[
{ tag_id: ancestor.id, parent_tag_id: parent.id,
created_at: Time.current, updated_at: Time.current }
]
)
implication = described_class.new(tag: child, parent_tag: parent)
expect(implication).to be_valid
end
end
+4
ファイルの表示
@@ -1,6 +1,10 @@
require 'rails_helper'
RSpec.describe TagNameSanitisationRule, type: :model do
before do
described_class.unscoped.delete_all
end
describe '.sanitise' do
before do
described_class.create!(priority: 10, source_pattern: '_', replacement: '')
+83
ファイルの表示
@@ -1,6 +1,89 @@
require 'rails_helper'
RSpec.describe Tag, type: :model do
describe '.normalise_tags!' do
it 'rejects deprecated tags when deny_deprecated is enabled' do
tag_name = TagName.create!(name: 'normalise deprecated tag')
deprecated_tag = Tag.create!(
tag_name:,
category: :general,
deprecated_at: 1.day.from_now
)
expect {
described_class.normalise_tags!(
[deprecated_tag.name],
deny_deprecated: true
)
}.to raise_error(Tag::DeprecatedTagNormalisationError) { |error|
expect(error.tag_names).to eq([deprecated_tag.name])
}
end
end
describe '.expand_parent_tags' do
it 'expands through multiple deprecated parents to an active ancestor' do
child = create(:tag, name: 'expand_child')
deprecated_parent = create(
:tag,
name: 'expand_deprecated_parent',
deprecated_at: Time.current
)
deprecated_grandparent = create(
:tag,
name: 'expand_deprecated_grandparent',
deprecated_at: Time.current
)
active_ancestor = create(:tag, name: 'expand_active_ancestor')
TagImplication.create!(tag: child, parent_tag: deprecated_parent)
TagImplication.create!(tag: deprecated_parent, parent_tag: deprecated_grandparent)
TagImplication.create!(tag: deprecated_grandparent, parent_tag: active_ancestor)
expanded = described_class.expand_parent_tags([child])
expect(expanded).to include(
child,
deprecated_parent,
deprecated_grandparent,
active_ancestor
)
expect(expanded.reject(&:deprecated?)).to contain_exactly(child, active_ancestor)
end
it 'terminates when implications contain a cycle' do
first = create(:tag, name: 'expand_cycle_first')
second = create(:tag, name: 'expand_cycle_second')
TagImplication.create!(tag: first, parent_tag: second)
now = Time.current
TagImplication.insert_all!(
[
{
tag_id: second.id,
parent_tag_id: first.id,
created_at: now,
updated_at: now
}
]
)
expect(described_class.expand_parent_tags([first])).to contain_exactly(first, second)
end
end
describe 'deprecated validation' do
it 'rejects deprecated nico tags' do
tag = build(
:tag,
name: 'nico:deprecated_validation',
category: :nico,
deprecated_at: Time.current
)
expect(tag).not_to be_valid
expect(tag.errors[:deprecated_at]).to include('ニコタグは廃止できません.')
end
end
describe '.merge_tags!' do
let!(:target_tag) { create(:tag, category: :general) }
let!(:source_tag) { create(:tag, category: :general) }
+287 -11
ファイルの表示
@@ -151,6 +151,188 @@ RSpec.describe 'Gekanator learning API', type: :request do
expect(response).to have_http_status(:unauthorized)
end
it 'learns accepted non-nico tag answers from camelCase main game logs' do
sign_in_as admin
tag_question = GekanatorQuestion.create!(
text: 'MAD 要素がある?',
kind: 'tag',
source: 'admin_curated',
status: 'accepted',
priority_weight: 1.0,
condition: { type: 'tag', key: 'meme:MAD' },
created_by: admin
)
expect {
post '/gekanator/games', params: {
guessed_post_id: guessed_post.id,
correct_post_id: correct_post.id,
answers: [
{
questionId: 'tag:meme:MAD',
question_text: 'MAD 要素がある?',
answer: 'yes',
original_answer: 'yes'
},
{
questionId: 'tag:meme:missing',
question_text: '存在しない質問?',
answer: 'yes',
original_answer: 'yes'
},
{
questionId: 'tag:meme:MAD',
question_text: 'MAD 要素がある?',
answer: 'unknown',
original_answer: 'unknown'
}
]
}
}.to change { GekanatorQuestionExample.count }.by(1)
expect(response).to have_http_status(:created)
expect(json['learned_example_count']).to eq(1)
example = GekanatorQuestionExample.last
expect(example).to have_attributes(
gekanator_question_id: tag_question.id,
post_id: correct_post.id,
user_id: admin.id,
answer: 'yes',
source: 'post_game_answer'
)
expect(example.gekanator_game_id).to eq(json['id'])
end
it 'learns accepted post_similarity answers from main game logs' do
sign_in_as admin
question = create_post_similarity_question!(text: '泣いてる?')
expect {
post '/gekanator/games', params: {
guessed_post_id: guessed_post.id,
correct_post_id: correct_post.id,
answers: [
{
question_id: "post-similarity:#{question.id}",
question_text: '泣いてる?',
answer: 'partial',
original_answer: 'partial'
}
]
}
}.to change { GekanatorQuestionExample.count }.by(1)
expect(response).to have_http_status(:created)
expect(json['learned_example_count']).to eq(1)
example = GekanatorQuestionExample.last
expect(example).to have_attributes(
gekanator_question_id: question.id,
post_id: correct_post.id,
user_id: admin.id,
answer: 'partial',
source: 'post_game_answer'
)
expect(example.gekanator_game_id).to eq(json['id'])
end
it 'does not learn fact questions or nico tag questions from main game logs' do
sign_in_as admin
[
{
text: 'example.com 由来?',
kind: 'source',
condition: { type: 'source', host: 'example.com' }
},
{
text: '題名に結束バンドを含む?',
kind: 'title',
condition: { type: 'title-contains', text: '結束バンド' }
},
{
text: '2024 年投稿?',
kind: 'original_date',
condition: { type: 'original-year', year: 2024 }
},
{
text: 'ニコニコにぼっちタグ?',
kind: 'tag',
condition: { type: 'tag', key: 'nico:ぼっち' }
}
].each do |attributes|
GekanatorQuestion.create!(
text: attributes[:text],
kind: attributes[:kind],
source: 'admin_curated',
status: 'accepted',
priority_weight: 1.0,
condition: attributes[:condition],
created_by: admin
)
end
expect {
post '/gekanator/games', params: {
guessed_post_id: guessed_post.id,
correct_post_id: correct_post.id,
answers: [
{ question_id: 'source:example.com', answer: 'yes' },
{ question_id: 'title:contains:結束バンド', answer: 'yes' },
{ question_id: 'original-year:2024', answer: 'yes' },
{ question_id: 'tag:nico:ぼっち', answer: 'yes' }
]
}
}.not_to change { GekanatorQuestionExample.count }
expect(response).to have_http_status(:created)
expect(json['learned_example_count']).to eq(0)
end
it 'updates an existing main game example instead of duplicating it' do
sign_in_as admin
tag_question = GekanatorQuestion.create!(
text: '喜多ちゃんが関係してる?',
kind: 'tag',
source: 'admin_curated',
status: 'accepted',
priority_weight: 1.0,
condition: { type: 'tag', key: 'character:喜多郁代' },
created_by: admin
)
existing = GekanatorQuestionExample.create!(
gekanator_question: tag_question,
post: correct_post,
user: admin,
answer: 'no',
source: 'post_game_answer',
weight: 1.0
)
expect {
post '/gekanator/games', params: {
guessed_post_id: guessed_post.id,
correct_post_id: correct_post.id,
answers: [
{
question_id: 'tag:character:喜多郁代',
answer: 'yes',
original_answer: 'yes'
}
]
}
}.not_to change { GekanatorQuestionExample.count }
expect(response).to have_http_status(:created)
expect(json['learned_example_count']).to eq(1)
expect(existing.reload.answer).to eq('yes')
expect(existing.sample_count).to eq(2)
end
end
describe 'POST /gekanator/question_suggestions' do
@@ -249,7 +431,7 @@ RSpec.describe 'Gekanator learning API', type: :request do
expect(GekanatorQuestionSuggestion.last.processed).to eq(false)
end
it 'limits suggestions to three per game' do
it 'allows more than three suggestions per game' do
sign_in_as admin
3.times do |i|
@@ -267,9 +449,10 @@ RSpec.describe 'Gekanator learning API', type: :request do
question_text: 'fourth question?',
answer: 'yes'
}
}.not_to change { GekanatorQuestionSuggestion.count }
}.to change { GekanatorQuestionSuggestion.count }.by(1)
expect(response).to have_http_status(:unprocessable_entity)
expect(response).to have_http_status(:created)
expect(json['count']).to eq(4)
end
it 'allows a non-admin user to suggest a question for their own game' do
@@ -326,28 +509,59 @@ RSpec.describe 'Gekanator learning API', type: :request do
end
describe 'GET /gekanator/games/:id/extra_questions' do
it 'returns at most two accepted user_suggested post_similarity questions without duplicates' do
it 'returns at most six accepted user_suggested post_similarity questions without duplicates' do
sign_in_as admin
lowest = create_post_similarity_question!(
text: 'lowest?',
priority_weight: 0.5
)
low = create_post_similarity_question!(
text: 'low?',
priority_weight: 1.0
)
high = create_post_similarity_question!(
text: 'high?',
priority_weight: 3.0
)
middle = create_post_similarity_question!(
text: 'middle?',
priority_weight: 1.5
)
medium_high = create_post_similarity_question!(
text: 'medium high?',
priority_weight: 2.0
)
high = create_post_similarity_question!(
text: 'high?',
priority_weight: 2.5
)
higher = create_post_similarity_question!(
text: 'higher?',
priority_weight: 2.8
)
highest = create_post_similarity_question!(
text: 'highest?',
priority_weight: 3.0
)
overflow = create_post_similarity_question!(
text: 'overflow?',
priority_weight: 2.2
)
get "/gekanator/games/#{game.id}/extra_questions"
expect(response).to have_http_status(:ok)
expect(json['questions'].length).to eq(2)
expect(json['questions'].map { _1['id'] }.uniq.length).to eq(2)
expect(json['questions'].map { _1['id'] }).to all(be_in([low.id, high.id, middle.id]))
expect(json['questions'].length).to eq(6)
expect(json['questions'].map { _1['id'] }.uniq.length).to eq(6)
expect(json['questions'].map { _1['id'] }).to all(
be_in([
lowest.id,
low.id,
middle.id,
medium_high.id,
high.id,
higher.id,
highest.id,
overflow.id,
])
)
end
it 'can return questions that already have an example for the correct post' do
@@ -370,6 +584,37 @@ RSpec.describe 'Gekanator learning API', type: :request do
expect(json['questions'].map { _1['id'] }).to include(existing.id)
end
it 'prioritizes questions the current user has not answered' do
sign_in_as admin
answered = create_post_similarity_question!(
text: 'already answered?',
priority_weight: 3.0
)
GekanatorQuestionExample.create!(
gekanator_question: answered,
post: other_post,
user: admin,
answer: 'yes',
source: 'post_game_extra'
)
unanswered =
6.times.map { |index|
create_post_similarity_question!(
text: "unanswered #{index}?",
priority_weight: 0.5
)
}
get "/gekanator/games/#{game.id}/extra_questions"
expect(response).to have_http_status(:ok)
expect(json['questions'].map { _1['id'] }).to match_array(
unanswered.map(&:id)
)
end
it 'can return questions already asked in the game using snake_case question_id' do
sign_in_as admin
@@ -652,6 +897,37 @@ RSpec.describe 'Gekanator learning API', type: :request do
end
describe 'GET /gekanator/questions' do
it 'omits questions for deprecated tags' do
active_tag = Tag.create!(name: 'active_question_tag', category: :general)
deprecated_tag = Tag.create!(
name: 'deprecated_question_tag',
category: :general,
deprecated_at: Time.current
)
[active_tag, deprecated_tag].each do |question_tag|
GekanatorQuestion.create!(
text: "#{ question_tag.name }?",
kind: 'tag',
source: 'admin_curated',
status: 'accepted',
priority_weight: 1.0,
condition: {
type: 'tag',
key: "#{ question_tag.category }:#{ question_tag.name }"
},
created_by: admin
)
end
get '/gekanator/questions'
expect(response).to have_http_status(:ok)
question_ids = json.fetch('questions').map { |question| question.fetch('id') }
expect(question_ids).to include('tag:general:active_question_tag')
expect(question_ids).not_to include('tag:general:deprecated_question_tag')
end
it 'returns accepted questions only and includes example_answers for post_similarity questions' do
sign_in_as admin
+33
ファイルの表示
@@ -0,0 +1,33 @@
require 'rails_helper'
RSpec.describe 'Gekanator posts API', type: :request do
describe 'GET /gekanator/posts' do
it 'omits deprecated tags and returns the stored similarity cosine' do
active_tag = Tag.create!(name: 'active tag', category: :general)
deprecated_tag = Tag.create!(
name: 'deprecated tag',
category: :general,
deprecated_at: Time.current
)
post_record = Post.create!(title: 'source', url: 'https://example.com/source')
target_post = Post.create!(title: 'target', url: 'https://example.com/target')
PostTag.create!(post: post_record, tag: active_tag)
PostTag.create!(post: post_record, tag: deprecated_tag)
PostTag.create!(post: target_post, tag: deprecated_tag)
PostSimilarity.create!(post: post_record, target_post:, cos: 0.375)
get '/gekanator/posts'
expect(response).to have_http_status(:ok)
post_json = json.fetch('posts').find { |post| post.fetch('id') == post_record.id }
expect(post_json.fetch('tags').map { |tag| tag.fetch('name') }).to eq(['active tag'])
expect(post_json.fetch('post_similarity_edges')).to contain_exactly(
'target_post_id' => target_post.id,
'cos' => 0.375
)
end
end
end
+377 -34
ファイルの表示
@@ -1,7 +1,10 @@
require 'rails_helper'
RSpec.describe 'Materials API', type: :request do
include ActiveJob::TestHelper
let!(:member_user) { create(:user, :member) }
let!(:admin_user) { create(:user, :admin) }
let!(:guest_user) { create(:user) }
def dummy_upload(filename: 'dummy.png', type: 'image/png', body: 'dummy')
@@ -13,22 +16,29 @@ RSpec.describe 'Materials API', type: :request do
end
def build_material(tag:, user:, parent: nil, file: dummy_upload, url: nil)
Material.new(tag:, parent:, url:, created_by_user: user, updated_by_user: user).tap do |material|
Material.new(tag:, parent:, url:,
created_by_user: user,
updated_by_user: user).tap do |material|
material.file.attach(file) if file
material.save!
end
end
describe 'GET /materials' do
let!(:tag_a) { Tag.create!(tag_name: TagName.create!(name: 'material_index_a'), category: :material) }
let!(:tag_b) { Tag.create!(tag_name: TagName.create!(name: 'material_index_b'), category: :material) }
let!(:tag_a) do
Tag.create!(tag_name: TagName.create!(name: 'material_index_a'), category: :material)
end
let!(:tag_b) do
Tag.create!(tag_name: TagName.create!(name: 'material_index_b'), category: :material)
end
let!(:material_a) do
build_material(tag: tag_a, user: member_user, file: dummy_upload(filename: 'a.png'))
end
let!(:material_b) do
build_material(tag: tag_b, user: member_user, parent: material_a, file: dummy_upload(filename: 'b.png'))
build_material(tag: tag_b, user: member_user, parent: material_a,
file: dummy_upload(filename: 'b.png'))
end
before do
@@ -54,27 +64,36 @@ RSpec.describe 'Materials API', type: :request do
'name' => 'material_index_b',
'category' => 'material'
)
expect(row['created_by_user']).to include(
'id' => member_user.id,
'name' => member_user.name
)
expect(row['content_type']).to eq('image/png')
expect(row['media_kind']).to eq('image')
end
it 'filters materials by tag_id' do
get '/materials', params: { tag_id: material_a.tag_id }
it 'filters materials by q' do
get '/materials', params: { q: 'material_index_a' }
expect(response).to have_http_status(:ok)
expect(json['count']).to eq(1)
expect(response_materials.map { |m| m['id'] }).to eq([material_a.id])
end
it 'filters materials by parent_id' do
get '/materials', params: { parent_id: material_a.id }
it 'filters materials by tag_state' do
untagged =
build_material(tag: nil, user: member_user,
file: dummy_upload(filename: 'untagged.png'))
get '/materials', params: { tag_state: 'untagged' }
expect(response).to have_http_status(:ok)
expect(json['count']).to eq(1)
expect(response_materials.map { |m| m['id'] }).to eq([material_b.id])
expect(response_materials.map { |m| m['id'] }).to eq([untagged.id])
end
it 'filters materials by media_kind' do
get '/materials', params: { media_kind: 'image' }
expect(response).to have_http_status(:ok)
expect(json['count']).to eq(2)
expect(response_materials.map { |m| m['media_kind'] }).to all(eq('image'))
end
it 'paginates and keeps total count' do
@@ -94,10 +113,74 @@ RSpec.describe 'Materials API', type: :request do
expect(response_materials.size).to eq(1)
expect(response_materials.first['id']).to eq(material_b.id)
end
it 'filters by descendant tags and returns stable parent tag groups' do
root =
Tag.create!(tag_name: TagName.create!(name: 'material_scope_root'),
category: :material)
child_b =
Tag.create!(tag_name: TagName.create!(name: 'material_scope_b'),
category: :material)
child_a =
Tag.create!(tag_name: TagName.create!(name: 'material_scope_a'),
category: :material)
deprecated =
Tag.create!(tag_name: TagName.create!(name: 'material_scope_old'),
category: :material,
deprecated_at: Time.current)
grandchild =
Tag.create!(tag_name: TagName.create!(name: 'material_scope_grandchild'),
category: :material)
root_material =
build_material(tag: root, user: member_user,
file: dummy_upload(filename: 'root.png'))
child_a_material =
build_material(tag: child_a, user: member_user,
file: dummy_upload(filename: 'child_a.png'))
grandchild_material =
build_material(tag: grandchild, user: member_user,
file: dummy_upload(filename: 'grandchild.png'))
TagImplication.create!(parent_tag: root, tag: child_b)
TagImplication.create!(parent_tag: root, tag: child_a)
TagImplication.create!(parent_tag: child_b, tag: deprecated)
TagImplication.create!(parent_tag: deprecated, tag: grandchild)
get '/materials',
params: { tag_id: root.id, include_descendants: '1',
group_by: 'parent_tag', sort: 'id', direction: 'asc' }
expect(response).to have_http_status(:ok)
expect(response_materials.map { |m| m['id'] })
.to include(root_material.id, child_a_material.id, grandchild_material.id)
expect(json['tag_scope']).to eq(
'tag' => {
'id' => root.id,
'name' => 'material_scope_root',
'category' => 'material'
},
'include_descendants' => true
)
expect(json['groups'].map { |group| group.dig('tag', 'name') })
.to eq(['material_scope_a', 'material_scope_b', 'material_scope_root'])
expect(json['groups'].find { |group| group.dig('tag', 'id') == child_b.id }
.fetch('material_ids')).to eq([grandchild_material.id])
end
it 'returns nil tag_scope when tag_id is unknown' do
get '/materials', params: { tag_id: 999_999, group_by: 'parent_tag' }
expect(response).to have_http_status(:ok)
expect(json['tag_scope']).to be_nil
expect(json['groups']).to eq([])
expect(response_materials).to eq([])
end
end
describe 'GET /materials/:id' do
let!(:tag) { Tag.create!(tag_name: TagName.create!(name: 'material_show'), category: :material) }
let!(:tag) do
Tag.create!(tag_name: TagName.create!(name: 'material_show'), category: :material)
end
let!(:material) do
build_material(tag:, user: member_user, file: dummy_upload(filename: 'show.png'))
end
@@ -138,9 +221,22 @@ RSpec.describe 'Materials API', type: :request do
end
end
context 'when logged in' do
context 'when logged in but not member' do
before { sign_in_as(guest_user) }
it 'returns 403' do
post '/materials', params: {
tag: 'material_create_guest_forbidden',
file: dummy_upload
}
expect(response).to have_http_status(:forbidden)
end
end
context 'when member' do
before { sign_in_as(member_user) }
it 'returns 422 when tag is blank' do
post '/materials', params: { tag: ' ', file: dummy_upload }
@@ -162,24 +258,49 @@ RSpec.describe 'Materials API', type: :request do
expect do
post '/materials', params: {
tag: 'material_create_new',
file: dummy_upload(filename: 'created.png')
file: dummy_upload(filename: 'created.png'),
export_paths: { legacy_drive: '伊地知ニジカ/created.png' }
}
end.to change(Material, :count).by(1)
.and change(Tag, :count).by(1)
.and change(TagName, :count).by(1)
.and change(MaterialVersion, :count).by(1)
expect(response).to have_http_status(:created)
material = Material.order(:id).last
expect(material.tag.name).to eq('material_create_new')
expect(material.tag.category).to eq('material')
expect(material.created_by_user).to eq(guest_user)
expect(material.updated_by_user).to eq(guest_user)
expect(material.created_by_user).to eq(member_user)
expect(material.updated_by_user).to eq(member_user)
expect(material.file.attached?).to be(true)
expect(material.version_no).to eq(1)
expect(material.material_versions.first.event_type).to eq('create')
expect(material.material_export_items.first.export_path).to eq('伊地知ニジカ/created.png')
expect(material.material_versions.first.export_paths_json).to eq(
'legacy_drive' => '伊地知ニジカ/created.png'
)
expect(json['id']).to eq(material.id)
expect(json.dig('tag', 'name')).to eq('material_create_new')
expect(json['content_type']).to eq('image/png')
expect(json.dig('export_paths', 'legacy_drive')).to eq('伊地知ニジカ/created.png')
end
it 'snapshots attached file metadata and sha256' do
post '/materials', params: {
tag: 'material_create_file_version',
file: dummy_upload(filename: 'created.png', body: 'sha-body')
}
expect(response).to have_http_status(:created)
version = Material.order(:id).last.material_versions.first
expect(version.file_blob_id).to be_present
expect(version.file_filename).to eq('created.png')
expect(version.file_content_type).to eq('image/png')
expect(version.file_byte_size).to eq('sha-body'.bytesize)
expect(version.file_sha256).to eq(Digest::SHA256.hexdigest('sha-body'))
end
it 'returns 422 when the existing tag is not material/character' do
@@ -219,11 +340,33 @@ RSpec.describe 'Materials API', type: :request do
expect(response).to have_http_status(:created)
expect(json['url']).to eq('https://example.com/material-source')
end
it 'rejects sha256-blocked file upload' do
sha256 = Digest::SHA256.hexdigest('blocked-body')
MaterialImportBlock.create!(match_kind: 'sha256',
sha256:,
reason: 'copyright_high_risk',
created_by_user: admin_user)
expect do
post '/materials', params: {
tag: 'material_blocked_create',
file: dummy_upload(filename: 'blocked.png', body: 'blocked-body')
}
end.not_to change(Material, :count)
expect(response).to have_http_status(:unprocessable_entity)
expect(json.fetch('errors')).to include(
'file' => ['抑止された素材です: copyright_high_risk']
)
end
end
end
describe 'PUT /materials/:id' do
let!(:tag) { Tag.create!(tag_name: TagName.create!(name: 'material_update_old'), category: :material) }
let!(:tag) do
Tag.create!(tag_name: TagName.create!(name: 'material_update_old'), category: :material)
end
let!(:material) do
build_material(tag:, user: member_user, file: dummy_upload(filename: 'old.png'))
end
@@ -277,25 +420,26 @@ RSpec.describe 'Materials API', type: :request do
'tag' => ['タグは必須です.'])
end
it 'returns 422 when both file and url are blank' do
it 'keeps the existing file when file and url are omitted' do
put "/materials/#{ material.id }", params: {
tag: 'material_update_no_payload'
}
expect(response).to have_http_status(:unprocessable_entity)
expect(json.fetch('errors')).to include(
'file' => ['ファイルまたは URL は必須です.'],
'url' => ['ファイルまたは URL は必須です.'])
expect(response).to have_http_status(:ok)
expect(material.reload.file.attached?).to be(true)
end
it 'updates tag, url, file, and updated_by_user' do
old_blob_id = material.file.blob.id
put "/materials/#{ material.id }", params: {
tag: 'material_update_new',
url: 'https://example.com/updated-source',
file: dummy_upload(filename: 'updated.jpg', type: 'image/jpeg')
}
expect do
put "/materials/#{ material.id }", params: {
tag: 'material_update_new',
url: 'https://example.com/updated-source',
file: dummy_upload(filename: 'updated.jpg', type: 'image/jpeg'),
export_paths: { legacy_drive: '伊地知ニジカ/updated.jpg' }
}
end.to change(MaterialVersion, :count).by(2)
expect(response).to have_http_status(:ok)
@@ -306,8 +450,15 @@ RSpec.describe 'Materials API', type: :request do
expect(material.updated_by_user).to eq(member_user)
expect(material.file.attached?).to be(true)
expect(material.file.blob.id).not_to eq(old_blob_id)
expect(ActiveStorage::Blob.where(id: old_blob_id).exists?).to be(true)
expect(material.file.blob.filename.to_s).to eq('updated.jpg')
expect(material.file.blob.content_type).to eq('image/jpeg')
expect(material.version_no).to eq(2)
expect(material.material_versions.order(:version_no).last.event_type).to eq('update')
expect(material.material_export_items.first.export_path).to eq('伊地知ニジカ/updated.jpg')
expect(material.material_versions.order(:version_no).last.export_paths_json).to eq(
'legacy_drive' => '伊地知ニジカ/updated.jpg'
)
expect(json['id']).to eq(material.id)
expect(json['file']).to be_present
@@ -315,7 +466,7 @@ RSpec.describe 'Materials API', type: :request do
expect(json.dig('tag', 'name')).to eq('material_update_new')
end
it 'purges the existing file when file is omitted and url is provided' do
it 'detaches the existing file without purging blob when url replaces file' do
old_blob_id = material.file.blob.id
put "/materials/#{ material.id }", params: {
@@ -331,9 +482,7 @@ RSpec.describe 'Materials API', type: :request do
expect(material.updated_by_user).to eq(member_user)
expect(material.file.attached?).to be(false)
expect(
ActiveStorage::Blob.where(id: old_blob_id).exists?
).to be(false)
expect(ActiveStorage::Blob.where(id: old_blob_id).exists?).to be(true)
expect(json['id']).to eq(material.id)
expect(json['file']).to be_nil
@@ -341,11 +490,205 @@ RSpec.describe 'Materials API', type: :request do
expect(json.dig('tag', 'name')).to eq('material_update_remove_file')
expect(json['url']).to eq('https://example.com/updated-source')
end
it 'does not increase version for the same snapshot update' do
MaterialVersionRecorder.record!(material:, event_type: :create,
created_by_user: member_user)
expect do
put "/materials/#{ material.id }", params: {
tag: 'material_update_old'
}
end.not_to change(MaterialVersion, :count)
expect(response).to have_http_status(:ok)
expect(material.reload.version_no).to eq(1)
end
it 'records update version when only export_path changes' do
MaterialVersionRecorder.record!(material:, event_type: :create,
created_by_user: member_user)
expect do
put "/materials/#{ material.id }", params: {
tag: 'material_update_old',
export_paths: { legacy_drive: '素材/only-path.png' }
}
end.to change(MaterialVersion, :count).by(1)
expect(response).to have_http_status(:ok)
expect(material.reload.material_export_items.first.export_path).to eq('素材/only-path.png')
expect(material.material_versions.order(:version_no).last.export_paths_json).to eq(
'legacy_drive' => '素材/only-path.png'
)
end
it 'removes export_path item when blank is submitted' do
MaterialExportItem.create!(material:, profile: 'legacy_drive',
export_path: '素材/remove.png',
created_by_user: member_user)
MaterialVersionRecorder.record!(material:, event_type: :create,
created_by_user: member_user)
expect do
put "/materials/#{ material.id }", params: {
tag: 'material_update_old',
export_paths: { legacy_drive: '' }
}
end.to change(MaterialExportItem, :count).by(-1)
.and change(MaterialVersion, :count).by(1)
expect(response).to have_http_status(:ok)
expect(material.reload.material_export_items).to be_empty
expect(material.material_versions.order(:version_no).last.export_paths_json).to eq({})
end
it 'rejects sha256-blocked replacement file' do
sha256 = Digest::SHA256.hexdigest('blocked-update')
MaterialImportBlock.create!(match_kind: 'sha256',
sha256:,
reason: 'source_owner_request',
created_by_user: admin_user)
put "/materials/#{ material.id }", params: {
tag: 'material_update_old',
file: dummy_upload(filename: 'blocked.png', body: 'blocked-update')
}
expect(response).to have_http_status(:unprocessable_entity)
expect(material.reload.file.blob.filename.to_s).to eq('old.png')
end
end
end
describe 'GET /materials/download.zip' do
let!(:tag_a) { Tag.create!(tag_name: TagName.create!(name: 'zip_a'), category: :material) }
let!(:tag_b) { Tag.create!(tag_name: TagName.create!(name: 'zip_b'), category: :material) }
let!(:material_a) do
build_material(tag: tag_a, user: member_user,
file: dummy_upload(filename: 'a.png', body: 'zip-a'))
end
let!(:material_b) do
build_material(tag: tag_b, user: member_user,
file: dummy_upload(filename: 'b.png', body: 'zip-b'))
end
before do
MaterialExportItem.create!(material: material_a, profile: 'legacy_drive',
export_path: '素材/a.png',
created_by_user: member_user)
MaterialExportItem.create!(material: material_b, profile: 'legacy_drive',
export_path: '素材/b.png',
created_by_user: member_user)
end
it 'uses material_export_items.export_path as ZIP entry paths' do
get '/materials/download.zip', params: { profile: 'legacy_drive' }
expect(response).to have_http_status(:ok)
expect(response.media_type).to eq('application/zip')
expect(response.body.b).to include('素材/a.png'.b)
expect(response.body.b).to include('素材/b.png'.b)
end
it 'filters by tag_id' do
get '/materials/download.zip', params: { profile: 'legacy_drive', tag_id: tag_a.id }
expect(response).to have_http_status(:ok)
expect(response.body.b).to include('素材/a.png'.b)
expect(response.body.b).not_to include('素材/b.png'.b)
end
it 'does not include disabled export items' do
material_b.material_export_items.first.update!(enabled: false)
get '/materials/download.zip', params: { profile: 'legacy_drive' }
expect(response).to have_http_status(:ok)
expect(response.body.b).to include('素材/a.png'.b)
expect(response.body.b).not_to include('素材/b.png'.b)
end
end
describe 'GET /materials/versions' do
let!(:tag) do
Tag.create!(tag_name: TagName.create!(name: 'material_history'), category: :material)
end
let!(:material) do
build_material(tag:, user: member_user, file: dummy_upload(filename: 'history.png'))
end
it 'returns material versions in reverse chronological order' do
MaterialVersionRecorder.record!(material:, event_type: :create,
created_by_user: member_user)
material.update!(url: 'https://example.com/history')
MaterialVersionRecorder.record!(material:, event_type: :update,
created_by_user: admin_user)
get '/materials/versions', params: { material_id: material.id }
expect(response).to have_http_status(:ok)
expect(json['count']).to eq(2)
versions = json.fetch('versions')
expect(versions.map { |v| v['version_no'] }).to eq([2, 1])
expect(versions.first).to include(
'material_id' => material.id,
'event_type' => 'update',
'tag_name' => 'material_history',
'file_filename' => 'history.png',
'url' => 'https://example.com/history'
)
expect(versions.first['created_by_user']).to include('id' => admin_user.id)
end
it 'filters material versions by tag and event_type' do
MaterialVersionRecorder.record!(material:, event_type: :create,
created_by_user: member_user)
get '/materials/versions', params: {
tag: 'material_history',
event_type: 'create'
}
expect(response).to have_http_status(:ok)
expect(json.fetch('versions').map { |v| v['material_id'] }).to include(material.id)
expect(json.fetch('versions').map { |v| v['event_type'] }).to all(eq('create'))
end
end
describe 'material sync suppressions' do
it 'requires member permission for index' do
sign_out
get '/materials/suppressions'
expect(response).to have_http_status(:unauthorized)
end
it 'creates source suppression as member' do
sign_in_as(member_user)
expect do
post '/materials/suppressions', params: {
source_kind: 'google_drive_path_prefix',
drive_path: '伊地知ニジカ/危険素材',
reason: 'copyright_high_risk'
}
end.to change(MaterialSyncSuppression, :count).by(1)
expect(response).to have_http_status(:created)
expect(json).to include(
'source_kind' => 'google_drive_path_prefix',
'drive_path' => '伊地知ニジカ/危険素材',
'normalized_source_key' => 'google_drive_path_prefix:伊地知ニジカ/危険素材'
)
end
end
describe 'DELETE /materials/:id' do
let!(:tag) { Tag.create!(tag_name: TagName.create!(name: 'material_destroy'), category: :material) }
let!(:tag) do
Tag.create!(tag_name: TagName.create!(name: 'material_destroy'), category: :material)
end
let!(:material) do
build_material(tag:, user: member_user, file: dummy_upload(filename: 'destroy.png'))
end
+168
ファイルの表示
@@ -1,4 +1,5 @@
require 'rails_helper'
require 'base64'
require 'set'
include ActiveSupport::Testing::TimeHelpers
@@ -8,6 +9,11 @@ RSpec.describe 'Posts API', type: :request do
# resized_thumbnail! が MiniMagick 依存でコケやすいので request spec ではスタブしとくのが無難。
before do
allow_any_instance_of(Post).to receive(:resized_thumbnail!).and_return(true)
allow(Post).to receive(:resized_thumbnail_attachment).and_return(
io: StringIO.new('dummy'),
filename: 'resized_thumbnail.jpg',
content_type: 'image/jpeg'
)
end
def create_nico_tag!(name)
@@ -19,6 +25,18 @@ RSpec.describe 'Posts API', type: :request do
Rack::Test::UploadedFile.new(StringIO.new('dummy'), 'image/jpeg', original_filename: 'dummy.jpg')
end
def real_thumbnail_upload
gif =
Base64.decode64(
'R0lGODdhAQABAIABAP///wAAACwAAAAAAQABAAACAkQBADs=')
Rack::Test::UploadedFile.new(
StringIO.new(gif),
'image/gif',
original_filename: 'thumbnail.gif'
)
end
def post_write_params params = { }
{ parent_post_ids: '' }.merge(params)
end
@@ -517,6 +535,24 @@ RSpec.describe 'Posts API', type: :request do
expect([true, false]).to include(json['viewed'])
end
it 'omits deprecated tags' do
deprecated_tag = Tag.create!(
name: 'deprecated_post_tag',
category: :general,
deprecated_at: Time.current
)
PostTag.create!(post: post_record, tag: deprecated_tag)
request
expect(response).to have_http_status(:ok)
tag_names = json.fetch('tags').flat_map { |node|
[node.fetch('name')] + node.fetch('children').map { |child| child.fetch('name') }
}
expect(tag_names).to include('spec_tag')
expect(tag_names).not_to include('deprecated_post_tag')
end
context 'when post has parent, child, and sibling posts' do
let!(:parent_post) do
create_parent_post!(
@@ -678,6 +714,66 @@ RSpec.describe 'Posts API', type: :request do
expect(json['tags'][0]).to have_key('name')
end
it '201 when posting manually with a thumbnail' do
sign_in_as(member)
allow(Post).to receive(:resized_thumbnail_attachment).and_call_original
post '/posts', params: post_write_params(
title: 'thumbnail post',
url: 'https://example.com/thumbnail-post',
tags: 'spec_tag',
thumbnail: real_thumbnail_upload
)
expect(response).to have_http_status(:created)
post_record = Post.find(json.fetch('id'))
expect(post_record.thumbnail).to be_attached
expect(post_record.thumbnail.blob.content_type).to eq('image/jpeg')
expect { post_record.thumbnail.download }.not_to raise_error
end
it 'resizes the thumbnail before the create transaction begins' do
sign_in_as(member)
open_transactions = []
baseline_open_transactions = Post.connection.open_transactions
allow(Post).to receive(:resized_thumbnail_attachment) do |_upload|
open_transactions << Post.connection.open_transactions
{ io: StringIO.new('dummy'),
filename: 'resized_thumbnail.jpg',
content_type: 'image/jpeg' }
end
post '/posts', params: post_write_params(
title: 'transaction post',
url: 'https://example.com/transaction-post',
tags: 'spec_tag',
thumbnail: dummy_upload
)
expect(response).to have_http_status(:created)
expect(open_transactions).to eq([baseline_open_transactions])
end
it 'returns 422 and does not create a post when thumbnail resize fails' do
sign_in_as(member)
allow(Post).to receive(:resized_thumbnail_attachment).and_raise(MiniMagick::Error)
expect {
post '/posts', params: post_write_params(
title: 'broken thumbnail post',
url: 'https://example.com/broken-thumbnail-post',
tags: 'spec_tag',
thumbnail: dummy_upload
)
}.not_to change(Post, :count)
expect(response).to have_http_status(:unprocessable_entity)
expect(json.fetch('errors')).to include(
'thumbnail' => ['サムネイル画像の変換に失敗しました.']
)
end
it '201 and creates post + tags when member and tags have aliases' do
sign_in_as(member)
@@ -697,6 +793,58 @@ RSpec.describe 'Posts API', type: :request do
expect(names).not_to include('manko')
end
it 'rejects a deprecated tag specified directly' do
Tag.create!(
name: 'deprecated_direct_tag',
category: :general,
deprecated_at: Time.current
)
sign_in_as(member)
post '/posts', params: post_write_params(
title: 'new post',
url: 'https://example.com/deprecated-direct-tag',
tags: 'deprecated_direct_tag',
thumbnail: dummy_upload
)
expect(response).to have_http_status(:unprocessable_entity)
expect(json.fetch('errors')).to include(
'tags' => ['廃止済みタグは付与できません.']
)
end
it 'expands through multiple deprecated parent tags and saves active ancestors' do
child = Tag.create!(name: 'active_child', category: :general)
deprecated_parent = Tag.create!(
name: 'deprecated_parent',
category: :general,
deprecated_at: Time.current
)
deprecated_grandparent = Tag.create!(
name: 'deprecated_grandparent',
category: :general,
deprecated_at: Time.current
)
active_grandparent = Tag.create!(name: 'active_grandparent', category: :general)
TagImplication.create!(tag: child, parent_tag: deprecated_parent)
TagImplication.create!(tag: deprecated_parent, parent_tag: deprecated_grandparent)
TagImplication.create!(tag: deprecated_grandparent, parent_tag: active_grandparent)
sign_in_as(member)
post '/posts', params: post_write_params(
title: 'expanded post',
url: 'https://example.com/expanded-deprecated-parent',
tags: 'active_child',
thumbnail: dummy_upload
)
expect(response).to have_http_status(:created)
saved_names = Post.find(json.fetch('id')).tags.map(&:name)
expect(saved_names).to include('active_child', 'active_grandparent')
expect(saved_names).not_to include('deprecated_parent', 'deprecated_grandparent')
end
context "when nico tag already exists in tags" do
before do
Tag.find_undiscard_or_create_by!(
@@ -930,6 +1078,26 @@ RSpec.describe 'Posts API', type: :request do
expect(names).to include('spec_tag_2')
end
it 'rejects a deprecated tag specified directly' do
Tag.create!(
name: 'deprecated_update_tag',
category: :general,
deprecated_at: Time.current
)
sign_in_as(member)
put "/posts/#{ post_record.id }", params: post_update_params(
post_record,
title: 'updated title',
tags: 'deprecated_update_tag'
)
expect(response).to have_http_status(:unprocessable_entity)
expect(json.fetch('errors')).to include(
'tags' => ['廃止済みタグは付与できません.']
)
end
context "when nico tag already exists in tags" do
before do
Tag.find_undiscard_or_create_by!(
+11
ファイルの表示
@@ -56,6 +56,17 @@ RSpec.describe "TagChildren", type: :request do
expect(response).to have_http_status(:no_content)
end
it 'returns 422 and does not create relation when the new link makes a cycle' do
TagImplication.create!(tag: parent, parent_tag: child)
expect {
do_request
}.not_to change(TagImplication, :count)
expect(response).to have_http_status(:unprocessable_entity)
expect(TagImplication.where(tag: child, parent_tag: parent)).not_to exist
end
end
context "when Tag.find raises (invalid ids)" do
+11
ファイルの表示
@@ -21,6 +21,7 @@ RSpec.describe 'TagVersions API', type: :request do
event_type:,
name:,
category:,
deprecated_at: nil,
aliases: [],
parent_tags: [],
created_by_user:,
@@ -33,6 +34,7 @@ RSpec.describe 'TagVersions API', type: :request do
event_type: event_type,
name: name,
category: category,
deprecated_at: deprecated_at,
aliases: Array(aliases).join(' '),
parent_tag_ids: Array(parent_tags).map(&:id).join(' '),
created_by_user: created_by_user,
@@ -65,6 +67,7 @@ RSpec.describe 'TagVersions API', type: :request do
event_type: 'update',
name: 'new_tag_name',
category: 'meme',
deprecated_at: t_v2,
aliases: ['alias_shared', 'alias_new'],
parent_tags: [parent_shared, parent_new],
created_by_user: member,
@@ -133,6 +136,10 @@ RSpec.describe 'TagVersions API', type: :request do
'current' => 'meme',
'prev' => 'general'
)
expect(latest.fetch('deprecated_at')).to eq(
'current' => t_v2.iso8601,
'prev' => nil
)
expect(latest.fetch('aliases')).to include(
{ 'name' => 'alias_shared', 'type' => 'context' },
{ 'name' => 'alias_new', 'type' => 'added' },
@@ -178,6 +185,10 @@ RSpec.describe 'TagVersions API', type: :request do
'current' => 'general',
'prev' => nil
)
expect(first.fetch('deprecated_at')).to eq(
'current' => nil,
'prev' => nil
)
expect(first.fetch('aliases')).to include(
{ 'name' => 'alias_shared', 'type' => 'added' },
{ 'name' => 'alias_old', 'type' => 'added' }
+3
ファイルの表示
@@ -89,6 +89,7 @@ RSpec.describe 'Tag and wiki history integrity', type: :request do
category: 'general',
aliases: '',
parent_tags: '',
deprecated: '0',
}
}
.to change(TagVersion, :count).by(2)
@@ -123,6 +124,7 @@ RSpec.describe 'Tag and wiki history integrity', type: :request do
category: 'meme',
aliases: '',
parent_tags: '',
deprecated: '0',
}
}
.to change(TagVersion, :count).by(2)
@@ -149,6 +151,7 @@ RSpec.describe 'Tag and wiki history integrity', type: :request do
category: 'general',
aliases: 'put_tag_alias_only_alias',
parent_tags: '',
deprecated: '0',
}
}
.to change(TagVersion, :count).by(2)
+375 -2
ファイルの表示
@@ -76,6 +76,27 @@ RSpec.describe 'Tags API', type: :request do
expect(response_tags.first['id']).to eq(meme.id)
end
it 'filters tags by deprecated state' do
deprecated_tag = Tag.create!(
name: 'deprecated_filter',
category: :general,
deprecated_at: 1.day.from_now
)
active_tag = Tag.create!(name: 'active_filter', category: :general)
get '/tags', params: { name: '_filter', deprecated: '1' }
expect(response).to have_http_status(:ok)
expect(response_names).to include(deprecated_tag.name)
expect(response_names).not_to include(active_tag.name)
get '/tags', params: { name: '_filter', deprecated: '0' }
expect(response).to have_http_status(:ok)
expect(response_names).to include(active_tag.name)
expect(response_names).not_to include(deprecated_tag.name)
end
it 'filters tags by post_count range' do
low = Tag.create!(tag_name: TagName.create!(name: 'pc_low'), category: :general)
mid = Tag.create!(tag_name: TagName.create!(name: 'pc_mid'), category: :general)
@@ -301,6 +322,21 @@ RSpec.describe 'Tags API', type: :request do
expect(t['matched_alias']).to eq('unko')
expect(json.map { |x| x['name'] }).not_to include('unknown')
end
it 'omits deprecated tags' do
deprecated_tag = Tag.create!(
name: 'spec_deprecated',
category: :general,
deprecated_at: Time.current
)
deprecated_tag.update_columns(post_count: 1)
get '/tags/autocomplete', params: { q: 'spec_', present: '0' }
expect(response).to have_http_status(:ok)
expect(json.map { |item| item.fetch('name') }).to include('spec_tag')
expect(json.map { |item| item.fetch('name') }).not_to include('spec_deprecated')
end
end
describe 'GET /tags/name/:name' do
@@ -437,6 +473,32 @@ RSpec.describe 'Tags API', type: :request do
expect(versions.second.created_by_user_id).to eq(member_user.id)
end
it 'updates deprecated state and records it in tag versions' do
expect {
patch "/tags/#{ tag.id }", params: { deprecated: '1' }
}.to change(TagVersion, :count).by(2)
expect(response).to have_http_status(:ok)
expect(tag.reload.deprecated_at).to be_present
versions = tag.tag_versions.order(:version_no)
expect(versions.first.deprecated_at).to be_nil
expect(versions.second.deprecated_at).to eq(tag.deprecated_at)
expect(json.fetch('deprecated_at')).to be_present
end
it 'rejects deprecating a nico tag' do
nico_tag = Tag.create!(name: 'nico:deprecated_update', category: :nico)
patch "/tags/#{ nico_tag.id }", params: { deprecated: '1' }
expect(response).to have_http_status(:unprocessable_entity)
expect(nico_tag.reload.deprecated_at).to be_nil
expect(json.fetch('errors')).to include(
'deprecated' => ['ニコタグは廃止できません.']
)
end
it 'returns 422 when changing normal tag category to nico' do
expect {
patch "/tags/#{tag.id}", params: { category: 'nico' }
@@ -519,6 +581,58 @@ RSpec.describe 'Tags API', type: :request do
expect(response).to have_http_status(:ok)
expect(wiki_page.reload.wiki_versions.count).to eq(before_wiki_version_count)
end
it 'full update で旧名 alias が残った tag を PATCH で旧名へ戻せる' do
put "/tags/#{ tag.id }", params: {
name: 'patch_roundtrip_target',
category: 'general',
aliases: 'unko',
parent_tags: '',
deprecated: '0',
}
expect(response).to have_http_status(:ok)
expect(TagName.find_by!(name: 'spec_tag').canonical).to eq(tag.reload.tag_name)
patch "/tags/#{ tag.id }", params: { name: 'spec_tag' }
expect(response).to have_http_status(:ok)
tag.reload
expect(tag.name).to eq('spec_tag')
expect(tag.tag_name.canonical_id).to be_nil
expect(TagName.find_by!(name: 'patch_roundtrip_target').canonical).to eq(tag.tag_name)
end
it '別 tag の正規名には変更できない' do
wiki_page =
Wiki::Commit.create_content!(
tag_name: tag.tag_name,
body: 'patch collision wiki',
created_by_user: member_user,
message: 'init')
patch "/tags/#{ tag.id }", params: { name: 'unknown' }
expect(response).to have_http_status(:unprocessable_entity)
expect(json.fetch('errors')).to include(
'name' => ['その名前は既に使はれてゐます.']
)
expect(tag.reload.name).to eq('spec_tag')
expect(tag.tag_name.aliases.map(&:name)).to contain_exactly('unko')
expect(wiki_page.reload.tag_name).to eq(tag.tag_name)
end
it 'system tag の name は変更できない' do
system_tag = Tag.bot
patch "/tags/#{ system_tag.id }", params: { name: 'patch_system_tag_renamed' }
expect(response).to have_http_status(:unprocessable_entity)
expect(system_tag.reload.name).to eq('bot操作')
end
end
end
@@ -585,6 +699,121 @@ RSpec.describe 'Tags API', type: :request do
expect(row['has_children']).to eq(true)
expect(row['children']).to eq([])
end
it 'passes through deprecated tags when finding children' do
deprecated_middle = Tag.create!(
name: 'depth_deprecated_middle',
category: :character,
deprecated_at: Time.current
)
visible_descendant = Tag.create!(
name: 'depth_visible_descendant',
category: :material
)
TagImplication.create!(parent_tag: root_material, tag: deprecated_middle)
TagImplication.create!(parent_tag: deprecated_middle, tag: visible_descendant)
get '/tags/with-depth', params: { parent: root_material.id }
expect(response).to have_http_status(:ok)
expect(json.map { |item| item.fetch('name') }).to eq(['depth_visible_descendant'])
expect(json.map { |item| item.fetch('name') }).not_to include('depth_deprecated_middle')
end
it 'passes through multiple deprecated tags for roots and has_children' do
active_child = Tag.create!(
name: 'depth_active_child_below_deprecated',
category: :character
)
deprecated_parent = Tag.create!(
name: 'depth_deprecated_parent',
category: :character,
deprecated_at: Time.current
)
deprecated_grandparent = Tag.create!(
name: 'depth_deprecated_grandparent',
category: :material,
deprecated_at: Time.current
)
active_ancestor = Tag.create!(
name: 'depth_active_ancestor',
category: :meme
)
TagImplication.create!(tag: active_child, parent_tag: deprecated_parent)
TagImplication.create!(tag: deprecated_parent, parent_tag: deprecated_grandparent)
TagImplication.create!(tag: deprecated_grandparent, parent_tag: active_ancestor)
get '/tags/with-depth'
root_names = json.map { |item| item.fetch('name') }
expect(root_names).to include('depth_active_ancestor')
expect(root_names).not_to include('depth_active_child_below_deprecated')
ancestor_json = json.find { |item| item.fetch('id') == active_ancestor.id }
expect(ancestor_json.fetch('has_children')).to eq(true)
get '/tags/with-depth', params: { parent: active_ancestor.id }
expect(json.map { |item| item.fetch('name') }).to include(
'depth_active_child_below_deprecated'
)
expect(json.map { |item| item.fetch('name') }).not_to include(
'depth_deprecated_parent',
'depth_deprecated_grandparent'
)
end
it 'treats an active tag with only deprecated ancestors as a root' do
active_child = Tag.create!(
name: 'depth_root_below_deprecated',
category: :character
)
deprecated_parent = Tag.create!(
name: 'depth_root_deprecated_parent',
category: :material,
deprecated_at: Time.current
)
TagImplication.create!(tag: active_child, parent_tag: deprecated_parent)
get '/tags/with-depth'
expect(json.map { |item| item.fetch('name') }).to include(
'depth_root_below_deprecated'
)
expect(json.map { |item| item.fetch('name') }).not_to include(
'depth_root_deprecated_parent'
)
end
it 'terminates when deprecated implications contain a cycle' do
first = Tag.create!(
name: 'depth_cycle_first',
category: :character,
deprecated_at: Time.current
)
second = Tag.create!(
name: 'depth_cycle_second',
category: :material,
deprecated_at: Time.current
)
TagImplication.create!(tag: first, parent_tag: root_material)
TagImplication.create!(tag: second, parent_tag: first)
now = Time.current
TagImplication.insert_all!(
[
{
tag_id: first.id,
parent_tag_id: second.id,
created_at: now,
updated_at: now
}
]
)
get '/tags/with-depth', params: { parent: root_material.id }
expect(response).to have_http_status(:ok)
expect(json).to eq([])
end
end
describe 'GET /tags/name/:name/materials' do
@@ -732,6 +961,20 @@ RSpec.describe 'Tags API', type: :request do
expect(tag.category).to eq('general')
end
it 'deprecated がなければ 422 を返す' do
put "/tags/#{ tag.id }", params: {
name: 'new',
category: 'general',
aliases: '',
parent_tags: '',
}
expect(response).to have_http_status(:unprocessable_entity)
expect(json.fetch('errors')).to include(
'deprecated' => ['廃止状態は必須です.']
)
end
it 'name, category, aliases, parent tags をまとめて更新できる' do
old_parent = Tag.create!(
tag_name: TagName.create!(name: 'put_old_parent'),
@@ -749,6 +992,7 @@ RSpec.describe 'Tags API', type: :request do
category: 'meme',
aliases: 'put_alias_a put_alias_b put_alias_a',
parent_tags: 'put_kept_parent put_new_parent',
deprecated: '0',
}
expect(response).to have_http_status(:ok)
@@ -793,6 +1037,7 @@ RSpec.describe 'Tags API', type: :request do
category: 'general',
aliases: 'spec_tag put_alias_self_test',
parent_tags: '',
deprecated: '0',
}
expect(response).to have_http_status(:ok)
@@ -810,6 +1055,7 @@ RSpec.describe 'Tags API', type: :request do
category: 'general',
aliases: 'unko',
parent_tags: 'spec_tag',
deprecated: '0',
}
expect(response).to have_http_status(:ok)
@@ -825,6 +1071,7 @@ RSpec.describe 'Tags API', type: :request do
category: 'meta',
aliases: '',
parent_tags: '',
deprecated: '0',
}
}.to change(TagVersion, :count).by(2)
@@ -844,6 +1091,42 @@ RSpec.describe 'Tags API', type: :request do
expect(versions.second.created_by_user_id).to eq(member_user.id)
end
it '同じ tag の旧 alias へ戻しても rename できる' do
put "/tags/#{ tag.id }", params: {
name: 'put_roundtrip_b',
category: 'general',
aliases: 'unko',
parent_tags: '',
deprecated: '0',
}
expect(response).to have_http_status(:ok)
put "/tags/#{ tag.id }", params: {
name: 'spec_tag',
category: 'general',
aliases: 'unko',
parent_tags: '',
deprecated: '0',
}
expect(response).to have_http_status(:ok)
tag.reload
expect(tag.name).to eq('spec_tag')
expect(TagName.find_by!(name: 'put_roundtrip_b').canonical).to eq(tag.tag_name)
expect(tag.tag_name.aliases.map(&:name)).to contain_exactly('put_roundtrip_b', 'unko')
expect(tag.tag_name.aliases.map(&:name)).not_to include('spec_tag')
expect(alias_tn.reload.canonical).to eq(tag.tag_name)
version = tag.tag_versions.order(:version_no).last
expect(version.event_type).to eq('update')
expect(version.name).to eq('spec_tag')
expect(version.aliases.split).to contain_exactly('put_roundtrip_b', 'unko')
end
it 'parent tag の snapshot も作成する' do
old_parent = Tag.create!(
tag_name: TagName.create!(name: 'put_snapshot_old_parent'),
@@ -860,6 +1143,7 @@ RSpec.describe 'Tags API', type: :request do
category: 'general',
aliases: 'unko',
parent_tags: new_parent.name,
deprecated: '0',
}
expect(response).to have_http_status(:ok)
@@ -875,6 +1159,7 @@ RSpec.describe 'Tags API', type: :request do
category: 'nico',
aliases: '',
parent_tags: '',
deprecated: '0',
}
}.not_to change(TagVersion, :count)
@@ -896,6 +1181,7 @@ RSpec.describe 'Tags API', type: :request do
category: 'nico',
aliases: '',
parent_tags: '',
deprecated: '0',
}
}.not_to change(NicoTagVersion, :count)
@@ -916,6 +1202,7 @@ RSpec.describe 'Tags API', type: :request do
category: old_category,
aliases: '',
parent_tags: '',
deprecated: '0',
}
}.not_to change(TagVersion, :count)
@@ -946,6 +1233,7 @@ RSpec.describe 'Tags API', type: :request do
category: 'meme',
aliases: 'unko',
parent_tags: '',
deprecated: '0',
}
}
.to change(TagVersion, :count).by(2)
@@ -963,6 +1251,48 @@ RSpec.describe 'Tags API', type: :request do
)
end
it 'wiki を持つ tag を旧 alias へ戻しても wiki を新 canonical へ移す' do
wiki_page =
Wiki::Commit.create_content!(
tag_name: tag.tag_name,
body: 'wiki body before',
created_by_user: member_user,
message: 'init')
expect {
put "/tags/#{ tag.id }", params: {
name: 'put_wiki_roundtrip_b',
category: 'general',
aliases: 'unko',
parent_tags: '',
deprecated: '0',
}
put "/tags/#{ tag.id }", params: {
name: 'spec_tag',
category: 'general',
aliases: 'unko',
parent_tags: '',
deprecated: '0',
}
}
.to change(TagVersion, :count).by(3)
.and change(WikiVersion, :count).by(2)
expect(response).to have_http_status(:ok)
tag.reload
expect(wiki_page.reload.tag_name).to eq(tag.tag_name)
expect(TagName.find_by!(name: 'put_wiki_roundtrip_b').wiki_page).to be_nil
expect(TagName.find_by!(name: 'put_wiki_roundtrip_b').canonical).to eq(tag.tag_name)
versions = wiki_page.wiki_versions.order(:version_no).last(2)
expect(versions.map(&:event_type)).to eq(['update', 'update'])
expect(versions.map(&:title)).to eq(['put_wiki_roundtrip_b', 'spec_tag'])
end
it '別名を他 tag から奪った場合、奪はれた側の tag version も作成する' do
old_owner = Tag.create!(
tag_name: TagName.create!(name: 'put_alias_old_owner'),
@@ -981,6 +1311,7 @@ RSpec.describe 'Tags API', type: :request do
category: 'general',
aliases: 'unko put_stolen_alias',
parent_tags: '',
deprecated: '0',
}
}
.to change { tag.reload.tag_versions.count }.by(2)
@@ -1000,9 +1331,49 @@ RSpec.describe 'Tags API', type: :request do
expect(old_owner_versions.second.aliases.split).not_to include('put_stolen_alias')
end
it 'parent_tags に指定すると循環する tag は 422 にする' do
pending '#332 で対応予定'
it '別 tag の alias 名を rename で奪へる' do
old_owner = Tag.create!(
tag_name: TagName.create!(name: 'put_alias_collision_owner'),
category: :general
)
stolen_alias = TagName.create!(
name: 'put_alias_collision_name',
canonical: old_owner.tag_name
)
wiki_page =
Wiki::Commit.create_content!(
tag_name: tag.tag_name,
body: 'put collision wiki',
created_by_user: member_user,
message: 'init')
put "/tags/#{ tag.id }", params: {
name: 'put_alias_collision_name',
category: 'general',
aliases: 'unko',
parent_tags: '',
deprecated: '0',
}
expect(response).to have_http_status(:ok)
tag.reload
old_owner.reload
stolen_alias.reload
expect(tag.name).to eq('put_alias_collision_name')
expect(stolen_alias.canonical_id).to be_nil
expect(TagName.find_by!(name: 'spec_tag').canonical).to eq(tag.tag_name)
expect(old_owner.tag_name.aliases.map(&:name)).not_to include('put_alias_collision_name')
old_owner_versions = old_owner.tag_versions.order(:version_no)
expect(old_owner_versions.last.event_type).to eq('update')
expect(old_owner_versions.last.aliases.split).not_to include('put_alias_collision_name')
expect(wiki_page.reload.tag_name).to eq(tag.tag_name)
end
it 'parent_tags に指定すると循環する tag は 422 にする' do
child = Tag.create!(
tag_name: TagName.create!(name: 'put_cycle_child'),
category: :general
@@ -1015,6 +1386,7 @@ RSpec.describe 'Tags API', type: :request do
category: 'general',
aliases: 'unko',
parent_tags: child.name,
deprecated: '0',
}
expect(response).to have_http_status(:unprocessable_entity)
@@ -1036,6 +1408,7 @@ RSpec.describe 'Tags API', type: :request do
category: 'general',
aliases: 'unko',
parent_tags: '',
deprecated: '0',
}
}
.to change(TagVersion, :count).by(2)
+17 -2
ファイルの表示
@@ -18,6 +18,13 @@ RSpec.describe 'Wiki API', type: :request do
created_by_user: user,
message: 'init')
end
let!(:tag) do
Tag.create!(
tag_name: tn,
category: :general,
deprecated_at: Time.zone.local(2026, 6, 1)
)
end
describe 'GET /wiki' do
it 'returns wiki pages with title' do
@@ -30,6 +37,8 @@ RSpec.describe 'Wiki API', type: :request do
expect(json[0]).to have_key('title')
expect(json.map { |p| p['title'] }).to include('spec_wiki_title')
wiki_json = json.find { |item| item.fetch('id') == page.id }
expect(wiki_json.fetch('deprecated_at')).to eq(tag.deprecated_at.iso8601(3))
end
end
@@ -48,7 +57,8 @@ RSpec.describe 'Wiki API', type: :request do
expect(json).to include(
'id' => page.id,
'title' => 'spec_wiki_title')
'title' => 'spec_wiki_title',
'deprecated_at' => tag.deprecated_at.iso8601(3))
end
end
@@ -409,7 +419,11 @@ RSpec.describe 'Wiki API', type: :request do
'kind' => 'content',
'message' => 'r2'
)
expect(top['wiki_page']).to include('id' => page.id, 'title' => 'spec_wiki_title')
expect(top['wiki_page']).to include(
'id' => page.id,
'title' => 'spec_wiki_title',
'deprecated_at' => tag.deprecated_at.iso8601(3)
)
expect(top['user']).to include('id' => user.id, 'name' => user.name)
expect(top).to have_key('timestamp')
@@ -479,6 +493,7 @@ RSpec.describe 'Wiki API', type: :request do
expect(json).to include(
'wiki_page_id' => page.id,
'title' => 'spec_wiki_title',
'deprecated_at' => tag.deprecated_at.iso8601(3),
'older_revision_id' => rev_a.id,
'newer_revision_id' => rev_b.id
)
+23
ファイルの表示
@@ -0,0 +1,23 @@
require 'rails_helper'
RSpec.describe MaterialFileSha256 do
describe '.metadata_sha256' do
it 'reads sha256 from Hash metadata' do
blob = instance_double(ActiveStorage::Blob, metadata: { 'sha256' => 'abc123' })
expect(described_class.metadata_sha256(blob)).to eq('abc123')
end
it 'reads sha256 from JSON string metadata' do
blob = instance_double(ActiveStorage::Blob, metadata: '{"sha256":"abc123"}')
expect(described_class.metadata_sha256(blob)).to eq('abc123')
end
it 'returns nil for invalid metadata' do
blob = instance_double(ActiveStorage::Blob, metadata: '{')
expect(described_class.metadata_sha256(blob)).to be_nil
end
end
end
+128
ファイルの表示
@@ -0,0 +1,128 @@
require 'rails_helper'
RSpec.describe MaterialSyncImporter do
let(:user) { create(:user, :member) }
let(:tag) { Tag.create!(tag_name: TagName.create!(name: 'sync_tag'), category: :material) }
def tempfile_for body
Tempfile.new(['material-sync-importer', '.png']).tap do |file|
file.binmode
file.write(body)
file.rewind
end
end
it 'creates an untagged material from a sync candidate with file_downloader' do
result =
described_class.import!(
source_kind: 'google_drive_file',
source_path: '素材/a.png',
source_file_id: 'drive-file-a',
normalized_source_key: 'google_drive_file:drive-file-a',
filename: 'a.png',
content_type: 'image/png',
file_sha256: Digest::SHA256.hexdigest('sync-body'),
file_downloader: -> { tempfile_for('sync-body') },
export_path: '素材/a.png',
tag: nil,
url: nil,
created_by_user: user,
updated_by_user: user)
material = result.material.reload
expect(result.action).to eq(:imported)
expect(material.tag).to be_nil
expect(material.url).to be_nil
expect(material.file).to be_attached
expect(material.file.blob.filename.to_s).to eq('a.png')
expect(material.material_export_items.first.export_path).to eq('素材/a.png')
end
it 'does not clear a human-assigned tag or URL on existing Drive sync' do
material =
Material.create!(tag:, url: 'https://example.com/human',
source_kind: 'google_drive_file',
source_path: '素材/a.png',
source_file_id: 'drive-file-a',
created_by_user: user,
updated_by_user: user)
material.file.attach(
io: StringIO.new('same-body'),
filename: 'a.png',
content_type: 'image/png')
material.file.blob.metadata['sha256'] = Digest::SHA256.hexdigest('same-body')
material.file.blob.save!
MaterialExportItem.create!(material:,
profile: 'legacy_drive',
export_path: '素材/a.png',
created_by_user: user)
result =
described_class.import!(
source_kind: 'google_drive_file',
source_path: '素材/a.png',
source_file_id: 'drive-file-a',
filename: 'a.png',
content_type: 'image/png',
file_sha256: Digest::SHA256.hexdigest('same-body'),
export_path: '素材/a.png',
tag: nil,
url: nil,
updated_by_user: user)
expect(result.action).to eq(:unchanged)
expect(material.reload.tag).to eq(tag)
expect(material.url).to eq('https://example.com/human')
end
it 'suppresses a Google Drive candidate by relative path prefix' do
MaterialSyncSuppression.create!(source_kind: 'google_drive_path_prefix',
drive_path: '素材/危険',
reason: 'copyright_high_risk',
created_by_user: user)
result =
described_class.import!(
source_kind: 'google_drive_file',
source_path: '素材/危険/a.png',
source_file_id: 'drive-file-b',
filename: 'a.png',
content_type: 'image/png',
file_sha256: Digest::SHA256.hexdigest('blocked-body'),
file_downloader: -> { tempfile_for('blocked-body') },
export_path: '素材/危険/a.png',
updated_by_user: user)
expect(result.action).to eq(:suppressed)
expect(result.suppression.normalized_source_key)
.to eq('google_drive_path_prefix:素材/危険')
expect(Material.find_by(source_file_id: 'drive-file-b')).to be_nil
end
it 'rechecks import blocks after downloaded file sha256 is known' do
MaterialImportBlock.create!(match_kind: 'sha256',
sha256: Digest::SHA256.hexdigest('blocked-body'),
reason: 'copyright_high_risk',
created_by_user: user)
blob_count = ActiveStorage::Blob.count
expect do
result =
described_class.import!(
source_kind: 'google_drive_file',
source_path: '素材/blocked.png',
source_file_id: 'drive-file-blocked',
filename: 'blocked.png',
content_type: 'image/png',
file_downloader: -> { tempfile_for('blocked-body') },
export_path: '素材/blocked.png',
updated_by_user: user)
expect(result.action).to eq(:suppressed)
expect(result.suppression.reason).to eq('copyright_high_risk')
end.not_to change(Material, :count)
expect(ActiveStorage::Blob.count).to eq(blob_count)
expect(Material.find_by(source_file_id: 'drive-file-blocked')).to be_nil
end
end
+89
ファイルの表示
@@ -0,0 +1,89 @@
require 'rails_helper'
RSpec.describe MaterialSyncRunner do
let(:user) { create(:user, :member) }
let(:source) do
MaterialSyncSource.create!(
name: 'Drive source',
source_kind: 'google_drive_path',
source_file_id: 'folder-123',
profile: 'legacy_drive',
created_by_user: user,
updated_by_user: user)
end
let(:drive_client) { instance_double(GoogleDrive::ApiClient) }
describe '#sync!' do
it 'imports google drive path candidates as they are yielded' do
first_entry = { id: 'file-1',
name: 'a.png',
mime_type: 'image/png',
relative_path: '素材/a.png',
sha256_checksum: 'sha-a',
web_view_link: 'https://drive.google.com/file/d/file-1/view',
web_content_link: nil }
second_entry = { id: 'file-2',
name: 'b.png',
mime_type: 'image/png',
relative_path: '素材/b.png',
sha256_checksum: 'sha-b',
web_view_link: 'https://drive.google.com/file/d/file-2/view',
web_content_link: nil }
yielded = []
allow(GoogleDrive::ApiClient).to receive(:new).and_return(drive_client)
allow(drive_client).to receive(:each_material_file_under_folder) do |folder_id, &block|
expect(folder_id).to eq('folder-123')
block.call(first_entry)
expect(yielded).to eq(['素材/a.png'])
block.call(second_entry)
end
allow(MaterialSyncImporter).to receive(:import!) do |candidate|
yielded << candidate.fetch(:source_path)
instance_double(MaterialSyncImporter::Result,
action: yielded.last == '素材/a.png' ? :imported : :updated)
end
result = described_class.new(source).sync!
expect(yielded).to eq(['素材/a.png', '素材/b.png'])
expect(result.imported).to eq(1)
expect(result.updated).to eq(1)
expect(result.unchanged).to eq(0)
expect(source.reload.last_synced_at).to be_present
end
it 'logs google drive path progress at first item and summary' do
entry = { id: 'file-1',
name: 'a.png',
mime_type: 'image/png',
relative_path: '素材/a.png',
sha256_checksum: 'sha-a',
web_view_link: 'https://drive.google.com/file/d/file-1/view',
web_content_link: nil }
logged = []
allow(GoogleDrive::ApiClient).to receive(:new).and_return(drive_client)
allow(drive_client).to receive(:each_material_file_under_folder) do |_folder_id, &block|
block.call(entry)
end
allow(MaterialSyncImporter).to receive(:import!)
.and_return(instance_double(MaterialSyncImporter::Result, action: :imported))
allow(Rails.logger).to receive(:info) { |message| logged << JSON.parse(message) }
described_class.new(source).sync!
progress_logs = logged.select { |row| row['folder_id'] == 'folder-123' }
expect(progress_logs.map { |row| row['progress'] }).to eq(['scan', 'summary'])
expect(progress_logs.last).to include(
'material_sync_source_id' => source.id,
'material_sync_source_name' => 'Drive source',
'scanned_count' => 1,
'imported' => 1,
'updated' => 0,
'unchanged' => 0,
'suppressed' => 0,
'failed' => 0)
end
end
end
+5 -2
ファイルの表示
@@ -4,11 +4,12 @@ require 'rails_helper'
RSpec.describe 'post_similarity:calc' do
include RakeTaskHelper
it 'calls Similarity::Calc with Post and :tags' do
it 'calculates similarities from active tags only' do
# 必要最低限のデータ
t1 = Tag.create!(name: "t1")
t2 = Tag.create!(name: "t2")
t3 = Tag.create!(name: "t3")
deprecated_tag = Tag.create!(name: 'deprecated', deprecated_at: Time.current)
p1 = Post.create!(url: "https://example.com/1")
p2 = Post.create!(url: "https://example.com/2")
@@ -22,6 +23,8 @@ RSpec.describe 'post_similarity:calc' do
PostTag.create!(post: p2, tag: t3)
PostTag.create!(post: p3, tag: t3)
PostTag.create!(post: p1, tag: deprecated_tag)
PostTag.create!(post: p2, tag: deprecated_tag)
expect { run_rake_task("post_similarity:calc") }
.to change { PostSimilarity.count }.from(0)
@@ -29,6 +32,6 @@ RSpec.describe 'post_similarity:calc' do
ps = PostSimilarity.find_by!(post_id: p1.id, target_post_id: p2.id)
ps_rev = PostSimilarity.find_by!(post_id: p2.id, target_post_id: p1.id)
expect(ps_rev.cos).to eq(ps.cos)
expect(ps.cos).to be_within(0.0001).of(0.5)
end
end
+5 -2
ファイルの表示
@@ -4,11 +4,12 @@ require 'rails_helper'
RSpec.describe 'tag_similarity:calc' do
include RakeTaskHelper
it 'calls Similarity::Calc with Tag and :posts' do
it 'calculates similarities for active tags only' do
# 必要最低限のデータ
t1 = Tag.create!(name: "t1")
t2 = Tag.create!(name: "t2")
t3 = Tag.create!(name: "t3")
deprecated_tag = Tag.create!(name: 'deprecated', deprecated_at: Time.current)
p1 = Post.create!(url: "https://example.com/1")
p2 = Post.create!(url: "https://example.com/2")
@@ -22,6 +23,7 @@ RSpec.describe 'tag_similarity:calc' do
PostTag.create!(post: p2, tag: t3)
PostTag.create!(post: p3, tag: t3)
PostTag.create!(post: p1, tag: deprecated_tag)
expect { run_rake_task("tag_similarity:calc") }
.to change { TagSimilarity.count }.from(0)
@@ -29,6 +31,7 @@ RSpec.describe 'tag_similarity:calc' do
ps = TagSimilarity.find_by!(tag_id: t1.id, target_tag_id: t2.id)
ps_rev = TagSimilarity.find_by!(tag_id: t2.id, target_tag_id: t1.id)
expect(ps_rev.cos).to eq(ps.cos)
expect(TagSimilarity.where(tag_id: deprecated_tag.id)).to be_empty
expect(TagSimilarity.where(target_tag_id: deprecated_tag.id)).to be_empty
end
end
+9
ファイルの表示
@@ -10,6 +10,15 @@ bundle exec rspec
bundle exec rails routes
```
Material video thumbnail generation requires `ffmpeg` on the host. If `ffmpeg`
is missing or frame extraction fails, `MaterialThumbnailGenerator` logs the
result and stderr, then leaves the existing thumbnail unchanged.
```sh
cd backend
bundle exec rails materials:thumbnails:backfill
```
## Frontend
```sh
バイナリファイルは表示されません.

変更前

幅:  |  高さ:  |  サイズ: 559 KiB

バイナリファイルは表示されません.

変更前

幅:  |  高さ:  |  サイズ: 146 KiB

バイナリファイルは表示されません.

変更前

幅:  |  高さ:  |  サイズ: 1.2 MiB

バイナリファイルは表示されません.

変更前

幅:  |  高さ:  |  サイズ: 188 KiB

バイナリファイルは表示されません.

変更前

幅:  |  高さ:  |  サイズ: 201 KiB

バイナリファイルは表示されません.

変更前

幅:  |  高さ:  |  サイズ: 196 KiB

バイナリファイルは表示されません.

変更前

幅:  |  高さ:  |  サイズ: 179 KiB

+4 -1
ファイルの表示
@@ -14,8 +14,10 @@ import { apiPost, isApiError } from '@/lib/api'
import DeerjikistDetailPage from '@/pages/deerjikists/DeerjikistDetailPage'
import MaterialBasePage from '@/pages/materials/MaterialBasePage'
import MaterialDetailPage from '@/pages/materials/MaterialDetailPage'
import MaterialHistoryPage from '@/pages/materials/MaterialHistoryPage'
import MaterialListPage from '@/pages/materials/MaterialListPage'
import MaterialNewPage from '@/pages/materials/MaterialNewPage'
import MaterialSyncSuppressionsPage from '@/pages/materials/MaterialSyncSuppressionsPage'
// import MaterialSearchPage from '@/pages/materials/MaterialSearchPage'
import MorePage from '@/pages/MorePage'
import GekanatorPage from '@/pages/GekanatorPage'
@@ -68,10 +70,11 @@ const RouteTransitionWrapper = ({ user, setUser }: {
<Route path="/theatres/:id" element={<TheatreDetailPage user={user}/>}/>
<Route path="/materials" element={<MaterialBasePage/>}>
<Route index element={<MaterialListPage/>}/>
<Route path="changes" element={<MaterialHistoryPage/>}/>
<Route path="new" element={<MaterialNewPage/>}/>
<Route path="suppressions" element={<MaterialSyncSuppressionsPage/>}/>
<Route path=":id" element ={<MaterialDetailPage/>}/>
</Route>
{/* <Route path="/materials/search" element={<MaterialSearchPage/>}/> */}
<Route path="/wiki" element={<WikiSearchPage/>}/>
<Route path="/wiki/:title" element={<WikiDetailPage/>}/>
<Route path="/wiki/new" element={<WikiNewPage user={user}/>}/>
+481 -74
ファイルの表示
@@ -1,23 +1,43 @@
import { Fragment, useEffect, useState } from 'react'
import { useQuery } from '@tanstack/react-query'
import { useEffect, useRef, useState } from 'react'
import { useLocation, useNavigate } from 'react-router-dom'
import TagLink from '@/components/TagLink'
import SidebarComponent from '@/components/layout/SidebarComponent'
import { apiGet } from '@/lib/api'
import TagLink from '@/components/TagLink'
import { fetchMaterialTagTree, parseMaterialFilter } from '@/lib/materials'
import { materialsKeys } from '@/lib/queryKeys'
import { cn } from '@/lib/utils'
import type { FC, ReactNode } from 'react'
import type { CSSProperties, Dispatch, FC, ReactNode, SetStateAction } from 'react'
import type { Tag } from '@/types'
import type { MaterialFilter, MaterialSidebarTag, Tag } from '@/types'
type TagWithDepth = Tag & {
hasChildren: boolean
children: TagWithDepth[] }
const FILTERS: MaterialFilter[] = ['missing', 'present', 'any']
const FILTER_LABELS: Record<MaterialFilter, string> = { present: '有', missing: '無', any: '全' }
const px = (value: string): number => {
const parsed = Number.parseFloat (value)
return Number.isFinite (parsed) ? parsed : 0
}
const verticalChrome = (el: HTMLElement): number => {
const style = window.getComputedStyle (el)
return (
px (style.paddingTop)
+ px (style.paddingBottom)
+ px (style.borderTopWidth)
+ px (style.borderBottomWidth))
}
const setChildrenById = (
tags: TagWithDepth[],
tags: MaterialSidebarTag[],
targetId: number,
children: TagWithDepth[],
): TagWithDepth[] => (
children: MaterialSidebarTag[],
): MaterialSidebarTag[] => (
tags.map (tag => {
if (tag.id === targetId)
return { ...tag, children }
@@ -25,75 +45,462 @@ const setChildrenById = (
if (tag.children.length === 0)
return tag
return { ...tag,
children: (setChildrenById (tag.children, targetId, children)
.filter (t => t.category !== 'meme' || t.hasChildren)) }
return { ...tag, children: setChildrenById (tag.children, targetId, children) }
}))
const MaterialSidebar: FC = () => {
const [tags, setTags] = useState<TagWithDepth[]> ([])
const [openTags, setOpenTags] = useState<Record<number, boolean>> ({ })
const [tagFetchedFlags, setTagFetchedFlags] = useState<Record<number, boolean>> ({ })
const materialPath = (
tagId: number,
materialFilter: MaterialFilter,
): string =>
`/materials?tag_id=${ tagId }&include_descendants=1&group_by=parent_tag`
+ `&material_filter=${ materialFilter }`
useEffect (() => {
void (async () => {
setTags ((await apiGet<TagWithDepth[]> ('/tags/with-depth'))
.filter (t => t.category !== 'meme' || t.hasChildren))
}) ()
}, [])
const renderTags = (ts: TagWithDepth[], nestLevel = 0): ReactNode => (
ts.map (t => (
<Fragment key={t.id}>
<li>
<div className="flex">
<div className="flex-none w-4">
{t.hasChildren && (
<a
href="#"
onClick={async e => {
e.preventDefault ()
if (!(tagFetchedFlags[t.id]))
{
try
{
const data =
await apiGet<TagWithDepth[]> (
'/tags/with-depth', { params: { parent: String (t.id) } })
setTags (prev => setChildrenById (prev, t.id, data))
setTagFetchedFlags (prev => ({ ...prev, [t.id]: true }))
}
catch
{
;
}
}
setOpenTags (prev => ({ ...prev, [t.id]: !(prev[t.id]) }))
}}>
{openTags[t.id] ? <>&minus;</> : '+'}
</a>)}
</div>
<div className="flex-1 truncate">
<TagLink
tag={t}
nestLevel={nestLevel}
title={t.name}
withCount={false}
withWiki={false}
to={`/materials?tag=${ encodeURIComponent (t.name) }`}/>
</div>
</div>
</li>
{openTags[t.id] && renderTags (t.children, nestLevel + 1)}
</Fragment>)))
const sidebarTagToTag = (tag: MaterialSidebarTag): Tag => ({
id: tag.id,
name: tag.name,
category: tag.category,
deprecatedAt: tag.deprecated ? '' : null,
aliases: [],
parents: [],
postCount: 0,
createdAt: '',
updatedAt: '',
hasWiki: false,
materialId: null,
hasDeerjikists: false,
matchedAlias: null })
return (
<SidebarComponent>
<ul>
{renderTags (tags)}
</ul>
</SidebarComponent>)
const tagSelectionShellClass = (selected: boolean): string =>
cn (selected
? ['rounded-md border border-sky-500 bg-sky-50 px-2 py-1 text-sky-700',
'dark:border-sky-400 dark:bg-sky-950 dark:text-sky-100']
: 'px-2 py-1')
const updateMaterialFilterQuery = (
pathname: string,
locationSearch: string,
navigate: ReturnType<typeof useNavigate>,
materialFilter: MaterialFilter,
) => {
const qs = new URLSearchParams (locationSearch)
qs.set ('material_filter', materialFilter)
navigate (`${ pathname }${ qs.toString () ? `?${ qs.toString () }` : '' }`)
}
const MaterialFilterButtons: FC<{ materialFilter: MaterialFilter
onChange: (materialFilter: MaterialFilter) => void }> = (
{ materialFilter, onChange },
) => (
<div className="flex flex-wrap gap-2 justify-end md:justify-start flex-center">
<label className="my-auto text-sm font-bold"></label>
{FILTERS.map (value => (
<button
key={value}
type="button"
onClick={() => onChange (value)}
className={cn (
'rounded-full border px-3 py-1 text-sm',
(materialFilter === value
? ['border-sky-500 bg-sky-50 text-sky-700 dark:border-sky-400',
'dark:bg-sky-950 dark:text-sky-100']
: ['border-neutral-300 bg-white text-neutral-700 dark:border-stone-700',
'dark:bg-stone-900 dark:text-stone-200']))}>
{FILTER_LABELS[value]}
</button>))}
</div>)
const MaterialTreeNode: FC<{
materialFilter: MaterialFilter
selectedTagId: number | null
nestLevel?: number
onChildren: (tagId: number, children: MaterialSidebarTag[]) => void
openTags: Record<number, boolean>
setOpenTags: Dispatch<SetStateAction<Record<number, boolean>>>
tag: MaterialSidebarTag
}> = ({ materialFilter, nestLevel = 0, onChildren, openTags, selectedTagId,
setOpenTags, tag }) => {
const open = Boolean (openTags[tag.id])
const { data } = useQuery ({
queryKey: materialsKeys.tree ({ parentId: tag.id, materialFilter }),
queryFn: () => fetchMaterialTagTree ({ parentId: tag.id, materialFilter }),
enabled: open && tag.hasChildren && tag.children.length === 0})
useEffect (() => {
if (open && data && tag.children.length === 0)
onChildren (tag.id, data)
}, [data, onChildren, open, tag.children.length, tag.id])
return (
<>
<li>
<div className="flex flex-center">
<div className="flex-none w-4 my-auto">
{tag.hasChildren && (
<button
type="button"
onClick={() => setOpenTags (prev => ({ ...prev, [tag.id]: !prev[tag.id] }))}
className="text-neutral-500 dark:text-stone-400">
{open ? <>&minus;</> : '+'}
</button>)}
</div>
<div className="min-w-0 flex-1 my-auto">
<div className={cn (tagSelectionShellClass (selectedTagId === tag.id),
'min-w-0 truncate')}>
<TagLink
tag={sidebarTagToTag (tag)}
nestLevel={nestLevel}
title={tag.name}
withCount={false}
withWiki={false}
to={materialPath (tag.id, materialFilter)}/>
</div>
</div>
</div>
</li>
{open && tag.children.length > 0 && (
<ul>
{tag.children.map (child => (
<MaterialTreeNode
key={child.id}
tag={child}
nestLevel={nestLevel + 1}
materialFilter={materialFilter}
selectedTagId={selectedTagId}
openTags={openTags}
setOpenTags={setOpenTags}
onChildren={onChildren}/>))}
</ul>)}
</>)
}
const MobileMaterialTreeNode: FC<{ depth?: number
availableInlineSizePx?: number | null
materialFilter: MaterialFilter
selectedTagId: number | null
onChildren: (tagId: number, children: MaterialSidebarTag[]) =>
void
openTags: Record<number, boolean>
setOpenTags: Dispatch<SetStateAction<Record<number, boolean>>>
tag: MaterialSidebarTag }> = (
{
depth = 0,
availableInlineSizePx = null,
materialFilter,
onChildren,
openTags,
selectedTagId,
setOpenTags,
tag,
},
) => {
const open = Boolean (openTags[tag.id])
const tagColumnRef = useRef<HTMLDivElement | null> (null)
const chipRef = useRef<HTMLDivElement | null> (null)
const buttonRef = useRef<HTMLButtonElement | null> (null)
const expansionSlotRef = useRef<HTMLDivElement | null> (null)
const expansionBorderRef = useRef<HTMLDivElement | null> (null)
const [tagChipInlineSizePx, setTagChipInlineSizePx] = useState<number | null> (null)
const [tagLinkInlineSizePx, setTagLinkInlineSizePx] = useState<number | null> (null)
const [childAvailableInlineSizePx, setChildAvailableInlineSizePx] = useState<number | null> (null)
const { data } = useQuery ({
queryKey: materialsKeys.tree ({ parentId: tag.id, materialFilter }),
queryFn: () => fetchMaterialTagTree ({ parentId: tag.id, materialFilter }),
enabled: open && tag.hasChildren && tag.children.length === 0})
useEffect (() => {
if (open && data && tag.children.length === 0)
onChildren (tag.id, data)
}, [data, onChildren, open, tag.children.length, tag.id])
useEffect (() => {
const tagColumn = tagColumnRef.current
const chip = chipRef.current
if (!(tagColumn) || !(chip))
return
const updateTagInlineSize = () => {
const buttonHeight = buttonRef.current?.offsetHeight ?? 0
const gap = tag.hasChildren ? px (window.getComputedStyle (tagColumn).rowGap) : 0
const columnHeight =
availableInlineSizePx == null
? tagColumn.clientHeight
: Math.min (tagColumn.clientHeight, availableInlineSizePx)
const nextChipInlineSize = Math.max (24, columnHeight - buttonHeight - gap)
const nextLinkInlineSize = Math.max (
16,
nextChipInlineSize - verticalChrome (chip),
)
setTagChipInlineSizePx (prev => prev === nextChipInlineSize ? prev : nextChipInlineSize)
setTagLinkInlineSizePx (prev => prev === nextLinkInlineSize ? prev : nextLinkInlineSize)
}
updateTagInlineSize ()
const resizeObserver = new ResizeObserver (() => {
updateTagInlineSize ()
})
resizeObserver.observe (tagColumn)
resizeObserver.observe (chip)
if (buttonRef.current)
resizeObserver.observe (buttonRef.current)
return () => {
resizeObserver.disconnect ()
}
}, [availableInlineSizePx, open, tag.hasChildren, tag.children.length])
useEffect (() => {
const expansionSlot = expansionSlotRef.current
const expansionBorder = expansionBorderRef.current
if (!(expansionSlot) || !(expansionBorder))
return
const updateChildInlineSize = () => {
const base =
availableInlineSizePx == null
? expansionSlot.clientHeight
: availableInlineSizePx
const chrome = verticalChrome (expansionSlot) + verticalChrome (expansionBorder)
const nextInlineSize = Math.max (24, base - chrome)
setChildAvailableInlineSizePx (prev => prev === nextInlineSize ? prev : nextInlineSize)
}
updateChildInlineSize ()
const resizeObserver = new ResizeObserver (() => {
updateChildInlineSize ()
})
resizeObserver.observe (expansionSlot)
resizeObserver.observe (expansionBorder)
return () => {
resizeObserver.disconnect ()
}
}, [availableInlineSizePx, open, tag.children.length])
return (
<div className="flex h-full min-h-0 max-h-full flex-row-reverse items-start gap-2
overflow-hidden">
<div
ref={tagColumnRef}
className="flex h-full min-h-0 max-h-full flex-col items-center gap-1
overflow-hidden">
<div
ref={chipRef}
className={cn (
tagSelectionShellClass (selectedTagId === tag.id),
'box-border rounded-xl border px-3 py-2 text-sm shadow-sm',
'min-h-0 overflow-hidden [max-inline-size:var(--tag-chip-inline-size)]',
'[max-height:var(--tag-chip-inline-size)]',
)}
style={{
writingMode: 'vertical-rl',
'--tag-chip-inline-size': (
tagChipInlineSizePx == null
? undefined
: `${ tagChipInlineSizePx }px`),
'--tag-link-inline-size': (
tagLinkInlineSizePx == null
? undefined
: `${ tagLinkInlineSizePx }px`),
} as CSSProperties}>
<TagLink
tag={sidebarTagToTag (tag)}
title={tag.name}
withCount={false}
withWiki={false}
to={materialPath (tag.id, materialFilter)}
className="block overflow-hidden text-ellipsis whitespace-nowrap
[max-inline-size:var(--tag-link-inline-size)]
[max-height:var(--tag-link-inline-size)]"/>
</div>
{tag.hasChildren && (
<button
ref={buttonRef}
type="button"
onClick={() => setOpenTags (prev => ({ ...prev, [tag.id]: !prev[tag.id] }))}
className="flex-none rounded-full border border-stone-300 bg-white
px-2 py-0.5 text-sm text-stone-700 dark:border-stone-700
dark:bg-stone-900 dark:text-stone-100">
{open ? <>&minus;</> : '+'}
</button>)}
</div>
{open && tag.children.length > 0 && (
<div
ref={expansionSlotRef}
className={cn (
'h-full min-h-0 max-h-full overflow-hidden box-border',
depth === 0 ? 'pt-5' : 'pt-3')}>
<div
ref={expansionBorderRef}
className="relative max-h-full overflow-hidden rounded-2xl border border-stone-200
bg-stone-100/70 py-2 pl-2 pr-2 text-stone-900
dark:border-stone-700 dark:bg-stone-900/70 dark:text-stone-100">
<div className="flex h-full min-h-0 max-h-full flex-row-reverse items-start gap-2
overflow-hidden">
<span
aria-hidden="true"
className="absolute -right-3 top-4 h-px w-3 bg-stone-300 dark:bg-stone-600"/>
{tag.children.map (child => (
<div
key={child.id}
className="relative h-full min-h-0 max-h-full overflow-hidden">
<MobileMaterialTreeNode
tag={child}
depth={depth + 1}
availableInlineSizePx={childAvailableInlineSizePx}
materialFilter={materialFilter}
selectedTagId={selectedTagId}
openTags={openTags}
setOpenTags={setOpenTags}
onChildren={onChildren}/>
</div>))}
</div>
</div>
</div>)}
</div>)
}
const MaterialSidebar: FC = () => {
const location = useLocation ()
const navigate = useNavigate ()
const qs = new URLSearchParams (location.search)
const materialFilter = parseMaterialFilter (qs.get ('material_filter'), 'any')
const selectedTagId = Number (qs.get ('tag_id') ?? 0) || null
const [desktopTags, setDesktopTags] = useState<MaterialSidebarTag[]> ([])
const [openTags, setOpenTags] = useState<Record<number, boolean>> ({ })
const mobileRailRef = useRef<HTMLDivElement | null> (null)
const [mobileAvailableInlineSizePx, setMobileAvailableInlineSizePx] =
useState<number | null> (null)
const { data: rootTags = [], isLoading, isError } = useQuery ({
queryKey: materialsKeys.tree ({ parentId: null, materialFilter }),
queryFn: () => fetchMaterialTagTree ({ parentId: null, materialFilter })})
useEffect (() => {
setDesktopTags (rootTags)
}, [rootTags])
useEffect (() => {
const el = mobileRailRef.current
if (!(el))
return
requestAnimationFrame (() => {
el.scrollLeft = el.scrollWidth
})
}, [rootTags, materialFilter])
useEffect (() => {
const el = mobileRailRef.current
if (!(el))
return
const updateAvailableInlineSize = () => {
const nextInlineSize = Math.max (24, el.clientHeight)
setMobileAvailableInlineSizePx (prev => prev === nextInlineSize ? prev : nextInlineSize)
}
updateAvailableInlineSize ()
const resizeObserver = new ResizeObserver (() => {
updateAvailableInlineSize ()
})
resizeObserver.observe (el)
return () => {
resizeObserver.disconnect ()
}
}, [rootTags, materialFilter])
const visibleRootTags = desktopTags.length > 0 ? desktopTags : rootTags
const setChildren = (tagId: number, children: MaterialSidebarTag[]) => {
setDesktopTags (prev => {
const base = prev.length > 0 ? prev : rootTags
return setChildrenById (base, tagId, children)
})
}
const handleFilterChange = (value: MaterialFilter) => {
setDesktopTags ([])
setOpenTags ({ })
updateMaterialFilterQuery (location.pathname, location.search, navigate, value)
}
const renderDesktopTree = (tags: MaterialSidebarTag[]): ReactNode => (
tags.map (tag => (
<MaterialTreeNode
key={tag.id}
tag={tag}
materialFilter={materialFilter}
selectedTagId={selectedTagId}
openTags={openTags}
setOpenTags={setOpenTags}
onChildren={setChildren}/>)))
return (
<>
<div className="border-b bg-stone-50 p-3 dark:border-stone-700 dark:bg-stone-950
dark:text-stone-100 md:hidden flex h-[25dvh] min-h-0 flex-col
overflow-hidden">
<MaterialFilterButtons
materialFilter={materialFilter}
onChange={handleFilterChange}/>
<div
ref={mobileRailRef}
className="mt-3 min-h-0 flex-1 overflow-x-auto overflow-y-hidden">
<div className="flex min-w-max flex-row-reverse items-start gap-3 pb-1 h-full
min-h-0 max-h-full">
{visibleRootTags.map (tag => (
<MobileMaterialTreeNode
key={tag.id}
tag={tag}
availableInlineSizePx={mobileAvailableInlineSizePx}
materialFilter={materialFilter}
selectedTagId={selectedTagId}
openTags={openTags}
setOpenTags={setOpenTags}
onChildren={setChildren}/>))}
</div>
</div>
</div>
<div className="hidden md:block">
<SidebarComponent>
<div className="space-y-4">
<MaterialFilterButtons
materialFilter={materialFilter}
onChange={handleFilterChange}/>
{isLoading && (
<p className="text-sm text-neutral-500 dark:text-stone-400"></p>)}
{isError && (
<p className="text-sm text-red-600 dark:text-red-300">
</p>)}
{(!isLoading && !isError) && (
<ul>
{renderDesktopTree (visibleRootTags)}
</ul>)}
</div>
</SidebarComponent>
</div>
</>)
}
export default MaterialSidebar
+1 -2
ファイルの表示
@@ -107,8 +107,7 @@ export default forwardRef ((props: Props, ref: ForwardedRef<NiconicoViewerHandle
loadCompleteTimerRef.current = setTimeout (() => {
onError?.({
eventName: 'loadCompleteTimeout',
reason: 'niconico video length was not reported by embed',
})
reason: 'niconico video length was not reported by embed'})
}, LOAD_COMPLETE_TIMEOUT_MS)
}, [clearLoadCompleteTimer, onError])
+1 -2
ファイルの表示
@@ -19,8 +19,7 @@ const PostOriginalCreatedTimeField: FC<Props> = (
setOriginalCreatedFrom,
originalCreatedBefore,
setOriginalCreatedBefore,
errors }: Props,
) => (
errors }: Props) => (
<FormField label="オリジナルの作成日時" messages={errors}>
{({ describedBy, invalid }) => (
<>
+5 -10
ファイルの表示
@@ -37,8 +37,7 @@ const renderTagTree = (
path: string,
suppressClickRef: MutableRefObject<boolean>,
parentTagId?: number,
sp?: boolean,
): ReactNode[] => {
sp?: boolean): ReactNode[] => {
const key = `${ path }-${ tag.id }`
const self = (
@@ -64,8 +63,7 @@ const renderTagTree = (
const isDescendant = (
root: Tag,
targetId: number,
): boolean => {
targetId: number): boolean => {
if (!(root.children))
return false
@@ -83,8 +81,7 @@ const isDescendant = (
const findTag = (
byCat: TagByCategory,
id: number,
): Tag | undefined => {
id: number): Tag | undefined => {
const walk = (nodes: Tag[]): Tag | undefined => {
for (const t of nodes)
{
@@ -130,8 +127,7 @@ const buildTagByCategory = (post: Post): TagByCategory => {
const changeCategory = async (
tagId: number,
category: Category,
): Promise<void> => {
category: Category): Promise<void> => {
await apiPatch (`/tags/${ tagId }`, { category })
}
@@ -294,8 +290,7 @@ const TagDetailSidebar: FC<Props> = ({ post, sp }) => {
addEventListener ('click', e => {
e.preventDefault ()
e.stopPropagation ()
suppressClickRef.current = false
}, { capture: true, once: true })
suppressClickRef.current = false}, { capture: true, once: true })
}}
onDragCancel={() => {
setActiveTagId (null)
+15
ファイルの表示
@@ -18,6 +18,21 @@ describe ('TagLink', () => {
expect (screen.getByText ('4')).toBeInTheDocument ()
})
it ('does not append deprecated state to the rendered tag name', () => {
renderWithProviders (
<TagLink
tag={buildTag ({
name: '旧タグ',
deprecatedAt: '2026-06-01T00:00:00.000Z',
})}
withWiki={false}
withCount={false}/>,
)
expect (screen.getByRole ('link', { name: '旧タグ' })).toBeInTheDocument ()
expect (screen.queryByText ('(廃止)')).not.toBeInTheDocument ()
})
it ('links wiki markers to the correct detail route', () => {
renderWithProviders (
<TagLink tag={buildTag ({ hasWiki: true, name: 'a/b' })}/>,
+9 -8
ファイルの表示
@@ -28,11 +28,12 @@ type Props =
const TagLink: FC<Props> = ({ tag,
nestLevel = 0,
linkFlg = true,
withWiki = true,
withCount = true,
...props }) => {
nestLevel = 0,
linkFlg = true,
withWiki = true,
withCount = true,
className,
...props }) => {
const spanClass = cn (
`text-${ TAG_COLOUR[tag.category] }-${ LIGHT_COLOUR_SHADE }`,
`dark:text-${ TAG_COLOUR[tag.category] }-${ DARK_COLOUR_SHADE }`)
@@ -105,7 +106,7 @@ const TagLink: FC<Props> = ({ tag,
</span>)}
{tag.matchedAlias != null && (
<>
<span className={spanClass} {...props}>
<span className={cn (spanClass, className)} {...props}>
{tag.matchedAlias}
</span>
<> </>
@@ -114,12 +115,12 @@ const TagLink: FC<Props> = ({ tag,
? (
<PrefetchLink
to={`/posts?${ (new URLSearchParams ({ tags: tag.name })).toString () }`}
className={linkClass}
className={cn (linkClass, className)}
{...props}>
{tag.name}
</PrefetchLink>)
: (
<span className={spanClass}
<span className={cn (spanClass, className)}
{...props}>
{tag.name}
</span>)}
+34 -14
ファイルの表示
@@ -7,30 +7,36 @@ import Separator from '@/components/MenuSeparator'
import PrefetchLink from '@/components/PrefetchLink'
import TopNavUser from '@/components/TopNavUser'
import { WikiIdBus } from '@/lib/eventBus/WikiIdBus'
import { tagsKeys, wikiKeys } from '@/lib/queryKeys'
import { materialsKeys, tagsKeys, wikiKeys } from '@/lib/queryKeys'
import { fetchTag, fetchTagByName } from '@/lib/tags'
import { fetchMaterial } from '@/lib/materials'
import { cn } from '@/lib/utils'
import { fetchWikiPage } from '@/lib/wiki'
import type { FC, MouseEvent } from 'react'
import type { Menu, MenuVisibleItem, Tag, User } from '@/types'
import type { Material, Menu, MenuVisibleItem, Tag, User } from '@/types'
type Props = { user: User | null }
export const menuOutline = ({ tag, wikiId, user, pathName }: {
tag?: Tag | null
wikiId: number | null
user: User | null,
pathName: string }): Menu => {
const postCount = tag?.postCount ?? 0
export const menuOutline = (
{ tag, material, wikiId, user, pathName }: {
tag?: Tag | null
material?: Material | null
wikiId: number | null
user: User | null,
pathName: string },
): Menu => {
const postCount = tag?.postCount ?? material?.tag?.postCount ?? 0
const wikiPageFlg = Boolean (/^\/wiki\/(?!new|changes)[^/]+/.test (pathName) && wikiId)
const wikiTitle = pathName.split ('/')[2] ?? ''
const tagFlg = /^\/tags\/\d+/.test (pathName)
const materialFlg = /^\/materials\/\d+/.test (pathName)
return [
{ name: '広場', to: '/posts', subMenu: [
{ name: '一覧', to: '/posts' },
@@ -49,12 +55,18 @@ export const menuOutline = ({ tag, wikiId, user, pathName }: {
visible: tagFlg },
{ name: '履歴', to: `/tags/changes?id=${ tag?.id }`,
visible: tagFlg && tag?.category !== 'nico' }] },
{ name: '素材', to: '/materials', visible: false, subMenu: [
{ name: '素材', to: '/materials', visible: true, subMenu: [
{ name: '一覧', to: '/materials' },
{ name: '検索', to: '/materials/search', visible: false },
{ name: '追加', to: '/materials/new' },
{ name: '全体履歴', to: '/materials/changes', visible: false },
{ name: 'ヘルプ', to: '/wiki/ヘルプ:素材集' }] },
{ name: '抑止', to: '/materials/suppressions' },
{ name: '全体履歴', to: '/materials/changes' },
{ name: 'ヘルプ', to: '/wiki/ヘルプ:素材管理' },
{ component: <Separator/>, visible: materialFlg },
{ name: `広場 (${ postCount || 0 })`,
to: `/posts?tags=${ encodeURIComponent (material?.tag?.name ?? '') }`,
visible: materialFlg && Boolean (material?.tag) },
{ name: '履歴', to: `/materials/changes?material_id=${ material?.id }`,
visible: materialFlg }] },
{ name: 'Wiki', to: '/wiki/ヘルプ:ホーム', base: '/wiki', subMenu: [
{ name: '検索', to: '/wiki' },
{ name: '新規', to: '/wiki/new' },
@@ -119,15 +131,23 @@ const TopNav: FC<Props> = ({ user }) => {
queryFn: () => fetchWikiPage (wikiIdStr, { }) })
const tagFlg = /^\/tags\/\d+/.test (location.pathname)
const effectiveTitle = (tagFlg ? location.pathname.split ('/')[2] : wikiPage?.title) ?? ''
const materialFlg = /^\/materials\/\d+/.test (location.pathname)
const effectiveTitle = (((tagFlg || materialFlg)
? location.pathname.split ('/')[2]
: wikiPage?.title)
?? '')
const { data: tag } = useQuery ({
enabled: Boolean (effectiveTitle),
queryKey: tagsKeys.show (effectiveTitle),
queryFn: () => (tagFlg ? fetchTag : fetchTagByName) (effectiveTitle) })
const { data: material } = useQuery ({
enabled: Boolean (effectiveTitle),
queryKey: materialsKeys.show (effectiveTitle),
queryFn: () => fetchMaterial (effectiveTitle) })
const menu = menuOutline ({ tag, wikiId, user, pathName: location.pathname })
const menu = menuOutline ({ tag, material, wikiId, user, pathName: location.pathname })
const visibleMenu = menu.filter ((item): item is MenuVisibleItem => item.visible ?? true)
const moreMenu = menu.filter (item =>
!(item.visible ?? true)
+5 -3
ファイルの表示
@@ -1,12 +1,14 @@
import React from 'react'
import { cn } from '@/lib/utils'
import type { FC } from 'react'
type Props = { children: React.ReactNode }
type Props = { children: React.ReactNode; className?: string }
const PageTitle: FC<Props> = ({ children }) => (
<h1 className="text-2xl font-bold mb-2">
const PageTitle: FC<Props> = ({ children, className, ...rest }) => (
<h1 className={cn ('text-2xl font-bold mb-2', className)} {...rest}>
{children}
</h1>)
+1 -2
ファイルの表示
@@ -16,8 +16,7 @@ const range = (start: number, end: number): number[] =>
const getPages = (
page: number,
total: number,
siblingCount: number,
): (number | '…')[] => {
siblingCount: number): (number | '…')[] => {
if (total <= 1)
return [1]
+1 -2
ファイルの表示
@@ -103,8 +103,7 @@ const DialogueProvider: FC<Props> = ({ children }) => {
choice: options => new Promise (resolve => {
push ({ kind: 'choice',
options: options as ChoiceOptions<string>,
resolve: resolve as (value: string | null) => void })
}) }), [push])
resolve: resolve as (value: string | null) => void })}) }), [push])
const active = queue[0]
+23 -31
ファイルの表示
@@ -10,41 +10,35 @@ const buttonVariants = cva (
'rounded-md text-sm font-medium transition-colors',
'focus-visible:outline-none focus-visible:ring-2 focus-visible:ring-slate-400',
'disabled:pointer-events-none disabled:opacity-50',
'[&_svg]:pointer-events-none [&_svg]:size-4 [&_svg]:shrink-0',
].join (' '),
'[&_svg]:pointer-events-none [&_svg]:size-4 [&_svg]:shrink-0'].join (' '),
{
variants: {
variant: {
default:
'bg-slate-900 text-white hover:bg-slate-700 dark:bg-slate-100 dark:text-slate-900 dark:hover:bg-slate-300',
default:
'bg-slate-900 text-white hover:bg-slate-700 dark:bg-slate-100 dark:text-slate-900 dark:hover:bg-slate-300',
destructive:
'bg-red-600 text-white hover:bg-red-700 dark:bg-red-700 dark:hover:bg-red-600',
destructive:
'bg-red-600 text-white hover:bg-red-700 dark:bg-red-700 dark:hover:bg-red-600',
outline:
'border border-slate-300 bg-white text-slate-900 hover:bg-slate-100 dark:border-slate-700 dark:bg-slate-900 dark:text-slate-100 dark:hover:bg-slate-800',
outline:
'border border-slate-300 bg-white text-slate-900 hover:bg-slate-100 dark:border-slate-700 dark:bg-slate-900 dark:text-slate-100 dark:hover:bg-slate-800',
secondary:
'bg-slate-100 text-slate-900 hover:bg-slate-200 dark:bg-slate-800 dark:text-slate-100 dark:hover:bg-slate-700',
secondary:
'bg-slate-100 text-slate-900 hover:bg-slate-200 dark:bg-slate-800 dark:text-slate-100 dark:hover:bg-slate-700',
ghost:
'text-slate-900 hover:bg-slate-100 dark:text-slate-100 dark:hover:bg-slate-800',
ghost:
'text-slate-900 hover:bg-slate-100 dark:text-slate-100 dark:hover:bg-slate-800',
link:
'text-blue-700 underline-offset-4 hover:underline dark:text-blue-300',
},
link:
'text-blue-700 underline-offset-4 hover:underline dark:text-blue-300'},
size: {
default: 'h-10 px-4 py-2',
sm: 'h-9 rounded-md px-3',
lg: 'h-11 rounded-md px-8',
icon: 'h-10 w-10',
},
},
default: 'h-10 px-4 py-2',
sm: 'h-9 rounded-md px-3',
lg: 'h-11 rounded-md px-8',
icon: 'h-10 w-10'}},
defaultVariants: {
variant: 'default',
size: 'default',
},
})
size: 'default'}})
export interface ButtonProps
extends React.ButtonHTMLAttributes<HTMLButtonElement>,
@@ -57,13 +51,11 @@ const Button = React.forwardRef<HTMLButtonElement, ButtonProps>(
const Comp = asChild ? Slot : "button"
return (
<Comp
className={cn(buttonVariants({ variant, size, className }))}
ref={ref}
{...props}
/>
)
}
)
className={cn(buttonVariants({ variant, size, className }))}
ref={ref}
{...props}
/>)
})
Button.displayName = "Button"
export { Button, buttonVariants }
+10 -20
ファイルの表示
@@ -22,11 +22,9 @@ const DialogOverlay = React.forwardRef<
ref={ref}
className={cn(
"fixed inset-0 z-50 bg-black/80 data-[state=open]:animate-in data-[state=closed]:animate-out data-[state=closed]:fade-out-0 data-[state=open]:fade-in-0",
className
)}
className)}
{...props}
/>
))
/>))
DialogOverlay.displayName = DialogPrimitive.Overlay.displayName
const DialogContent = React.forwardRef<
@@ -62,8 +60,7 @@ const DialogContent = React.forwardRef<
<span className="sr-only"></span>
</DialogPrimitive.Close>
</DialogPrimitive.Content>
</DialogPortal>
))
</DialogPortal>))
DialogContent.displayName = DialogPrimitive.Content.displayName
const DialogHeader = ({
@@ -73,11 +70,9 @@ const DialogHeader = ({
<div
className={cn(
"flex flex-col space-y-1.5 text-center sm:text-left",
className
)}
className)}
{...props}
/>
)
/>)
DialogHeader.displayName = "DialogHeader"
const DialogFooter = ({
@@ -87,11 +82,9 @@ const DialogFooter = ({
<div
className={cn(
"flex flex-col-reverse sm:flex-row sm:justify-end sm:space-x-2",
className
)}
className)}
{...props}
/>
)
/>)
DialogFooter.displayName = "DialogFooter"
const DialogTitle = React.forwardRef<
@@ -102,11 +95,9 @@ const DialogTitle = React.forwardRef<
ref={ref}
className={cn(
"text-lg font-semibold leading-none tracking-tight",
className
)}
className)}
{...props}
/>
))
/>))
DialogTitle.displayName = DialogPrimitive.Title.displayName
const DialogDescription = React.forwardRef<
@@ -117,8 +108,7 @@ const DialogDescription = React.forwardRef<
ref={ref}
className={cn("text-sm text-muted-foreground", className)}
{...props}
/>
))
/>))
DialogDescription.displayName = DialogPrimitive.Description.displayName
export {
+12 -15
ファイルの表示
@@ -1,22 +1,19 @@
import * as React from "react"
import * as React from 'react'
import { cn } from "@/lib/utils"
import { cn } from '@/lib/utils'
const Input = React.forwardRef<HTMLInputElement, React.ComponentProps<"input">>(
const Input = React.forwardRef<HTMLInputElement, React.ComponentProps<'input'>>(
({ className, type, ...props }, ref) => {
return (
<input
type={type}
className={cn(
"flex h-10 w-full rounded-md border border-input bg-background px-3 py-2 text-base ring-offset-background file:border-0 file:bg-transparent file:text-sm file:font-medium file:text-foreground placeholder:text-muted-foreground focus-visible:outline-none focus-visible:ring-2 focus-visible:ring-ring focus-visible:ring-offset-2 disabled:cursor-not-allowed disabled:opacity-50 md:text-sm",
className
)}
ref={ref}
{...props}
/>
)
}
)
Input.displayName = "Input"
type={type}
className={cn(
'flex h-10 w-full rounded-md border border-input bg-background px-3 py-2 text-base ring-offset-background file:border-0 file:bg-transparent file:text-sm file:font-medium file:text-foreground placeholder:text-muted-foreground focus-visible:outline-none focus-visible:ring-2 focus-visible:ring-ring focus-visible:ring-offset-2 disabled:cursor-not-allowed disabled:opacity-50 md:text-sm',
className)}
ref={ref}
{...props}
/>)
})
Input.displayName = 'Input'
export { Input }
+8 -11
ファイルの表示
@@ -1,9 +1,9 @@
"use client"
'use client'
import * as React from "react"
import * as SwitchPrimitives from "@radix-ui/react-switch"
import * as React from 'react'
import * as SwitchPrimitives from '@radix-ui/react-switch'
import { cn } from "@/lib/utils"
import { cn } from '@/lib/utils'
const Switch = React.forwardRef<
React.ElementRef<typeof SwitchPrimitives.Root>,
@@ -11,19 +11,16 @@ const Switch = React.forwardRef<
>(({ className, ...props }, ref) => (
<SwitchPrimitives.Root
className={cn(
"peer inline-flex h-6 w-11 shrink-0 cursor-pointer items-center rounded-full border-2 border-transparent transition-colors focus-visible:outline-none focus-visible:ring-2 focus-visible:ring-ring focus-visible:ring-offset-2 focus-visible:ring-offset-background disabled:cursor-not-allowed disabled:opacity-50 data-[state=checked]:bg-primary data-[state=unchecked]:bg-input",
className
)}
'peer inline-flex h-6 w-11 shrink-0 cursor-pointer items-center rounded-full border-2 border-transparent transition-colors focus-visible:outline-none focus-visible:ring-2 focus-visible:ring-ring focus-visible:ring-offset-2 focus-visible:ring-offset-background disabled:cursor-not-allowed disabled:opacity-50 data-[state=checked]:bg-primary data-[state=unchecked]:bg-input',
className)}
{...props}
ref={ref}
>
<SwitchPrimitives.Thumb
className={cn(
"pointer-events-none block h-5 w-5 rounded-full bg-background shadow-lg ring-0 transition-transform data-[state=checked]:translate-x-5 data-[state=unchecked]:translate-x-0"
)}
'pointer-events-none block h-5 w-5 rounded-full bg-background shadow-lg ring-0 transition-transform data-[state=checked]:translate-x-5 data-[state=unchecked]:translate-x-0')}
/>
</SwitchPrimitives.Root>
))
</SwitchPrimitives.Root>))
Switch.displayName = SwitchPrimitives.Root.displayName
export { Switch }
+13 -27
ファイルの表示
@@ -17,11 +17,9 @@ const ToastViewport = React.forwardRef<
ref={ref}
className={cn(
"fixed top-0 z-[100] flex max-h-screen w-full flex-col-reverse p-4 sm:bottom-0 sm:right-0 sm:top-auto sm:flex-col md:max-w-[420px]",
className
)}
className )}
{...props}
/>
))
/> ))
ToastViewport.displayName = ToastPrimitives.Viewport.displayName
const toastVariants = cva(
@@ -29,16 +27,12 @@ const toastVariants = cva(
{
variants: {
variant: {
default: "border bg-background text-foreground",
destructive:
"destructive group border-destructive bg-destructive text-destructive-foreground",
default: "border bg-background text-foreground",
destructive:
"destructive group border-destructive bg-destructive text-destructive-foreground",
},
},
defaultVariants: {
variant: "default",
},
}
)
defaultVariants: { variant: "default" } } )
const Toast = React.forwardRef<
React.ElementRef<typeof ToastPrimitives.Root>,
@@ -50,9 +44,7 @@ const Toast = React.forwardRef<
ref={ref}
className={cn(toastVariants({ variant }), className)}
{...props}
/>
)
})
/> ) })
Toast.displayName = ToastPrimitives.Root.displayName
const ToastAction = React.forwardRef<
@@ -63,11 +55,9 @@ const ToastAction = React.forwardRef<
ref={ref}
className={cn(
"inline-flex h-8 shrink-0 items-center justify-center rounded-md border bg-transparent px-3 text-sm font-medium ring-offset-background transition-colors hover:bg-secondary focus:outline-none focus:ring-2 focus:ring-ring focus:ring-offset-2 disabled:pointer-events-none disabled:opacity-50 group-[.destructive]:border-muted/40 group-[.destructive]:hover:border-destructive/30 group-[.destructive]:hover:bg-destructive group-[.destructive]:hover:text-destructive-foreground group-[.destructive]:focus:ring-destructive",
className
)}
className )}
{...props}
/>
))
/> ))
ToastAction.displayName = ToastPrimitives.Action.displayName
const ToastClose = React.forwardRef<
@@ -78,14 +68,12 @@ const ToastClose = React.forwardRef<
ref={ref}
className={cn(
"absolute right-2 top-2 rounded-md p-1 text-foreground/50 opacity-0 transition-opacity hover:text-foreground focus:opacity-100 focus:outline-none focus:ring-2 group-hover:opacity-100 group-[.destructive]:text-red-300 group-[.destructive]:hover:text-red-50 group-[.destructive]:focus:ring-red-400 group-[.destructive]:focus:ring-offset-red-600",
className
)}
className )}
toast-close=""
{...props}
>
<X className="h-4 w-4" />
</ToastPrimitives.Close>
))
</ToastPrimitives.Close> ))
ToastClose.displayName = ToastPrimitives.Close.displayName
const ToastTitle = React.forwardRef<
@@ -96,8 +84,7 @@ const ToastTitle = React.forwardRef<
ref={ref}
className={cn("text-sm font-semibold", className)}
{...props}
/>
))
/> ))
ToastTitle.displayName = ToastPrimitives.Title.displayName
const ToastDescription = React.forwardRef<
@@ -108,8 +95,7 @@ const ToastDescription = React.forwardRef<
ref={ref}
className={cn("text-sm opacity-90", className)}
{...props}
/>
))
/> ))
ToastDescription.displayName = ToastPrimitives.Description.displayName
type ToastProps = React.ComponentPropsWithoutRef<typeof Toast>
+1 -2
ファイルの表示
@@ -26,6 +26,5 @@ export const Toaster = () => {
<ToastClose />
</Toast>))}
<ToastViewport />
</ToastProvider>
)
</ToastProvider> )
}

変更されたファイルが多すぎるため,一部のファイルは表示されません さらに表示