#434

Reviewed-on: #433
Co-authored-by: miteruzo <miteruzo@naver.com>

AGENTS.md

@@ -1,5 +1,18 @@
 # AGENTS.md
 
+## Mandatory security and URL contracts
+
+- URL クエリへの JSON 格納を禁止する。Base64・16進数等による迂回も禁止。
+  単純な配列は半角スペース区切り、構造化条件は番号付きの個別キーで表す。
+  未指定と明示的な空選択を区別し、自由入力を区切り文字で分割しない。
+  Frontend / Backend で同じ文法・上限・検証を使い、新しい URL 状態の追加前に確認する。
+- 一度除票されたリソースの除票以前（除票版を含む）の履歴は admin 以外へ公開しない。
+  恢復しても解除しない。すべての後続除票を確定版番号と正しい複合キーで判定する。
+  本文・属性・ID・版番号・関連参照・旧値を API / SSR / キャッシュ / 出力から漏らさない。
+  サーバーが確定した current_user の role のみを認可に使用する。
+  秘匿版も安全な全体件数には含めるが、公開明細・内訳・検索件数とは分離する。
+  秘匿属性を条件・軸・ソート・差分件数から推測できる実装を禁止する。
+
 ## Project overview
 
 BTRC Hub / タグ広場 is a split Rails API and React frontend repository.
@@ -124,7 +137,7 @@ npm run preview
 - For arrays, never put whitespace or a line break immediately before `]`.
 - Keep the first element on the same line as `[` by default.
 - If an array would exceed the line limit, break after `[` and indent
-  elements by 4 spaces.
+  elements 4 spaces deeper than the statement's base indentation.
 
 ### Ruby delimiter and wrapping rules
 
@@ -140,6 +153,32 @@ npm run preview
   99 文字を超えるなら block 形式へ切り替へるか、message 定数化などで縮める。
 - Ruby の method chain や call argument を折り返す際、call-site の `)` を
   block close のやうに独立させない。
+- Ruby では、行末の `\` を用途を問はず一切使用しない。
+- Ruby では、文字列連結、logger message、method call、条件式、SQL 断片、
+  正規表現その他すべての式で、行末バックスラッシュによる継続を禁止する。
+- Ruby の block body は、その基準位置から 2 空白深くする。
+- Ruby の wrapped expression、method argument、array element、hash pair などの
+  continuation indentation は、その statement の基準位置から 4 空白深くする。
+- Ruby の continuation indentation を、行頭からの絶対空白数として扱はない。
+- Ruby では、暗黙的に継続可能な構文を優先し、method call、array、Hash 及び
+  括弧内ではバックスラッシュなしで改行する。
+- Ruby では、行長制限を守るために行末バックスラッシュを導入してはならない。
+- Ruby で行末バックスラッシュが必要に見える場合は、括弧内で自然に改行する、
+  一つの文字列補間へまとめる、中間変数へ分ける、`format` を使ふ、heredoc を
+  使ふ、array 又は Hash を組み立ててから処理する、条件式全体を括弧で囲む、
+  method へ抽出する、のいづれかへ書き換へる。
+- Ruby では、一つの文字列を、改行をまたいだ隣接文字列 literal として記述しない。
+- Ruby では、method argument 内でも、複数の文字列 literal を区切りなしで縦に
+  並べない。
+- RSpec の `describe`、`context`、`it` 等の description が長い場合は、意味を
+  保ったまま一行へ収まる文言へ短縮する。
+- 文字列を短縮できない場合は、用途に応じて `format`、heredoc 又は中間変数を
+  検討する。
+- ただし RSpec description では、原則として簡潔な一行の文字列を使ふ。
+- formatter 又は自動修正にも、Ruby の行末バックスラッシュを生成させない。
+- 新規 code だけでなく、今回触れる Ruby code にも行末バックスラッシュを残さない。
+- 例へば class body 内の array は、class body の 2 空白を基準に、更に 4 空白
+  深くするため、結果として行頭から 6 空白になる。
 
 Bad:
 
@@ -191,6 +230,63 @@ end
 
 Bad:
 
+```rb
+Rails.logger.info(
+    "post_import_metadata_fetch_failure "\
+    "#{ payload.to_json }")
+```
+
+Good:
+
+```rb
+payload = {
+    error: e.class.name,
+    message: e.message }
+
+Rails.logger.info(
+    "post_import_metadata_fetch_failure #{ payload.to_json }")
+```
+
+Bad:
+
+```rb
+message = "first "\
+          "second"
+```
+
+Good:
+
+```rb
+message = format(
+    '%<first>s %<second>s',
+    first: 'first',
+    second: 'second')
+```
+
+Bad:
+
+```rb
+result = first_value + \
+         second_value
+```
+
+Bad:
+
+```rb
+it(
+    'returns 409 when stale changes '
+    'do not conflict'
+) do
+```
+
+Good:
+
+```rb
+it 'returns mergeable 409 for stale non-conflicting changes' do
+```
+
+Bad:
+
 ```rb
 records.each {
   do_work(_1) }
@@ -225,20 +321,37 @@ records.each {
   formatting as the local reference shape: component and callback bodies use
   2-space block indentation, single-line bodies do not gain unnecessary
   braces, wrapped expressions use 4-space continuation indentation, multi-
-  stage ternaries use explicit parentheses, and only complete leading runs
+  stage ternaries use explicit parentheses, and TypeScript / TSX indentation
-  of 8 spaces are compressed to tabs.
+  and alignment whitespace compress every complete run of 8 spaces to tabs.
-- In TypeScript and TSX only, tabs are for leading 8-column compression only.
 - A tab does not represent one indentation level.
-- Do not replace 2-space or 4-space indentation with tabs.
+- In TypeScript and TSX, first determine visible indentation using 2-space
-- First determine visible indentation using 2-space block indentation and
+  block indentation and 4-space continuation indentation, then compress every
-  4-space continuation indentation, then compress only complete leading runs
+  complete run of 8 spaces used for indentation or column alignment to tabs.
-  of 8 spaces into tabs.
+- In TypeScript and TSX, 8-space compression is mandatory, not optional.
-- Tabs are only for leading indentation, never for spaces after non-space
+- In TypeScript and TSX, this applies both to leading indentation and to
-  text.
+  alignment whitespace after non-space text, such as aligned inline type
-- Keep residual leading 2, 4, or 6 spaces after any tab compression.
+  columns.
+- In TypeScript and TSX, keep residual 2, 4, or 6 spaces after each tab
+  compression.
+- In TypeScript and TSX, do not treat a tab as one logical indentation level.
+- In TypeScript and TSX, do not alter string literals, template-literal
+  contents, regular expressions, or user-facing text merely to apply tab
+  compression.
 - Examples: 2 columns = 2 spaces, 4 columns = 4 spaces, 6 columns = 6
   spaces, 8 columns = 1 tab, 10 columns = 1 tab + 2 spaces, 12 columns = 1
-  tab + 4 spaces, 16 columns = 2 tabs.
+  tab + 4 spaces, 14 columns = 1 tab + 6 spaces, 16 columns = 2 tabs.
+- When TypeScript or TSX code already uses column alignment, apply the same
+  8-space compression rule to that alignment whitespace.
+- Example:
+
+```ts
+type Props = {
+    row:	      PostImportRow
+    displayNumber?:   number
+    onEdit?:	  () => void
+    onRetry?:	  () => void
+    onToggleSkip?:    (checked: boolean) => void }
+```
 - TypeScript and TSX imports may stay on one line if they remain within the
   line limit; do not expand short type-only imports mechanically.
 - Keep runtime value imports and type imports in separate declarations.
@@ -473,9 +586,33 @@ and layout reuse, follow `frontend/AGENTS.md`.
 - When new user-facing wording appears necessary, ask the user for the exact
   wording and placement before implementing it.
 - Do not invent replacement copy when removing unrequested wording.
-- Do not create, modify, or run tests unless the user explicitly asks for
+- 明示的な個別指示がない限り、テストの追加・修正・削除・実行を一切行わない。
-  test work. When the user asks for tests, keep working and rerun them until
+  fixture、mock、snapshot、テスト補助コードも対象とする。
-  they pass or the remaining failure is clearly blocked.
+  既存のテスト差分を勝手に修正・差戻ししない。
+  この規約はディレクトリ固有の自動的なテスト実行推奨より優先する。
+- Test-only work includes adding, updating, deleting, reorganising, or fixing
+  SyntaxError in tests. During test-only work, do not modify production code.
+- During test-only work, do not change production constants, behaviour, API
+  contracts, validation, routes, authentication, permissions, UI, copy,
+  dependencies, limits, thresholds, defaults, migrations, schema, or
+  environment settings to satisfy tests.
+- Do not make production code match failing tests, mock assumptions, fixtures,
+  snapshots, old expectations, or stale setup. This includes changing
+  production constants merely because a test expects a different value.
+- If test work reveals a production bug, spec mismatch, or missing behaviour,
+  stop without modifying production code and report: the failing test or
+  discovered issue, the related production file, the actual behaviour, the
+  expected behaviour, and why a production change appears necessary.
+- Modify production code for test failures only when the user explicitly asks
+  for that production change. Do not expand a test task into a production task
+  on your own authority.
+- If the user explicitly asks for both production implementation and test
+  updates, implement production code to the confirmed specification first,
+  then add or update tests to verify that specification. Never roll production
+  behaviour back to satisfy old tests.
+- If it is unclear whether the test or the production implementation is stale,
+  or a test cannot be corrected without changing production code, ask the user
+  instead of guessing.
 
 ## Backend rules
 
@@ -507,6 +644,19 @@ and layout reuse, follow `frontend/AGENTS.md`.
 
 ## Frontend rules
 
+- 原則として、ユーザ向け UI 文言を TSX / TypeScript に直接記述しない。
+  新しい文言は既存の i18n システムで管理し、適切な既存キーを先に確認する。
+  不必要な重複定義を避け、新規機能固有の文言は適切な namespace にまとめる。
+- 対応 locale の辞書を整合させる。未翻訳項目には既存の日本語 fallback と
+  `_untranslated` 規約を使い、language metadata を維持する。
+  翻訳を捏造せず、未翻訳文字列を正式な翻訳として登録しない。
+- UI 文言の追加・変更時はビルド、静的検査、実画面の確認等で、辞書と
+  SSR / hydration の整合及び翻訳キーの非露出を検証する。
+  i18n の検証を理由にテスト関連作業を開始しない。
+- API の内部識別子、パラメータ、ユーザ入力データ、外部 URL は機械的に
+  翻訳しない。ボタン、項目名、CSV ヘッダ、エラー、空状態、ツールチップ、
+  アクセシビリティ用ラベルも UI 文言として扱う。
+
 - Use `frontend/src/lib/api.ts` for API calls so headers and camelCase conversion stay consistent.
 - Add or reuse TanStack Query keys through `frontend/src/lib/queryKeys.ts`;
   avoid ad hoc query key arrays.
@@ -529,6 +679,11 @@ and layout reuse, follow `frontend/AGENTS.md`.
 - Mobile UI must be checked as a first-class layout. Avoid wide fixed content,
   make dense controls wrap or scroll intentionally, and keep tag/filter
   controls usable without horizontal page overflow.
+- Frontend のスマホ／PC表示境界は原則 `md` とする。
+- button stack、footer action、dialogue action は `md` 未満で縦並び、
+  `md` 以上で横並びとする。
+- 同じ画面内で `sm` と `md` を混在させて中間 layout を作らない。
+- 明確に別の responsive 要件がある component だけを例外とする。
 - For mobile horizontal scrollers, make the scroll direction and item sizing
   explicit, and ensure chip text remains readable in both light and dark modes.
 - In TypeScript and TSX, prefer direct comparison operators such as `===` and
@@ -593,10 +748,10 @@ and layout reuse, follow `frontend/AGENTS.md`.
   beginning of a line.
 - The TSX-specific self-review must confirm JSX closing markers and closing
   parentheses keep the surrounding compact style.
-- The TypeScript/TSX self-review must confirm leading block indentation uses
+- The TypeScript/TSX self-review must confirm block indentation uses 2 spaces
-  2 spaces per level, wrapped continuations use the repository's 4-space
+  per level, wrapped continuations use the repository's 4-space continuation
-  continuation alignment, and complete leading runs of 8 spaces may be
+  alignment, and every complete run of 8 spaces used for indentation or
-  compressed to tabs.
+  alignment has been compressed to tabs.
 - Prefer `const` arrow functions for TypeScript/TSX component and helper declarations.
 - Put two blank lines before and after top-level `const` function
   declarations, unless imports, exports, or file boundaries make that awkward.
@@ -609,9 +764,18 @@ and layout reuse, follow `frontend/AGENTS.md`.
 - Indent the block body 2 spaces deeper than the keyword and opening brace.
 - Put the closing `}` on its own line at the same indentation as the keyword.
 - Do not write `try {`, `catch {`, or `finally {`.
-- In TypeScript and TSX, convert every complete leading run of 8 spaces to a
+- In TypeScript and TSX, convert every complete run of 8 spaces used for
-  tab character.
+  indentation or alignment to a tab character.
-- A leading tab is exactly equivalent to 8 leading spaces.
+- In TypeScript and TSX, a tab is exactly equivalent to 8 columns, whether it
+  appears at the beginning of a line or in alignment whitespace after
+  non-space text.
+- In TSX, JSX nesting uses 2-space block indentation and wrapped JSX
+  attributes use 4-space continuation indentation; after visible columns are
+  determined, compress every complete run of 8 spaces in the resulting
+  indentation or alignment to tabs.
+- In TSX, do not leave JSX subtree indentation at 8, 10, 12, 14, or 16
+  columns as spaces alone; convert each complete run of 8 spaces to tabs and
+  keep only the residual 2, 4, or 6 spaces.
 - In TypeScript and TSX function declarations, including `const` arrow
   function declarations, classify the parameter list before placing the closing
   `)`.
@@ -1254,9 +1418,9 @@ to `.ts` and `.tsx`:
 7. JSX `>` and `/>` stay with the final prop unless nearby code proves
    otherwise.
 8. JSX closing parentheses keep the compact local style.
-9. Leading block indentation uses 2 spaces per level, wrapped continuations
+9. Block indentation uses 2 spaces per level, wrapped continuations use the
-   use the repository's 4-space continuation alignment, and complete leading
+   repository's 4-space continuation alignment, and every complete run of 8
-   runs of 8 spaces may be compressed to tabs.
+   spaces used for indentation or alignment has been compressed to tabs.
 10. No line has trailing whitespace.
 
 Preferred:

backend/AGENTS.md

@@ -1,5 +1,20 @@
 # backend/AGENTS.md
 
+## Mandatory security and URL contracts
+
+- URL クエリへの JSON 格納を禁止する。Base64・16進数等による迂回も禁止。
+  単純な配列は半角スペース区切り、構造化条件は番号付きの個別キーで表す。
+  未指定と明示的な空選択を区別し、自由入力を区切り文字で分割しない。
+  Frontend / Backend で同じ文法・上限・検証を使い、新しい URL 状態の追加前に確認する。
+- 一度除票されたリソースの除票以前（除票版を含む）の履歴は admin 以外へ公開しない。
+  恢復しても解除しない。すべての後続除票を確定版番号と正しい複合キーで判定する。
+  本文・属性・ID・版番号・関連参照・旧値を API / SSR / キャッシュ / 出力から漏らさない。
+  サーバーが確定した current_user の role のみを認可に使用する。
+  秘匿版も安全な全体件数には含めるが、公開明細・内訳・検索件数とは分離する。
+  秘匿属性を条件・軸・ソート・差分件数から推測できる実装を禁止する。
+- 履歴読み取りは HistoryAccess の共通認可を必須とする。書込用 relation を公開しない。
+  export は生成時と取得時に認可し、権限変更・除票後に古いファイルを再利用しない。
+
 ## Scope
 
 These rules apply to work under `backend/`.

backend/app/controllers/application_controller.rb

@@ -1,20 +1,63 @@
 class ApplicationController < ActionController::API
+  include RequestAccess
+  include ActionController::Cookies
+  include ActionController::RequestForgeryProtection
+
+  class UnsupportedLocale < StandardError
+    ;
+  end
+
   rescue_from ActiveRecord::RecordInvalid, with: :render_record_invalid
   rescue_from ActiveRecord::RecordNotUnique, with: :render_record_not_unique
+  rescue_from(ActiveRecord::RecordNotFound) { head :not_found }
+  rescue_from(UnsupportedLocale) { head :not_found }
 
   before_action :reject_banned_ip_address!
   before_action :authenticate_user
   before_action :reject_banned_user!
-
+  before_action :validate_explicit_locale!
-  def current_user = @current_user
+  before_action :enforce_query_contract!
+  around_action :with_history_authority
+  rescue_from(HistoryAccess::Forbidden) { head :not_found }
+  protect_from_forgery with: :exception
+  self.forgery_protection_origin_check = true
+  rescue_from ActionController::InvalidAuthenticityToken, with: :reject_invalid_csrf
+  after_action :prevent_authenticated_caching
 
   private
 
-  def authenticate_user
+  def enforce_query_contract!
-    code = request.headers['X-Transfer-Code'] || request.headers['HTTP_X_TRANSFER_CODE']
+    pairs = URI.decode_www_form(request.query_string)
-    return if code.blank?
+    return head :bad_request if pairs.length > 1000 || pairs.map(&:first).uniq.length != pairs.length
+    return head :bad_request if request.query_parameters.values.any? { !_1.is_a?(String) }
+    if (controller_name.start_with?('counts_') &&
+        (request.query_parameters.keys & ['headers', 'labels']).any?)
+      return head :bad_request
+    end
+  rescue ArgumentError
+    head :bad_request
+  end
 
-    @current_user = User.find_by(inheritance_code: code)
+  def with_history_authority(&action)
+    response.headers['Cache-Control'] = 'private, no-store'
+    HistoryAccess.set(user: current_user, &action)
+  end
+
+  def reject_invalid_csrf
+    response.headers['X-CSRF-Invalid'] = '1'
+    response.headers['Cache-Control'] = 'private, no-store'
+    head :forbidden
+  end
+
+  def prevent_authenticated_caching
+    response.headers['Cache-Control'] = 'private, no-store' if current_user
+  end
+
+  def establish_user_session user
+    reset_session
+    session[:inheritance_code] = user.inheritance_code
+    @current_user = user
+    response.headers['Cache-Control'] = 'private, no-store'
   end
 
   def bool? key, default: false
@@ -49,19 +92,6 @@ class ApplicationController < ActionController::API
     render_validation_error base: ['すでに存在してゐます．']
   end
 
-  def reject_banned_ip_address!
-    ip_address = IpAddress.find_by(ip_address: IPAddr.new(request.remote_ip).hton)
-    return unless ip_address&.banned?
-
-    head :forbidden
-  end
-
-  def reject_banned_user!
-    return unless current_user&.banned?
-
-    head :forbidden
-  end
-
   def render_validation_error record = nil, fields: { }, base: [], status: :unprocessable_entity
     errors = { }
 
@@ -85,4 +115,29 @@ class ApplicationController < ActionController::API
                    base_errors: },
            status:
   end
+
+  def normalise_json value
+    return nil if value.nil?
+    return JSON.parse(value) if value.is_a?(String)
+
+    value
+  end
+
+  def resolve_locale!
+    return @resolved_locale if defined?(@resolved_locale)
+
+    code = params.fetch(:locale, 'ja')
+    unless code.is_a?(String) && Locale.codes.value?(code)
+      raise UnsupportedLocale
+    end
+
+    locale = Locale.find_by(code:)
+    raise UnsupportedLocale unless locale && locale.deprecated_at.nil?
+
+    @resolved_locale = locale
+  end
+
+  def validate_explicit_locale!
+    resolve_locale! if params.key?(:locale)
+  end
 end

backend/app/controllers/concerns/counts_listing.rb

@@ -0,0 +1,21 @@
+module CountsListing
+  extend ActiveSupport::Concern
+
+  def index
+    locale = resolve_locale!
+    resource = is_a?(CountsPostsController) ? 'posts' : 'tags'
+    query = CountsQuery.new(version_scope, resource, request.query_parameters.except('locale')
+        .merge('locale' => locale.code))
+    result = { population_count_scope: 'all_recorded_versions', population_count: HistoryAccess.population_count(query.relation.klass), count: query.count, page: query.page, limit: query.limit,
+               at: request.query_parameters['at'].presence || Time.current.utc.iso8601(6) }
+    if query.mode == 'summary'
+      result.merge!(query.summary)
+    else
+      result[:versions] = CountsVersionRepr.many(
+          query.details, resource:, language_code: locale.language_code)
+    end
+    render json: result
+  rescue ArgumentError, TypeError
+    head :bad_request
+  end
+end

backend/app/controllers/concerns/history_storage_access.rb

@@ -0,0 +1,52 @@
+# Permanent storage URLs must not bypass a resource's later discard boundary.
+module HistoryStorageAccess
+  extend ActiveSupport::Concern
+
+  included do
+    include RequestAccess
+    prepend_before_action :authorise_history_storage!
+  end
+
+  private
+
+  def authorise_history_storage!
+    response.headers['Cache-Control'] = 'private, no-store'
+    return unless request.get? || request.head?
+
+    reject_banned_ip_address!
+    return if performed?
+
+    authenticate_user
+    reject_banned_user!
+    return if performed?
+
+    signed = params[:signed_blob_id] || params[:signed_id]
+    blob = if signed
+             ActiveStorage::Blob.find_signed(signed)
+           elsif params[:encoded_key]
+             payload = ActiveStorage.verifier.verified(params[:encoded_key], purpose: :blob_key)
+             key = payload && (payload[:key] || payload['key'])
+             if key
+               original = key.start_with?('variants/') ? key.split('/')[1] : key
+               ActiveStorage::Blob.find_by(key: original)
+             end
+           end
+    return head :not_found unless blob
+
+    allowed = HistoryAccess.set(user: current_user) { HistoryAccess.blob_visible?(blob) }
+    head :not_found unless allowed
+  rescue ActiveSupport::MessageVerifier::InvalidSignature
+    head :not_found
+  end
+
+  # Override the engine's public-forever caching before a streaming response commits.
+  def http_cache_forever(**_options)
+    response.headers['Cache-Control'] = 'private, no-store'
+    yield
+  end
+
+  def expires_in(*arguments, **options)
+    super
+    response.headers['Cache-Control'] = 'private, no-store'
+  end
+end

backend/app/controllers/concerns/request_access.rb

@@ -0,0 +1,28 @@
+# Canonical read-side session and BAN boundary, shared by API and storage controllers.
+module RequestAccess
+  extend ActiveSupport::Concern
+
+  def current_user = @current_user
+
+  private
+
+  def authenticate_user
+    code = session[:inheritance_code]
+    return if code.blank?
+
+    @current_user = User.find_by(inheritance_code: code)
+  end
+
+  def reject_banned_ip_address!
+    ip_address = IpAddress.find_by(ip_address: IPAddr.new(request.remote_ip).hton)
+    return unless ip_address&.banned?
+
+    head :forbidden
+  end
+
+  def reject_banned_user!
+    return unless current_user&.banned?
+
+    head :forbidden
+  end
+end

backend/app/controllers/counts_exports_controller.rb

@@ -0,0 +1,83 @@
+class CountsExportsController < ApplicationController
+  wrap_parameters false
+
+  def create_job
+    locale = resolve_locale!
+    payload = request.request_parameters.merge(request.query_parameters)
+    session[:counts_export_owner] ||= SecureRandom.hex(32)
+    job = CountsExportJobs.start(payload, session[:counts_export_owner], locale.language_code)
+    response.headers['Cache-Control'] = 'private, no-store'
+    render json: { job: }, status: :accepted
+  rescue CountsExportDownload::Busy
+    head :too_many_requests
+  rescue ArgumentError, TypeError, KeyError, JSON::ParserError
+    head :bad_request
+  end
+
+  def job_status
+    job_state(cancel: false)
+  end
+
+  def cancel_job
+    job_state(cancel: true)
+  end
+
+  def prepare
+    locale = resolve_locale!
+    payload = request.request_parameters.merge(request.query_parameters)
+    exporter = CountsExporter.new(payload, language_code: locale.language_code)
+    session[:counts_export_owner] ||= SecureRandom.hex(32)
+    token = CountsExportDownload.prepare(exporter, session[:counts_export_owner])
+    response.headers['Cache-Control'] = 'private, no-store'
+    render json: { token: }
+  rescue CountsExportDownload::Busy
+    head :too_many_requests
+  rescue CountsExportDownload::TooLarge
+    head 413
+  rescue CountsExportDownload::TimedOut
+    head :gateway_timeout
+  rescue CountsExportDownload::StorageFull, Errno::ENOSPC
+    head :insufficient_storage
+  rescue ArgumentError, TypeError, KeyError, JSON::ParserError
+    head :bad_request
+  end
+
+  def download
+    file = CountsExportDownload.find(params[:token], session[:counts_export_owner])
+    return head :not_found unless file
+
+    response.headers['Cache-Control'] = 'private, no-store'
+    send_file file.fetch('path'),
+              filename: "counts-#{ file.fetch('resource') }.#{ file.fetch('format') }",
+              type: file['format'] == 'json' ? 'application/json' : 'text/csv',
+              disposition: 'attachment'
+  end
+
+  def job_state(cancel:)
+    result = CountsExportJobs.status(params[:token], session[:counts_export_owner], cancel:)
+    return head :not_found unless result
+
+    response.headers['Cache-Control'] = 'private, no-store'
+    render json: result
+  end
+  private :job_state
+
+  def index
+    locale = resolve_locale!
+    payload = request.request_parameters.except('authenticity_token')
+        .merge(request.query_parameters)
+    exporter = CountsExporter.new(payload,
+                                  language_code: locale.language_code)
+    session[:counts_export_owner] ||= SecureRandom.hex(32)
+    token = CountsExportDownload.prepare(exporter, session[:counts_export_owner])
+    file = CountsExportDownload.find(token, session[:counts_export_owner])
+    return head :not_found unless file
+
+    send_file file.fetch('path'),
+        filename: "counts-#{ exporter.resource }.#{ exporter.format }",
+        type: exporter.format == 'json' ? 'application/json' : 'text/csv',
+        disposition: 'attachment'
+  rescue ArgumentError, TypeError, KeyError, JSON::ParserError
+    head :bad_request
+  end
+end

backend/app/controllers/counts_posts_controller.rb

@@ -0,0 +1,3 @@
+class CountsPostsController < PostVersionsController
+  include CountsListing
+end

backend/app/controllers/counts_states_controller.rb

@@ -0,0 +1,10 @@
+class CountsStatesController < ApplicationController
+  def index
+    locale = resolve_locale!
+    result = CountsStateQuery.new(request.query_parameters,
+                                  language_code: locale.language_code).call
+    render json: result
+  rescue ArgumentError, TypeError
+    head :bad_request
+  end
+end

backend/app/controllers/counts_tables_controller.rb

@@ -0,0 +1,21 @@
+class CountsTablesController < ApplicationController
+  def definition
+    resolve_locale!
+    definition = CountsResourceDefinition.new(params[:resource])
+    render json: { fields: definition.metadata,
+                   axes: { latest: definition.axes(false), history: definition.axes(true) },
+                   grains: CountsAxes::GRAINS, defaults: CountsDefaults.metadata(params[:resource]) }
+  rescue ArgumentError
+    head :bad_request
+  end
+
+  def index
+    resolve_locale!
+    render json: CountsTableQuery.new(request.query_parameters).call
+  rescue CountsTableQuery::TooManyColumns
+    render json: { error: 'too_many_columns', max_columns: CountsTableQuery::MAX_COLUMNS },
+           status: :unprocessable_entity
+  rescue ArgumentError, TypeError, KeyError, JSON::ParserError
+    head :bad_request
+  end
+end

backend/app/controllers/counts_tags_controller.rb

@@ -0,0 +1,3 @@
+class CountsTagsController < TagVersionsController
+  include CountsListing
+end

backend/app/controllers/deerjikists_controller.rb

@@ -1,5 +1,7 @@
 class DeerjikistsController < ApplicationController
   def show
+    locale = resolve_locale!
+
     platform = params[:platform].to_s.strip
     code = params[:code].to_s.strip
     return render_bad_request('platform は必須です．') if platform.blank?
@@ -7,10 +9,10 @@ class DeerjikistsController < ApplicationController
 
     deerjikist = Deerjikist
                  .joins(:tag)
-                 .includes(tag: :tag_name)
+                 .includes(tag: :tag_names)
                  .find_by(platform:, code:)
     if deerjikist
-      render json: DeerjikistRepr.base(deerjikist)
+      render json: DeerjikistRepr.base(deerjikist, language_code: locale.language_code)
     else
       head :not_found
     end
@@ -20,6 +22,7 @@ class DeerjikistsController < ApplicationController
     return head :unauthorized unless current_user
     return head :forbidden unless current_user.gte_member?
 
+    locale = resolve_locale!
     platform = params[:platform].to_s.strip
     code = params[:code].to_s.strip
     tag_id = params[:tag_id].to_i
@@ -32,7 +35,7 @@ class DeerjikistsController < ApplicationController
       d.save!
     end
 
-    render json: DeerjikistRepr.base(deerjikist)
+    render json: DeerjikistRepr.base(deerjikist, language_code: locale.language_code)
   end
 
   def destroy

backend/app/controllers/gekanator_posts_controller.rb

@@ -2,7 +2,7 @@ class GekanatorPostsController < ApplicationController
   def index
     posts =
         Post
-        .preload(:post_similarities, tags: :tag_name)
+        .preload(:post_similarities, tags: :tag_names)
         .with_attached_thumbnail
         .order(Arel.sql(
             'COALESCE(posts.original_created_before - INTERVAL 1 MINUTE, ' \
@@ -56,7 +56,7 @@ class GekanatorPostsController < ApplicationController
   def tag_json tag
     {
       id: tag.id,
-      name: tag.name,
+      name: tag.name('ja'),
       category: tag.category
     }
   end

backend/app/controllers/gekanator_questions_controller.rb

@@ -126,9 +126,9 @@ class GekanatorQuestionsController < ApplicationController
     end
 
     Tag
-        .joins(:tag_name)
+        .joins(:tag_names)
         .where(category: categories.uniq)
-        .where(tag_names: { name: names.uniq })
+        .where(tag_names: { language_code: 'ja', primary_flg: true, name: names.uniq })
         .where.not(deprecated_at: nil)
         .pluck('tags.category', 'tag_names.name')
         .each_with_object({ }) do |(category, name), h|

backend/app/controllers/locales_controller.rb

@@ -0,0 +1,10 @@
+class LocalesController < ApplicationController
+  def index
+    locales = Locale.where(deprecated_at: nil).order(:created_at).map do |locale|
+      { code: locale.code_before_type_cast, name: locale.name }
+    end
+
+    response.headers['Cache-Control'] = 'no-store'
+    render json: { locales: }
+  end
+end

backend/app/controllers/material_versions_controller.rb

@@ -8,7 +8,7 @@ class MaterialVersionsController < ApplicationController
 
     offset = (page - 1) * limit
 
-    q = MaterialVersion.includes(:created_by_user)
+    q = HistoryAccess.visible(MaterialVersion.includes(:created_by_user))
     q = q.where(material_id: params[:material_id]) if params[:material_id].present?
     q = q.where(tag_name: params[:tag].to_s.strip) if params[:tag].present?
     q = q.where(event_type: params[:event_type]) if params[:event_type].present?
@@ -20,7 +20,7 @@ class MaterialVersionsController < ApplicationController
         .limit(limit)
         .offset(offset)
 
-    render json: { versions: serialise_versions(versions), count: }
+    render json: { versions: serialise_versions(versions), count:, population_count: HistoryAccess.population_count(MaterialVersion) }
   end
 
   private

backend/app/controllers/materials_controller.rb

@@ -7,6 +7,8 @@ class MaterialsController < ApplicationController
   rescue_from MaterialZipExporter::MissingFileError, with: :render_zip_missing_file
 
   def index
+    locale = resolve_locale!
+
     page = (params[:page].presence || 1).to_i
     limit = (params[:limit].presence || 20).to_i
 
@@ -20,7 +22,7 @@ class MaterialsController < ApplicationController
     q = Material.includes(:material_export_items,
                           thumbnail_attachment: :blob,
                           file_attachment: :blob,
-                          tag: :tag_name)
+                          tag: :tag_names)
     q = q.where(tag_id: nil) if filters[:tag_state] == 'untagged'
     q = q.where.not(tag_id: nil) if filters[:tag_state] == 'tagged'
     q = apply_material_tag_filter(q, filters, tag_graph)
@@ -42,7 +44,10 @@ class MaterialsController < ApplicationController
         .offset(offset)
         .to_a
 
-    response = { materials: MaterialRepr.list_many(materials, host: request.base_url),
+    response = { materials: MaterialRepr.list_many(
+                   materials,
+                   host: request.base_url,
+                   language_code: locale.language_code),
                  count: }
     if filters[:tag_id]
       response[:tag_scope] = material_index_tag_scope(filters, tag_graph)
@@ -55,6 +60,8 @@ class MaterialsController < ApplicationController
   end
 
   def show
+    locale = resolve_locale!
+
     material =
         Material
         .includes(:tag, :material_export_items)
@@ -63,15 +70,20 @@ class MaterialsController < ApplicationController
         .find_by(id: params[:id])
     return head :not_found unless material
 
-    wiki_page_body = material.tag&.tag_name&.wiki_page&.current_revision&.body
+    wiki_page_body =
+        TagDescription
+        .find_by(tag_id: material.tag_id, language_code: locale.language_code)
+        &.body
 
-    render json: MaterialRepr.base(material, host: request.base_url).merge(wiki_page_body:)
+    render json: MaterialRepr.base(material, host: request.base_url,
+                                   language_code: locale.language_code).merge(wiki_page_body:)
   end
 
   def create
     return head :unauthorized unless current_user
     return head :forbidden unless current_user.gte_member?
 
+    locale = resolve_locale!
     tag_name_raw = params[:tag].to_s.strip
     file = params[:file]
     file_sha256 = MaterialFileSha256.from_upload(file)
@@ -89,13 +101,14 @@ class MaterialsController < ApplicationController
 
     begin
       Material.transaction do
-        tag = resolve_material_tag!(tag_name_raw)
+        tag = resolve_material_tag!(locale, tag_name_raw)
         material = Material.new(tag:, url:,
                                 created_by_user: current_user,
                                 updated_by_user: current_user)
         material.file.attach(uploaded_blob) if uploaded_blob
         material.save!
-        TagVersioning.record_tag_snapshot!(tag, created_by_user: current_user)
+        TagVersioning.record_tag_snapshot!(tag, created_by_user: current_user,
+                                           language_code: locale.language_code)
         upsert_export_paths!(material)
         MaterialVersionRecorder.record!(material:, event_type: :create,
                                         created_by_user: current_user)
@@ -107,7 +120,8 @@ class MaterialsController < ApplicationController
 
     if material
       log_thumbnail_generation(material, MaterialThumbnailGenerator.generate!(material))
-      render json: MaterialRepr.base(material, host: request.base_url), status: :created
+      render json: MaterialRepr.base(material, host: request.base_url,
+                                     language_code: locale.language_code), status: :created
     else
       render_validation_error material
     end
@@ -132,12 +146,14 @@ class MaterialsController < ApplicationController
     block = MaterialImportBlockMatcher.match_for_sha256(file_sha256)
     return render_material_import_block(block) if block
 
+    locale = resolve_locale!
+
     uploaded_blob = build_uploaded_material_blob!(file, file_sha256)
 
     begin
       Material.transaction do
         MaterialVersionRecorder.ensure_snapshot!(material, created_by_user: current_user)
-        tag = resolve_material_tag!(tag_name_raw)
+        tag = resolve_material_tag!(locale, tag_name_raw)
         material.assign_attributes(tag:, url:, updated_by_user: current_user)
         if uploaded_blob
           material.file.attach(uploaded_blob)
@@ -145,7 +161,8 @@ class MaterialsController < ApplicationController
           material.file.detach
         end
         material.save!
-        TagVersioning.record_tag_snapshot!(tag, created_by_user: current_user)
+        TagVersioning.record_tag_snapshot!(tag, created_by_user: current_user,
+                                           language_code: locale.language_code)
         upsert_export_paths!(material)
         MaterialVersionRecorder.record!(material:, event_type: :update,
                                         created_by_user: current_user)
@@ -157,7 +174,8 @@ class MaterialsController < ApplicationController
 
     log_thumbnail_generation(material, MaterialThumbnailGenerator.generate!(material))
 
-    render json: MaterialRepr.base(material, host: request.base_url)
+    render json: MaterialRepr.base(material, host: request.base_url,
+                                   language_code: locale.language_code)
   end
 
   def destroy
@@ -236,10 +254,9 @@ class MaterialsController < ApplicationController
     nil
   end
 
-  def resolve_material_tag! tag_name_raw
+  def resolve_material_tag! locale, tag_name_raw
-    tag_name = TagName.find_undiscard_or_create_by!(name: tag_name_raw)
+    Tag.find_or_create_by_tag_name!(locale, tag_name_raw, category: :material,
-    tag = tag_name.tag
+                                   created_by_user: current_user)
-    tag || Tag.create!(tag_name:, category: :material)
   end
 
   def material_index_needs_tag_name? filters
@@ -253,7 +270,12 @@ class MaterialsController < ApplicationController
   end
 
   def material_index_join_tag_name q
-    q.left_joins(tag: :tag_name)
+    q.left_joins(:tag).joins(<<~SQL.squish)
+      LEFT JOIN tag_names
+        ON tag_names.tag_id = tags.id
+       AND tag_names.language_code = 'ja'
+       AND tag_names.primary_flg = TRUE
+    SQL
   end
 
   def material_index_join_file_blob q
@@ -269,9 +291,11 @@ class MaterialsController < ApplicationController
 
   def apply_material_query q, term
     like = "%#{ ActiveRecord::Base.sanitize_sql_like(term) }%"
-    q.where('tag_names.name LIKE :q OR materials.url LIKE :q OR ' \
+    q.where(<<~SQL.squish, q: like)
-            'material_file_blobs.filename LIKE :q',
+      tag_names.name LIKE :q
-            q: like)
+      OR materials.url LIKE :q
+      OR material_file_blobs.filename LIKE :q
+    SQL
   end
 
   def apply_material_media_kind q, media_kind
@@ -323,8 +347,9 @@ class MaterialsController < ApplicationController
         end
     tags_by_id =
         Tag
-        .joins(:tag_name)
+        .joins(:tag_names)
-        .where(id: scope_tag_ids)
+        .where(id: scope_tag_ids,
+               tag_names: { language_code: 'ja', primary_flg: true })
         .pluck('tags.id', 'tag_names.name', 'tags.category', 'tags.deprecated_at')
         .each_with_object({ }) do |(id, name, category, deprecated_at), hash|
           hash[id] = { id:, name:, category:, deprecated: deprecated_at.present? }

backend/app/controllers/nico_tags_controller.rb

@@ -1,5 +1,7 @@
 class NicoTagsController < ApplicationController
   def index
+    locale = resolve_locale!
+
     name = params[:name].presence
     linked_tag = params[:linked_tag].presence
     link_status = params[:link_status].presence
@@ -15,30 +17,36 @@ class NicoTagsController < ApplicationController
     limit = 1 if limit < 1
 
     post_tag_max_sql =
-        PostTag
+        PostExternalTag
-        .select('tag_id, MAX(created_at) AS max_created_at')
+        .select('external_tag_id, MAX(created_at) AS max_created_at')
-        .group('tag_id')
+        .group('external_tag_id')
         .to_sql
 
-    q = Tag.nico_tags
+    q =
-           .joins(:tag_name)
+        ExternalTag
-           .joins("LEFT JOIN (#{ post_tag_max_sql }) post_tag_max " \
+        .joins("LEFT JOIN (#{ post_tag_max_sql }) post_tag_max " +
-                  'ON post_tag_max.tag_id = tags.id')
+               'ON post_tag_max.external_tag_id = external_tags.id')
-           .includes(:tag_name, tag_name: :wiki_page, linked_tags: { tag_name: :wiki_page })
+        .includes(linked_tags: [:tag_names, :tag_descriptions])
-    q = q.where('tag_names.name LIKE ?', "%#{ name }%") if name
+    if name
+      q = q.where(('external_tags.name LIKE ? ' +
+                   "OR CONCAT(external_tags.platform, ':', external_tags.name) LIKE ?"),
+                  "%#{ name }%", "%#{ name }")
+    end
+
     if linked_tag
       linked_tag_ids =
-        Tag
+          Tag
-        .joins(:tag_name)
+          .joins(:tag_names)
-        .where('tag_names.name LIKE ?', "%#{ linked_tag }%")
+          .where(tag_names: { language_code: locale.language_code, primary_flg: true })
-        .pluck(:id)
+          .where('tag_names.name LIKE ?', "%#{ linked_tag }%")
+          .pluck(:id)
       linked_nico_tag_ids = NicoTagRelation.where(tag_id: linked_tag_ids).pluck(:nico_tag_id)
       q = q.where(id: linked_nico_tag_ids)
     end
     if link_status.in?(['linked', 'unlinked'])
       exists_sql =
-          'EXISTS (SELECT 1 FROM nico_tag_relations ' \
+          'EXISTS (SELECT 1 FROM nico_tag_relations ' +
-          'WHERE nico_tag_relations.nico_tag_id = tags.id)'
+          'WHERE nico_tag_relations.nico_tag_id = external_tags.id)'
       q = link_status == 'linked' ? q.where(exists_sql) : q.where("NOT #{ exists_sql }")
     end
 
@@ -46,54 +54,69 @@ class NicoTagsController < ApplicationController
     sort_sql =
         case order[0]
         when 'name'
-          'tag_names.name'
+          'external_tags.name'
         when 'updated_at'
           'post_tag_max.max_created_at'
         else
-          "tags.#{ order[0] }"
+          "external_tags.#{ order[0] }"
         end
-    tags = q.reselect('tags.*',
+    tags = q.reselect('external_tags.*',
                       Arel.sql('post_tag_max.max_created_at AS recent_post_tag_created_at'))
-            .order(Arel.sql("#{ sort_sql } #{ order[1] }, tags.id #{ order[1] }"))
+            .order(Arel.sql("#{ sort_sql } #{ order[1] }, external_tags.id #{ order[1] }"))
             .limit(limit)
             .offset((page - 1) * limit)
             .to_a
 
     render json: { tags: tags.map { |tag|
-      TagRepr.base(tag).merge(
+      external_tag_json(tag).merge(
           recent_post_tag_created_at: tag.recent_post_tag_created_at,
-          linked_tags: tag.linked_tags.map { |lt| TagRepr.base(lt) })
+          linked_tags: tag.linked_tags.map { |lt|
+            TagRepr.base(lt, language_code: locale.language_code)
+          })
     }, count: }
   end
 
+  def show
+    tag = ExternalTag.find(params[:id])
+
+    render json: external_tag_json(tag)
+  end
+
   def update
     return head :unauthorized unless current_user
     return head :forbidden unless current_user.gte_member?
 
+    locale = resolve_locale!
+
     id = params[:id].to_i
 
-    tag = Tag.find(id)
+    tag = ExternalTag.find(id)
-    return render_bad_request('ニコニコ・タグを指定してください．') unless tag.nico?
 
     linked_tag_names = params[:tags].to_s.split
     linked_tags = nil
 
     ApplicationRecord.transaction do
-      linked_tags = Tag.normalise_tags!(linked_tag_names, with_tagme: false,
+      linked_tags = Tag.normalise_tags!(locale, linked_tag_names,
-                                                          with_no_deerjikist: false)
+                                        with_tagme: false,
-      if linked_tags.any? { |t| t.nico? }
+                                        with_no_deerjikist: false,
-        raise Tag::NicoTagNormalisationError
+                                        created_by_user: current_user)
-      end
 
-      TagVersioning.record_tag_snapshots!(linked_tags, created_by_user: current_user)
+      TagVersioning.record_tag_snapshots!(
+          linked_tags,
+          created_by_user: current_user,
+          language_code: locale.language_code)
 
       tag.linked_tags = linked_tags
       tag.save!
 
-      NicoTagVersionRecorder.record!(tag:, event_type: :update, created_by_user: current_user)
+      NicoTagVersionRecorder.record!(external_tag: tag,
+                                     event_type: :update,
+                                     created_by_user: current_user)
     end
 
-    render json: tag.linked_tags.map { |t| TagRepr.base(t) }, status: :ok
+    render json: tag.linked_tags.map { |t|
+      TagRepr.base(t, language_code: locale.language_code)
+    }, status: :ok
   rescue Tag::NicoTagNormalisationError
     render_validation_error fields: { tags: ['ニコニコ・タグ同士は連携できません．'] }
   rescue ActiveRecord::RecordInvalid => e
@@ -102,11 +125,39 @@ class NicoTagsController < ApplicationController
 
   private
 
+  def external_tag_json tag
+    { id: tag.id,
+      name: "#{ tag.platform }:#{ tag.name }",
+      category: 'nico',
+      post_count: tag.post_count,
+      created_at: tag.created_at,
+      updated_at: tag.created_at,
+      deprecated_at: nil,
+      aliases: [],
+      parents: [],
+      has_wiki: false,
+      material_id: nil,
+      has_deerjikists: false }
+  end
+
   def render_nico_tag_form_record_invalid record
-    if record.is_a?(TagName) || record.is_a?(Tag)
+    names =
-      render_validation_error fields: { tags: record.errors.full_messages.map { |message|
+        case record
-        "タグ名 “#{ record.name }”: #{ message }"
+        when TagName
-      } }
+          [record]
+        when Tag
+          record.tag_names.to_a
+        else
+          []
+        end
+    messages = names.flat_map do |tag_name|
+      tag_name.errors.full_messages.map do |message|
+        "タグ名 “#{ tag_name.name }”: #{ message }"
+      end
+    end
+
+    if messages.any?
+      render_validation_error fields: { tags: messages }
     else
       render_validation_error record
     end

backend/app/controllers/post_imports_controller.rb

@@ -1,49 +0,0 @@
-class PostImportsController < ApplicationController
-  before_action :require_member!
-
-  def preview
-    rows = PostImportPreviewer.new.preview_rows(
-        rows: PostImportUrlListParser.parse(params[:source]))
-    render json: { rows: }
-  rescue ArgumentError => e
-    render_bad_request e.message
-  end
-
-  def validate
-    rows = normalised_import_rows allow_warning_fields: true
-    changed_row =
-        if params[:changed_row].to_s == 'all'
-          true
-        else
-          Integer(params[:changed_row], exception: false)
-        end
-    result =
-        PostImportPreviewer.new.preview_rows(rows:,
-                                             fetch_metadata: changed_row,
-                                             metadata_cache: { })
-    render json: { rows: result }
-  rescue ArgumentError => e
-    render_bad_request e.message
-  end
-
-  def create
-    result = PostImportRunner.new(actor: current_user,
-                                  rows: normalised_import_rows).run
-    render json: result, status: result[:created].positive? ? :created : :ok
-  rescue ArgumentError => e
-    render_bad_request e.message
-  end
-
-  private
-
-  def require_member!
-    return head :unauthorized unless current_user
-    return if current_user.gte_member?
-
-    head :forbidden
-  end
-
-  def normalised_import_rows allow_warning_fields: false
-    PostImportRowNormaliser.normalise!(params[:rows], allow_warning_fields:)
-  end
-end

backend/app/controllers/post_versions_controller.rb

@@ -1,7 +1,11 @@
 class PostVersionsController < ApplicationController
   def index
+    locale = resolve_locale!
     post_id = params[:post].presence
-    tag_id = params[:tag].presence
+    tag_id = params[:tag].presence&.to_i
+    external_tag_id = params[:external_tag].presence&.to_i
+    return head :bad_request if tag_id && external_tag_id
+
     page = (params[:page].presence || 1).to_i
     limit = (params[:limit].presence || 20).to_i
 
@@ -10,30 +14,16 @@ class PostVersionsController < ApplicationController
 
     offset = (page - 1) * limit
 
-    tag_name =
+    q = HistoryAccess.visible(version_scope)
-        if tag_id
-          TagName.joins(:tag).find_by(tag: { id: tag_id })
-        end
-    return render json: { versions: [], count: 0 } if tag_id && tag_name.blank?
-
-    q = PostVersion.joins(<<~SQL.squish)
-          LEFT JOIN
-            post_versions prev
-            ON
-              prev.post_id = post_versions.post_id
-              AND prev.version_no = post_versions.version_no - 1
-        SQL
-        .select('post_versions.*', 'prev.title AS prev_title', 'prev.url AS prev_url',
-                'prev.thumbnail_base AS prev_thumbnail_base', 'prev.tags AS prev_tags',
-                'prev.video_ms AS prev_video_ms',
-                'prev.original_created_from AS prev_original_created_from',
-                'prev.original_created_before AS prev_original_created_before')
     q = q.where('post_versions.post_id = ?', post_id) if post_id
-    if tag_name
+    if external_tag_id || (tag_id && !(Tag.exists?(id: tag_id)))
-      escaped = ActiveRecord::Base.sanitize_sql_like(tag_name.name)
+      q = q.where('JSON_CONTAINS(post_versions.tags_json,' +
-      q = q.where(("CONCAT(' ', post_versions.tags, ' ') LIKE :kw " +
+                  "JSON_OBJECT('external_tag_id', #{ external_tag_id || tag_id })) " +
-                   "OR CONCAT(' ', prev.tags, ' ') LIKE :kw"),
+                  'OR JSON_CONTAINS(prev.tags_json,' +
-                  kw: "% #{ escaped } %")
+                  "JSON_OBJECT('external_tag_id', #{ external_tag_id || tag_id }))")
+    elsif tag_id
+      q = q.where("JSON_CONTAINS(post_versions.tags_json, JSON_OBJECT('tag_id', #{ tag_id })) " +
+                  "OR JSON_CONTAINS(prev.tags_json, JSON_OBJECT('tag_id', #{ tag_id }))")
     end
 
     count = q.except(:select, :order, :limit, :offset).count
@@ -42,83 +32,129 @@ class PostVersionsController < ApplicationController
                 .limit(limit)
                 .offset(offset)
 
-    render json: { versions: serialise_versions(versions), count: }
+    render json: { versions: serialise_versions(versions, locale.language_code), count:, population_count: HistoryAccess.population_count(PostVersion) }
   end
 
   private
 
-  def serialise_versions rows
+  def version_scope
+    PostVersion.with_previous_version
+        .select('post_versions.*',
+                'prev.title AS prev_title',
+                'prev.url AS prev_url',
+                'prev.thumbnail_base AS prev_thumbnail_base',
+                'prev.tags_json AS prev_tags_json',
+                'prev.video_ms AS prev_video_ms',
+                'prev.original_created_from AS prev_original_created_from',
+                'prev.original_created_before AS prev_original_created_before')
+  end
+
+  def serialise_versions rows, language_code
+    rows = rows.to_a
     user_ids = rows.map(&:created_by_user_id).compact.uniq
     users_by_id = User.where(id: user_ids).pluck(:id, :name).to_h
+    snapshots = rows.flat_map { |row|
+      [normalise_json(row.tags_json),
+       normalise_json(row.attributes['prev_tags_json']) || []]
+    }
+    tag_ids = snapshots.flatten.filter_map { _1['tag_id'] }.uniq
+    tag_names = TagName.where(tag_id: tag_ids, language_code:, primary_flg: true)
+                       .pluck(:tag_id, :name).to_h
+    external_tag_names = external_tag_names_for(snapshots)
 
     rows.map do |row|
-      cur_tags = split_tags(row.tags)
+      cur_tags = snapshot_tag_literals(
-      prev_tags = split_tags(row.attributes['prev_tags'])
+          normalise_json(row.tags_json), tag_names, external_tag_names)
+      prev_tags = snapshot_tag_literals(
+          normalise_json(row.attributes['prev_tags_json']) || [], tag_names, external_tag_names)
 
-      {
+      { post_id: row.post_id,
-        post_id: row.post_id,
         version_no: row.version_no,
         event_type: row.event_type,
-        title: {
+        title: { current: row.title, prev: row.attributes['prev_title'] },
-          current: row.title,
+        url: { current: row.url, prev: row.attributes['prev_url'] },
-          prev: row.attributes['prev_title']
+        thumbnail: { current: nil, prev: nil },
-        },
+        thumbnail_base: { current: row.thumbnail_base,
-        url: {
+                          prev: row.attributes['prev_thumbnail_base'] },
-          current: row.url,
+        video_ms: { current: row.video_ms, prev: row.attributes['prev_video_ms'] },
-          prev: row.attributes['prev_url']
-        },
-        thumbnail: {
-          current: nil,
-          prev: nil
-        },
-        thumbnail_base: {
-          current: row.thumbnail_base,
-          prev: row.attributes['prev_thumbnail_base']
-        },
-        video_ms: {
-          current: row.video_ms,
-          prev: row.attributes['prev_video_ms']
-        },
         tags: build_version_tags(cur_tags, prev_tags),
+        parent_post_ids: row.parent_post_ids,
         original_created_from: {
-          current: row.original_created_from&.iso8601,
+            current: row.original_created_from&.iso8601,
-          prev: row.attributes['prev_original_created_from']&.iso8601
+            prev: row.attributes['prev_original_created_from']&.iso8601 },
-        },
         original_created_before: {
-          current: row.original_created_before&.iso8601,
+            current: row.original_created_before&.iso8601,
-          prev: row.attributes['prev_original_created_before']&.iso8601
+            prev: row.attributes['prev_original_created_before']&.iso8601 },
-        },
         created_at: row.created_at.iso8601,
         created_by_user:
-          if row.created_by_user_id
+            if row.created_by_user_id
-            {
+              { id: row.created_by_user_id,
-              id: row.created_by_user_id,
+                name: users_by_id[row.created_by_user_id] }
-              name: users_by_id[row.created_by_user_id]
+            end }
-            }
+    end
+  end
+
+  def external_tag_names_for snapshots
+    ids = snapshots.flatten.filter_map { _1['external_tag_id'] }.uniq
+    names = ExternalTag.where(id: ids).pluck(:id, :platform, :name).to_h { |id, platform, name|
+      [id, "#{ platform }:#{ name }"]
+    }
+    missing_ids = ids - names.keys
+
+    HistoryAccess.visible(NicoTagVersion.all)
+        .where(tag_id: missing_ids)
+        .order(:tag_id, version_no: :desc)
+        .pluck(:tag_id, :name)
+        .each { |id, name| names[id] ||= name }
+
+    names
+  end
+
+  def snapshot_tag_literals snapshots, tag_names, external_tag_names
+    snapshots
+        .filter_map { |snapshot|
+          if snapshot.key?('tag_id')
+            sections = snapshot.fetch('sections', []).map { |section|
+              [section.fetch('begin_ms'), section['end_ms']]
+            }.sort_by { |begin_ms, end_ms| [begin_ms, end_ms || Float::INFINITY] }
+            name = tag_names[snapshot['tag_id']] || snapshot.fetch('name')
+            [tag_category_order(snapshot['category']),
+             { identity: [:tag, snapshot['tag_id'], sections],
+               name: Post.tag_snapshot_literal(snapshot.merge('name' => name)),
+               restore_name: tag_names[snapshot['tag_id']] &&
+                   Post.tag_snapshot_literal(snapshot.merge('name' => name)),
+               external: false }]
+          elsif external_tag_names[snapshot['external_tag_id']]
+            [6, { identity: [:external_tag, snapshot['external_tag_id']],
+                  name: external_tag_names[snapshot['external_tag_id']],
+                  restore_name: nil, external: true }]
           end
-      }
+        }
-    end
+        .sort_by { |order, tag| [order, tag[:name].downcase] }
+        .map(&:second)
   end
 
-  def build_version_tags(cur_tags, prev_tags)
+  def tag_category_order category
-    (cur_tags | prev_tags).map do |name|
+    ['deerjikist', 'meme', 'character', 'general', 'material', 'meta'].index(category) || 6
+  end
+
+  def build_version_tags cur_tags, prev_tags
+    current = cur_tags.index_by { _1[:identity] }
+    previous = prev_tags.index_by { _1[:identity] }
+
+    (current.keys | previous.keys).map do |identity|
       type =
-        if cur_tags.include?(name) && prev_tags.include?(name)
+          if current.key?(identity) && previous.key?(identity)
-          'context'
+            'context'
-        elsif cur_tags.include?(name)
+          elsif current.key?(identity)
-          'added'
+            'added'
-        else
+          else
-          'removed'
+            'removed'
-        end
+          end
 
-      {
+      entry = current[identity] || previous[identity]
-        name:,
+      { name: entry[:name], restore_name: entry[:restore_name],
-        type:
+        external: entry[:external], type: }
-      }
     end
   end
-
-  def split_tags(tags)
-    tags.to_s.split(/\s+/).reject(&:blank?)
-  end
 end

backend/app/controllers/posts_controller.rb

@@ -1,11 +1,14 @@
 class PostsController < ApplicationController
   Event = Struct.new(:post, :tag, :user, :change_type, :timestamp, keyword_init: true)
+  MAX_BULK_REQUEST_BYTES = 40 * 1024 * 1024
 
   class VideoMsParseError < ArgumentError
     ;
   end
 
   def index
+    locale = resolve_locale!
+
     url = params[:url].presence
     title = params[:title].presence
     original_created_from = params[:original_created_from].presence
@@ -35,8 +38,8 @@ class PostsController < ApplicationController
     offset = (page - 1) * limit
 
     pt_max_sql =
-        PostTag
+        HistoryAccess.visible(PostVersion.all)
-        .select('post_id, MAX(updated_at) AS max_updated_at')
+        .select('post_id, MAX(created_at) AS max_updated_at')
         .group('post_id')
         .to_sql
 
@@ -45,13 +48,12 @@ class PostsController < ApplicationController
         'COALESCE(pt_max.max_updated_at, posts.updated_at))'
 
     q =
-        filtered_posts
+        filtered_posts(locale)
         .joins("LEFT JOIN (#{ pt_max_sql }) pt_max ON pt_max.post_id = posts.id")
         .reselect('posts.*', Arel.sql("#{ updated_at_all_sql } AS updated_at_all"))
-        .preload(:uploaded_user, :parents, :children,
+        .preload(:external_tags, :uploaded_user, :parents, :children,
-                 active_post_tags: [:sections,
+                 post_tags: [:sections, { tag: [:deerjikists, :materials,
-                                    { tag: [:deerjikists, :materials,
+                                                :tag_names, :tag_descriptions] }])
-                                            { tag_name: :wiki_page }] }])
         .with_attached_thumbnail
 
     q = q.where('posts.url LIKE ?', "%#{ url }%") if url
@@ -90,7 +92,8 @@ class PostsController < ApplicationController
     q = q.except(:select, :order)
 
     render json: { posts: posts.map { |post|
-      PostRepr.base(post).merge(updated_at: post.updated_at_all).tap do |json|
+      PostRepr.base(post, language_code: locale.language_code)
+              .merge(updated_at: post.updated_at_all).tap do |json|
         json['thumbnail'] =
             if post.thumbnail.attached?
               rails_storage_proxy_url(post.thumbnail, only_path: false)
@@ -102,25 +105,100 @@ class PostsController < ApplicationController
   end
 
   def random
-    post = filtered_posts.preload(:uploaded_user, :parents, :children,
+    locale = resolve_locale!
-                                  active_post_tags: [:sections,
+
-                                                     { tag: [:deerjikists, :materials,
+    post =
-                                                             { tag_name: :wiki_page }] }])
+        filtered_posts(locale)
-                         .with_attached_thumbnail
+        .preload(:uploaded_user, :parents, :children,
-                         .order('RAND()')
+                 post_tags: [:sections, { tag: [:deerjikists, :materials,
-                         .first
+                                                :tag_names, :tag_descriptions] }])
+        .with_attached_thumbnail
+        .order('RAND()')
+        .first
     return head :not_found unless post
 
-    render json: PostRepr.base(post, current_user)
+    render json: PostRepr.base(post, current_user, language_code: locale.language_code)
+  end
+
+  def metadata
+    return head :unauthorized unless current_user
+    return head :forbidden unless current_user.gte_member?
+    return render_bad_request('URL は必須です．') if params[:url].blank?
+
+    normal_url = PostUrlNormaliser.normalise(params[:url].to_s)
+    return render_validation_error(fields: { url: ['URL が不正です．'] }) if normal_url.blank?
+
+    Preview::UrlSafety.validate(normal_url)
+    existing_post = Post.with_attached_thumbnail.find_by(url: normal_url)
+    if existing_post.present?
+      return render json: {
+          url: normal_url,
+          title: nil,
+          thumbnail_base: nil,
+          tags: nil,
+          display_tags: [],
+          original_created_from: nil,
+          original_created_before: nil,
+          duration: nil,
+          video_ms: nil,
+          field_warnings: { },
+          base_warnings: [],
+          existing_post_id: existing_post.id,
+          existing_post: compact_post(existing_post.id) }
+    end
+
+    metadata = PostMetadataFetcher.fetch(normal_url)
+    field_warnings = { }
+    field_warnings[:title] = ['タイトルを取得できませんでした．'] if metadata[:title].blank?
+    if metadata[:thumbnail_base].blank?
+      field_warnings[:thumbnail_base] = ['サムネールを取得できませんでした．']
+    end
+
+    render json: {
+        url: normal_url,
+        title: metadata[:title],
+        thumbnail_base: metadata[:thumbnail_base],
+        tags: metadata[:tags],
+        display_tags: metadata[:display_tags],
+        original_created_from: metadata[:original_created_from],
+        original_created_before: metadata[:original_created_before],
+        duration: metadata[:duration],
+        video_ms: metadata[:video_ms],
+        field_warnings: field_warnings,
+        base_warnings: [],
+        existing_post_id: nil,
+        existing_post: nil }
+  rescue ArgumentError => e
+    render_bad_request e.message
+  rescue Preview::UrlSafety::UnsafeUrl => e
+    render_validation_error fields: { url: [e.message] }
+  rescue Preview::HttpFetcher::FetchFailed,
+         Preview::HttpFetcher::FetchTimeout,
+         Preview::HttpFetcher::ResponseTooLarge
+    render json: {
+        url: normal_url,
+        title: nil,
+        thumbnail_base: nil,
+        tags: nil,
+        display_tags: [],
+        original_created_from: nil,
+        original_created_before: nil,
+        duration: nil,
+        video_ms: nil,
+        field_warnings: { url: ['自動取得に失敗しました．'] },
+        base_warnings: [],
+        existing_post_id: nil,
+        existing_post: nil }
   end
 
   def show
+    locale = resolve_locale!
+
     post =
         Post
         .includes(:uploaded_user, :parents, :children,
-                  active_post_tags: [:sections,
+                  post_tags: [:sections, { tag: [:deerjikists, :materials,
-                                     { tag: [:deerjikists, :materials,
+                                                 :tag_names, :tag_descriptions] }])
-                                             { tag_name: :wiki_page }] }])
         .with_attached_thumbnail
         .find_by(id: params[:id])
     return head :not_found unless post
@@ -134,25 +212,52 @@ class PostsController < ApplicationController
                                  parent_posts:,
                                  child_posts:,
                                  sibling_posts:,
-                                 related:)
+                                 related:,
-                         .merge(tags: build_tag_tree_for(post))
+                                 language_code: locale.language_code)
+                         .merge(tags: build_tag_tree_for(post, locale.language_code))
   end
 
   def create
     return head :unauthorized unless current_user
     return head :forbidden unless current_user.gte_member?
 
+    locale = resolve_locale!
+
+    preflight = PostCreatePreflight.new(
+        attributes: post_create_attributes,
+        thumbnail: params[:thumbnail],
+        host: request.base_url).run(locale)
+    return render json: dry_run_json(preflight) if bool?(:dry)
+    if preflight[:existing_post_id].present?
+      post = Post.new(url: preflight[:url])
+      post.errors.add :url, :taken
+      return render_post_form_record_invalid post
+    end
+
     post = PostCreator.new(actor: current_user,
-                           attributes: { title: params[:title], url: params[:url],
+                           attributes: post_create_attributes.merge(
-                                         thumbnail: params[:thumbnail], tags: params[:tags],
+                               preflight.slice(
-                                         original_created_from: params[:original_created_from],
+                                   :url,
-                                         original_created_before: params[:original_created_before],
+                                   :title,
-                                         parent_post_ids: parse_parent_post_ids,
+                                   :thumbnail_base,
-                                         video_ms: params[:video_ms],
+                                   :tags,
-                                         duration: params[:duration] }).create!
+                                   :parent_post_ids,
+                                   :original_created_from,
+                                   :original_created_before,
+                                   :duration,
+                                   :video_ms,
+                                   :direct_tag_specs,
+                                   :default_tag_specs,
+                                   :snapshot_tag_specs,
+                                   :post_tag_specs,
+                                   :tag_sections,
+                                   :normalised_parent_post_ids).symbolize_keys).merge(
+                               thumbnail: params[:thumbnail])).create!(locale)
 
     post.reload
-    render json: PostRepr.base(post), status: :created
+    render json: PostRepr.base(post, language_code: locale.language_code), status: :created
+  rescue PostCreatePreflight::ValidationFailed => e
+    render_validation_error fields: e.fields, base: e.base_errors
   rescue Tag::NicoTagNormalisationError
     render_validation_error fields: { tags: 'ニコニコ・タグは直接指定できません．' }
   rescue Tag::DeprecatedTagNormalisationError
@@ -161,6 +266,8 @@ class PostsController < ApplicationController
     render_validation_error fields: { tags: ['タグ区間の記法が不正です．'] }
   rescue PostCreator::VideoMsParseError
     render_validation_error fields: { video_ms: ['動画時間の記法が不正です．'] }
+  rescue Post::RemoteThumbnailFetchFailed
+    render_validation_error fields: { thumbnail_base: ['サムネイル画像の取得に失敗しました．'] }
   rescue MiniMagick::Error
     render_validation_error fields: { thumbnail: ['サムネイル画像の変換に失敗しました．'] }
   rescue ArgumentError => e
@@ -169,6 +276,30 @@ class PostsController < ApplicationController
     render_post_form_record_invalid e.record
   end
 
+  def bulk
+    return head :unauthorized unless current_user
+    return head :forbidden unless current_user.gte_member?
+    return head :unsupported_media_type unless request.content_mime_type == Mime[:multipart_form]
+    return head :payload_too_large if request.content_length.to_i > MAX_BULK_REQUEST_BYTES
+
+    locale = resolve_locale!
+
+    posts = parse_bulk_posts_manifest
+    thumbnails = parse_bulk_thumbnails(posts.length)
+
+    result = PostBulkCreator.new(
+        actor: current_user,
+        posts:,
+        thumbnails:,
+        host: request.base_url).run(locale)
+
+    render json: result
+  rescue JSON::ParserError
+    render_bad_request 'posts manifest の JSON が不正です．'
+  rescue ArgumentError => e
+    render_validation_error base: [e.message]
+  end
+
   def viewed
     return head :unauthorized unless current_user
 
@@ -194,6 +325,7 @@ class PostsController < ApplicationController
     base_version_no = parse_base_version_no
     return render_bad_request('base_version_no は必須です．') if !(force) && !(base_version_no)
 
+    locale = Locale.find_by(code: params[:locale].presence) || Locale.nipponese
     title = params[:title].presence
     tag_names = params[:tags].to_s.split
     original_created_from = params[:original_created_from]
@@ -212,10 +344,12 @@ class PostsController < ApplicationController
       unless force
         base_version = post.post_versions.find_by!(version_no: base_version_no)
 
+        HistoryAccess.require!(base_version)
         base_snapshot = post_snapshot_from_version(base_version)
-        current_snapshot = post_snapshot_from_record(post)
+        current_snapshot = post_snapshot_from_record(post, locale)
       end
-      incoming_snapshot = post_incoming_snapshot(title:,
+      incoming_snapshot = post_incoming_snapshot(locale:,
+                                                 title:,
                                                  original_created_from:,
                                                  original_created_before:,
                                                  tag_names:,
@@ -244,14 +378,14 @@ class PostsController < ApplicationController
             end
           end
 
-      apply_post_snapshot!(post, snapshot_to_apply)
+      apply_post_snapshot!(locale, post, snapshot_to_apply)
     end
 
     return render json: conflict_json, status: :conflict if conflict_json
 
     post.reload
-    json = PostRepr.base(post, current_user)
+    json = PostRepr.base(post, current_user, language_code: locale.language_code)
-    json['tags'] = build_tag_tree_for(post)
+    json['tags'] = build_tag_tree_for(post, locale.language_code)
     render json:, status: :ok
   rescue Tag::NicoTagNormalisationError
     render_validation_error fields: { tags: ['ニコニコ・タグは直接指定できません．'] }
@@ -267,77 +401,84 @@ class PostsController < ApplicationController
     render_post_form_record_invalid e.record
   end
 
-  def changes
+  def tag_filters
-    id = params[:id].presence
+    source = resolve_locale!
-    tag_id = params[:tag].presence
+    code = params[:target_locale]
-    page = (params[:page].presence || 1).to_i
+    return head :not_found unless code.is_a?(String) && Locale.codes.value?(code)
-    limit = (params[:limit].presence || 20).to_i
 
-    page = 1 if page < 1
+    target = Locale.find_by(code:, deprecated_at: nil)
-    limit = 1 if limit < 1
+    return head :not_found unless target
 
-    offset = (page - 1) * limit
+    literals = params[:tags].to_s.split
-
+    references = tag_filter_references(literals)
-    pts = PostTag.with_discarded
+    names = literals.each_with_index.map do |literal, index|
-    pts = pts.where(post_id: id) if id.present?
+      reference = references&.[](index)
-    pts = pts.where(tag_id:) if tag_id.present?
+      (reference && reference['negative'] == false) ? literal : literal.sub(/\Anot:/i, '')
-    pts = pts.includes(:post, :created_user, :deleted_user,
-                       tag: [:deerjikists, :materials, { tag_name: :wiki_page }])
-
-    events = []
-    pts.each do |pt|
-      tag = TagRepr.base(pt.tag)
-      post = pt.post
-
-      events << Event.new(
-          post:,
-          tag:,
-          user: pt.created_user && { id: pt.created_user.id, name: pt.created_user.name },
-          change_type: 'add',
-          timestamp: pt.created_at)
-
-      if pt.discarded_at
-        events << Event.new(
-            post:,
-            tag:,
-            user: pt.deleted_user && { id: pt.deleted_user.id, name: pt.deleted_user.name },
-            change_type: 'remove',
-            timestamp: pt.discarded_at)
-      end
     end
-    events.sort_by!(&:timestamp)
+    internal = references ? {} : TagName.where(
-    events.reverse!
+        language_code: source.language_code, name: names).pluck(:name, :tag_id).to_h
+    canonical = references ? {} : TagName.where(
+        language_code: source.language_code, primary_flg: true,
+        tag_id: internal.values).pluck(:tag_id, :name).to_h
+    external_names = names.map { canonical[internal[_1]] || _1 }
+    external = references ? {} : ExternalTag.where(
+        "CONCAT(platform, ':', name) IN (?)", external_names).pluck(
+        Arel.sql("CONCAT(platform, ':', name)"), :id).to_h
+    ids = references ? references.filter_map { _1['tag_id'] } : internal.values
+    translated = TagName.where(language_code: target.language_code,
+                               primary_flg: true, tag_id: ids).pluck(:tag_id, :name).to_h
 
-    render json: { changes: (events.slice(offset, limit) || []).as_json, count: events.size }
+    filters = literals.each_with_index.map do |literal, index|
+      name = names[index]
+      reference = if references
+                    references[index].except('name')
+                  else
+                    matches = {}
+                    matches['tag_id'] = internal[name] if internal[name]
+                    external_id = external[external_names[index]]
+                    matches['external_tag_id'] = external_id if external_id
+                    matches.empty? ? { 'missing' => true } : matches
+                  end
+      name = translated[reference['tag_id']] || name
+      negative = reference.fetch('negative', literal.downcase.start_with?('not:'))
+      prefix = negative ? literal[/\Anot:/i] || 'not:' : ''
+      reference.merge('name' => "#{ prefix }#{ name }", 'negative' => negative)
+    end
+    render json: { references: filters }
   end
 
   private
 
-  def filtered_posts
+  def tag_filter_references literals
+    PostTagSearch.parse_references(params.to_unsafe_h, literals)
+  rescue ArgumentError, JSON::ParserError, TypeError
+    raise ActionController::BadRequest
+  end
+
+  def filtered_posts locale
     tag_names = params[:tags].to_s.split
     match_type = params[:match]
     if tag_names.present?
-      filter_posts_by_tags(tag_names, match_type)
+      filter_posts_by_tags(locale, tag_names, match_type)
     else
       Post.all
     end
   end
 
-  def filter_posts_by_tags tag_names, match_type
+  def filter_posts_by_tags locale, tag_names, match_type
-    literals = tag_names.map do |raw_name|
+    references = tag_filter_references(tag_names)
-      { name: TagName.canonicalise(raw_name.sub(/\Anot:/i, '')).first,
+    literals = PostTagSearch.literals(locale, tag_names.join(' '), references:)
-        negative: raw_name.downcase.start_with?('not:') }
-    end
 
     return Post.all if literals.empty?
 
     if match_type == 'any'
       literals.reduce(Post.none) do |posts, literal|
-        posts.or(tag_literal_relation(literal[:name], negative: literal[:negative]))
+        posts.or(tag_literal_relation(locale, literal[:name], negative: literal[:negative],
+                                      reference: literal[:reference]))
       end
     else
       literals.reduce(Post.all) do |posts, literal|
-        ids = tagged_post_ids_for(literal[:name])
+        ids = tagged_post_ids_for(locale, literal[:name], reference: literal[:reference])
         if literal[:negative]
           posts.where.not(id: ids)
         else
@@ -347,8 +488,8 @@ class PostsController < ApplicationController
     end
   end
 
-  def tag_literal_relation name, negative:
+  def tag_literal_relation locale, name, negative:, reference: nil
-    ids = tagged_post_ids_for(name)
+    ids = tagged_post_ids_for(locale, name, reference:)
     if negative
       Post.where.not(id: ids)
     else
@@ -356,8 +497,12 @@ class PostsController < ApplicationController
     end
   end
 
-  def tagged_post_ids_for(name) =
+  def tagged_post_ids_for locale, name, reference: nil
-      Post.joins(tags: :tag_name).where(tag_names: { name: }).select(:id)
+    internal, external = PostTagSearch.references(locale, { name:, reference: })
+    Post.where(id: PostTag.where(tag_id: Array(internal)).select(:post_id))
+        .or(Post.where(id: PostExternalTag.where(external_tag_id: Array(external))
+            .select(:post_id))).select(:id)
+  end
 
   def sync_post_tags! post, desired_tags, sections
     desired_tags.each do |t|
@@ -385,13 +530,13 @@ class PostsController < ApplicationController
       end
     end
 
-    PostTag.where(post_id: post.id, tag_id: to_remove.to_a).kept.find_each do |pt|
+    PostTag.where(post_id: post.id, tag_id: to_remove.to_a).find_each do |pt|
-      pt.discard_by!(current_user)
+      pt.destroy!
     end
   end
 
-  def build_tag_tree_for post
+  def build_tag_tree_for post, language_code = 'ja'
-    post_tags = post.active_post_tags.reject { |post_tag| post_tag.tag.deprecated? }
+    post_tags = post.post_tags.reject { |post_tag| post_tag.tag.deprecated? }
     tags = post_tags.map(&:tag)
     tag_ids = tags.map(&:id)
 
@@ -420,7 +565,7 @@ class PostsController < ApplicationController
       sections = sections_by_tag_id.fetch(tag_id, [])
 
       if path.include?(tag_id)
-        return TagRepr.inline(tag).merge(children: [], sections:)
+        return TagRepr.inline(tag, language_code:).merge(children: [], sections:)
       end
 
       if memo.key?(tag_id)
@@ -432,10 +577,13 @@ class PostsController < ApplicationController
 
       children = child_ids.filter_map { |cid| build_node.(cid, new_path) }
 
-      memo[tag_id] = TagRepr.inline(tag).merge(children:, sections:)
+      memo[tag_id] = TagRepr.inline(tag, language_code:).merge(children:, sections:)
     end
 
-    root_ids.filter_map { |id| build_node.call(id, []) }
+    internal_tags = root_ids.filter_map { |id| build_node.call(id, []) }
+    external_tags =
+        post.external_tags.map { ExternalTagRepr.inline(_1).merge(children: [], sections: []) }
+    internal_tags + external_tags
   end
 
   def sibling_posts_by_parent parent_post_ids
@@ -463,6 +611,79 @@ class PostsController < ApplicationController
     }.uniq
   end
 
+  def post_create_attributes
+    { title: params[:title],
+      url: params[:url],
+      thumbnail_base: params[:thumbnail_base],
+      tags: params[:tags],
+      original_created_from: params[:original_created_from],
+      original_created_before: params[:original_created_before],
+      parent_post_ids: parse_parent_post_ids,
+      video_ms: params[:video_ms],
+      duration: params[:duration] }
+  end
+
+  def parse_bulk_posts_manifest
+    manifest = params[:posts]
+    raise ArgumentError, 'posts は必須です．' if manifest.blank?
+    raise ArgumentError, 'posts は JSON 文字列で指定してください．' unless manifest.is_a?(String)
+
+    posts = JSON.parse(manifest)
+    raise ArgumentError, 'posts は配列で指定してください．' unless posts.is_a?(Array)
+    raise ArgumentError, '投稿件数は 1 件以上必要です．' if posts.empty?
+    raise ArgumentError, '投稿件数が多すぎます．' if posts.length > 100
+    raise ArgumentError, 'posts 要素の形式が不正です．' unless posts.all? { _1.is_a?(Hash) }
+
+    posts
+  end
+
+  def parse_bulk_thumbnails post_count
+    thumbnails = { }
+    raw = params[:thumbnails]
+    return thumbnails if raw.blank?
+    raise ArgumentError, 'thumbnail key が不正です．' unless raw.respond_to?(:to_unsafe_h)
+
+    raw.to_unsafe_h.each do |key, value|
+      raise ArgumentError, 'thumbnail key が不正です．' unless key.to_s.match?(/\A\d+\z/)
+
+      index = Integer(key, 10)
+      raise ArgumentError, 'thumbnail index が範囲外です．' if index.negative? || index >= post_count
+      raise ArgumentError, 'thumbnail index が重複しています．' if thumbnails.key?(index)
+      unless value.is_a?(ActionDispatch::Http::UploadedFile)
+        raise ArgumentError, 'thumbnail upload が不正です．'
+      end
+
+      thumbnails[index] = value
+    end
+
+    thumbnails
+  end
+
+  def compact_post post_id
+    return nil if post_id.blank?
+
+    post = Post.with_attached_thumbnail.find_by(id: post_id)
+    PostCompactRepr.base(post, host: request.base_url)
+  end
+
+  def dry_run_json preflight
+    preflight.slice(
+        :url,
+        :title,
+        :thumbnail_base,
+        :tags,
+        :display_tags,
+        :parent_post_ids,
+        :original_created_from,
+        :original_created_before,
+        :duration,
+        :video_ms,
+        :field_warnings,
+        :base_warnings,
+        :existing_post_id,
+        :existing_post)
+  end
+
   def sync_parent_posts! post, parent_post_ids
     if parent_post_ids.include?(post.id)
       post.errors.add :parent_post_ids, '自分自身を親投稿にはできません．'
@@ -509,42 +730,50 @@ class PostsController < ApplicationController
   end
 
   def editable_tag_names_from_version version
-    version.tags.to_s.split.reject { |name| name.downcase.start_with?('nico:') }.sort
+    version.tags_json
+        .select { _1.key?('tag_id') }
+        .map { Post.tag_snapshot_literal(_1) }
+        .compact
+        .sort
   end
 
-  def post_snapshot_from_record post
+  def post_snapshot_from_record post, locale
     { title: post.title,
       video_ms: post.video_ms,
       original_created_from: snapshot_time(post.original_created_from),
       original_created_before: snapshot_time(post.original_created_before),
-      tag_names: editable_tag_names_from_post(post),
+      tag_names: editable_tag_names_from_post(post, locale.language_code),
       parent_post_ids: post.parent_posts.order(:id).pluck(:id) }
   end
 
-  def editable_tag_names_from_post post
+  def editable_tag_names_from_post post, language_code
     post
         .post_tags
-        .kept
+        .joins(tag: :tag_names)
-        .joins(tag: :tag_name)
-        .merge(Tag.not_nico)
         .merge(Tag.where(deprecated_at: nil))
-        .includes(:sections, tag: :tag_name)
+        .where(tag_names: { language_code:, primary_flg: true })
+        .includes(:sections, tag: :tag_names)
         .order('tag_names.name')
         .map do |post_tag|
-      name = post_tag.tag.tag_name.name
+      name = post_tag.tag.name(language_code)
       sections = post_tag.sections.sort_by(&:begin_ms)
 
       next name if sections.empty?
 
       "#{ name }#{ sections.map { Post.section_literal(_1) }.join }"
-    end
+    end.compact.sort
   end
 
-  def post_incoming_snapshot title:, original_created_from:, original_created_before:,
+  def post_incoming_snapshot locale:, title:, original_created_from:, original_created_before:,
                              tag_names:, video_ms_param:, duration_param:, parent_post_ids:
-    Tag.normalise_tags!(tag_names, with_tagme: false, deny_deprecated: true,
+    validate_original_created_values!(original_created_from, original_created_before)
-                                        with_sections: true) =>
+
-        { tags:, sections: }
+    Tag.normalise_tags!(locale, tag_names,
+                        with_tagme: false,
+                        with_no_deerjikist: false,
+                        deny_deprecated: true,
+                        with_sections: true,
+                        created_by_user: current_user) => { tags:, sections: }
 
     tags = Tag.expand_parent_tags(tags).reject(&:deprecated?)
     video_ms = normalise_video_ms(tags, video_ms_param:, duration_param:)
@@ -555,7 +784,7 @@ class PostsController < ApplicationController
       original_created_from: snapshot_time(original_created_from),
       original_created_before: snapshot_time(original_created_before),
       tag_names: tags.uniq(&:id).map { |tag|
-        "#{ tag.name }#{ sections[tag.id].to_a.map { section_literal(_1) }.join }"
+        "#{ tag.name(locale.language_code) }#{ sections[tag.id].to_a.map { section_literal(_1) }.join }"
       }.sort,
       parent_post_ids: parent_post_ids.sort }
   end
@@ -579,6 +808,23 @@ class PostsController < ApplicationController
     value.to_s
   end
 
+  def validate_original_created_values! original_created_from, original_created_before
+    candidate = Post.new(
+        url: 'https://example.invalid/original-created-validation',
+        original_created_from:,
+        original_created_before:)
+    candidate.valid?
+    fields = [:original_created_from, :original_created_before, :original_created_at]
+    relevant_errors = candidate.errors.select { fields.include?(_1.attribute) }
+    return if relevant_errors.empty?
+
+    invalid_post = Post.new
+    relevant_errors.each { |error|
+      invalid_post.errors.add(error.attribute, error.message)
+    }
+    raise ActiveRecord::RecordInvalid, invalid_post
+  end
+
   def section_literal section
     "[#{ Post.ms_to_time(section[0]) }-#{ section[1] ? Post.ms_to_time(section[1]) : '' }]"
   end
@@ -662,7 +908,7 @@ class PostsController < ApplicationController
     (added_by_current & removed_by_me).present? || (removed_by_current & added_by_me).present?
   end
 
-  def apply_post_snapshot! post, snapshot
+  def apply_post_snapshot! locale, post, snapshot
     PostVersionRecorder.ensure_snapshot!(post, created_by_user: current_user)
 
     post.update!(title: snapshot[:title],
@@ -670,15 +916,14 @@ class PostsController < ApplicationController
                  original_created_from: snapshot[:original_created_from],
                  original_created_before: snapshot[:original_created_before])
 
-    Tag.normalise_tags!(snapshot[:tag_names], with_tagme: false,
+    Tag.normalise_tags!(locale, snapshot[:tag_names],
-                                               deny_deprecated: true,
+                        with_tagme: false,
-                                               with_sections: true) =>
+                        deny_deprecated: true,
-        { tags: editable_tags, sections: }
+                        with_sections: true,
-    TagVersioning.record_tag_snapshots!(editable_tags, created_by_user: current_user)
+                        created_by_user: current_user) => { tags:, sections: }
+    TagVersioning.record_tag_snapshots!(tags, created_by_user: current_user,
+                                        language_code: locale.language_code)
 
-    readonly_tags = post.tags.nico.to_a
-
-    tags = readonly_tags + editable_tags
     tags = Tag.expand_parent_tags(tags).reject(&:deprecated?)
 
     post.video_ms = tags.any? { _1.id == Tag.video.id } ? snapshot[:video_ms] : nil

backend/app/controllers/preview_controller.rb

@@ -18,32 +18,9 @@ class PreviewController < ApplicationController
   def thumbnail
     return render_bad_request('URL は必須です．') if params[:url].blank?
 
-    image = MiniMagick::Image.read(Preview::ThumbnailFetcher.fetch(params[:url]))
+    attachment =
-    image.auto_orient
+        Post.resized_thumbnail_attachment(
-    image.resize '180x180>'
+            StringIO.new(Preview::ThumbnailFetcher.fetch(params[:url])))
-    image.format 'png'
-    width, height = image.dimensions
-    raise Preview::ThumbnailFetcher::GenerationFailed, 'サムネール画像の変換に失敗しました．' if width > 180 || height > 180
-
-    send_data image.to_blob, type: 'image/png', disposition: 'inline'
-  rescue Preview::UrlSafety::UnsafeUrl => e
-    render_bad_request(e.message)
-  rescue Preview::HttpFetcher::FetchTimeout => e
-    render_preview_error(e.message, :gateway_timeout)
-  rescue Preview::HttpFetcher::ResponseTooLarge => e
-    render_preview_error(e.message, :payload_too_large)
-  rescue Preview::HttpFetcher::FetchFailed => e
-    render_preview_error(e.message, :bad_gateway)
-  rescue Preview::ThumbnailFetcher::GenerationFailed, MiniMagick::Error => e
-    render_unprocessable_entity(e.message)
-  end
-
-  def image
-    return render_bad_request('URL は必須です．') if params[:url].blank?
-
-    attachment = Post.resized_thumbnail_attachment(
-        StringIO.new(Preview::ThumbnailFetcher.fetch_image_response(params[:url]).body))
-    attachment[:io].rewind
     send_data attachment[:io].read,
               type: attachment[:content_type],
               disposition: 'inline'

backend/app/controllers/sitemap_entries_controller.rb

@@ -0,0 +1,58 @@
+class SitemapEntriesController < ApplicationController
+  PAGE_SIZE = 1_000
+  RESOURCES = [
+      'posts', 'tags', 'deerjikists', 'materials',
+      'descriptions', 'tag_groups', 'theatres'].freeze
+
+  def index
+    locale = resolve_locale!
+    resources = RESOURCES.map { |resource| { resource:, count: entries(resource, locale).count } }
+    render json: { resources:, page_size: PAGE_SIZE }
+  end
+
+  def show
+    locale = resolve_locale!
+    resource = params[:resource]
+    return head :not_found unless RESOURCES.include?(resource)
+
+    page = Integer(params.fetch(:page, '1'), exception: false)
+    return head :bad_request unless page && page.positive?
+
+    scope = entries(resource, locale)
+    return head :not_found if page > [(scope.count.to_f / PAGE_SIZE).ceil, 1].max
+
+    ids = scope.order(:id).offset((page - 1) * PAGE_SIZE).limit(PAGE_SIZE).pluck(:id)
+    render json: { ids: }
+  end
+
+  private
+
+  def entries resource, locale
+    named_tags = TagName.where(language_code: locale.language_code, primary_flg: true)
+        .select(:tag_id)
+    case resource
+    when 'posts'
+      Post.all
+    when 'tags'
+      Tag.where(id: named_tags)
+    when 'deerjikists'
+      Tag.where(id: named_tags).where(id: Deerjikist.select(:tag_id))
+    when 'materials'
+      Material.all
+    when 'descriptions'
+      TagDescription.where(language_code: locale.language_code, tag_id: named_tags)
+          .where.not(body: [nil, ''])
+          .where(HistoryAccess.visible(TagDescriptionVersion.all)
+              .where('tag_description_versions.tag_id = tag_descriptions.tag_id')
+              .where('tag_description_versions.language_code = tag_descriptions.language_code')
+              .where('tag_description_versions.version_no = tag_descriptions.version_no')
+              .arel.exists)
+    when 'tag_groups'
+      TagGroup.joins(:versions)
+          .where('tag_group_versions.version_no = tag_groups.version_no')
+          .where(Arel.sql(HistoryAccess.sql(TagGroupVersion)))
+    when 'theatres'
+      Theatre.kept
+    end
+  end
+end

backend/app/controllers/tag_children_controller.rb

@@ -10,7 +10,6 @@ class TagChildrenController < ApplicationController
 
     parent = Tag.find(parent_id)
     child = Tag.find(child_id)
-    return render_bad_request('ニコニコ・タグの階層は変更できません．') if parent.nico? || child.nico?
 
     ApplicationRecord.transaction do
       TagVersioning.ensure_snapshot!(child, created_by_user: current_user)
@@ -33,7 +32,6 @@ class TagChildrenController < ApplicationController
 
     parent = Tag.find(parent_id)
     child = Tag.find(child_id)
-    return render_bad_request('ニコニコ・タグの階層は変更できません．') if parent.nico? || child.nico?
 
     ApplicationRecord.transaction do
       TagVersioning.ensure_snapshot!(child, created_by_user: current_user)

backend/app/controllers/tag_descriptions_controller.rb

@@ -0,0 +1,63 @@
+class TagDescriptionsController < ApplicationController
+  def show
+    locale = resolve_locale!
+    description = TagDescription.find_by(tag_id: params[:id], language_code: locale.language_code)
+    return head :not_found unless description
+
+    render json: representation(description)
+  end
+
+  def show_by_title
+    locale = resolve_locale!
+    tag_name = TagName.find_by(
+        language_code: locale.language_code, name: params[:name].to_s.strip)
+    description = tag_name&.tag && TagDescription.find_by(
+        tag: tag_name.tag, language_code: locale.language_code)
+    return head :not_found unless description
+
+    render json: representation(description)
+  end
+
+  def update
+    return head :unauthorized unless current_user
+    return head :forbidden unless current_user.gte_member?
+
+    body = params[:body].to_s
+    return render_unprocessable_entity('本文は必須です．', field: :body) if body.blank?
+
+    locale = resolve_locale!
+    tag = Tag.find_by(id: params[:id])
+    return head :not_found unless tag
+
+    description = TagDescription.find_by(tag:, language_code: locale.language_code)
+
+    if description
+      TagDescriptionOperation.update!(locale, tag:, body:, updated_by_user: current_user)
+      description.reload
+      render json: representation(description)
+    else
+      TagDescriptionOperation.create!(locale, tag:, body:, created_by_user: current_user)
+      description = TagDescription.find_by!(tag:, language_code: locale.language_code)
+      render json: representation(description), status: :created
+    end
+  rescue ActiveRecord::RecordInvalid => e
+    render_validation_error e.record
+  rescue ActiveRecord::RecordNotUnique
+    render_record_not_unique
+  end
+
+  private
+
+  def representation description
+    HistoryAccess.require_current!(description, TagDescriptionVersion)
+    description.as_json(only: [
+        :tag_id,
+        :language_code,
+        :body,
+        :version_no,
+        :created_by_user_id,
+        :updated_by_user_id,
+        :created_at,
+        :updated_at])
+  end
+end

backend/app/controllers/tag_groups_controller.rb

@@ -0,0 +1,43 @@
+class TagGroupsController < ApplicationController
+  def index
+    locale = resolve_locale!
+    groups = HistoryAccess.current(TagGroup.all, TagGroupVersion).includes(:created_by_user, :updated_by_user).order(:id)
+    render json: TagGroupRepr.many(locale, groups.to_a)
+  end
+
+  def show
+    locale = resolve_locale!
+    group = TagGroup.includes(:created_by_user, :updated_by_user).find_by(id: params[:id])
+    return head :not_found unless group
+
+    render json: TagGroupRepr.one(locale, group)
+  end
+
+  def create
+    return head :unauthorized unless current_user
+    return head :forbidden unless current_user.gte_member?
+
+    locale = resolve_locale!
+    group = TagGroupOperation.create!(locale, **write_attributes, actor: current_user)
+    render json: TagGroupRepr.one(locale, group), status: :created
+  end
+
+  def update
+    return head :unauthorized unless current_user
+    return head :forbidden unless current_user.gte_member?
+
+    locale = resolve_locale!
+    group = TagGroup.find_by(id: params[:id])
+    return head :not_found unless group
+
+    TagGroupOperation.update!(locale, group:, **write_attributes, actor: current_user)
+    render json: TagGroupRepr.one(locale, group.reload)
+  end
+
+  private
+
+  def write_attributes
+    { name: params[:name], description: params[:description],
+      members: params[:members], placements: params[:placements] }
+  end
+end

backend/app/controllers/tag_versions_controller.rb

@@ -1,5 +1,6 @@
 class TagVersionsController < ApplicationController
   def index
+    locale = resolve_locale!
     tag_id = params[:id].presence
     page = (params[:page].presence || 1).to_i
     limit = (params[:limit].presence || 20).to_i
@@ -9,16 +10,7 @@ class TagVersionsController < ApplicationController
 
     offset = (page - 1) * limit
 
-    q = TagVersion.joins(<<~SQL.squish)
+    q = HistoryAccess.visible(version_scope)
-          LEFT JOIN
-            tag_versions prev
-            ON
-              prev.tag_id = tag_versions.tag_id
-              AND prev.version_no = tag_versions.version_no - 1
-        SQL
-      .select('tag_versions.*', 'prev.name AS prev_name', 'prev.category AS prev_category',
-              'prev.deprecated_at AS prev_deprecated_at',
-              'prev.aliases AS prev_aliases', 'prev.parent_tag_ids AS prev_parent_tag_ids')
     q = q.where('tag_versions.tag_id = ?', tag_id) if tag_id
 
     count = q.except(:select, :order, :limit, :offset).count
@@ -27,41 +19,68 @@ class TagVersionsController < ApplicationController
                 .limit(limit)
                 .offset(offset)
 
-    render json: { versions: serialise_versions(versions), count: }
+    render json: { versions: serialise_versions(versions, locale.language_code), count:, population_count: HistoryAccess.population_count(TagVersion) }
   end
 
   private
 
-  def serialise_versions rows
+  def version_scope
+    TagVersion.joins(<<~SQL.squish)
+          LEFT JOIN
+            tag_versions prev
+            ON
+              prev.tag_id = tag_versions.tag_id
+              AND prev.version_no = tag_versions.version_no - 1
+              AND #{ HistoryAccess.sql(TagVersion, 'prev') }
+        SQL
+      .select('tag_versions.*', 'prev.name AS prev_name', 'prev.category AS prev_category',
+              'prev.deprecated_at AS prev_deprecated_at',
+              'prev.aliases AS prev_aliases', 'prev.names AS prev_names',
+              'prev.parent_tag_ids AS prev_parent_tag_ids')
+  end
+
+  def serialise_versions rows, language_code
+    rows = rows.to_a
+    tag_ids = rows.map(&:tag_id).uniq
+    current_names_by_tag_id = TagName.where(tag_id: tag_ids, language_code:, primary_flg: true)
+        .pluck(:tag_id, :name, :auto_generated).to_h do |id, name, automatic|
+      [id, { name:, auto_generated: automatic }]
+    end
     user_ids = rows.map(&:created_by_user_id).compact.uniq
     users_by_id = User.where(id: user_ids).pluck(:id, :name).to_h
+    parent_ids = rows.flat_map do |row|
+      split_parent_tag_ids(row.parent_tag_ids) +
+          split_parent_tag_ids(row.attributes['prev_parent_tag_ids'])
+    end.uniq
+    parents_by_id = Tag.includes(
+        :tag_names, :materials, :deerjikists, :tag_descriptions,
+        parents: [:tag_names, :materials, :deerjikists, :tag_descriptions])
+        .where(id: parent_ids).index_by(&:id)
+    parent_json = parents_by_id.transform_values { TagRepr.base(_1, language_code:) }
 
     rows.map do |row|
-      cur_aliases = split_values(row.aliases)
+      current_names = names_for_locale(row.names, language_code)
-      prev_aliases = split_values(row.attributes['prev_aliases'])
+      previous_names = names_for_locale(row.attributes['prev_names'], language_code)
+      cur_aliases = current_names.fetch('aliases', []).map { _1.fetch('name') }
+      prev_aliases = previous_names.fetch('aliases', []).map { _1.fetch('name') }
 
       cur_parent_tag_ids = split_parent_tag_ids(row.parent_tag_ids)
       prev_parent_tag_ids = split_parent_tag_ids(row.attributes['prev_parent_tag_ids'])
 
-      all_parent_tag_ids = (cur_parent_tag_ids | prev_parent_tag_ids)
-
-      tags_by_id =
-          Tag
-          .includes(:tag_name, :materials, { tag_name: :wiki_page })
-          .where(id: all_parent_tag_ids)
-          .index_by(&:id)
-
       parent_tags =
           build_version_values(cur_parent_tag_ids, prev_parent_tag_ids, key: :tag_id)
           .map do |h|
-            { tag: TagRepr.base(tags_by_id[h[:tag_id]]),
+            { tag: parent_json.fetch(h[:tag_id]),
               type: h[:type] }
           end
 
       { tag_id: row.tag_id,
         version_no: row.version_no,
         event_type: row.event_type,
-        name: { current: row.name, prev: row.attributes['prev_name'] },
+        names: row.names,
+        name_changes: name_changes(row.names, row.attributes['prev_names']),
+        current_name_reference: current_names_by_tag_id[row.tag_id],
+        name: { current: current_names['name'], prev: previous_names['name'] },
         category: { current: row.category, prev: row.attributes['prev_category'] },
         deprecated_at: { current: row.deprecated_at&.iso8601,
                          prev: row.attributes['prev_deprecated_at']&.iso8601 },
@@ -91,5 +110,45 @@ class TagVersionsController < ApplicationController
 
   def split_values(values) = values.to_s.split(/\s+/).reject(&:blank?)
 
+  def names_for_locale names, language_code
+    values = snapshot_names(names)[language_code]
+    values.is_a?(Hash) ? values : {}
+  end
+
+  def snapshot_names names
+    values = names.is_a?(String) ? JSON.parse(names) : names
+    values.is_a?(Hash) ? values : {}
+  rescue JSON::ParserError
+    {}
+  end
+
+  def name_changes current_names, previous_names
+    current = snapshot_names(current_names)
+    previous = snapshot_names(previous_names)
+    (current.keys | previous.keys).sort.filter_map do |language|
+      current_name = language_snapshot(current[language])
+      previous_name = language_snapshot(previous[language])
+      next if current_name == previous_name
+
+      { language_code: language, current: current_name, prev: previous_name }
+    end
+  end
+
+  def language_snapshot entry
+    return nil unless entry.is_a?(Hash)
+
+    aliases = Array(entry['aliases']).filter_map do |value|
+      value = { 'name' => value } if value.is_a?(String)
+      next unless value.is_a?(Hash) && value['name'].is_a?(String)
+
+      { name: value['name'], script_code: value['script_code'] }
+    end
+    aliases = aliases.uniq.sort_by { [_1[:name], _1[:script_code].to_s] }
+    { name: entry['name'],
+      script_code: entry['script_code'],
+      auto_generated: entry['auto_generated'],
+      aliases: }
+  end
+
   def split_parent_tag_ids(values) = split_values(values).map(&:to_i)
 end

backend/app/controllers/tags_controller.rb

@@ -5,6 +5,8 @@ require 'set'
 
 class TagsController < ApplicationController
   def index
+    locale = resolve_locale!
+
     post_id = params[:post]
 
     name = params[:name].presence
@@ -34,49 +36,162 @@ class TagsController < ApplicationController
 
     offset = (page - 1) * limit
 
-    q =
+    tags =
         if post_id.present?
-          Tag.joins(:posts, :tag_name)
+          Tag
+              .joins(:posts, :tag_names)
+              .where(tag_names: { language_code: locale.language_code, primary_flg: true },
+                     posts: { id: post_id })
         else
-          Tag.joins(:tag_name)
+          Tag
+              .joins(:tag_names)
+              .where(tag_names: { language_code: locale.language_code, primary_flg: true })
         end
-        .includes(:tag_name, :materials, tag_name: :wiki_page)
-    q = q.where(posts: { id: post_id }) if post_id.present?
 
-    q = q.where('tag_names.name LIKE ?', "%#{ name }%") if name
+    external_tags =
-    q = q.where(category:) if category
+        if post_id.present?
-    q = q.where('tags.post_count >= ?', post_count_between[0]) if post_count_between[0]
+          ExternalTag.joins(:posts).where(posts: { id: post_id })
-    q = q.where('tags.post_count <= ?', post_count_between[1]) if post_count_between[1]
+        else
-    q = q.where('tags.created_at >= ?', created_between[0]) if created_between[0]
+          ExternalTag.all
-    q = q.where('tags.created_at <= ?', created_between[1]) if created_between[1]
+        end
-    q = q.where('tags.updated_at >= ?', updated_between[0]) if updated_between[0]
+
-    q = q.where('tags.updated_at <= ?', updated_between[1]) if updated_between[1]
+    if name
-    if deprecated_given
+      tags = tags.where('tag_names.name LIKE ?', "%#{ name }%")
-      q = deprecated ? q.where.not(deprecated_at: nil) : q.where(deprecated_at: nil)
+      external_tags =
+          external_tags.where("CONCAT(external_tags.platform, ':', external_tags.name) LIKE ?",
+                              "%#{ name }%")
     end
 
+    if category == 'nico'
+      tags = tags.none
+    elsif category
+      tags = tags.where(category:)
+      external_tags = external_tags.none
+    end
+
+    if post_count_between[0]
+      tags = tags.where('tags.post_count >= ?', post_count_between[0])
+      external_tags = external_tags.where('external_tags.post_count >= ?', post_count_between[0])
+    end
+
+    if post_count_between[1]
+      tags = tags.where('tags.post_count <= ?', post_count_between[1])
+      external_tags = external_tags.where('external_tags.post_count <= ?', post_count_between[1])
+    end
+
+    if created_between[0]
+      tags = tags.where('tags.created_at >= ?', created_between[0])
+      external_tags = external_tags.where('external_tags.created_at >= ?', created_between[0])
+    end
+
+    if created_between[1]
+      tags = tags.where('tags.created_at <= ?', created_between[1])
+      external_tags = external_tags.where('external_tags.created_at <= ?', created_between[1])
+    end
+
+    if updated_between[0]
+      tags = tags.where('tags.updated_at >= ?', updated_between[0])
+      external_tags = external_tags.where('external_tags.created_at >= ?', updated_between[0])
+    end
+
+    if updated_between[1]
+      tags = tags.where('tags.updated_at <= ?', updated_between[1])
+      external_tags = external_tags.where('external_tags.created_at <= ?', updated_between[1])
+    end
+
+    if deprecated_given
+      if deprecated
+        tags = tags.where.not(deprecated_at: nil)
+        external_tags = external_tags.none
+      else
+        tags = tags.where(deprecated_at: nil)
+      end
+    end
+
+    tag_sql = tags.select("'tag' AS source",
+                          'tags.id',
+                          'tag_names.name',
+                          'tags.category',
+                          'tags.post_count',
+                          'tags.created_at',
+                          'tags.updated_at',
+                          'tags.deprecated_at').to_sql
+
+    external_tag_sql = external_tags.select(
+        "'external_tag' AS source",
+        'external_tags.id',
+        "CONCAT(external_tags.platform, ':', external_tags.name) AS name",
+        "'nico' AS category",
+        'external_tags.post_count',
+        'external_tags.created_at',
+        'external_tags.created_at AS updated_at',
+        'NULL AS deprecated_at').to_sql
+
+    union_sql = "#{ tag_sql } UNION ALL #{ external_tag_sql }"
+
     sort_sql =
-        case order[0]
+        if order[0] == 'category'
-        when 'name'
+          'CASE category ' +
-          'tag_names.name'
-        when 'category'
-          'CASE tags.category ' +
           "WHEN 'deerjikist' THEN 0 " +
           "WHEN 'meme' THEN 1 " +
           "WHEN 'character' THEN 2 " +
           "WHEN 'general' THEN 3 " +
           "WHEN 'material' THEN 4 " +
           "WHEN 'meta' THEN 5 " +
-          "WHEN 'nico' THEN 6 END"
+          "WHEN 'nico' THEN 6 " +
+          'END'
         else
-          "tags.#{ order[0] }"
+          order[0]
         end
-    tags = q.order(Arel.sql("#{ sort_sql } #{ order[1] }, tags.id #{ order[1] }"))
-            .limit(limit)
-            .offset(offset)
-            .to_a
 
-    render json: { tags: TagRepr.many(tags), count: q.size }
+    connection = ApplicationRecord.connection
+
+    count = connection.select_value(<<~SQL)
+      SELECT
+        COUNT(0)
+      FROM
+        (#{ union_sql }) legacy_tags
+    SQL
+
+    rows = connection.select_all(<<~SQL).to_a
+      SELECT
+        source
+      , id
+      FROM
+        (#{ union_sql }) legacy_tags
+      ORDER BY
+        #{ sort_sql } #{ order[1] }
+      , id #{ order[1] }
+      , source #{ order[1] }
+      LIMIT
+        #{ limit }
+      OFFSET
+        #{ offset }
+    SQL
+
+    tag_ids = rows.filter { _1['source'] == 'tag' }.map { _1['id'] }
+    external_tag_ids = rows.filter { _1['source'] == 'external_tag' }.map { _1['id'] }
+
+    tags_by_id =
+        Tag
+        .joins(:tag_names)
+        .preload(:tag_names, :materials, :deerjikists, :tag_descriptions,
+                 parents: [:tag_names, :materials, :deerjikists, :tag_descriptions])
+        .where(tag_names: { language_code: locale.language_code, primary_flg: true },
+               id: tag_ids)
+        .index_by(&:id)
+    external_tags_by_id =
+        ExternalTag
+        .where(id: external_tag_ids)
+        .index_by(&:id)
+
+    render json: { tags: rows.map { |row|
+      if row['source'] == 'tag'
+        TagRepr.base(tags_by_id.fetch(row['id']), language_code: locale.language_code)
+      else
+        external_tag_json(external_tags_by_id.fetch(row['id']))
+      end
+    }, count: }
   end
 
   def with_depth
@@ -84,7 +199,8 @@ class TagsController < ApplicationController
     parent_tag_id = nil if parent_tag_id <= 0
     material_filter = material_filter_param(default: 'any')
 
-    graph = build_with_depth_graph(material_filter)
+    locale = resolve_locale!
+    graph = build_with_depth_graph(material_filter, locale.language_code)
 
     tag_ids =
         if parent_tag_id
@@ -99,137 +215,231 @@ class TagsController < ApplicationController
   end
 
   def autocomplete
+    locale = resolve_locale!
+
     q = params[:q].to_s.strip.sub(/\Anot:/i, '')
+    prefix = "#{ ActiveRecord::Base.sanitize_sql_like(q) }%"
 
     with_nico = bool?(:nico, default: true)
     present_only = bool?(:present, default: true)
 
     alias_rows =
         TagName
-        .where('name LIKE ?', "#{ q }%")
+        .where(language_code: locale.language_code, primary_flg: false)
-        .where.not(canonical_id: nil)
+        .where('name LIKE ?', prefix)
-        .pluck(:canonical_id, :name)
+        .pluck(:tag_id, :name)
 
-    matched_alias_by_tag_name_id = { }
+    matched_alias_by_tag_id = { }
-    canonical_ids = []
+    tag_ids = []
-
+    alias_rows.each do |tag_id, alias_name|
-    alias_rows.each do |canonical_id, alias_name|
+      tag_ids << tag_id
-      canonical_ids << canonical_id
+      matched_alias_by_tag_id[tag_id] ||= alias_name
-      matched_alias_by_tag_name_id[canonical_id] ||= alias_name
     end
 
-    base = Tag.joins(:tag_name)
+    base =
-              .includes(:tag_name, :materials, tag_name: :wiki_page)
+        Tag
-              .where(deprecated_at: nil)
+        .joins(:tag_names)
+        .preload(:tag_names, :materials, :deerjikists, :tag_descriptions,
+                 parents: [:tag_names, :materials, :deerjikists, :tag_descriptions])
+        .where(tag_names: { language_code: locale.language_code, primary_flg: true },
+               deprecated_at: nil)
     base = base.where('tags.post_count > 0') if present_only
 
-    canonical_hit =
+    canonical_hit = base.where('tag_names.name LIKE ?', prefix)
-        base
-        .where(((with_nico ? '(tags.category = ? AND tag_names.name LIKE ?) OR ' : '') +
-                'tag_names.name LIKE ?'),
-               *(with_nico ? ['nico', "nico:#{ q }%"] : []), "#{ q }%")
 
-    tags =
+    internal_tags = canonical_hit.or(base.where(id: tag_ids.uniq))
-        if canonical_ids.present?
-          canonical_hit.or(base.where(tag_name_id: canonical_ids.uniq))
-        else
-          canonical_hit
-        end
 
-    tags = tags.order(Arel.sql('post_count DESC, tag_names.name')).limit(20).to_a
+    internal_rows =
+        internal_tags
+        .order(Arel.sql('tags.post_count DESC, tag_names.name'))
+        .limit(20)
+        .map { |tag|
+          TagRepr.base(tag, language_code: locale.language_code)
+                 .merge(matched_alias: matched_alias_by_tag_id[tag.id])
+        }
 
-    render json: tags.map { |tag|
+    return render json: internal_rows unless with_nico
-      TagRepr.base(tag).merge(matched_alias: matched_alias_by_tag_name_id[tag.tag_name_id])
+
-    }
+    external_base = ExternalTag.all
+    external_base = external_base.where('post_count > 0') if present_only
+
+    external_rows =
+        external_base
+        .where("CONCAT(platform, ':', name) LIKE ? OR name LIKE ?", prefix, prefix)
+        .order(post_count: :desc, name: :asc)
+        .limit(20)
+        .map { external_tag_json(_1) }
+
+    rows =
+    (internal_rows + external_rows)
+        .sort_by { |row| [-row['post_count'], row['name'] || row[:name]] }
+        .first(20)
+
+    render json: rows
   end
 
   def show
-    tag = Tag.joins(:tag_name)
+    locale = resolve_locale!
-             .includes(:tag_name, :materials, tag_name: :wiki_page)
+
-             .find_by(id: params[:id])
+    tag =
-    if tag
+        Tag
-      render json: TagRepr.base(tag)
+        .joins(:tag_names)
-    else
+        .preload(:tag_names, :materials, :deerjikists, :tag_descriptions,
-      head :not_found
+                 parents: [:tag_names, :materials, :deerjikists, :tag_descriptions])
-    end
+        .find_by(id: params[:id],
+                 tag_names: { language_code: locale.language_code, primary_flg: true })
+    return render json: TagRepr.base(tag, language_code: locale.language_code) if tag
+
+    external_tag = ExternalTag.find_by(id: params[:id])
+    return render json: ExternalTagRepr.base(external_tag) if external_tag
+
+    head :not_found
   end
 
   def show_by_name
     name = params[:name].to_s.strip
     return render_bad_request('name は必須です．') if name.blank?
 
-    tag = Tag.joins(:tag_name)
+    locale = resolve_locale!
-             .includes(:tag_name, :materials, tag_name: :wiki_page)
+
-             .find_by(tag_names: { name: })
+    tag =
-    if tag
+        Tag
-      render json: TagRepr.base(tag)
+        .joins(:tag_names)
-    else
+        .preload(:tag_names, :materials, :deerjikists, :tag_descriptions,
-      head :not_found
+                 parents: [:tag_names, :materials, :deerjikists, :tag_descriptions])
-    end
+        .find_by(tag_names: { name:, language_code: locale.language_code })
+    return render json: TagRepr.base(tag, language_code: locale.language_code) if tag
+
+    platform, external_name = name.split(':', 2)
+    return head :not_found unless external_name
+
+    external_tag = ExternalTag.find_by(platform:, name: external_name)
+    return head :not_found unless external_tag
+
+    render json: ExternalTagRepr.base(external_tag)
   end
 
   def deerjikists
-    tag = Tag.joins(:tag_name)
+    locale = resolve_locale!
-             .includes(:tag_name, tag_name: :wiki_page)
+
-             .find_by(id: params[:id])
+    tag = 
+        Tag.joins(:tag_names)
+        .preload(:tag_names, :materials, :deerjikists, :tag_descriptions,
+                 parents: [:tag_names, :materials, :deerjikists, :tag_descriptions])
+        .find_by(id: params[:id],
+                 tag_names: { language_code: locale.language_code, primary_flg: true })
     return head :not_found unless tag
 
-    render json: { tag: TagRepr.base(tag),
+    render json: { tag: TagRepr.base(tag, language_code: locale.language_code),
-                   deerjikists: DeerjikistRepr.many(tag.deerjikists) }
+                   deerjikists: DeerjikistRepr.many(
+                       tag.deerjikists,
+                       language_code: locale.language_code) }
   end
 
   def deerjikists_by_name
     name = params[:name].to_s.strip
     return render_bad_request('name は必須です．') if name.blank?
 
-    tag = Tag.joins(:tag_name)
+    locale = resolve_locale!
-             .includes(:tag_name, tag_name: :wiki_page)
+
-             .find_by(tag_names: { name: })
+    tag =
+        Tag
+        .joins(:tag_names)
+        .preload(:tag_names, :materials, :deerjikists, :tag_descriptions,
+                 parents: [:tag_names, :materials, :deerjikists, :tag_descriptions])
+        .find_by(tag_names: { name:, language_code: locale.language_code, primary_flg: true })
     return head :not_found unless tag
 
-    render json: { tag: TagRepr.base(tag),
+    render json: { tag: TagRepr.base(tag, language_code: locale.language_code),
-                   deerjikists: DeerjikistRepr.many(tag.deerjikists) }
+                   deerjikists: DeerjikistRepr.many(
+                       tag.deerjikists,
+                       language_code: locale.language_code) }
   end
 
   def update_deerjikists
     return head :unauthorized unless current_user
     return head :forbidden unless current_user.gte_member?
 
-    tag = Tag.joins(:tag_name)
+    locale = resolve_locale!
-             .includes(:tag_name, tag_name: :wiki_page)
+
-             .find_by(id: params[:id])
+    tag =
+        Tag
+        .joins(:tag_names)
+        .preload(:tag_names, :materials, :deerjikists, :tag_descriptions,
+                 parents: [:tag_names, :materials, :deerjikists, :tag_descriptions])
+        .find_by(id: params[:id],
+                 tag_names: { language_code: locale.language_code, primary_flg: true })
     return head :not_found unless tag
 
+    rows = normalise_deerjikist_rows(tag)
+    return if performed?
+
     ApplicationRecord.transaction do
-      tag.deerjikists = []
+      SharedResourceIdAllocator.lock_space!(Tag, 'tags')
-      params[:_json].each.with_index do |item, i|
+      tag.lock!
-        platform = item[:platform]
+
-        code = normalise_deerjikist_code(platform, item[:code])
+      requested_keys = rows.map { |row| [row[:platform], row[:code]] }.uniq
-        deerjikist = Deerjikist.find_or_initialize_by(platform:, code:)
+      row_indexes_by_key = rows_by_key(rows)
-        deerjikist.tag = tag
+      locked_deerjikists = lock_deerjikists_for_tag_update(tag.id, requested_keys)
-        render_deerjikist_form_record_invalid(deerjikist, i) unless deerjikist.save
+      current_deerjikists = locked_deerjikists.filter { |deerjikist|
+        deerjikist.tag_id == tag.id
+      }
+      requested_deerjikists = locked_deerjikists.filter { |deerjikist|
+        row_indexes_by_key.key?([deerjikist.platform, deerjikist.code])
+      }
+
+      render_deerjikist_conflicts(locale, requested_deerjikists, row_indexes_by_key, tag)
+      raise ActiveRecord::Rollback if performed?
+
+      requested_keys_set = requested_keys.to_set
+      current_deerjikists.each do |deerjikist|
+        key = [deerjikist.platform, deerjikist.code]
+        deerjikist.destroy! unless requested_keys_set.include?(key)
+      end
+
+      existing_keys = requested_deerjikists.to_h { |deerjikist|
+        [[deerjikist.platform, deerjikist.code], true]
+      }
+      requested_keys.each do |platform, code|
+        next if existing_keys[[platform, code]]
+
+        deerjikist = Deerjikist.new(platform:, code:, tag:)
+        row_index = row_indexes_by_key[[platform, code]].first
+        begin
+          render_deerjikist_form_record_invalid(deerjikist, row_index) unless deerjikist.save
+        rescue ActiveRecord::RecordNotUnique
+          conflicts = lock_deerjikists_for_tag_update(tag.id, [[platform, code]])
+          render_deerjikist_conflicts(locale, conflicts, row_indexes_by_key, tag)
+        end
         raise ActiveRecord::Rollback if performed?
       end
     end
 
     return if performed?
 
-    render json: DeerjikistRepr.many(tag.reload.deerjikists)
+    render json: DeerjikistRepr.many(tag.reload.deerjikists,
+                                     language_code: locale.language_code)
   end
 
   def materials_by_name
     name = params[:name].to_s.strip
     return render_bad_request('name は必須です．') if name.blank?
+
+    locale = resolve_locale!
+
     material_filter = material_filter_param(default: 'any')
 
-    tag = Tag.joins(:tag_name)
+    tag =
-             .includes(:tag_name, :materials, tag_name: :wiki_page)
+        Tag
-             .find_by(tag_names: { name: })
+        .joins(:tag_names)
+        .preload(:tag_names, :materials, :deerjikists, :tag_descriptions,
+                 parents: [:tag_names, :materials, :deerjikists, :tag_descriptions])
+        .find_by(tag_names: { name:, language_code: locale.language_code, primary_flg: true })
     return head :not_found unless tag
 
-    graph = build_with_depth_graph(material_filter)
+    graph = build_with_depth_graph(material_filter, locale.language_code)
 
-    render json: build_tag_children(tag, graph:)
+    render json: build_tag_children(tag, graph:, language_code: locale.language_code)
   end
 
   def update_all
@@ -245,39 +455,37 @@ class TagsController < ApplicationController
     return render_unprocessable_entity('カテゴリは必須です．', field: :category) if category.blank?
     return render_unprocessable_entity '廃止状態は必須です．', field: :deprecated unless params.key?(:deprecated)
 
-    return unless validate_tag_rename(tag, name)
+    locale = resolve_locale!
+
+    return unless validate_tag_rename(locale, tag, name)
 
     alias_names = params[:aliases].to_s.split.uniq
     parent_names = params[:parent_tags].to_s.split.uniq
     deprecated = bool?(:deprecated)
 
-    if tag.nico? && deprecated
+    if category == 'nico'
-      return render_unprocessable_entity 'ニコタグは廃止できません．', field: :deprecated
-    end
-
-    if tag.nico? || category == 'nico'
       return render_unprocessable_entity 'ニコタグは変更できません．', field: :category
     end
 
     ApplicationRecord.transaction do
-      TagVersioning.ensure_snapshot!(tag, created_by_user: current_user)
+      TagVersioning.ensure_snapshot!(tag, created_by_user: current_user,
+                                     language_code: locale.language_code)
 
-      old_name = tag.name
+      old_name = tag.name(locale.language_code)
       name_changed = name != old_name
-      wiki_page = tag.tag_name.wiki_page if name_changed
 
       if tag.deprecated? == deprecated
         tag.update!(category:)
       else
         tag.update!(category:, deprecated_at: deprecated ? Time.current : nil)
       end
-      rename_tag_name!(tag, name) if name_changed
+      rename_tag_name!(locale, tag, name) if name_changed
 
-      alias_names << old_name if name_changed
+      alias_names << old_name if name_changed && old_name.present?
       alias_names.delete(name)
 
-      update_aliases!(tag, alias_names)
+      update_aliases!(locale, tag, alias_names)
-      update_parent_tags!(tag, parent_names)
+      update_parent_tags!(locale, tag, parent_names)
 
       tag.reload
 
@@ -285,17 +493,18 @@ class TagsController < ApplicationController
           tag,
           event_type: :update,
           created_by_user: current_user,
-          name_changed:,
+          language_code: locale.language_code)
-          wiki_page:)
     end
 
-    render json: TagRepr.base(tag.reload)
+    render json: TagRepr.base(tag.reload, language_code: locale.language_code)
   end
 
   def update
     return head :unauthorized unless current_user
     return head :forbidden unless current_user.gte_member?
 
+    locale = resolve_locale!
+
     name = params[:name].presence
     category = params[:category].presence
     deprecated_given = params.key?(:deprecated)
@@ -303,24 +512,20 @@ class TagsController < ApplicationController
 
     tag = Tag.find(params[:id])
 
-    if tag.nico? && deprecated_given && deprecated
+    return unless validate_tag_rename(locale, tag, name)
-      return render_unprocessable_entity 'ニコタグは廃止できません．', field: :deprecated
-    end
 
-    return unless validate_tag_rename(tag, name)
+    if category.present? && category == 'nico'
-
-    if tag.nico? || (category.present? && category == 'nico')
       return render_unprocessable_entity 'ニコタグは変更できません．', field: :category
     end
 
     ApplicationRecord.transaction do
-      TagVersioning.ensure_snapshot!(tag, created_by_user: current_user)
+      TagVersioning.ensure_snapshot!(tag, created_by_user: current_user,
+                                     language_code: locale.language_code)
 
-      old_name = tag.name
+      old_name = tag.name(locale.language_code)
       name_changed = name.present? && name != old_name
-      wiki_page = tag.tag_name.wiki_page if name_changed
 
-      rename_tag_name!(tag, name) if name_changed
+      rename_tag_name!(locale, tag, name) if name_changed
       tag.update!(category:) if category.present?
       if deprecated_given && tag.deprecated? != deprecated
         tag.update!(deprecated_at: deprecated ? Time.current : nil)
@@ -332,11 +537,10 @@ class TagsController < ApplicationController
           tag,
           event_type: :update,
           created_by_user: current_user,
-          name_changed:,
+          language_code: locale.language_code)
-          wiki_page:)
     end
 
-    render json: TagRepr.base(tag.reload)
+    render json: TagRepr.base(tag.reload, language_code: locale.language_code)
   end
 
   private
@@ -348,7 +552,7 @@ class TagsController < ApplicationController
     value
   end
 
-  def build_with_depth_graph material_filter
+  def build_with_depth_graph material_filter, language_code
     children_by_parent_id = Hash.new { |h, k| h[k] = [] }
     parent_ids_by_child_id = Hash.new { |h, k| h[k] = [] }
 
@@ -363,10 +567,13 @@ class TagsController < ApplicationController
 
     material_tag_ids = Material.unscoped.kept.where.not(tag_id: nil).distinct.pluck(:tag_id).to_set
 
-    tags_by_id = Tag.joins(:tag_name)
+    tags_by_id =
-                    .where(id: tag_ids)
+        Tag
-                    .pluck('tags.id', 'tag_names.name', 'tags.category', 'tags.deprecated_at')
+        .joins(:tag_names)
-                    .each_with_object({ }) do |(id, name, category, deprecated_at), h|
+        .where(id: tag_ids,
+               tag_names: { language_code:, primary_flg: true })
+        .pluck('tags.id', 'tag_names.name', 'tags.category', 'tags.deprecated_at')
+        .each_with_object({ }) do |(id, name, category, deprecated_at), h|
       h[id] = { name:, category:, deprecated: deprecated_at.present?,
                 has_material: material_tag_ids.include?(id) }
     end
@@ -498,145 +705,147 @@ class TagsController < ApplicationController
       children: [] }
   end
 
-  def build_tag_children tag, graph: nil
+  def build_tag_children tag, graph: nil, language_code: 'ja'
     material = tag.materials.first
     tag_graph = graph && graph[:tags_by_id][tag.id]
     material =
         nil if tag_graph && !with_depth_visible_tag?(tag_graph, graph[:material_filter])
 
-    children = tag.children.sort_by(&:name)
+    children = tag.children.sort_by { _1.name(language_code) }
     if graph
       children = children.filter { |child_tag| visible_subtree?(child_tag.id, graph) }
     end
 
-    TagRepr.base(tag).merge(
+    TagRepr.base(tag, language_code:).merge(
-        children: children.map { build_tag_children(_1, graph:) },
+        children: children.map { build_tag_children(_1, graph:, language_code:) },
         has_material: tag_graph ? tag_graph[:has_material] : material.present?,
-        material: material && MaterialRepr.base(material, host: request.base_url))
+        material: material && MaterialRepr.base(
+            material,
+            host: request.base_url,
+            language_code:))
   end
 
-  def record_tag_version! tag, event_type:, created_by_user:, name_changed: false, wiki_page: nil
+  def record_tag_version! tag, event_type:, created_by_user:, language_code: 'ja'
-    if tag.nico?
+    TagVersionRecorder.record!(tag:, event_type:, created_by_user:, language_code:)
-      NicoTagVersionRecorder.record!(tag:, event_type:, created_by_user:)
-      return
-    end
-
-    TagVersionRecorder.record!(tag:, event_type:, created_by_user:)
-
-    return unless name_changed
-
-    wiki_page ||= tag.tag_name.wiki_page
-    return unless wiki_page&.wiki_versions&.exists?
-
-    WikiVersionRecorder.record!(
-        page: wiki_page,
-        event_type: :update,
-        created_by_user:)
   end
 
-  def validate_tag_rename tag, name
+  def validate_tag_rename locale, tag, name
-    return true if name.blank? || name == tag.name
+    return true if name.blank? || name == tag.name(locale.language_code)
 
-    if tag.in?([Tag.tagme, Tag.bot, Tag.no_deerjikist, Tag.video, Tag.niconico])
+    system_tag_names = ['タグ希望', 'bot操作', 'ニジラー情報不詳', '動画', 'ニコニコ']
+    if TagName.exists?(tag_id: tag.id,
+                       language_code: 'ja',
+                       primary_flg: true,
+                       name: system_tag_names)
       render_unprocessable_entity 'システム・タグの名称は変更できません．', field: :name
       return false
     end
 
-    target_tag_name = TagName.with_discarded.find_by(name:)
+    target_tag_name = TagName.find_by(language_code: locale.language_code, name:)
-    return true if target_tag_name.nil?
+    return true unless target_tag_name&.primary_flg
-    return true if target_tag_name.canonical_id?
 
     render_unprocessable_entity 'その名前は既に使はれてゐます．', field: :name
     false
   end
 
-  def rename_tag_name! tag, name
+  def rename_tag_name! locale, tag, name
-    return if name == tag.name
+    return if name == tag.name(locale.language_code)
 
-    current_tag_name = tag.tag_name
+    current_tag_name = tag.tag_name(locale.language_code)
-    target_tag_name = TagName.with_discarded.find_by(name:)
+    target_tag_name = TagName.find_by(language_code: locale.language_code, name:)
 
-    if target_tag_name.nil?
+    unless target_tag_name
-      current_tag_name.update!(name:)
+      if current_tag_name
+        current_tag_name.update!(name:, auto_generated: false)
+      else
+        current_tag_name = TagName.create!(
+            tag:,
+            language_code: locale.language_code,
+            script_code: locale.script_code,
+            name:,
+            primary_flg: true)
+        tag.association(:tag_names).reset
+      end
       return
     end
 
-    promote_tag_alias!(
+    promote_tag_alias!(tag, current_tag_name:, promoted_tag_name: target_tag_name)
-        tag,
-        current_tag_name:,
-        promoted_tag_name: target_tag_name)
   end
 
   def promote_tag_alias! tag, current_tag_name:, promoted_tag_name:
-    old_owner_tag = promoted_tag_name.canonical&.tag
+    old_owner_tag = promoted_tag_name.tag
 
     if old_owner_tag && old_owner_tag != tag
       TagVersioning.ensure_snapshot!(old_owner_tag, created_by_user: current_user)
     end
 
-    promoted_tag_name.undiscard! if promoted_tag_name.discarded?
+    current_tag_name&.update!(primary_flg: false, auto_generated: false)
-    promoted_tag_name.update!(canonical: nil)
+    promoted_tag_name.update!(tag:, primary_flg: true, auto_generated: false)
-
+    tag.association(:tag_names).reset
-    TagName.with_discarded
-           .where(canonical_id: current_tag_name.id)
-           .where.not(id: promoted_tag_name.id)
-           .find_each do |alias_tag_name|
-      alias_tag_name.update!(canonical: promoted_tag_name)
-    end
-
-    current_tag_name.wiki_page&.update!(tag_name: promoted_tag_name)
-    tag.update!(tag_name: promoted_tag_name)
-    current_tag_name.association(:wiki_page).reset
-    current_tag_name.association(:tag).reset
-    current_tag_name.reload.update!(canonical: promoted_tag_name)
 
     return unless old_owner_tag && old_owner_tag != tag
 
-    record_tag_version!(old_owner_tag.reload, event_type: :update, created_by_user: current_user)
+    record_tag_version!(old_owner_tag.reload, event_type: :update,
+                        created_by_user: current_user,
+                        language_code: promoted_tag_name.language_code)
   end
 
-  def update_aliases! tag, alias_names
+  def update_aliases! locale, tag, alias_names
     alias_names = alias_names.uniq
 
     affected_tags = [tag]
 
-    current_aliases = tag.tag_name.aliases.to_a
+    current_aliases = tag.tag_names.where(language_code: locale.language_code,
+                                          primary_flg: false).to_a
 
-    current_aliases.each do |alias_tag_name|
+    requested_aliases = TagName.where(
-      next if alias_names.include?(alias_tag_name.name)
+        language_code: locale.language_code, name: alias_names)
-
+        .includes(:tag).index_by(&:name)
-      affected_tags << alias_tag_name.canonical&.tag
-    end
 
     alias_names.each do |alias_name|
-      alias_tag_name = TagName.find_undiscard_or_create_by!(name: alias_name)
+      alias_tag_name = requested_aliases[alias_name]
-      affected_tags << alias_tag_name.canonical&.tag
+      if alias_tag_name&.primary? && alias_tag_name.tag_id && alias_tag_name.tag_id != tag.id
+        alias_tag_name.errors.add(:name, 'その名前は既に使はれてゐます．')
+        raise ActiveRecord::RecordInvalid, alias_tag_name
+      end
+      affected_tags << alias_tag_name&.tag
     end
 
-    affected_tags.compact.uniq.each do |affected_tag|
+    affected_tags = affected_tags.compact.uniq(&:id)
-      TagVersioning.ensure_snapshot!(affected_tag, created_by_user: current_user)
+    affected_tags.each do |affected_tag|
+      TagVersioning.ensure_snapshot!(affected_tag, created_by_user: current_user,
+                                     language_code: locale.language_code)
     end
 
     current_aliases.each do |alias_tag_name|
       next if alias_names.include?(alias_tag_name.name)
 
-      alias_tag_name.update!(canonical: nil)
+      alias_tag_name.destroy!
     end
 
     alias_names.each do |alias_name|
-      alias_tag_name = TagName.find_undiscard_or_create_by!(name: alias_name)
+      alias_tag_name = requested_aliases[alias_name] || TagName.new(
-      alias_tag_name.update!(canonical: tag.tag_name)
+          language_code: locale.language_code,
+          name: alias_name)
+      alias_tag_name.script_code = locale.script_code
+      alias_tag_name.update!(tag:, primary_flg: false, auto_generated: false)
     end
 
-    affected_tags.compact.uniq.each do |affected_tag|
+    affected_tags.each do |affected_tag|
-      record_tag_version!(affected_tag, event_type: :update, created_by_user: current_user)
+      # update_all records this Tag after its parent relationships are updated.
+      next if affected_tag.id == tag.id
+
+      record_tag_version!(affected_tag, event_type: :update,
+                          created_by_user: current_user,
+                          language_code: locale.language_code)
     end
   end
 
-  def update_parent_tags! tag, parent_names
+  def update_parent_tags! locale, tag, parent_names
-    parent_tags = Tag.normalise_tags!(parent_names, with_tagme: false,
+    parent_tags = Tag.normalise_tags!(locale, parent_names,
-                                                    with_no_deerjikist: false,
+                                      with_tagme: false,
-                                                    deny_nico: true)
+                                      with_no_deerjikist: false,
+                                      created_by_user: current_user)
 
     old_parent_tags = tag.parents.to_a
 
@@ -653,6 +862,7 @@ class TagsController < ApplicationController
   end
 
   def normalise_deerjikist_code platform, code
+    code = code.to_s
     return code if platform != 'youtube' || code[0] != '@'
 
     url = "https://www.youtube.com/#{ code }"
@@ -669,6 +879,84 @@ class TagsController < ApplicationController
     nil
   end
 
+  def normalise_deerjikist_rows tag
+    rows = []
+
+    params[:_json].each.with_index do |item, index|
+      platform = item[:platform]
+      unless Deerjikist.platforms.key?(platform)
+        render_deerjikist_platform_invalid(index)
+        return rows
+      end
+
+      code = normalise_deerjikist_code(platform, item[:code])
+      deerjikist = Deerjikist.new(platform:, code:, tag:)
+      unless deerjikist.valid?
+        render_deerjikist_form_record_invalid(deerjikist, index)
+        return rows
+      end
+
+      rows << { index:, platform:, code: }
+    end
+
+    rows
+  end
+
+  def lock_deerjikists_for_tag_update tag_id, keys
+    clauses = ['tag_id = ?']
+    values = [tag_id]
+
+    keys.each do |platform, code|
+      clauses << '(platform = ? AND code = ?)'
+      values << platform << code
+    end
+
+    Deerjikist
+      .where(clauses.join(' OR '), *values)
+      .order(:platform, :code)
+      .lock
+      .to_a
+  end
+
+  def render_deerjikist_conflicts locale, deerjikists, row_indexes_by_key, tag
+    conflicts = deerjikists.filter { |deerjikist| deerjikist.tag_id != tag.id }
+    return if conflicts.empty?
+
+    tag_names_by_id =
+        Tag
+        .joins(:tag_names)
+        .where(tag_names: { language_code: locale.language_code, primary_flg: true },
+               id: conflicts.map(&:tag_id).uniq)
+        .pluck('tags.id', 'tag_names.name')
+        .to_h
+    fields = { }
+
+    conflicts.each do |deerjikist|
+      message = "この情報は既に「#{ tag_names_by_id[deerjikist.tag_id] }」に紐づいてゐます．"
+      row_indexes_by_key[[deerjikist.platform, deerjikist.code]].each do |index|
+        field = :"deerjikists.#{ index }.code"
+        fields[field] ||= []
+        fields[field] << message
+      end
+    end
+
+    render_validation_error fields:
+  end
+
+  def render_deerjikist_platform_invalid index
+    render_validation_error fields: {
+      :"deerjikists.#{ index }.platform" => ['値が不正です．'],
+    }
+  end
+
+  def rows_by_key rows
+    rows.each_with_object({ }) do |row, result|
+      key = [row[:platform], row[:code]]
+      result[key] ||= []
+      result[key] << row[:index]
+    end
+  end
+
   def render_deerjikist_form_record_invalid deerjikist, index
     fields = { }
 
@@ -687,4 +975,20 @@ class TagsController < ApplicationController
 
     render_validation_error fields:
   end
+
+  def external_tag_json tag
+    { 'id' => tag.id,
+      'name' => "#{ tag.platform }:#{ tag.name }",
+      'category' => 'nico',
+      'deprecated_at' => nil,
+      'created_at' => tag.created_at,
+      'updated_at' => tag.created_at,
+      'post_count' => tag.post_count,
+      'matched_alias' => nil,
+      'aliases' => [],
+      'parents' => [],
+      'has_wiki' => false,
+      'material_id' => nil,
+      'has_deerjikists' => false }
+  end
 end

backend/app/controllers/theatre_skip_events_controller.rb

@@ -6,7 +6,7 @@ class TheatreSkipEventsController < ApplicationController
     events =
         TheatreSkipEvent
         .where(theatre_id: params[:theatre_id])
-        .includes(:post, tags: :tag_name)
+        .includes(:post, tags: :tag_names)
         .order(created_at: :desc)
         .limit(limit)
 
@@ -14,7 +14,7 @@ class TheatreSkipEventsController < ApplicationController
       { id: event.id,
         theatre_id: event.theatre_id,
         post: { id: event.post.id, title: event.post.title, url: event.post.url },
-        tags: event.tags.map { |tag| { id: tag.id, name: tag.name } },
+        tags: event.tags.map { |tag| { id: tag.id, name: tag.name('ja') } },
         programme_position: event.programme_position,
         created_at: event.created_at }
     }

backend/app/controllers/users_controller.rb

@@ -1,39 +1,69 @@
 class UsersController < ApplicationController
   def create
+    if current_user
+      return render json: { code: current_user.inheritance_code,
+                            user: current_user.slice(:id, :name, :inheritance_code, :role) }
+    end
+
     user = nil
     User.transaction do
       user = User.create!(inheritance_code: SecureRandom.uuid, role: :guest)
       attach_ip_address!(user)
     end
 
+    establish_user_session(user)
     render json: { code: user.inheritance_code,
                    user: user.slice(:id, :name, :inheritance_code, :role) },
            status: :created
   end
 
   def verify
-    user = User.find_by(inheritance_code: params[:code])
+    user = if bool?(:restore) && current_user
+             current_user
+           else
+             User.find_by(inheritance_code: params[:code])
+           end
     return render json: { valid: false } unless user
     return head :forbidden if user.banned?
 
     attach_ip_address!(user)
+    establish_user_session(user)
 
     render json: { valid: true, user: user.slice(:id, :name, :inheritance_code, :role) }
   end
 
+  def show_session
+    return head :unauthorized unless current_user
+
+    render json: current_user.slice(:id, :name, :role)
+  end
+
+  def csrf
+    response.headers['Cache-Control'] = 'private, no-store'
+    render json: { token: form_authenticity_token }
+  end
+
+  def destroy_session
+    reset_session
+    response.headers['Cache-Control'] = 'private, no-store'
+    head :no_content
+  end
+
   def renew
     return head :unauthorized unless current_user
 
     user = current_user
     user.inheritance_code = SecureRandom.uuid
     user.save!
+    establish_user_session(user)
     render json: { code: user.inheritance_code }
   end
 
   def me
-    user = User.find_by(inheritance_code: params[:code])
+    user = current_user
-    return head :not_found unless user
+    return head :unauthorized unless user
 
+    attach_ip_address!(user)
     render json: user.slice(:id, :name, :inheritance_code, :role)
   end
 

backend/app/controllers/wiki_pages_controller.rb

@@ -1,86 +1,168 @@
 class WikiPagesController < ApplicationController
-  rescue_from Wiki::Commit::Conflict, with: :render_wiki_conflict
+  # 互換性のための Facade API
+
+  rescue_from TagDescriptionOperation::Conflict, with: :render_wiki_conflict
 
   def index
+    locale = resolve_locale!
+
     title = params[:title].to_s.strip
-    if title.blank?
+    descriptions =
-      return render json: WikiPageRepr.base(
+        HistoryAccess.current(TagDescription.all, TagDescriptionVersion)
-          WikiPage.joins(:tag_name).includes(tag_name: :tag))
+        .where(language_code: locale.language_code)
+        .includes(tag: :tag_names)
+
+    descriptions = descriptions.order(updated_at: :desc, id: :asc)
+    tag_groups = HistoryAccess.current(TagGroup.all, TagGroupVersion).order(updated_at: :desc, id: :asc)
+    if title.present?
+      pattern = "%#{ ApplicationRecord.sanitize_sql_like(title) }%"
+      descriptions =
+          descriptions.joins(tag: :tag_names)
+          .where(tag_names: { language_code: locale.language_code, primary_flg: true })
+          .where('tag_names.name LIKE ?', pattern).limit(20)
+      tag_groups = tag_groups.where(
+          'CONCAT(?, name) LIKE ? OR CONCAT(?, name) LIKE ?',
+          TagGroupRepr::PREFIX, pattern, 'tag_group:', pattern).limit(20)
     end
 
-    q = WikiPage.joins(:tag_name).includes(tag_name: :tag)
+    json = descriptions.map { |description| WikiPageRepr.tag_description(locale, description) }
-        .where('tag_names.name LIKE ?', "%#{ WikiPage.sanitize_sql_like(title) }%")
+    json.concat(tag_groups.map { WikiPageRepr.tag_group(_1) })
-    render json: WikiPageRepr.base(q.limit(20))
+    json.sort_by! do |row|
+      [-row['updated_at'].to_f, row.fetch('resource_type'), row.fetch('id')]
+    end
+    json = json.first(20) if title.present?
+    render json:
+  end
+
+  def autolink_candidates
+    locale = resolve_locale!
+    language = params.fetch(:language, locale.language_code)
+    language = 'ja' if language == 'und'
+    return head :not_found unless language.is_a?(String)
+    return head :not_found unless Language.exists?(code: language)
+
+    names = HistoryAccess.current(TagDescription.all, TagDescriptionVersion).joins(tag: :tag_names)
+        .where(language_code: locale.language_code, tag_names: { language_code: language })
+        .pluck('tag_descriptions.tag_id', 'tag_descriptions.id', 'tag_names.name')
+    candidates = names.map do |tag_id, description_id, name|
+      WikiPageRepr.autolink_tag(locale, tag_id:, description_id:, name:, language:)
+    end
+    candidates.concat(HistoryAccess.current(TagGroup.all, TagGroupVersion).order(:id).pluck(:id, :name).map do |id, name|
+      WikiPageRepr.autolink_group(locale, id:, name:, language:)
+    end)
+    render json: candidates
   end
 
   def show
-    page = WikiPage.joins(:tag_name)
+    locale = resolve_locale!
-                   .includes(tag_name: :tag)
+    reference = TagDescription.find_by(id: params[:id])
-                   .find_by(id: params[:id])
+    tag_description = reference && TagDescription.find_by(
-    render_wiki_page_or_404 page
+        tag_id: reference.tag_id, language_code: locale.language_code)
+
+    return render_wiki_page_by_tag_description locale, tag_description if tag_description
+    return head :not_found if reference
+
+    tag_group = TagGroup.find_by(id: params[:id])
+    return render_tag_group locale, tag_group if tag_group
+
+    head :not_found
   end
 
   def show_by_title
+    locale = resolve_locale!
     title = params[:title].to_s.strip
-    page = WikiPage.joins(:tag_name)
+
-                   .includes(tag_name: :tag)
+    if (name = TagGroupRepr.name_from_title(title))
-                   .find_by(tag_name: { name: title })
+      tag_group = TagGroup.find_by(name:)
-    render_wiki_page_or_404 page
+      return render_tag_group locale, tag_group if tag_group
+      return head :not_found
+    end
+
+    tag_description = tag_description_by_title(locale, title)
+
+    return render_wiki_page_by_tag_description locale, tag_description if tag_description
+
+    head :not_found
+  end
+
+  def tag_reference
+    locale = resolve_locale!
+    description = TagDescription.find_by(id: params[:id])
+    return head :not_found unless description
+
+    tag = Tag.preload(:tag_names, :materials, :deerjikists, :tag_descriptions,
+                      parents: [:tag_names, :materials, :deerjikists, :tag_descriptions])
+        .find(description.tag_id)
+    render json: TagRepr.base(tag, language_code: locale.language_code)
   end
 
   def exists
-    if WikiPage.exists?(id: params[:id])
+    locale = resolve_locale!
-      head :no_content
+    if TagDescription.exists?(id: params[:id], language_code: locale.language_code)
-    else
+      return head :no_content
-      head :not_found
     end
+    return head :no_content if TagGroup.exists?(id: params[:id])
+
+    head :not_found
   end
 
   def exists_by_title
+    locale = resolve_locale!
     title = params[:title].to_s.strip
-    if WikiPage.joins(:tag_name).exists?(tag_names: { name: title })
+    if (name = TagGroupRepr.name_from_title(title))
-      head :no_content
+      exists = TagGroup.exists?(name:)
-    else
+      return head exists ? :no_content : :not_found
-      head :not_found
     end
+
+    return head :no_content if tag_description_by_title(locale, title)
+
+    head :not_found
   end
 
   def diff
     id = params[:id]
     return render_bad_request('id は必須です．') if id.blank?
 
+    locale = resolve_locale!
+
     from = params[:from].presence
     to = params[:to].presence
 
-    page = WikiPage.joins(:tag_name).includes(tag_name: :tag).find(id)
+    description = TagDescription.find_by(id:, language_code: locale.language_code)
+    if description
+      versions = description_versions(description)
+      from_rev = from && versions.find(from)
+      to_rev = to ? versions.find(to) : versions.find_by!(version_no: description.version_no)
+      resource = WikiPageRepr.tag_description(locale, description)
+    else
+      tag_group = TagGroup.find_by(id:)
+      if tag_group
+        from_rev = from && group_version(tag_group, from)
+        to_rev = to ? group_version(tag_group, to) : current_group_version(tag_group)
+        return head :not_found if (from && !from_rev) || !to_rev
 
-    from_rev = from && page.wiki_revisions.find(from)
+        from_body = from_rev && render_group_version(locale, from_rev)
-    to_rev = to ? page.wiki_revisions.find(to) : page.current_revision
+        to_body = render_group_version(locale, to_rev)
-    if ((from_rev && !(from_rev.content?)) || !(to_rev&.content?))
+        resource = { 'id' => tag_group.id,
-      return render_unprocessable_entity('差分を表示できない版です．')
+                     'title' => "#{ TagGroupRepr::PREFIX }#{ tag_group.name }",
+                     'deprecated_at' => nil }
+      else
+        return head :not_found
+      end
     end
 
-    diffs = Diff::LCS.sdiff(from_rev&.body&.lines || [], to_rev.body.lines)
+    from_body ||= from_rev&.body
-    diff_json = diffs.map { |change|
+    to_body ||= to_rev.body
-      case change.action
+    diff_json = wiki_diff(from_body, to_body)
-      when ?=
-        { type: 'context', content: change.old_element }
-      when ?!
-        [{ type: 'removed', content: change.old_element },
-         { type: 'added', content: change.new_element }]
-      when ?+
-        { type: 'added', content: change.new_element }
-      when ?-
-        { type: 'removed', content: change.old_element }
-      end
-    }.flatten.compact
 
-    render json: { wiki_page_id:      page.id,
+    render json: { wiki_page_id:      resource.fetch('id'),
-                   title:             page.title,
+                   title:             resource.fetch('title'),
-                   deprecated_at:     page.deprecated_at,
+                   deprecated_at:     resource.fetch('deprecated_at'),
                    older_revision_id: from_rev&.id,
                    newer_revision_id: to_rev.id,
                    diff:              diff_json }
+  rescue TagGroupMarkdownRenderer::UnsupportedStructure => error
+    render_unprocessable_entity(error.message)
   end
 
   def create
@@ -94,20 +176,37 @@ class WikiPagesController < ApplicationController
     return render_unprocessable_entity('タイトルは必須です．', field: :title) if title.blank?
     return render_unprocessable_entity('本文は必須です．', field: :body) if body.blank?
 
-    tag_name = TagName.find_undiscard_or_create_by!(name: title)
+    locale = resolve_locale!
 
-    page =
+    if (name = TagGroupRepr.name_from_title(title))
-        Wiki::Commit.create_content!(
+      parsed = TagGroupMarkdownParser.parse(locale, body:, group: nil)
-            tag_name:,
+      group = TagGroupOperation.create!(
-            body:,
+          locale, name:, **parsed.except(:compatibility_body), actor: current_user,
-            created_by_user: current_user,
+          require_compatibility: true)
-            message:)
+      version = current_group_version(group)
+      return render_group_representation(TagGroupRepr.one(locale, group), version,
+                                         status: :created)
+    end
 
-    render json: WikiPageRepr.base(page), status: :created
+    description =
+        if title.include?(':')
+          return render_unprocessable_entity field: :title
+        else
+          tag_name = TagName.find_by(language_code: locale.language_code, name: title)
+          return render_unprocessable_entity field: :title unless tag_name&.tag
+
+          TagDescriptionOperation.create!(
+              locale,
+              tag: tag_name.tag, body:, created_by_user: current_user, message:)
+        end
+
+    render json: WikiPageRepr.tag_description(locale, description), status: :created
   rescue ActiveRecord::RecordInvalid => e
     render_validation_error e.record
   rescue ActiveRecord::RecordNotUnique
     render_record_not_unique
+  rescue TagGroupMarkdownParser::InvalidDocument => error
+    render_unprocessable_entity(error.message)
   end
 
   def update
@@ -120,91 +219,229 @@ class WikiPagesController < ApplicationController
     return render_unprocessable_entity('タイトルは必須です．', field: :title) if title.blank?
     return render_unprocessable_entity('本文は必須です．', field: :body) if body.blank?
 
-    page = WikiPage.find(params[:id])
+    locale = resolve_locale!
+
+    group = TagGroup.find_by(id: params[:id])
+    if (name = TagGroupRepr.name_from_title(title))
+      return head :not_found unless group
+
+      parsed = TagGroupMarkdownParser.parse(locale, body:, group:)
+      TagGroupOperation.update!(
+          locale, group:, name:, **parsed.except(:compatibility_body), actor: current_user,
+          require_compatibility: true)
+      return head :ok
+    end
+
+    return render_unprocessable_entity field: :title if title.include?(':')
+    description = TagDescription.find_by(id: params[:id], language_code: locale.language_code)
+    unless description
+      return render_unprocessable_entity field: :title if group
+      return head :not_found
+    end
+
+    tag = description.tag
+    tag_name = tag.tag_name(locale.language_code)
+    return render_unprocessable_entity field: :title unless tag_name
+    return render_unprocessable_entity field: :title if tag_name.name.include?(':')
+
     base_revision_id = params[:base_revision_id].presence
 
     ApplicationRecord.transaction do
-      page.lock!
+      SharedResourceIdAllocator.lock_space!(Tag, 'tags')
-
+      tag.lock!
-      old_title = page.title
+      old_title = tag_name.name
-
+      if title != old_title
-      tag = Tag.find_by(tag_name_id: page.tag_name_id)
-
-      if tag && title != old_title
         TagVersioning.ensure_snapshot!(tag, created_by_user: current_user)
       end
 
-      page.tag_name.update!(name: title) if title != old_title
+      tag_name.update!(name: title) if title != old_title
 
       message = params[:message].presence
-      Wiki::Commit.content!(page:,
+      TagDescriptionOperation.update!(locale, tag:, body:, updated_by_user: current_user,
-                            body:,
+                                      message:, base_revision_id:)
-                            created_user: current_user,
-                            message:,
-                            base_revision_id:)
 
-      if tag && title != old_title
+      if title != old_title
         tag.reload
         TagVersionRecorder.record!(tag:, event_type: :update, created_by_user: current_user)
       end
     end
 
     head :ok
-  end
+  rescue ActiveRecord::RecordInvalid => e
-
+    render_validation_error e.record
-  def search
+  rescue ActiveRecord::RecordNotUnique
-    index
+    render_record_not_unique
+  rescue TagGroupMarkdownParser::InvalidDocument => error
+    render_unprocessable_entity(error.message)
   end
 
   def changes
-    id = params[:id].presence
+    locale = resolve_locale!
-    q = WikiRevision.joins(wiki_page: :tag_name)
-                    .includes(:created_user, wiki_page: { tag_name: :tag })
-                    .order(id: :desc)
-    q = q.where(wiki_page_id: id) if id
 
-    render json: q.limit(200).map { |rev|
+    id = params[:id].presence
-      { revision_id:  rev.id,
+    description = id && TagDescription.find_by(id:, language_code: locale.language_code)
-        pred:         rev.base_revision_id,
+    if description
-        succ:         nil,
+      return render json: description_changes(locale, TagDescription.where(id: description.id))
-        wiki_page:    { id: rev.wiki_page_id,
+    end
-                        title: rev.wiki_page.title,
+
-                        deprecated_at: rev.wiki_page.deprecated_at },
+    tag_group = id && TagGroup.find_by(id:)
-        user:         rev.created_user && { id: rev.created_user.id, name: rev.created_user.name },
+    if tag_group
-        kind:         rev.kind,
+      return render json: tag_group_changes(HistoryAccess.visible(TagGroupVersion.all).where(tag_group_id: tag_group.id))
-        message:      rev.message,
+    end
-        timestamp:    rev.created_at }
+
-    }.compact
+    json = []
+    unless id
+      json.concat(tag_group_changes(HistoryAccess.visible(TagGroupVersion.all)))
+      json.concat(description_changes(locale,
+                                      TagDescription.where(language_code: locale.language_code)))
+      json = json.each_with_index.sort_by do |row, index|
+        [-row.fetch(:timestamp).to_f, index]
+      end.map(&:first)
+    end
+    render json: json.first(200)
   end
 
   private
 
-  def render_wiki_page_or_404 page
+  def render_tag_group locale, group
-    return head :not_found unless page
+    version = if params[:version].present?
-
+      group_version(group, params[:version])
-    rev = find_revision(page)
+    else
-    return head :not_found unless rev
+      current_group_version(group)
-
-    if rev.redirect?
-      return (
-          redirect_to wiki_page_by_title_path(title: rev.redirect_page.title),
-                      status: :moved_permanently)
     end
+    return head :not_found unless version
 
-    body = rev.body
+    representation =
-    revision_id = rev.id
+        if params[:version].present?
-    pred = page.pred_revision_id(revision_id)
+          TagGroupRepr.historical(locale, version)
-    succ = page.succ_revision_id(revision_id)
+        else
-    updated_at = rev.created_at
+          TagGroupRepr.one(locale, group)
-
+        end
-    render json: WikiPageRepr.base(page).merge(body:, revision_id:, pred:, succ:, updated_at:)
+    render_group_representation(representation, version)
   end
 
-  def find_revision page
+  def render_group_version locale, version
-    if params[:version].present?
+    representation = TagGroupRepr.historical(locale, version)
-      page.wiki_revisions.find_by(id: params[:version])
+    TagGroupMarkdownRenderer.render(representation)
-    else
+  end
-      page.current_revision
+
+  def render_group_representation representation, version, body: nil, status: :ok
+    body ||= TagGroupMarkdownRenderer.render(representation)
+    versions = HistoryAccess.visible(TagGroupVersion.all).where(tag_group_id: version.tag_group_id)
+    pred = versions.find_by(version_no: version.version_no - 1)&.id
+    succ = versions.find_by(version_no: version.version_no + 1)&.id
+    updated_at = version.created_at
+    render json: {
+        'id' => version.tag_group_id,
+        'resource_type' => 'tag_group',
+        'title' => "#{ TagGroupRepr::PREFIX }#{ representation.fetch('name') }",
+        'deprecated_at' => nil,
+        'body' => body,
+        'revision_id' => version.id,
+        'pred' => pred,
+        'succ' => succ,
+        'updated_at' => updated_at }, status:
+  rescue TagGroupMarkdownRenderer::UnsupportedStructure => error
+    render_unprocessable_entity(error.message)
+  end
+
+  def current_group_version group
+    HistoryAccess.visible(TagGroupVersion.all).find_by(tag_group_id: group.id, version_no: group.version_no)
+  end
+
+  def group_version group, version_id
+    HistoryAccess.visible(TagGroupVersion.all).find_by(tag_group_id: group.id, id: version_id)
+  end
+
+  def wiki_diff from_body, to_body
+    diffs = Diff::LCS.sdiff(from_body.to_s.lines, to_body.to_s.lines)
+    diffs.map { |change|
+      case change.action
+      when ?=
+        { type: 'context', content: change.old_element }
+      when ?!
+        [{ type: 'removed', content: change.old_element },
+         { type: 'added', content: change.new_element }]
+      when ?+
+        { type: 'added', content: change.new_element }
+      when ?-
+        { type: 'removed', content: change.old_element }
+      end
+    }.flatten.compact
+  end
+
+  def tag_group_changes versions
+    rows = versions.includes(:created_by_user).order(id: :desc).limit(200).to_a
+    groups = TagGroup.where(id: rows.map(&:tag_group_id).uniq).index_by(&:id)
+    predecessors = HistoryAccess.visible(TagGroupVersion.all).where(tag_group_id: groups.keys,
+                                         version_no: rows.map { _1.version_no - 1 })
+        .pluck(:tag_group_id, :version_no, :id)
+        .to_h { |group_id, number, version_id| [[group_id, number], version_id] }
+    rows.map do |version|
+      group = groups.fetch(version.tag_group_id)
+      user = version.created_by_user
+      { revision_id: version.id,
+        pred: predecessors[[group.id, version.version_no - 1]],
+        succ: nil,
+        wiki_page: { id: group.id, title: "#{ TagGroupRepr::PREFIX }#{ version.name }",
+                     deprecated_at: nil },
+        user: user && { id: user.id, name: user.name },
+        kind: 'content', message: nil, timestamp: version.created_at }
+    end
+  end
+
+  def render_wiki_page_by_tag_description locale, description
+    versions = description_versions(description)
+    historical = params[:version].present?
+    rev = historical ? versions.find_by(id: params[:version]) :
+        versions.find_by(version_no: description.version_no)
+    return head :not_found unless rev
+
+    pred = versions.find_by(version_no: rev.version_no - 1)&.id
+    succ = versions.find_by(version_no: rev.version_no + 1)&.id
+    body = historical ? rev.body : description.body
+    updated_at = historical ? rev.created_at : description.updated_at
+    render json: WikiPageRepr.tag_description(locale, description).merge(
+        body:, revision_id: rev.id, pred:, succ:, updated_at:)
+  end
+
+  def description_versions description
+    HistoryAccess.visible(TagDescriptionVersion.all).where(tag_id: description.tag_id,
+                                language_code: description.language_code)
+  end
+
+  def tag_description_by_title locale, title
+    HistoryAccess.current(TagDescription.all, TagDescriptionVersion).joins(tag: :tag_names)
+        .find_by(language_code: locale.language_code,
+                 tag_names: { language_code: locale.language_code, name: title })
+  end
+
+  def description_changes locale, descriptions
+    versions = HistoryAccess.visible(TagDescriptionVersion.all).where(language_code: locale.language_code,
+                                           tag_id: descriptions.select(:tag_id))
+        .order(id: :desc).limit(200).to_a
+    return [] if versions.empty?
+
+    by_tag = descriptions.where(tag_id: versions.map(&:tag_id))
+        .includes(tag: :tag_names).index_by(&:tag_id)
+    users = User.where(id: versions.map(&:created_by_user_id)).index_by(&:id)
+    predecessors = HistoryAccess.visible(TagDescriptionVersion.all).where(
+        language_code: locale.language_code, tag_id: by_tag.keys,
+        version_no: versions.map { |version| version.version_no - 1 })
+        .pluck(:tag_id, :version_no, :id)
+        .to_h { |tag_id, version_no, id| [[tag_id, version_no], id] }
+    versions.map do |version|
+      description = by_tag.fetch(version.tag_id)
+      user = users[version.created_by_user_id]
+      { revision_id: version.id,
+        pred: predecessors[[version.tag_id, version.version_no - 1]],
+        succ: nil,
+        wiki_page: { id: description.id, title: description.tag.name(locale.language_code),
+                     deprecated_at: description.tag.deprecated_at },
+        user: user && { id: user.id, name: user.name },
+        kind: 'content',
+        message: nil,
+        timestamp: version.created_at }
     end
   end
 

backend/app/models/external_tag.rb

@@ -0,0 +1,26 @@
+class ExternalTag < ApplicationRecord
+  before_create { SharedResourceIdAllocator.assign!(self) }
+
+  enum :platform, nico: 'nico'
+  validates :platform, presence: true, inclusion: { in: ExternalTag.platforms.keys }
+
+  has_many :post_external_tags, dependent: :delete_all
+  has_many :posts, through: :post_external_tags
+
+  has_many :nico_tag_versions, foreign_key: :tag_id, inverse_of: :external_tag
+
+  has_many :nico_tag_relations,
+           foreign_key: :nico_tag_id,
+           inverse_of: :nico_tag,
+           dependent: :destroy
+
+  has_many :linked_tags, through: :nico_tag_relations, source: :tag
+
+  def snapshot_linked_tag_names language_code = 'ja'
+    linked_tags
+        .joins(:tag_names)
+        .where(tag_names: { language_code:, primary_flg: true })
+        .order('tag_names.name')
+        .pluck('tag_names.name')
+  end
+end

backend/app/models/history_access.rb

@@ -0,0 +1,151 @@
+require 'digest'
+
+# Request/worker authority comes only from a server-resolved user. Default is public.
+class HistoryAccess < ActiveSupport::CurrentAttributes
+  attribute :user
+  class Forbidden < StandardError; end
+
+  KEYS = { 'post_versions' => ['post_id'], 'tag_versions' => ['tag_id'],
+           'nico_tag_versions' => ['tag_id'],
+           'tag_description_versions' => ['tag_id', 'language_code'],
+           'tag_group_versions' => ['tag_group_id'], 'material_versions' => ['material_id'] }.freeze
+
+  def self.revision
+    values = KEYS.keys.map do |table|
+      table.classify.constantize.unscoped.where(event_type: 'discard').maximum(:id)
+    end
+    Digest::SHA256.hexdigest(values.to_json)
+  end
+
+  def self.authority
+    { 'policy_revision' => revision, 'admin' => admin?, 'actor_id' => user&.id }
+  end
+
+  def self.authorised_artifact?(metadata)
+    return false unless metadata['policy_revision'].is_a?(String)
+    if metadata['admin']
+      admin? && user.id == metadata['actor_id']
+    else
+      metadata['policy_revision'] == revision
+    end
+  end
+
+  def self.admin? = user&.admin? == true
+
+  def self.sql(model, row = model.table_name, references: true)
+    return 'TRUE' if admin?
+
+    table = model.table_name
+    keys = KEYS.fetch(table)
+    matches = keys.map { "boundary.#{ _1 } = #{ row }.#{ _1 }" }.join(' AND ')
+    clauses = ["NOT EXISTS (SELECT 1 FROM #{ table } boundary WHERE #{ matches } " +
+        "AND boundary.event_type = 'discard' AND boundary.version_no >= #{ row }.version_no)"]
+    clauses += reference_sql(table, row) if references
+    clauses.map { "(#{ _1 })" }.join(' AND ')
+  end
+
+  def self.discard_reference(table, key, expression)
+    "NOT EXISTS (SELECT 1 FROM #{ table } blocked WHERE blocked.event_type = 'discard' " +
+        "AND #{ expression.call("blocked.#{ key }") })"
+  end
+
+  def self.reference_sql(table, row)
+    case table
+    when 'post_versions'
+      [['tag_versions', 'tag_id', 'tag_id'],
+       ['nico_tag_versions', 'tag_id', 'external_tag_id']].map do |target, key, field|
+        discard_reference(target, key, ->(id) {
+          "JSON_CONTAINS(#{ row }.tags_json, JSON_OBJECT('#{ field }', #{ id }))"
+        })
+      end + [discard_reference('post_versions', 'post_id', ->(id) {
+        "FIND_IN_SET(#{ id }, REPLACE(#{ row }.parent_post_ids, ' ', ',')) > 0"
+      })]
+    when 'tag_versions'
+      [discard_reference('tag_versions', 'tag_id', ->(id) {
+        "FIND_IN_SET(#{ id }, REPLACE(#{ row }.parent_tag_ids, ' ', ',')) > 0"
+      })]
+    when 'tag_description_versions'
+      [discard_reference('tag_versions', 'tag_id', ->(id) { "#{ row }.tag_id = #{ id }" })]
+    when 'material_versions'
+      [discard_reference('tag_versions', 'tag_id', ->(id) { "#{ row }.tag_id = #{ id }" }),
+       discard_reference('material_versions', 'material_id', ->(id) {
+         "#{ row }.parent_id = #{ id }"
+       })]
+    when 'tag_group_versions'
+      [['tag_versions', 'tag_id'], ['tag_group_versions', 'tag_group_id']].map do |target, field|
+        discard_reference(target, field, ->(id) {
+          ['members', 'placements'].map do |column|
+            "JSON_CONTAINS(COALESCE(JSON_EXTRACT(#{ row }.#{ column }, " +
+                "'$**.#{ field }'), JSON_ARRAY()), CAST(#{ id } AS CHAR))"
+          end.join(' OR ')
+        })
+      end
+    else []
+    end
+  end
+
+  def self.visible(scope, row = scope.klass.table_name)
+    scope.where(Arel.sql(sql(scope.klass, row)))
+  end
+
+  def self.current(scope, version_model)
+    return scope if admin?
+
+    table = scope.klass.table_name
+    keys = KEYS.fetch(version_model.table_name)
+    matches = keys.map do |key|
+      column = scope.klass.column_names.include?(key) ? key : 'id'
+      "current_snapshot.#{ key } = #{ table }.#{ column }"
+    end.join(' AND ')
+    boundary = "#{ matches } AND current_snapshot.version_no >= #{ table }.version_no " +
+        "AND current_snapshot.event_type = 'discard'"
+    scope = scope.where(Arel.sql("NOT EXISTS (SELECT 1 FROM #{ version_model.table_name } " +
+        "current_snapshot WHERE #{ boundary })"))
+    condition = "#{ matches } AND current_snapshot.version_no = #{ table }.version_no"
+    scope.where(Arel.sql("NOT EXISTS (SELECT 1 FROM #{ version_model.table_name } " +
+        "current_snapshot WHERE #{ condition } AND NOT (#{ sql(version_model, 'current_snapshot') }))"))
+  end
+
+  def self.require_current!(record, version_model)
+    return if admin?
+    raise Forbidden unless current(record.class.where(id: record.id), version_model).exists?
+  end
+
+  def self.blob_visible?(blob, seen = [])
+    return true if admin?
+    return false if seen.include?(blob.id)
+    seen = seen + [blob.id]
+    private_versions = MaterialVersion.where(file_blob_id: blob.id)
+        .where(Arel.sql("NOT (#{ sql(MaterialVersion) })"))
+    return false if private_versions.exists?
+
+    blob.attachments.each do |attachment|
+      model, key = { 'Post' => [PostVersion, 'post_id'],
+                     'Material' => [MaterialVersion, 'material_id'] }[attachment.record_type]
+      if model && model.where(key => attachment.record_id, event_type: 'discard').exists?
+        return false
+      end
+      if attachment.record_type == 'ActiveStorage::VariantRecord'
+        original = ActiveStorage::VariantRecord.find_by(id: attachment.record_id)&.blob
+        return false unless original && blob_visible?(original, seen)
+      end
+    end
+    true
+  end
+
+  def self.require_many!(records)
+    return if admin? || records.empty?
+
+    ids = records.map(&:id).uniq
+    raise Forbidden unless visible(records.first.class.where(id: ids)).count == ids.length
+  end
+
+  def self.require!(record)
+    return record if admin? || visible(record.class.where(id: record.id)).exists?
+
+    raise Forbidden
+  end
+
+  # This number deliberately ignores time, identity and content filters to prevent count probes.
+  def self.population_count(model) = model.unscoped.count
+end

backend/app/models/language.rb

@@ -0,0 +1,5 @@
+class Language < ApplicationRecord
+  has_many :locales, foreign_key: :language_code, primary_key: :code
+
+  enum :code, ja: 'ja', ain: 'ain'
+end

backend/app/models/locale.rb

@@ -0,0 +1,38 @@
+class Locale < ApplicationRecord
+  attr_accessor :certified_tag_names_path
+
+  around_save :save_with_tag_names!, if: :will_generate_tag_names?
+
+  belongs_to :language, foreign_key: :language_code, primary_key: :code
+  belongs_to :script, foreign_key: :script_code, primary_key: :code
+
+  # Enum deserialisation uses the first key for each stored value.
+  # Keep the legacy key for existing predicates, scopes and query arguments.
+  enum :code, { ja: 'ja', 'ain-Kana' => 'ain-Kana', ain_kana: 'ain-Kana' }
+
+  def code= value
+    super(self.class.codes.fetch(value.to_s, value))
+  end
+
+  def self.nipponese = Locale.find('ja')
+
+  private
+
+  def will_generate_tag_names?
+    deprecated_at.nil? && (will_save_change_to_code? || will_save_change_to_deprecated_at?)
+  end
+
+  def save_with_tag_names!
+    completed = self.class.transaction(requires_new: true) do
+      SharedResourceIdAllocator.lock_space!(Tag, 'tags')
+      saved = yield
+      raise ActiveRecord::RecordNotSaved, 'Locale save was halted' unless saved
+
+      TagNameCompletion.for_locale!(self)
+      true
+    end
+    unless completed
+      raise ActiveRecord::RecordNotSaved, 'Locale tag completion was rolled back'
+    end
+  end
+end

backend/app/models/nico_tag_relation.rb

@@ -1,24 +1,10 @@
 class NicoTagRelation < ApplicationRecord
-  belongs_to :nico_tag, class_name: 'Tag'
+  belongs_to :nico_tag,
-  belongs_to :tag, class_name: 'Tag'
+             class_name: 'ExternalTag',
+             foreign_key: :nico_tag_id,
+             inverse_of: :nico_tag_relations
+  belongs_to :tag, class_name: 'Tag', foreign_key: :tag_id
 
   validates :nico_tag_id, presence: true
   validates :tag_id, presence: true
-
-  validate :nico_tag_must_be_nico
-  validate :tag_mustnt_be_nico
-
-  private
-
-  def nico_tag_must_be_nico
-    if nico_tag && nico_tag.category != 'nico'
-      errors.add :nico_tag_id, 'タグのカテゴリがニコニコである必要があります．'
-    end
-  end
-
-  def tag_mustnt_be_nico
-    if tag && tag.category == 'nico'
-      errors.add :tag_id, '連携先タグのカテゴリはニコニコであってはなりません．'
-    end
-  end
 end

backend/app/models/nico_tag_version.rb

@@ -1,7 +1,7 @@
 class NicoTagVersion < ApplicationRecord
   include VersionRecord
 
-  belongs_to :tag
+  belongs_to :external_tag, foreign_key: :tag_id, inverse_of: :nico_tag_versions
 
   validates :name, presence: true
 end

backend/app/models/post.rb

@@ -1,7 +1,9 @@
 class Post < ApplicationRecord
   require 'date'
   require 'mini_magick'
+  require 'nokogiri'
   require 'stringio'
+  require 'timeout'
 
   class RemoteThumbnailFetchFailed < StandardError; end
 
@@ -11,44 +13,69 @@ class Post < ApplicationRecord
   ORIGINAL_CREATED_ORDER_MESSAGE = 'オリジナルの作成日時の順番がをかしぃです．'.freeze
   ORIGINAL_CREATED_MINIMUM_RANGE_MESSAGE =
       'オリジナルの作成日時の範囲は1分以上必要です．'.freeze
-
+  REMOTE_SVG_CONTENT_TYPE = 'image/svg+xml'.freeze
-  def self.resized_thumbnail_attachment(upload)
+  MAX_SVG_DIMENSION = 4_096
+  MAX_SVG_PIXELS = 16_777_216
+  THUMBNAIL_PROCESS_TIMEOUT = 5.seconds
+  def self.resized_thumbnail_attachment(upload, content_type: nil)
     upload.rewind
-    image = MiniMagick::Image.read(upload.read)
+    bytes = upload.read
-    image.resize '180x180^'
+    blob = Timeout.timeout(THUMBNAIL_PROCESS_TIMEOUT) do
-    image.gravity 'Center'
+      image = image_for_thumbnail_upload(bytes, content_type:)
-    image.extent '180x180'
+      image.auto_orient
-    image.format 'jpg'
+      image.resize '180x180'
+      image.format 'jpg'
+      image.to_blob
+    end
 
-    { io: StringIO.new(image.to_blob),
+    { io: StringIO.new(blob),
       filename: 'resized_thumbnail.jpg',
       content_type: 'image/jpeg' }
+  rescue Timeout::Error
+    raise MiniMagick::Error, 'サムネイル画像の変換に失敗しました．'
   ensure
     upload.rewind
   end
 
+  def self.remote_thumbnail_attachment(raw_url)
+    response = Preview::ThumbnailFetcher.fetch_image_response(raw_url)
+    resized_thumbnail_attachment(
+        StringIO.new(response.body),
+        content_type: response.content_type)
+  rescue Preview::UrlSafety::UnsafeUrl,
+         Preview::ThumbnailFetcher::GenerationFailed,
+         Preview::HttpFetcher::FetchFailed,
+         Preview::HttpFetcher::FetchTimeout,
+         Preview::HttpFetcher::ResponseTooLarge,
+         Timeout::Error,
+         MiniMagick::Error => e
+    raise RemoteThumbnailFetchFailed, e.message
+  end
+
   belongs_to :uploaded_user, class_name: 'User', optional: true
 
   has_many :post_tags, dependent: :destroy, inverse_of: :post
-  has_many :active_post_tags, -> { kept }, class_name: 'PostTag', inverse_of: :post
+  has_many :tags, through: :post_tags
-  has_many :post_tags_with_discarded, -> { with_discarded }, class_name: 'PostTag'
-  has_many :tags, through: :active_post_tags
   has_many :active_tags, -> { where(tags: { deprecated_at: nil }) },
-           through: :active_post_tags, source: :tag
+           through: :post_tags,
+           source: :tag
 
   has_many :user_post_views, dependent: :delete_all
   has_many :post_similarities, dependent: :delete_all
   has_many :post_versions
+
   has_many :gekanator_guessed_games,
            class_name: 'GekanatorGame',
            foreign_key: :guessed_post_id,
            dependent: :delete_all,
            inverse_of: :guessed_post
+
   has_many :gekanator_correct_games,
            class_name: 'GekanatorGame',
            foreign_key: :correct_post_id,
            dependent: :delete_all,
            inverse_of: :correct_post
+
   has_many :gekanator_question_examples, dependent: :delete_all
 
   has_many :parent_post_implications,
@@ -65,6 +92,9 @@ class Post < ApplicationRecord
            inverse_of: :parent_post
   has_many :children, through: :child_post_implications, source: :post
 
+  has_many :post_external_tags, dependent: :destroy
+  has_many :external_tags, through: :post_external_tags
+
   has_one_attached :thumbnail
 
   attribute :version_no, :integer, default: 1
@@ -96,14 +126,14 @@ class Post < ApplicationRecord
     super(options).merge(thumbnail: nil)
   end
 
-  def snapshot_tag_names
+  def snapshot_tag_names language_code = 'ja'
     post_tags
-        .kept
+        .joins(tag: :tag_names)
-        .joins(tag: :tag_name)
+        .where(tag_names: { language_code:, primary_flg: true })
-        .includes(:sections, tag: :tag_name)
+        .includes(:sections, tag: :tag_names)
         .order('tag_names.name')
         .map do |post_tag|
-      name = post_tag.tag.tag_name.name
+      name = post_tag.tag.name(language_code)
       sections = post_tag.sections.sort_by(&:begin_ms)
 
       next name if sections.empty?
@@ -112,8 +142,47 @@ class Post < ApplicationRecord
     end
   end
 
+  def self.tag_snapshot_literal tag
+    sections = tag.fetch('sections', []).map do |sec|
+      begin_ms = sec.fetch('begin_ms')
+      end_ms = sec['end_ms']
+
+      "[#{ Post.ms_to_time(begin_ms) }-#{ end_ms ? Post.ms_to_time(end_ms) : '' }]"
+    end
+
+    "#{ tag.fetch('name') }#{ sections.join }"
+  end
+
+  def snapshot_tags_json language_code = 'ja'
+    tag_snapshots =
+        post_tags
+            .joins(tag: :tag_names)
+            .where(tag_names: { language_code:, primary_flg: true })
+            .includes(:sections, tag: :tag_names)
+            .order('tags.id')
+            .map { |pt|
+          { 'tag_id' => pt.tag.id,
+            'version_no' => pt.tag.version_no,
+            'name' => pt.tag.name(language_code),
+            'category' => pt.tag.category,
+            'sections' => pt.sections.sort_by(&:begin_ms).map {
+                            { 'begin_ms' => _1.begin_ms, 'end_ms' => _1.end_ms }
+                          } }
+        }
+
+    external_tag_snapshots =
+        post_external_tags.order(:external_tag_id).map {
+          { 'external_tag_id' => _1.external_tag_id }
+        }
+
+    tag_snapshots + external_tag_snapshots
+  end
+
   def self.section_literal section
-    "[#{ Post.ms_to_time(section.begin_ms) }-#{ section.end_ms ? Post.ms_to_time(section.end_ms) : '' }]"
+    end_ms =
+        section.end_ms ? Post.ms_to_time(section.end_ms) : ''
+
+    "[#{ Post.ms_to_time(section.begin_ms) }-#{ end_ms }]"
   end
 
   def self.ms_to_time ms
@@ -157,17 +226,7 @@ class Post < ApplicationRecord
   end
 
   def attach_thumbnail_from_url! raw_url
-    response = Preview::ThumbnailFetcher.fetch_image_response(raw_url)
+    thumbnail.attach(self.class.remote_thumbnail_attachment(raw_url))
-    thumbnail.attach(
-        self.class.resized_thumbnail_attachment(
-            StringIO.new(response.body)))
-  rescue Preview::UrlSafety::UnsafeUrl,
-         Preview::ThumbnailFetcher::GenerationFailed,
-         Preview::HttpFetcher::FetchFailed,
-         Preview::HttpFetcher::FetchTimeout,
-         Preview::HttpFetcher::ResponseTooLarge,
-         MiniMagick::Error => e
-    raise RemoteThumbnailFetchFailed, e.message
   end
 
   private
@@ -209,6 +268,179 @@ class Post < ApplicationRecord
     self.url = PostUrlNormaliser.normalise(url) || url.strip
   end
 
+  def self.image_for_thumbnail_upload(bytes, content_type: nil)
+    if svg_content_type?(content_type) || svg_document_bytes?(bytes)
+      return decode_svg_thumbnail(bytes)
+    end
+
+    raise MiniMagick::Error, 'サムネイル画像の形式が不正です．' unless raster_thumbnail_bytes?(bytes)
+
+    decode_raster_thumbnail(bytes)
+  end
+
+  def self.raster_thumbnail_bytes?(bytes)
+    raster_thumbnail_format(bytes).present?
+  end
+
+  def self.remote_thumbnail_image_bytes?(bytes, content_type: nil)
+    return true if svg_content_type?(content_type) || svg_document_bytes?(bytes)
+
+    raster_thumbnail_bytes?(bytes)
+  end
+
+  def self.svg_content_type?(content_type)
+    content_type.to_s.split(';', 2).first.to_s.downcase.strip == REMOTE_SVG_CONTENT_TYPE
+  end
+
+  def self.svg_document_bytes?(bytes)
+    document = Nokogiri::XML(
+        bytes,
+        nil,
+        nil,
+        Nokogiri::XML::ParseOptions::STRICT |
+        Nokogiri::XML::ParseOptions::NONET)
+    document.root&.name == 'svg'
+  rescue Nokogiri::XML::SyntaxError
+    false
+  end
+
+  def self.decode_raster_thumbnail(bytes)
+    MiniMagick::Image.read(bytes)
+  end
+
+  def self.decode_svg_thumbnail(bytes)
+    MiniMagick::Image.read(sanitised_svg_bytes(bytes))
+  end
+
+  def self.raster_thumbnail_format(bytes)
+    binary = bytes.to_s.b
+    return 'jpeg' if binary.start_with?("\xFF\xD8\xFF".b)
+    return 'png' if binary.start_with?("\x89PNG\r\n\x1A\n".b)
+    return 'gif' if binary.start_with?('GIF87a'.b) || binary.start_with?('GIF89a'.b)
+    return 'webp' if binary.bytesize >= 12 &&
+                     binary.start_with?('RIFF'.b) &&
+                     binary.byteslice(8, 4) == 'WEBP'
+
+    nil
+  end
+
+  def self.sanitised_svg_bytes(bytes)
+    parse_options =
+        Nokogiri::XML::ParseOptions::STRICT |
+        Nokogiri::XML::ParseOptions::NONET
+    document = Nokogiri::XML(
+        bytes,
+        nil,
+        nil,
+        parse_options)
+    root = document.root
+    raise MiniMagick::Error, 'SVG が不正です．' if root == nil || root.name != 'svg'
+    raise MiniMagick::Error, 'SVG が不正です．' if document.internal_subset != nil
+    raise MiniMagick::Error, 'SVG が不正です．' if svg_uses_disallowed_features?(document)
+
+    width, height = svg_dimensions(root)
+    raise MiniMagick::Error, 'SVG が大きすぎます．' if width == nil || height == nil
+    if width > MAX_SVG_DIMENSION || height > MAX_SVG_DIMENSION || width * height > MAX_SVG_PIXELS
+      raise MiniMagick::Error, 'SVG が大きすぎます．'
+    end
+
+    document.to_xml
+  rescue Nokogiri::XML::SyntaxError
+    raise MiniMagick::Error, 'SVG が不正です．'
+  end
+
+  def self.svg_uses_disallowed_features?(document)
+    document.traverse.any? do |node|
+      next false unless node.element?
+
+      name = node.name.to_s.downcase
+      next true if name == 'script' || name == 'foreignobject'
+      next style_contains_disallowed_urls?(node.text.to_s) if name == 'style'
+
+      node.attribute_nodes.any? do |attribute|
+        attribute_name = attribute.name.to_s.downcase
+        attribute_value = attribute.value.to_s
+        attribute_name.start_with?('on') ||
+            external_svg_reference?(attribute_name, attribute_value)
+      end
+    end
+  end
+
+  def self.external_svg_reference?(attribute_name, attribute_value)
+    if ['href', 'xlink:href', 'src'].include?(attribute_name)
+      return external_svg_url?(attribute_value)
+    end
+    return style_contains_disallowed_urls?(attribute_value) if attribute_name == 'style'
+    return svg_url_function_disallowed?(attribute_value) if attribute_value.match?(/url\s*\(/i)
+
+    false
+  end
+
+  def self.external_svg_url?(value)
+    stripped = value.to_s.strip
+    return false if stripped.blank? || stripped.start_with?('#')
+
+    true
+  end
+
+  def self.style_contains_disallowed_urls?(value)
+    text = value.to_s
+    text.match?(/@import/i) || svg_url_function_disallowed?(text)
+  end
+
+  def self.svg_url_function_disallowed?(value)
+    value.to_s.scan(/url\s*\(([^)]*)\)/i).flatten.any? do |entry|
+      reference =
+          entry.to_s.strip
+               .delete_prefix("'")
+               .delete_prefix('"')
+               .delete_suffix("'")
+               .delete_suffix('"')
+      reference.present? && !(reference.start_with?('#'))
+    end
+  end
+
+  def self.svg_dimensions(root)
+    width = svg_length_to_pixels(root['width'])
+    height = svg_length_to_pixels(root['height'])
+    return [width, height] if width && height
+
+    view_box = root['viewBox'].to_s.strip.split(/\s+/).map { Float(_1) rescue nil }
+    return [nil, nil] if view_box.length != 4 || view_box.any?(&:nil?)
+    return [nil, nil] unless view_box[2].finite? && view_box[2].positive?
+    return [nil, nil] unless view_box[3].finite? && view_box[3].positive?
+
+    [view_box[2], view_box[3]]
+  end
+
+  def self.svg_length_to_pixels(value)
+    return nil if value.blank?
+
+    matched = /\A([0-9]+(?:\.[0-9]+)?)(px)?\z/i.match(value.to_s.strip)
+    return nil if matched == nil
+
+    pixels = Float(matched[1])
+    return nil unless pixels.finite? && pixels.positive?
+
+    pixels
+  rescue ArgumentError
+    nil
+  end
+
+  private_class_method :image_for_thumbnail_upload,
+                       :svg_content_type?,
+                       :decode_raster_thumbnail,
+                       :decode_svg_thumbnail,
+                       :raster_thumbnail_format,
+                       :sanitised_svg_bytes,
+                       :svg_uses_disallowed_features?,
+                       :external_svg_reference?,
+                       :external_svg_url?,
+                       :style_contains_disallowed_urls?,
+                       :svg_url_function_disallowed?,
+                       :svg_dimensions,
+                       :svg_length_to_pixels
+
   def parse_original_created_value field
     raw_value = public_send("#{ field }_before_type_cast")
     value = public_send(field)
@@ -252,7 +484,12 @@ class Post < ApplicationRecord
 
     match =
         value.match(
-            /\A(\d{4})-(\d{2})-(\d{2})T(\d{2}):(\d{2})(?::(\d{2})(?:\.(\d+))?)?(Z|[+-]\d{2}:?\d{2})?\z/)
+            /
+              \A(\d{4})-(\d{2})-(\d{2})T(\d{2}):(\d{2})
+              (?::(\d{2})(?:\.(\d+))?)?
+              (Z|[+-]\d{2}:?\d{2})?
+              \z
+            /x)
     return nil if match.nil?
 
     year = match[1].to_i

backend/app/models/post_external_tag.rb

@@ -0,0 +1,4 @@
+class PostExternalTag < ApplicationRecord
+  belongs_to :post
+  belongs_to :external_tag, counter_cache: :post_count
+end

backend/app/models/post_tag.rb

@@ -1,14 +1,7 @@
 class PostTag < ApplicationRecord
-  include Discard::Model
-
-  before_destroy do
-    raise ActiveRecord::ReadOnlyRecord, '消さないでください．'
-  end
-
   belongs_to :post
   belongs_to :tag, counter_cache: :post_count
   belongs_to :created_user, class_name: 'User', optional: true
-  belongs_to :deleted_user, class_name: 'User', optional: true
 
   has_many :sections, -> { order(:begin_ms) }, class_name: 'PostTagSection',
                                                foreign_key: [:post_id, :tag_id],
@@ -18,18 +11,5 @@ class PostTag < ApplicationRecord
 
   validates :post_id, presence: true
   validates :tag_id, presence: true
-  validates :post_id, uniqueness: {
+  validates :post_id, uniqueness: { scope: :tag_id }
-      scope: :tag_id,
-      conditions: -> { where(discarded_at: nil) } }
-
-  def discard_by! deleted_user
-    return self if discarded?
-
-    transaction do
-      update!(discarded_at: Time.current, deleted_user:)
-      Tag.where(id: tag_id).update_all('post_count = GREATEST(post_count - 1, 0)')
-    end
-
-    self
-  end
 end

backend/app/models/post_tag_section.rb

@@ -4,10 +4,10 @@ class PostTagSection < ApplicationRecord
   belongs_to :post
   belongs_to :tag
 
-  belongs_to :post_tag, -> { kept }, foreign_key: [:post_id, :tag_id],
+  belongs_to :post_tag, foreign_key: [:post_id, :tag_id],
-                                     primary_key: [:post_id, :tag_id],
+                        primary_key: [:post_id, :tag_id],
-                                     inverse_of: :sections,
+                        inverse_of: :sections,
-                                     optional: true
+                        optional: true
 
   validates :post_id, presence: true
   validates :tag_id, presence: true

backend/app/models/post_version.rb

@@ -1,6 +1,12 @@
 class PostVersion < ApplicationRecord
   include VersionRecord
 
+  scope :with_previous_version, -> {
+    joins('LEFT JOIN post_versions prev ON prev.post_id = post_versions.post_id ' +
+          'AND prev.version_no = post_versions.version_no - 1 AND ' +
+          HistoryAccess.sql(PostVersion, 'prev'))
+  }
+
   belongs_to :post
   belongs_to :parent, class_name: 'Post', optional: true
 

backend/app/models/region.rb

@@ -0,0 +1,3 @@
+class Region < ApplicationRecord
+  ;
+end

backend/app/models/script.rb

@@ -0,0 +1,11 @@
+class Script < ApplicationRecord
+  has_many :locales, foreign_key: :script_code, primary_key: :code
+
+  # Canonical keys come first so reads and plucks retain ISO 15924 spelling.
+  # Legacy keys preserve the existing enum predicates, scopes and queries.
+  enum :code, { 'Jpan' => 'Jpan', 'Kana' => 'Kana', jpan: 'Jpan', kana: 'Kana' }
+
+  def code= value
+    super(self.class.codes.fetch(value.to_s, value))
+  end
+end

backend/app/models/tag.rb

@@ -2,7 +2,7 @@ require 'set'
 
 
 class Tag < ApplicationRecord
-  include MyDiscard
+  before_create { SharedResourceIdAllocator.assign!(self) }
 
   class NicoTagNormalisationError < ArgumentError
     ;
@@ -28,15 +28,12 @@ class Tag < ApplicationRecord
   end
 
   has_many :post_tags, inverse_of: :tag
-  has_many :active_post_tags, -> { kept }, class_name: 'PostTag', inverse_of: :tag
+  has_many :posts, through: :post_tags
-  has_many :post_tags_with_discarded, -> { with_discarded }, class_name: 'PostTag'
-  has_many :posts, through: :active_post_tags
-
-  has_many :nico_tag_relations, foreign_key: :nico_tag_id, dependent: :destroy
-  has_many :linked_tags, through: :nico_tag_relations, source: :tag
 
   has_many :reversed_nico_tag_relations,
-           class_name: 'NicoTagRelation', foreign_key: :tag_id, dependent: :destroy
+           class_name: 'NicoTagRelation',
+           foreign_key: :tag_id,
+           dependent: :destroy
   has_many :linked_nico_tags, through: :reversed_nico_tag_relations, source: :nico_tag
 
   has_many :tag_implications, foreign_key: :parent_tag_id, dependent: :destroy
@@ -54,32 +51,23 @@ class Tag < ApplicationRecord
   has_many :materials
 
   has_many :tag_versions
-  has_many :nico_tag_versions
 
-  belongs_to :tag_name
+  has_many :tag_names
-  delegate :wiki_page, to: :tag_name
+  has_many :tag_descriptions
 
   attribute :version_no, :integer, default: 1
 
-  delegate :name, to: :tag_name, allow_nil: true
-  validates :tag_name, presence: true
-
   enum :category, deerjikist:  'deerjikist',
                   meme:        'meme',
                   character:   'character',
                   general:     'general',
                   material:    'material',
-                  nico:        'nico',
                   meta:        'meta'
 
   validates :category, presence: true, inclusion: { in: Tag.categories.keys }
 
-  validate :nico_tag_name_must_start_with_nico
-  validate :tag_name_must_be_canonical
   validate :category_must_be_deerjikist_with_deerjikists
-  validate :nico_tags_cannot_be_deprecated
+  validate :tag_names_must_be_valid
-
-  scope :nico_tags, -> { nico }
 
   CATEGORY_PREFIXES = {
       'general:'    => :general,
@@ -93,233 +81,296 @@ class Tag < ApplicationRecord
       'mtr:'        => :material,
       'meta:'       => :meta }.freeze
 
-  def name= val
+  SYSTEM_TAG_NAMES = ['タグ希望', 'bot操作', 'ニジラー情報不詳', '動画', 'ニコニコ', 'YouTube'].freeze
-    (self.tag_name ||= build_tag_name).name = val
+
+  def tag_name language_code
+    if tag_names.loaded?
+      tag_names.find { _1.language_code == language_code && _1.primary_flg }
+    else
+      tag_names.find_by(language_code:, primary_flg: true)
+    end
+  end
+  def name(language_code) = tag_name(language_code)&.name
+
+  def system?
+    tag_name('ja')&.name.in?(SYSTEM_TAG_NAMES)
   end
 
   def deprecated? = deprecated_at?
 
-  def has_wiki = wiki_page.present?
+  def has_description? language_code:
+    if tag_descriptions.loaded?
+      tag_descriptions.any? { _1.language_code == language_code }
+    else
+      tag_descriptions.exists?(language_code:)
+    end
+  end
 
   def material_id = materials.first&.id
 
   def has_deerjikists = deerjikists.loaded? ? deerjikists.any? : deerjikists.exists?
 
-  def self.tagme = find_or_create_by_tag_name!('タグ希望', category: :meta)
+  def snapshot_aliases language_code
-  def self.bot = find_or_create_by_tag_name!('bot操作', category: :meta)
+    if tag_names.loaded?
-  def self.no_deerjikist = find_or_create_by_tag_name!('ニジラー情報不詳', category: :meta)
+      return tag_names.select { _1.language_code == language_code && !_1.primary? }
-  def self.video = find_or_create_by_tag_name!('動画', category: :meta)
+          .sort_by(&:name).map(&:name)
-  def self.niconico = find_or_create_by_tag_name!('ニコニコ', category: :meta)
-  def self.youtube = find_or_create_by_tag_name!('YouTube', category: :meta)
-
-  def self.normalise_tags! tag_names, with_tagme: true,
-                                      with_no_deerjikist: true,
-                                      deny_nico: true,
-                                      deny_deprecated: false,
-                                      with_sections: false
-    if deny_nico && tag_names.any? { |n| n.downcase.start_with?('nico:') }
-      raise NicoTagNormalisationError
     end
 
-    sections = { }
+    tag_names.where(language_code:, primary_flg: false).order(:name).pluck(:name)
-    tags = tag_names.map do |name|
-      raw_name = name
-      pf, cat = CATEGORY_PREFIXES.find { |p, _| name.downcase.start_with?(p) } || ['', nil]
-
-      name = name.sub(/\A#{ pf }/i, '')
-
-      sections_by_tag = []
-      while (match = name.match(/\A(\S*?)\[([^\[\]\s]*)-([^\[\]\s]*)\](\S*)\z/))
-        name = "#{ match[1] }#{ match[4] }"
-        next if match[2].empty? && match[3].empty?
-
-        sections_by_tag << normalise_section_range!(
-            begin_raw: match[2],
-            end_raw: match[3],
-            tag_name: name)
-      end
-
-      if name.include?('[') || name.include?(']')
-        raise SectionLiteralParseError.new(raw_name, raw_name)
-      end
-
-      name = TagName.canonicalise(name).first
-
-      find_or_create_by_tag_name!(name, category: (cat || :general)).tap do |tag|
-        if deny_deprecated && tag.deprecated?
-          raise DeprecatedTagNormalisationError, [tag.name]
-        end
-
-        tag.update!(category: cat) if cat && tag.category != cat
-        next if sections_by_tag.empty?
-
-        sections[tag.id] ||= []
-        sections[tag.id].concat(sections_by_tag)
-        sections[tag.id] = merge_section_ranges(sections[tag.id])
-        sections.delete(tag.id) if sections[tag.id] == [[0, nil]]
-      end
-    end
-
-    tags << Tag.tagme if with_tagme && tags.size < 10 && tags.none?(Tag.tagme)
-    tags << Tag.no_deerjikist if with_no_deerjikist && tags.all? { |t| !(t.deerjikist?) }
-    tags.uniq!(&:id)
-
-    if with_sections
-      { tags:, sections: }
-    else
-      tags
-    end
   end
 
-  def self.expand_parent_tags tags
-    return [] if tags.blank?
-
-    seen = Set.new
-    result = []
-    stack = tags.compact.dup
-
-    until stack.empty?
-      tag = stack.pop
-      next unless tag
-
-      tag.parents.each do |parent|
-        next if seen.include?(parent.id)
-
-        seen << parent.id
-        result << parent
-        stack << parent
-      end
-    end
-
-    (result + tags).uniq { |t| t.id }
-  end
-
-  def self.normalise_section_range! begin_raw:, end_raw:, tag_name:
-    begin_ms = begin_raw.empty? ? 0 : time_to_ms!(begin_raw, tag_name:)
-    end_ms = end_raw.empty? ? nil : time_to_ms!(end_raw, tag_name:)
-
-    if end_ms
-      begin_ms, end_ms = end_ms, begin_ms if begin_ms > end_ms
-      end_ms = begin_ms + 1 if begin_ms == end_ms
-    end
-
-    [begin_ms, end_ms]
-  end
-
-  def self.merge_section_ranges ranges
-    sorted_ranges = ranges.sort_by { |begin_ms, end_ms| [begin_ms, end_ms || Float::INFINITY] }
-    merged = []
-
-    sorted_ranges.each do |begin_ms, end_ms|
-      if merged.empty?
-        merged << [begin_ms, end_ms]
-        next
-      end
-
-      last_begin_ms, last_end_ms = merged[-1]
-      if last_end_ms.nil? || begin_ms <= last_end_ms
-        merged[-1] = [last_begin_ms, merge_section_end(last_end_ms, end_ms)]
-      else
-        merged << [begin_ms, end_ms]
-      end
-    end
-
-    merged
-  end
-
-  def self.merge_section_end left_end_ms, right_end_ms
-    return nil if left_end_ms.nil? || right_end_ms.nil?
-
-    [left_end_ms, right_end_ms].max
-  end
-
-  def self.find_or_create_by_tag_name! name, category:
-    tn = TagName.find_undiscard_or_create_by!(name: name.to_s.strip)
-    tn = tn.canonical if tn.canonical_id?
-
-    Tag.find_undiscard_or_create_by!(tag_name_id: tn.id) do |t|
-      t.category = category
-    end
-  rescue ActiveRecord::RecordNotUnique
-    retry
-  end
-
-  def self.merge_tags! target_tag, source_tags, created_by_user: nil
-    target_tag => Tag
-
-    affected_post_ids = Set.new
-
-    Tag.transaction do
-      TagVersioning.ensure_snapshot!(target_tag, created_by_user:)
-
-      Array(source_tags).compact.uniq.each do |source_tag|
-        source_tag => Tag
-
-        next if source_tag == target_tag
-
-        TagVersioning.ensure_snapshot!(source_tag, created_by_user:)
-
-        source_tag.post_tags.kept.find_each do |source_pt|
-          post_id = source_pt.post_id
-          affected_post_ids << post_id
-          source_pt.discard_by!(created_by_user)
-          unless PostTag.kept.exists?(post_id:, tag: target_tag)
-            PostTag.create!(post_id:, tag: target_tag)
-          end
-        end
-
-        source_tag_name = source_tag.tag_name
-
-        if source_tag_name.wiki_page.present?
-          raise ActiveRecord::RecordInvalid.new(source_tag_name)
-        end
-
-        TagVersioning.record!(source_tag, event_type: :discard, created_by_user:)
-        source_tag.discard!
-
-        if source_tag.nico?
-          source_tag_name.discard!
-        else
-          source_tag_name.update_columns(canonical_id: target_tag.tag_name_id,
-                                         updated_at: Time.current)
-        end
-
-        TagVersioning.record!(target_tag, event_type: :update, created_by_user:)
-      end
-
-      Post.where(id: affected_post_ids.to_a).find_each do |post|
-        PostVersionRecorder.ensure_snapshot!(post, created_by_user:)
-        PostVersionRecorder.record!(post:, event_type: :update, created_by_user:)
-      end
-
-      # 投稿件数を再集計
-      target_tag.update_columns(post_count: PostTag.kept.where(tag: target_tag).count)
-    end
-
-    target_tag.reload
-  end
-
-  def snapshot_aliases = tag_name.aliases.kept.order(:name).pluck(:name)
-
   def snapshot_parent_tag_ids = parents.order(:id).pluck(:id)
 
-  def snapshot_linked_tag_names
+  def snapshot_linked_tag_names language_code
-    linked_tags.joins(:tag_name).order('tag_names.name').pluck('tag_names.name')
+    linked_tags
+        .joins(:tag_names)
+        .where(tag_names: { language_code:, primary_flg: true })
+        .order('tag_names.name')
+        .pluck('tag_names.name')
+  end
+
+  class << self
+    def tagme = find_or_create_by_tag_name!(Locale.nipponese, 'タグ希望', category: :meta)
+    def bot = find_or_create_by_tag_name!(Locale.nipponese, 'bot操作', category: :meta)
+    def no_deerjikist =
+        find_or_create_by_tag_name!(Locale.nipponese, 'ニジラー情報不詳', category: :meta)
+    def video = find_or_create_by_tag_name!(Locale.nipponese, '動画', category: :meta)
+    def niconico = find_or_create_by_tag_name!(Locale.nipponese, 'ニコニコ', category: :meta)
+    def youtube = find_or_create_by_tag_name!(Locale.nipponese, 'YouTube', category: :meta)
+
+    def normalise_tags! locale, tag_names,
+                             with_tagme: true,
+                             with_no_deerjikist: true,
+                             deny_deprecated: false,
+                             with_sections: false,
+                             created_by_user: nil
+      if tag_names.any? { |n| n.downcase.start_with?('nico:') }
+        raise NicoTagNormalisationError
+      end
+
+      sections = { }
+      tags = tag_names.map do |name|
+        raw_name = name
+        pf, cat = CATEGORY_PREFIXES.find { |p, _| name.downcase.start_with?(p) } || ['', nil]
+
+        name = name.sub(/\A#{ pf }/i, '')
+
+        sections_by_tag = []
+        while (match = name.match(/\A(\S*?)\[([^\[\]\s]*)-([^\[\]\s]*)\](\S*)\z/))
+          name = "#{ match[1] }#{ match[4] }"
+          next if match[2].empty? && match[3].empty?
+
+          sections_by_tag << normalise_section_range!(
+              begin_raw: match[2],
+              end_raw: match[3],
+              tag_name: name)
+        end
+
+        if name.include?('[') || name.include?(']')
+          raise SectionLiteralParseError.new(raw_name, raw_name)
+        end
+
+        name = TagName.canonicalise(locale, name).first
+
+        find_or_create_by_tag_name!(locale, name, category: (cat || :general),
+                                   created_by_user:).tap do |tag|
+          if deny_deprecated && tag.deprecated?
+            raise DeprecatedTagNormalisationError, [tag.name(locale.language_code)]
+          end
+
+          tag.update!(category: cat) if cat && tag.category != cat
+          next if sections_by_tag.empty?
+
+          sections[tag.id] ||= []
+          sections[tag.id].concat(sections_by_tag)
+          sections[tag.id] = merge_section_ranges(sections[tag.id])
+          sections.delete(tag.id) if sections[tag.id] == [[0, nil]]
+        end
+      end
+
+      tags << Tag.tagme if with_tagme && tags.size < 10 && tags.none?(Tag.tagme)
+      tags << Tag.no_deerjikist if with_no_deerjikist && tags.all? { |t| !(t.deerjikist?) }
+      tags.uniq!(&:id)
+
+      if with_sections
+        { tags:, sections: }
+      else
+        tags
+      end
+    end
+
+    def expand_parent_tags tags
+      return [] if tags.blank?
+
+      seen = Set.new
+      result = []
+      stack = tags.compact.dup
+
+      until stack.empty?
+        tag = stack.pop
+        next unless tag
+
+        tag.parents.each do |parent|
+          next if seen.include?(parent.id)
+
+          seen << parent.id
+          result << parent
+          stack << parent
+        end
+      end
+
+      (result + tags).uniq { |t| t.id }
+    end
+
+    def normalise_section_range! begin_raw:, end_raw:, tag_name:
+      begin_ms = begin_raw.empty? ? 0 : time_to_ms!(begin_raw, tag_name:)
+      end_ms = end_raw.empty? ? nil : time_to_ms!(end_raw, tag_name:)
+
+      if end_ms
+        begin_ms, end_ms = end_ms, begin_ms if begin_ms > end_ms
+        end_ms = begin_ms + 1 if begin_ms == end_ms
+      end
+
+      [begin_ms, end_ms]
+    end
+
+    def merge_section_ranges ranges
+      sorted_ranges = ranges.sort_by { |begin_ms, end_ms| [begin_ms, end_ms || Float::INFINITY] }
+      merged = []
+
+      sorted_ranges.each do |begin_ms, end_ms|
+        if merged.empty?
+          merged << [begin_ms, end_ms]
+          next
+        end
+
+        last_begin_ms, last_end_ms = merged[-1]
+        if last_end_ms.nil? || begin_ms <= last_end_ms
+          merged[-1] = [last_begin_ms, merge_section_end(last_end_ms, end_ms)]
+        else
+          merged << [begin_ms, end_ms]
+        end
+      end
+
+      merged
+    end
+
+    def merge_section_end left_end_ms, right_end_ms
+      return nil if left_end_ms.nil? || right_end_ms.nil?
+
+      [left_end_ms, right_end_ms].max
+    end
+
+    def find_or_create_by_tag_name! locale, name, category:, created_by_user: nil
+      language_code = locale.language_code
+      name = name.to_s.strip
+
+      ApplicationRecord.transaction do
+        SharedResourceIdAllocator.lock_space!(Tag, 'tags')
+        tn = TagName.lock.find_by(language_code:, name:)
+        next tn.tag if tn
+
+        tag = Tag.new(category:)
+        tag.tag_names.build(language_code:, name:, script_code: locale.script_code,
+                            primary_flg: true, auto_generated: false)
+        TagNameCompletion.for_tag!(tag, created_by_user:)
+        tag.association(:tag_names).reset
+
+        tag
+      end
+    rescue ActiveRecord::RecordNotUnique
+      existing = TagName.find_by(language_code:, name:)
+      raise unless existing
+
+      existing.tag
+    end
+
+    def merge_tags! target_tag, source_tags, created_by_user: nil
+      target_tag => Tag
+
+      affected_post_ids = Set.new
+
+      Tag.transaction do
+        TagVersioning.ensure_snapshot!(target_tag, created_by_user:)
+
+        Array(source_tags).compact.uniq.each do |source_tag|
+          source_tag => Tag
+
+          next if source_tag == target_tag
+
+          TagVersioning.ensure_snapshot!(source_tag, created_by_user:)
+
+          source_tag.post_tags.find_each do |source_pt|
+            post_id = source_pt.post_id
+            affected_post_ids << post_id
+            source_pt.destroy!
+            unless PostTag.exists?(post_id:, tag: target_tag)
+              PostTag.create!(post_id:, tag: target_tag)
+            end
+          end
+
+          if source_tag.tag_descriptions.exists?
+            source_tag.errors.add(:base, :invalid)
+            raise ActiveRecord::RecordInvalid, source_tag
+          end
+
+          TagVersioning.record!(source_tag, event_type: :discard, created_by_user:)
+          target_languages = target_tag.tag_names.where(primary_flg: true).pluck(:language_code)
+          source_tag.tag_names.where(language_code: target_languages)
+              .update_all(primary_flg: false, auto_generated: false, updated_at: Time.current)
+          source_tag.tag_names.update_all(tag_id: target_tag.id, updated_at: Time.current)
+          source_tag.destroy!
+
+          TagVersioning.record!(target_tag, event_type: :update, created_by_user:)
+        end
+
+        Post.where(id: affected_post_ids.to_a).find_each do |post|
+          PostVersionRecorder.ensure_snapshot!(post, created_by_user:)
+          PostVersionRecorder.record!(post:, event_type: :update, created_by_user:)
+        end
+
+        # 投稿件数を再集計
+        target_tag.update_columns(post_count: PostTag.where(tag: target_tag).count)
+      end
+
+      target_tag.reload
+    end
+
+    def time_to_ms! str, tag_name:
+      match =
+          case str
+          when /\A(?<seconds>\d+)(?:\.(?<ms>\d{1,3}))?\z/
+            { hours: nil, minutes: nil, seconds: Regexp.last_match[:seconds],
+              ms: Regexp.last_match[:ms] }
+          when /\A(?<minutes>\d+):(?<seconds>[0-5]?\d)(?:\.(?<ms>\d{1,3}))?\z/
+            { hours: nil, minutes: Regexp.last_match[:minutes],
+              seconds: Regexp.last_match[:seconds],
+              ms: Regexp.last_match[:ms] }
+          when /\A(?<hours>\d+):(?<minutes>[0-5]?\d):(?<seconds>[0-5]?\d)(?:\.(?<ms>\d{1,3}))?\z/
+            { hours: Regexp.last_match[:hours],
+              minutes: Regexp.last_match[:minutes],
+              seconds: Regexp.last_match[:seconds],
+              ms: Regexp.last_match[:ms] }
+          end
+
+      raise SectionLiteralParseError.new(tag_name, str) unless match
+
+      total_s = match[:seconds].to_i
+      total_s += match[:minutes].to_i * 60 if match[:minutes]
+      total_s += match[:hours].to_i * 3_600 if match[:hours]
+
+      total_s * 1_000 + match[:ms].to_s.ljust(3, '0')[0, 3].to_i
+    end
+
   end
 
   private
 
-  def nico_tag_name_must_start_with_nico
+  def tag_names_must_be_valid
-    n = name.to_s
+    tag_names.each do |tag_name|
-    if ((nico? && !(n.downcase.start_with?('nico:'))) || 
+      next if tag_name.valid?
-        (!(nico?) && n.downcase.start_with?('nico:')))
-      errors.add :name, 'ニコニコ・タグの命名規則に反してゐます．'
-    end
-  end
 
-  def tag_name_must_be_canonical
+      tag_name.errors[:name].each { errors.add :name, _1 }
-    if tag_name&.canonical_id?
-      errors.add :tag_name, 'tag_names へは実体を示す必要があります．'
     end
   end
 
@@ -328,36 +379,4 @@ class Tag < ApplicationRecord
       errors.add :category, 'ニジラーと紐づいてゐるタグはニジラー・カテゴリである必要があります．'
     end
   end
-
-  def self.time_to_ms! str, tag_name:
-    match =
-        case str
-        when /\A(?<seconds>\d+)(?:\.(?<ms>\d{1,3}))?\z/
-          { hours: nil, minutes: nil, seconds: Regexp.last_match[:seconds],
-            ms: Regexp.last_match[:ms] }
-        when /\A(?<minutes>\d+):(?<seconds>[0-5]?\d)(?:\.(?<ms>\d{1,3}))?\z/
-          { hours: nil, minutes: Regexp.last_match[:minutes],
-            seconds: Regexp.last_match[:seconds],
-            ms: Regexp.last_match[:ms] }
-        when /\A(?<hours>\d+):(?<minutes>[0-5]?\d):(?<seconds>[0-5]?\d)(?:\.(?<ms>\d{1,3}))?\z/
-          { hours: Regexp.last_match[:hours],
-            minutes: Regexp.last_match[:minutes],
-            seconds: Regexp.last_match[:seconds],
-            ms: Regexp.last_match[:ms] }
-        end
-
-    raise SectionLiteralParseError.new(tag_name, str) unless match
-
-    total_s = match[:seconds].to_i
-    total_s += match[:minutes].to_i * 60 if match[:minutes]
-    total_s += match[:hours].to_i * 3_600 if match[:hours]
-
-    total_s * 1_000 + match[:ms].to_s.ljust(3, '0')[0, 3].to_i
-  end
-
-  def nico_tags_cannot_be_deprecated
-    if nico? && deprecated_at.present?
-      errors.add :deprecated_at, 'ニコタグは廃止できません．'
-    end
-  end
 end

backend/app/models/tag_description.rb

@@ -0,0 +1,5 @@
+class TagDescription < ApplicationRecord
+  before_create { SharedResourceIdAllocator.assign!(self) }
+
+  belongs_to :tag
+end

backend/app/models/tag_description_version.rb

@@ -0,0 +1,5 @@
+class TagDescriptionVersion < ApplicationRecord
+  include VersionRecord
+
+  before_create { SharedResourceIdAllocator.assign!(self) }
+end

backend/app/models/tag_group.rb

@@ -0,0 +1,27 @@
+class TagGroup < ApplicationRecord
+  before_create { SharedResourceIdAllocator.assign!(self) }
+  before_destroy :delete_current_structure!
+
+  belongs_to :created_by_user, class_name: 'User', optional: true
+  belongs_to :updated_by_user, class_name: 'User', optional: true
+
+  has_many :tag_group_tags, dependent: :delete_all
+  has_many :tags, through: :tag_group_tags
+  has_many :tag_group_groups, dependent: :delete_all
+  has_many :child_tag_groups, through: :tag_group_groups
+  has_many :placements, -> { order(:position) },
+           class_name: 'TagGroupPlacement', dependent: :delete_all
+  has_many :versions, -> { order(:version_no) },
+           class_name: 'TagGroupVersion'
+
+  validates :name, presence: true, uniqueness: true, length: { maximum: 255 }
+  validates :version_no, numericality: { only_integer: true, greater_than: 0 }
+
+  private
+
+  def delete_current_structure!
+    placements.reorder(position: :desc).each(&:delete)
+    tag_group_tags.delete_all
+    tag_group_groups.delete_all
+  end
+end

backend/app/models/tag_group_group.rb

@@ -0,0 +1,7 @@
+class TagGroupGroup < ApplicationRecord
+  self.primary_key = [:tag_group_id, :child_tag_group_id]
+
+  belongs_to :tag_group
+  belongs_to :child_tag_group, class_name: 'TagGroup'
+  belongs_to :created_by_user, class_name: 'User', optional: true
+end

backend/app/models/tag_group_placement.rb

@@ -0,0 +1,9 @@
+class TagGroupPlacement < ApplicationRecord
+  self.primary_key = [:tag_group_id, :position]
+
+  belongs_to :tag_group
+  belongs_to :tag, optional: true
+  belongs_to :child_tag_group, class_name: 'TagGroup', optional: true
+  belongs_to :created_by_user, class_name: 'User', optional: true
+  belongs_to :updated_by_user, class_name: 'User', optional: true
+end

backend/app/models/tag_group_tag.rb

@@ -0,0 +1,7 @@
+class TagGroupTag < ApplicationRecord
+  self.primary_key = [:tag_group_id, :tag_id]
+
+  belongs_to :tag_group
+  belongs_to :tag
+  belongs_to :created_by_user, class_name: 'User', optional: true
+end

backend/app/models/tag_group_version.rb

@@ -0,0 +1,9 @@
+class TagGroupVersion < ApplicationRecord
+  include VersionRecord
+
+  before_create { SharedResourceIdAllocator.assign!(self) }
+
+  belongs_to :tag_group
+
+  scope :ordered, -> { order(:version_no) }
+end

backend/app/models/tag_name.rb

@@ -1,51 +1,82 @@
 class TagName < ApplicationRecord
-  include MyDiscard
+  belongs_to :language, foreign_key: :language_code, primary_key: :code
+  belongs_to :tag
 
-  has_one :tag
+  validates :name, presence: true,
-  has_one :wiki_page
+                   length: { maximum: 255 },
+                   uniqueness: { scope: :language_code }
 
-  belongs_to :canonical, class_name: 'TagName', optional: true
-  has_many :aliases, class_name: 'TagName', foreign_key: :canonical_id
-
-  validates :name, presence: true, length: { maximum: 255 }, uniqueness: true
-
-  validate :canonical_must_be_canonical
   validate :alias_name_must_not_have_prefix
-  validate :canonical_must_not_be_present_with_tag_or_wiki_page
   validate :name_must_be_sanitised
+  validate :name_mustnt_start_with_nico
+  validate :generated_name_must_be_reserved
 
-  def self.canonicalise names
+  def primary? = primary_flg
+
+  def self.canonicalise locale, names
     names = Array(names).map { |n| n.to_s.strip }.reject(&:blank?)
     return [] if names.blank?
 
-    tns = TagName.includes(:canonical).where(name: names).index_by(&:name)
+    tns = TagName.where(language_code: locale.language_code, name: names).index_by(&:name)
 
-    names.map { |name| tns[name]&.canonical&.name || name }.uniq
+    primary_names = TagName.where(
+        language_code: locale.language_code, tag_id: tns.values.map(&:tag_id),
+        primary_flg: true).pluck(:tag_id, :name).to_h
+
+    names.map { |name|
+      tag_name = tns[name]
+      if !tag_name || tag_name.primary?
+        name
+      else
+        primary_names[tag_name.tag_id] || name
+      end
+    }.uniq
+  end
+
+  def self.generate_name locale, tag, base_tag_name
+    fallback = "Tag_##{ tag.id }"
+    name =
+        case [locale.language_code, base_tag_name&.language_code]
+        when ['ain', 'ja']
+          TagNameTranscription.ainu(base_tag_name.name)
+        when ['ja', 'ain']
+          base_tag_name.name
+        else
+          return fallback
+        end
+    candidate = new(tag:, name:, language_code: locale.language_code,
+                    script_code: locale.script_code,
+                    primary_flg: true, auto_generated: true)
+    candidate.valid? ? name : fallback
+  rescue TagNameTranscription::Unconvertible
+    fallback
   end
 
   private
 
-  def canonical_must_be_canonical
+  def generated_name_must_be_reserved
-    if canonical&.canonical_id?
+    if !auto_generated && name.to_s.start_with?('Tag_#')
-      errors.add :canonical, 'canonical は実体を示す必要があります．'
+      errors.add :name, 'タグの命名規則に反してゐます．'
     end
   end
 
   def alias_name_must_not_have_prefix
-    if canonical_id? && name.to_s.include?(':')
+    if !(primary?) && name.to_s.include?(':')
       errors.add :name, 'エーリアス名にプレフィクスを含むことはできません．'
     end
   end
 
-  def canonical_must_not_be_present_with_tag_or_wiki_page
-    if canonical_id? && (tag || wiki_page)
-      errors.add :canonical, 'タグもしくは Wiki の参照がある名前はエーリアスになれません．'
-    end
-  end
-
   def name_must_be_sanitised
+    return if auto_generated && name == "Tag_##{ tag_id }"
+
     if name? && name != TagNameSanitisationRule.sanitise(name)
       errors.add :name, '名前に使用できない文字が含まれてゐます．'
     end
   end
+
+  def name_mustnt_start_with_nico
+    if name.to_s.downcase.start_with?('nico:')
+      errors.add :name, 'タグの命名規則に反してゐます．'
+    end
+  end
 end

backend/app/models/tag_name_sanitisation_rule.rb

@@ -10,35 +10,43 @@ class TagNameSanitisationRule < ApplicationRecord
   validate :source_pattern_must_be_regexp
 
   class << self
-    def sanitise(name) =
+    def sanitise(name)
-        rules.reduce(name.dup) { |name, (pattern, replacement)| name.gsub(pattern, replacement) }
+      rules.reduce(name.dup) { |name, (pattern, replacement)| name.gsub(pattern, replacement) }
+    end
 
     def apply!
-      TagName.find_each do |tn|
+      Language.find_each do |language|
-        name = sanitise(tn.name)
+        TagName.where(language:).find_each do |candidate|
-        next if name == tn.name
+          TagName.transaction do
+            tn = TagName.find_by(id: candidate.id)
+            next unless tn
 
-        TagName.transaction do
+            name = sanitise(tn.name)
-          existing_tn = TagName.find_by(name:)
+            next if name == tn.name
-          if existing_tn
-            existing_tn = existing_tn.canonical || existing_tn
-            next if existing_tn.id == tn.id
 
-            existing_tag = Tag.find_by(tag_name_id: existing_tn.id)
+            existing_tn = TagName.find_by(language:, name:)
-            source_tag = Tag.find_by(tag_name_id: tn.id)
+            source_tag = tn.tag
+            target_tag = existing_tn ? existing_tn.tag : source_tag
+            [source_tag, target_tag].uniq(&:id).sort_by(&:id).each(&:lock!)
+            TagVersioning.ensure_snapshot!(source_tag, created_by_user: nil)
 
-            if existing_tag
+            if existing_tn && target_tag != source_tag
-              Tag.merge_tags!(existing_tag, source_tag) if tn.tag
+              Tag.merge_tags!(target_tag, source_tag)
-            elsif source_tag
+              tn.reload.destroy!
-              source_tag.update_columns(tag_name_id: existing_tn.id, updated_at: Time.current)
+            elsif existing_tn
+              if tn.primary?
+                existing_tn.destroy!
+                tn.update_columns(name:, updated_at: Time.current)
+              else
+                tn.destroy!
+              end
+            else
+              # TagName 側の自動サニタイズを回避
+              tn.update_columns(name:, updated_at: Time.current)
             end
-            tn.discard!
 
-            next
+            TagVersioning.record!(target_tag, event_type: :update, created_by_user: nil)
           end
-
-          # TagName 側の自動サニタイズを回避
-          tn.update_columns(name:, updated_at: Time.current)
         end
       end
     end

backend/app/models/tag_version.rb

@@ -11,5 +11,6 @@ class TagVersion < ApplicationRecord
                     meta: 'meta' }, validate: true
 
   validates :name, presence: true
+  validates :names, presence: true
   validates :category, presence: true
 end

backend/app/models/user.rb

@@ -13,10 +13,10 @@ class User < ApplicationRecord
   has_many :ip_addresses, through: :user_ips
   has_many :user_post_views, dependent: :destroy
   has_many :viewed_posts, through: :user_post_views, source: :post
-  has_many :created_wiki_pages,
+  has_many :created_tag_descriptions, class_name: 'TagDescription',
-           class_name: 'WikiPage', foreign_key: :created_user_id, dependent: :nullify
+           foreign_key: :created_by_user_id, dependent: :nullify
-  has_many :updated_wiki_pages,
+  has_many :updated_tag_descriptions, class_name: 'TagDescription',
-           class_name: 'WikiPage', foreign_key: :updated_user_id, dependent: :nullify
+           foreign_key: :updated_by_user_id, dependent: :nullify
 
   def viewed?(post) = user_post_views.exists?(post_id: post.id)
 

backend/app/models/version_record.rb

@@ -6,6 +6,8 @@ module VersionRecord
                              discard: 'discard',
                              restore: 'restore' }.freeze
 
+  def require_history_access! = HistoryAccess.require!(self)
+
   def readonly? = persisted?
 
   included do

backend/app/models/wiki_line.rb

@@ -1,15 +0,0 @@
-class WikiLine < ApplicationRecord
-  has_many :wiki_revision_lines, dependent: :restrict_with_exception
-
-  validates :sha256, presence: true, uniqueness: true, length: { is: 64 }
-  validates :body, presence: true
-
-  def self.upsert_by_body! body
-    sha = Digest::SHA256.hexdigest(body)
-    now = Time.current
-
-    upsert(sha256: sha, body:, created_at: now, updated_at: now)
-
-    find_by!(sha256: sha)
-  end
-end

backend/app/models/wiki_page.rb

@@ -1,55 +0,0 @@
-require 'set'
-
-
-class WikiPage < ApplicationRecord
-  include MyDiscard
-
-  has_many :wiki_revisions, dependent: :destroy
-  belongs_to :created_user, class_name: 'User'
-  belongs_to :updated_user, class_name: 'User'
-
-  has_many :redirected_from_revisions,
-           class_name: 'WikiRevision',
-           foreign_key: :redirect_page_id,
-           dependent: :nullify
-
-  has_many :wiki_versions
-
-  attribute :version_no, :integer, default: 1
-
-  belongs_to :tag_name
-  validates :tag_name, presence: true
-  validates :body, presence: true
-
-  def title = tag_name.name
-  def deprecated_at = tag_name.tag&.deprecated_at
-
-  def title= val
-    (self.tag_name ||= build_tag_name).name = val
-  end
-
-  def current_revision = wiki_revisions.order(id: :desc).first
-
-  def resolve_redirect limit: 10
-    page = self
-    visited = Set.new
-
-    limit.times do
-      return page if visited.include?(page.id)
-
-      visited.add(page.id)
-
-      rev = page.current_revision
-      return page if !(rev&.redirect?) || !(rev.redirect_page)
-
-      page = rev.redirect_page
-    end
-
-    page
-  end
-
-  def pred_revision_id(revision_id) =
-      wiki_revisions.where('id < ?', revision_id).order(id: :desc).limit(1).pick(:id)
-  def succ_revision_id(revision_id) =
-      wiki_revisions.where('id > ?', revision_id).order(id: :asc).limit(1).pick(:id)
-end

backend/app/models/wiki_revision.rb

@@ -1,55 +0,0 @@
-class WikiRevision < ApplicationRecord
-  belongs_to :wiki_page
-  belongs_to :base_revision, class_name: 'WikiRevision', optional: true
-  belongs_to :created_user, class_name: 'User'
-  belongs_to :redirect_page, class_name: 'WikiPage', optional: true
-
-  has_many :wiki_revision_lines, dependent: :delete_all
-  has_many :wiki_lines, through: :wiki_revision_lines
-
-  enum :kind, content: 0, redirect: 1
-
-  validates :kind, presence: true
-  validates :lines_count, numericality: { only_integer: true, greater_than_or_equal_to: 0 }
-  validates :tree_sha256, length: { is: 64 }, allow_nil: true
-
-  validate :kind_consistency
-
-  def body
-    return unless content?
-
-    wiki_revision_lines
-        .includes(:wiki_line)
-        .order(:position)
-        .map { |rev| rev.wiki_line.body }
-        .join("\n")
-  end
-
-  private
-
-  def kind_consistency
-    if content?
-      if tree_sha256.blank?
-        errors.add(:tree_sha256, '種類がページの場合は必須です．') 
-      end
-
-      if redirect_page_id.present?
-        errors.add(:redirect_page_id, '種類がページの場合は空である必要があります．')
-      end
-    end
-
-    if redirect?
-      if redirect_page_id.blank?
-        errors.add(:redirect_page_id, '種類がリダイレクトの場合は必須です．')
-      end
-
-      if tree_sha256.present?
-        errors.add(:tree_sha256, '種類がリダイレクトの場合は空である必要があります．')
-      end
-
-      if lines_count.to_i > 0
-        errors.add(:lines_count, '種類がリダイレクトの場合は 0 である必要があります．')
-      end
-    end
-  end
-end

backend/app/models/wiki_revision_line.rb

@@ -1,8 +0,0 @@
-class WikiRevisionLine < ApplicationRecord
-  belongs_to :wiki_revision
-  belongs_to :wiki_line
-
-  validates :position, presence: true,
-                       numericality: { only_integer: true, greater_than_or_equal_to: 0 }
-  validates :position, uniqueness: { scope: :wiki_revision_id }
-end

backend/app/models/wiki_version.rb

@@ -1,8 +0,0 @@
-class WikiVersion < ApplicationRecord
-  include VersionRecord
-
-  belongs_to :wiki_page
-
-  validates :title, presence: true
-  validates :body, presence: true
-end

backend/app/representations/counts_version_repr.rb

@@ -0,0 +1,135 @@
+# Counts uses recorded values only; history/diff endpoints retain their own contract.
+class CountsVersionRepr
+  def self.many(rows, resource:, language_code:, at: nil, inclusive: true)
+    new(rows.to_a, resource:, language_code:, at:, inclusive:).serialise
+  end
+
+  def initialize(rows, resource:, language_code:, at:, inclusive:)
+    @rows = rows
+    HistoryAccess.require_many!(rows)
+    @at = at
+    @inclusive = inclusive
+    @resource = resource
+    @language = language_code
+    @users = User.where(id: rows.map(&:created_by_user_id).compact).pluck(:id, :name).to_h
+    @languages = master_names(Language)
+    @scripts = master_names(Script)
+    @tag_versions = {}
+    load_tag_versions if resource == 'posts'
+    load_parent_versions if resource == 'tags'
+  end
+
+  def serialise
+    @rows.map do |row|
+      resource_id = row.public_send(@resource == 'posts' ? :post_id : :tag_id)
+      common = { id: row.id, resource_id:,
+                 version_no: row.version_no, is_latest: row.attributes['counts_latest'].to_i == 1,
+                 created_at: row.created_at.iso8601,
+                 created_by_user: row.created_by_user_id &&
+                     { id: row.created_by_user_id, name: @users[row.created_by_user_id] } }
+      common.merge(@resource == 'posts' ? post(row) : tag(row))
+    end
+  end
+
+  private
+
+  def master_names(model)
+    model.select('code AS counts_code', :name).to_h do |record|
+      [record.attributes['counts_code'], record.name]
+    end
+  end
+
+  def json(value)
+    value.is_a?(String) ? JSON.parse(value) : value
+  end
+
+  def load_tag_versions
+    pairs = @rows.flat_map { Array(json(_1.tags_json)) }
+        .filter_map { |entry| [entry['tag_id'], entry['version_no']] if entry['tag_id'] }
+        .uniq
+    pairs.each_slice(500) do |batch|
+      placeholders = batch.map { '(?, ?)' }.join(', ')
+      scope = HistoryAccess.visible(TagVersion.all)
+      if @at
+        scope = scope.where("created_at #{ @inclusive ? '<=' : '<' } ?", @at)
+      end
+      scope.where(["(tag_id, version_no) IN (#{ placeholders })", *batch.flatten])
+          .select(:tag_id, :version_no, :name, :category, :names).each do |version|
+        @tag_versions[[version.tag_id, version.version_no]] = version
+      end
+    end
+  end
+
+  def names(value)
+    (json(value) || {}).sort.map do |language, entry|
+      aliases = entry.fetch('aliases', []).map do |alias_entry|
+        alias_entry.merge('script_name' => @scripts[alias_entry['script_code']])
+      end
+      { language_code: language, language_name: @languages[language],
+        script_code: entry['script_code'], script_name: @scripts[entry['script_code']],
+        name: entry['name'], auto_generated: entry['auto_generated'], aliases: }
+    end
+  end
+
+  def post(row)
+    snapshots = Array(json(row.tags_json)).map do |entry|
+      version = @tag_versions[[entry['tag_id'], entry['version_no']]]
+      { id: entry['tag_id'] || entry['external_tag_id'],
+        kind: entry['tag_id'] ? 'internal' : 'external',
+        tag_id: entry['tag_id'], external_tag_id: entry['external_tag_id'],
+        version_no: entry['version_no'], name: entry['name'] || version&.name,
+        category: entry['category'] || version&.category,
+        sections: entry.fetch('sections', []), names: version ? names(version.names) : [] }
+    end
+    { title: row.title, url: row.url, thumbnail_base: row.thumbnail_base,
+      tags: snapshots, parents: row.parent_post_ids.to_s.split.map { { id: _1.to_i } },
+      video_ms: row.video_ms,
+      original_created_from: row.original_created_from&.iso8601,
+      original_created_before: row.original_created_before&.iso8601 }
+  end
+
+  def parent_point(row) = @at || row.created_at
+
+  def load_parent_versions
+    @parent_versions = {}
+    requests = @rows.flat_map do |row|
+      row.parent_tag_ids.to_s.split.map { [_1.to_i, parent_point(row)] }
+    end.uniq
+    requests.each_slice(100) do |batch|
+      queries = batch.map do |id, point|
+        scope = HistoryAccess.visible(TagVersion.where(tag_id: id))
+            .where('created_at <= ?', point)
+            .where('NOT EXISTS (SELECT 1 FROM tag_versions newer ' +
+                'WHERE newer.tag_id = tag_versions.tag_id AND newer.created_at <= ? ' +
+                'AND newer.version_no > tag_versions.version_no)', point)
+        quoted = TagVersion.connection.quote(point.iso8601(6))
+        scope.select("tag_id, name, names, #{ quoted } AS reference_point").to_sql
+      end
+      next if queries.empty?
+
+      TagVersion.connection.select_all(queries.map { "(#{ _1 })" }.join(' UNION ALL ')).each do |value|
+        @parent_versions[[value['tag_id'], value['reference_point']]] = value
+      end
+    end
+  end
+
+  def tag(row)
+    recorded_names = names(row.names)
+    selected = recorded_names.find { _1[:language_code] == @language }
+    aliases = recorded_names.flat_map do |entry|
+      entry[:aliases].map {
+        _1.merge('language_code' => entry[:language_code],
+                 'language_name' => entry[:language_name])
+      }
+    end
+    { name: selected&.fetch(:name) || row.name, names: recorded_names,
+      category: row.category, aliases:,
+      parents: row.parent_tag_ids.to_s.split.map do |id|
+        value = @parent_versions[[id.to_i, parent_point(row).iso8601(6)]]
+        snapshots = value && json(value['names'])
+        name = snapshots&.dig(@language, 'name') || value&.fetch('name')
+        { id: id.to_i, name: }
+      end,
+      deprecated: row.deprecated_at&.iso8601 }
+  end
+end

backend/app/representations/deerjikist_repr.rb

@@ -6,11 +6,12 @@ module DeerjikistRepr
 
   module_function
 
-  def base deerjikist
+  def base deerjikist, language_code: 'ja'
-    deerjikist.as_json(BASE)
+    deerjikist.as_json(BASE).merge(
+        tag: deerjikist.tag && TagRepr.base(deerjikist.tag, language_code:))
   end
 
-  def many deerjikists
+  def many deerjikists, language_code: 'ja'
-    deerjikists.map { |d| base(d) }
+    deerjikists.map { |d| base(d, language_code:) }
   end
 end

backend/app/representations/external_tag_repr.rb

@@ -0,0 +1,20 @@
+module ExternalTagRepr
+  module_function
+
+  def base tag
+    { 'id' => tag.id,
+      'name' => "#{ tag.platform }:#{ tag.name }",
+      'category' => 'nico',
+      'post_count' => tag.post_count,
+      'created_at' => tag.created_at,
+      'updated_at' => tag.created_at,
+      'deprecated_at' => nil,
+      'aliases' => [],
+      'parents' => [],
+      'has_wiki' => false,
+      'material_id' => nil,
+      'has_deerjikists' => false }
+  end
+
+  def inline(tag) = base(tag)
+end

backend/app/representations/material_repr.rb

@@ -12,14 +12,15 @@ module MaterialRepr
 
   module_function
 
-  def base material, host:
+  def base material, host:, language_code: 'ja'
     material.as_json(BASE).merge(
+        tag: material.tag && TagRepr.base(material.tag, language_code:),
         file: if material.file.attached?
                 Rails.application.routes.url_helpers.rails_storage_proxy_url(
                     material.file, host:)
               end,
         thumbnail: thumbnail_url(material, host:),
-        thumbnail_fallback_text: thumbnail_fallback_text(material),
+        thumbnail_fallback_text: thumbnail_fallback_text(material, language_code:),
         thumbnail_fallback_kind: thumbnail_fallback_kind(material),
         media_kind: media_kind(material),
         file_byte_size: material.file_byte_size,
@@ -27,11 +28,11 @@ module MaterialRepr
         export_items: export_items(material))
   end
 
-  def many materials, host:
+  def many materials, host:, language_code: 'ja'
-    materials.map { |m| base(m, host:) }
+    materials.map { |m| base(m, host:, language_code:) }
   end
 
-  def list material, host:
+  def list material, host:, language_code: 'ja'
     { id: material.id,
       version_no: material.version_no,
       url: material.url,
@@ -40,9 +41,9 @@ module MaterialRepr
       source_path: material.source_path,
       source_file_id: material.source_file_id,
       normalized_source_key: material.normalized_source_key,
-      tag: compact_tag(material.tag),
+      tag: compact_tag(material.tag, language_code:),
       thumbnail: thumbnail_url(material, host:),
-      thumbnail_fallback_text: thumbnail_fallback_text(material),
+      thumbnail_fallback_text: thumbnail_fallback_text(material, language_code:),
       thumbnail_fallback_kind: thumbnail_fallback_kind(material),
       media_kind: media_kind(material),
       content_type: material.content_type,
@@ -53,8 +54,8 @@ module MaterialRepr
       export_items: export_items(material) }
   end
 
-  def list_many materials, host:
+  def list_many materials, host:, language_code: 'ja'
-    materials.map { |m| list(m, host:) }
+    materials.map { |m| list(m, host:, language_code:) }
   end
 
   def export_paths material
@@ -79,8 +80,8 @@ module MaterialRepr
         material.thumbnail, host:)
   end
 
-  def thumbnail_fallback_text material
+  def thumbnail_fallback_text material, language_code:
-    material.tag&.name || material.created_at&.strftime('%Y-%m-%d')
+    material.tag&.name(language_code) || material.created_at&.strftime('%Y-%m-%d')
   end
 
   def thumbnail_fallback_kind material
@@ -98,11 +99,11 @@ module MaterialRepr
     'file_other'
   end
 
-  def compact_tag tag
+  def compact_tag tag, language_code:
     return nil unless tag
 
     { id: tag.id,
-      name: tag.name,
+      name: tag.name(language_code),
       category: tag.category,
       deprecated_at: tag.deprecated_at }
   end

backend/app/representations/post_compact_repr.rb

@@ -0,0 +1,19 @@
+# frozen_string_literal: true
+
+
+module PostCompactRepr
+  module_function
+
+  def base post, host: nil
+    return nil if post.nil?
+
+    PostRepr
+        .common(post, host:)
+        .slice(
+            'id',
+            'title',
+            'url',
+            'thumbnail',
+            'thumbnail_base')
+  end
+end

backend/app/representations/post_repr.rb

@@ -17,59 +17,112 @@ module PostRepr
 
   module_function
 
-  def base post, current_user = nil
+  def base post, current_user = nil, host: nil, language_code: 'ja'
-    json = common(post)
+    json =
-    json['tags'] = tag_json(post)
+        if host.present?
+          common(post, host:)
+        else
+          common(post)
+        end
+    json['tags'] = tag_json(post, language_code:)
     json['uploaded_user'] = post.uploaded_user && UserRepr.base(post.uploaded_user)
     json['viewed'] = current_user ? current_user.viewed?(post) : false
     json
   end
 
   def detail post, current_user = nil, parent_posts: [], child_posts: [],
-             sibling_posts: { }, related: []
+             sibling_posts: { }, related: [], host: nil, language_code: 'ja'
-    base(post, current_user).merge(
+    if host.present?
-        'parent_posts' => cards(parent_posts),
+      base(post, current_user, host:, language_code:).merge(
-        'child_posts' => cards(child_posts),
+          'parent_posts' => cards(parent_posts, host:),
-        'sibling_posts' => sibling_posts.transform_keys(&:to_s).transform_values { |posts|
+          'child_posts' => cards(child_posts, host:),
-          cards(posts)
+          'sibling_posts' => sibling_posts.transform_keys(&:to_s).transform_values { |posts|
-        },
+            cards(posts, host:)
-        'related' => cards(related))
+          },
+          'related' => cards(related, host:))
+    else
+      base(post, current_user, language_code:).merge(
+          'parent_posts' => cards(parent_posts),
+          'child_posts' => cards(child_posts),
+          'sibling_posts' => sibling_posts.transform_keys(&:to_s).transform_values { |posts|
+            cards(posts)
+          },
+          'related' => cards(related))
+    end
   end
 
-  def card post
+  def card post, host: nil
-    common(post).merge('parent_posts' => [], 'child_posts' => [])
+    if host.present?
+      common(post, host:).merge('parent_posts' => [], 'child_posts' => [])
+    else
+      common(post).merge('parent_posts' => [], 'child_posts' => [])
+    end
   end
 
-  def cards posts
+  def cards posts, host: nil
-    posts.map { |post| card(post) }
+    if host.present?
+      posts.map { |post| card(post, host:) }
+    else
+      posts.map { |post| card(post) }
+    end
   end
 
-  def many posts, current_user = nil
+  def many posts, current_user = nil, host: nil
-    posts.map { |p| base(p, current_user) }
+    if host.present?
+      posts.map { |p| base(p, current_user, host:) }
+    else
+      posts.map { |p| base(p, current_user) }
+    end
   end
 
-  def common post
+  def common post, host: nil
     BASE_FIELDS.to_h { |field| [field.to_s, post.public_send(field)] }
-               .merge('thumbnail' => thumbnail_url(post))
+               .merge(
+                   'thumbnail' =>
+                       if host.present?
+                         thumbnail_url(post, host:)
+                       else
+                         thumbnail_url(post)
+                       end)
   end
 
-  def tag_json post
+  def tag_json post, language_code:
-    post
+    internal_tags =
-        .active_post_tags
+        post
+        .post_tags
         .reject { _1.tag.deprecated? }
-        .sort_by { _1.tag.name }
+        .sort_by { TagRepr.name_for(_1.tag, language_code) }
         .map { |post_tag|
-      TagRepr.inline(post_tag.tag).merge(
+          TagRepr.inline(post_tag.tag, language_code:).merge(
-          'children' => [],
+              'children' => [],
-          'sections' => post_tag.sections.as_json(only: [:begin_ms, :end_ms]))
+              'sections' => post_tag.sections.as_json(only: [:begin_ms, :end_ms]))
-    }
+        }
+
+    external_tags =
+        post
+        .external_tags
+        .sort_by { _1.name }
+        .map { ExternalTagRepr.base(_1).merge('children' => [], 'sections' => []) }
+
+    internal_tags + external_tags
   end
 
-  def thumbnail_url post
+  def thumbnail_url post, host: nil
     return nil unless post.thumbnail.attached?
 
-    Rails.application.routes.url_helpers.rails_blob_url(post.thumbnail, only_path: false)
+    options = { only_path: false }
-  rescue
+    options[:host] = host if host.present?
+
+    Rails.application.routes.url_helpers.rails_storage_proxy_url(post.thumbnail, **options)
+  rescue ActionController::UrlGenerationError, ArgumentError, URI::InvalidURIError => e
+    payload = {
+        post_id: post.id,
+        attachment_id: post.thumbnail.attachment&.id,
+        blob_id: post.thumbnail.blob&.id,
+        error_class: e.class,
+        message: e.message }
+
+    Rails.logger.warn("PostRepr.thumbnail_url failed #{ payload.to_json }")
     nil
   end
 end

backend/app/representations/tag_group_repr.rb

@@ -0,0 +1,207 @@
+module TagGroupRepr
+  PREFIX = 'タググループ:'
+
+  module_function
+
+  def name_from_title title
+    prefix = [PREFIX, 'tag_group:'].find { title.downcase.start_with?(_1) }
+    title[prefix.length..] if prefix
+  end
+
+  def many locale, groups
+    ids = groups.map(&:id)
+    return [] if ids.empty?
+
+    tag_members = TagGroupTag.where(tag_group_id: ids)
+        .group_by(&:tag_group_id)
+    group_members = TagGroupGroup.where(tag_group_id: ids)
+        .group_by(&:tag_group_id)
+    placements = TagGroupPlacement.where(tag_group_id: ids).order(:position)
+        .group_by(&:tag_group_id)
+
+    target_ids =
+        tag_members.values.flatten.map(&:tag_id) +
+        placements.values.flatten.filter_map(&:tag_id)
+    child_ids =
+        group_members.values.flatten.map(&:child_tag_group_id) +
+        placements.values.flatten.filter_map(&:child_tag_group_id)
+    tags = Tag.select(:id, :category).where(id: target_ids.uniq).index_by(&:id)
+    names =
+        TagName
+        .where(tag_id: target_ids.uniq,
+               language_code: locale.language_code,
+               primary_flg: true)
+        .pluck(:tag_id, :name)
+        .to_h
+    child_groups = TagGroup.where(id: child_ids.uniq).index_by(&:id)
+
+    groups.map do |group|
+      group_json(group,
+                 tag_members.fetch(group.id, []),
+                 group_members.fetch(group.id, []),
+                 placements.fetch(group.id, []), tags, names, child_groups)
+    end
+  end
+
+  def one locale, group
+    HistoryAccess.require_current!(group, TagGroupVersion)
+    many(locale, [group]).first
+  end
+
+  def tag_group wiki_page
+    { 'id' => wiki_page.id,
+      'title' => "#{ PREFIX }#{ wiki_page.name }",
+      'deprecated_at' => nil,
+      'created_at' => wiki_page.created_at,
+      'updated_at' => wiki_page.updated_at }
+  end
+
+  def current_version group
+    HistoryAccess.visible(TagGroupVersion.all).find_by(tag_group_id: group.id, version_no: group.version_no)
+  end
+
+  def historical locale, version
+    HistoryAccess.require!(version)
+    members = Array(version.members)
+    placements = Array(version.placements)
+    tag_references = members.filter_map { |member| member['tag_id'] } +
+        collect_ids(placements, 'tag_id')
+    group_references = members.filter_map { |member| member['tag_group_id'] } +
+        collect_ids(placements, 'tag_group_id')
+    tag_versions = historical_tag_versions(tag_references)
+    group_versions = historical_group_versions(group_references)
+    tag_version_numbers = members.filter_map do |member|
+      next unless member['tag_id']
+
+      [member.fetch('tag_id'), member.fetch('version_no')]
+    end.to_h
+    group_version_numbers = members.filter_map do |member|
+      next unless member['tag_group_id']
+
+      [member.fetch('tag_group_id'), member.fetch('version_no')]
+    end.to_h
+    historical_members = members.map do |member|
+      historical_member(locale, member, tag_versions, group_versions)
+    end
+    historical_placements = placements.map do |node|
+      historical_placement(locale, node, tag_versions, group_versions,
+                           tag_version_numbers, group_version_numbers)
+    end
+    { 'id' => version.tag_group_id,
+      'name' => version.name,
+      'description' => version.description,
+      'version_no' => version.version_no,
+      'members' => historical_members,
+      'placements' => historical_placements,
+      'created_at' => version.created_at,
+      'updated_at' => version.created_at,
+      'created_by_user' => user_json(version.created_by_user),
+      'updated_by_user' => user_json(version.created_by_user) }
+  end
+
+  def group_json group, tag_members, group_members, rows, tags, names, child_groups
+    tree = placement_tree(rows, tags, names, child_groups)
+    members = tag_members.sort_by(&:tag_id).map do |member|
+      { 'tag' => tag_json(tags.fetch(member.tag_id), names[member.tag_id]) }
+    end
+    members.concat(group_members.sort_by(&:child_tag_group_id).map do |member|
+      { 'tag_group' => group_json_reference(child_groups.fetch(member.child_tag_group_id)) }
+    end)
+    { 'id' => group.id,
+      'name' => group.name,
+      'description' => group.description,
+      'version_no' => group.version_no,
+      'members' => members,
+      'placements' => tree,
+      'created_at' => group.created_at,
+      'updated_at' => group.updated_at,
+      'created_by_user' => user_json(group.created_by_user),
+      'updated_by_user' => user_json(group.updated_by_user) }
+  end
+
+  def user_json user
+    UserRepr.base(user) if user
+  end
+
+  def placement_tree rows, tags, names, child_groups
+    by_parent = rows.group_by(&:parent_position)
+    build = lambda do |parent|
+      by_parent.fetch(parent, []).sort_by(&:position).map do |row|
+        node = if row.tag_id
+          { 'tag' => tag_json(tags.fetch(row.tag_id), names[row.tag_id]) }
+        elsif row.child_tag_group_id
+          { 'tag_group' => group_json_reference(child_groups.fetch(row.child_tag_group_id)) }
+        else
+          { 'section_name' => row.section_name, 'description' => row.description }
+        end
+        node.merge('children' => build.call(row.position))
+      end
+    end
+    build.call(nil)
+  end
+
+  def tag_json tag, name
+    { 'id' => tag.id, 'name' => name, 'category' => tag.category }
+  end
+
+  def group_json_reference group
+    { 'id' => group.id, 'name' => group.name }
+  end
+
+  def collect_ids nodes, key
+    nodes.flat_map do |node|
+      [node[key], *collect_ids(Array(node['children']), key)].compact
+    end
+  end
+
+  def historical_tag_versions ids
+    HistoryAccess.visible(TagVersion.all).where(tag_id: ids.uniq).to_a.index_by do |version|
+      [version.tag_id, version.version_no]
+    end
+  end
+
+  def historical_group_versions ids
+    HistoryAccess.visible(TagGroupVersion.all).where(tag_group_id: ids.uniq).to_a.index_by do |version|
+      [version.tag_group_id, version.version_no]
+    end
+  end
+
+  def historical_member locale, member, tag_versions, group_versions
+    if member['tag_id']
+      id = member.fetch('tag_id')
+      version = tag_versions.fetch([id, member.fetch('version_no')])
+      { 'tag' => { 'id' => id, 'name' => version_name(locale, version),
+                   'category' => version.category } }
+    else
+      id = member.fetch('tag_group_id')
+      version = group_versions.fetch([id, member.fetch('version_no')])
+      { 'tag_group' => { 'id' => id, 'name' => version.name } }
+    end
+  end
+
+  def historical_placement locale, node, tag_versions, group_versions,
+                           tag_version_numbers, group_version_numbers
+    result = if node['tag_id']
+      id = node.fetch('tag_id')
+      version_no = tag_version_numbers.fetch(id)
+      version = tag_versions.fetch([id, version_no])
+      { 'tag' => { 'id' => id, 'name' => version_name(locale, version),
+                   'category' => version.category } }
+    elsif node['tag_group_id']
+      id = node.fetch('tag_group_id')
+      version_no = group_version_numbers.fetch(id)
+      version = group_versions.fetch([id, version_no])
+      { 'tag_group' => { 'id' => id, 'name' => version.name } }
+    else
+      { 'section_name' => node.fetch('section_name'), 'description' => node['description'] }
+    end
+    result.merge('children' => Array(node['children']).map do |child|
+      historical_placement(locale, child, tag_versions, group_versions,
+                           tag_version_numbers, group_version_numbers)
+    end)
+  end
+
+  def version_name locale, version
+    version.names[locale.language_code]&.fetch('name')
+  end
+end

backend/app/representations/tag_repr.rb

@@ -3,18 +3,30 @@
 
 module TagRepr
   BASE = { only: [:id, :category, :post_count, :created_at, :updated_at, :deprecated_at],
-           methods: [:name, :has_wiki, :material_id, :has_deerjikists] }.freeze
+           methods: [:material_id, :has_deerjikists] }.freeze
 
   module_function
 
-  def base tag
+  def base tag, language_code: 'ja'
-    tag.as_json(BASE).merge(aliases: tag.snapshot_aliases,
+    tag.as_json(BASE).merge('has_wiki' => tag.has_description?(language_code:),
-                            parents: tag.parents.map { _1.as_json(BASE) })
+                            name: name_for(tag, language_code),
+                            aliases: tag.snapshot_aliases(language_code),
+                            parents: tag.parents.map { inline(_1, language_code:) })
   end
 
-  def inline tag
+  def inline tag, language_code: 'ja'
-    tag.as_json(BASE).merge(aliases: [], parents: [])
+    tag.as_json(BASE).merge('has_wiki' => tag.has_description?(language_code:),
+                            name: name_for(tag, language_code), aliases: [], parents: [])
   end
 
-  def many(tags) = tags.map { |t| base(t) }
+  def many tags, language_code: 'ja'
+    records = tags.to_a
+    ActiveRecord::Associations::Preloader.new(
+        records:, associations: [:tag_descriptions, { parents: :tag_descriptions }]).call
+    records.map { |tag| base(tag, language_code:) }
+  end
+
+  def name_for tag, language_code
+    tag.name(language_code) || (tag.system? ? tag.name('ja') : nil)
+  end
 end

backend/app/representations/wiki_page_repr.rb

@@ -2,15 +2,49 @@
 
 
 module WikiPageRepr
-  BASE = { methods: [:title, :deprecated_at] }.freeze
-
   module_function
 
-  def base wiki_page
+  def tag_description locale, description
-    wiki_page.as_json(BASE)
+    HistoryAccess.require_current!(description, TagDescriptionVersion)
+    tag = description.tag
+
+    { 'id' => description.id,
+      'resource_type' => 'tag_description',
+      'tag_id' => tag.id,
+      'language_code' => description.language_code,
+      'tag_name_id' => tag.tag_name(locale.language_code)&.id,
+      'body' => description.body,
+      'created_user_id' => description.created_by_user_id,
+      'updated_user_id' => description.updated_by_user_id,
+      'created_at' => description.created_at,
+      'updated_at' => description.updated_at,
+      'discarded_at' => nil,
+      'version_no' => description.version_no,
+      'title' => tag.name(locale.language_code),
+      'deprecated_at' => tag.deprecated_at }
   end
 
-  def many wiki_pages
+  def tag_group tag_group
-    wiki_pages.map { |p| base(p) }
+    TagGroupRepr.tag_group(tag_group).merge('resource_type' => 'tag_group')
+  end
+
+  def autolink_tag locale, tag_id:, description_id:, name:, language:
+    { resource_type: 'tag_description',
+      resource_id: description_id,
+      tag_id:,
+      tag_description_id: description_id,
+      name:,
+      language_code: language,
+      locale_code: locale.code_before_type_cast }
+  end
+
+  def autolink_group locale, id:, name:, language:
+    { resource_type: 'tag_group',
+      resource_id: id,
+      tag_id: nil,
+      tag_description_id: nil,
+      name: "#{ TagGroupRepr::PREFIX }#{ name }",
+      language_code: language,
+      locale_code: locale.code_before_type_cast }
   end
 end

backend/app/services/certified_tag_name_translations.rb

@@ -0,0 +1,105 @@
+require 'json'
+require 'pathname'
+
+
+class CertifiedTagNameTranslations
+  FORMAT = 'btrc-hub.certified-tag-name-translations.v1'.freeze
+
+  class Error < ArgumentError
+    ;
+  end
+
+  def self.load! path, locale:
+    new(path, locale:).load!
+  end
+
+  def initialize path, locale:
+    @path = path
+    @locale = locale
+  end
+
+  def load!
+    document = read_document
+    unless document.is_a?(Hash) && document['format'] == FORMAT
+      fail!('invalid format or version')
+    end
+
+    source_code = document['source_locale']
+    target_code = document['target_locale']
+    unless target_code.is_a?(String) && target_code == Locale.codes.fetch(@locale.code)
+      fail!("target_locale #{ target_code.inspect } does not match the new Locale")
+    end
+    unless source_code.is_a?(String) && Locale.codes.value?(source_code)
+      fail!("invalid source_locale #{ source_code.inspect }")
+    end
+    source_locale = Locale.find_by(code: source_code)
+    fail!("source Locale #{ source_code.inspect } does not exist") unless source_locale
+    fail!('source_locale and target_locale must differ') if source_code == target_code
+
+    entries = validate_entries(document['entries'], source_code, target_code)
+    resolve_tags(entries, source_locale.language_code)
+  end
+
+  private
+
+  def read_document
+    unless (@path.is_a?(String) && !@path.include?("\0") &&
+            Pathname.new(@path).absolute?)
+      fail!('certified_tag_names_path must be an absolute file path')
+    end
+    fail!('JSON file does not exist or is not a regular file') unless File.file?(@path)
+
+    text = File.read(@path, encoding: Encoding::UTF_8)
+    fail!('JSON file is not valid UTF-8') unless text.valid_encoding?
+    JSON.parse(text)
+  rescue JSON::ParserError, SystemCallError, IOError, EncodingError => error
+    fail!("cannot read JSON: #{ error.message }")
+  end
+
+  def validate_entries entries, source_code, target_code
+    fail!('entries must be an array') unless entries.is_a?(Array)
+
+    sources = {}
+    targets = {}
+    entries.each_with_index.map do |entry, index|
+      fail!("entry #{ index + 1 } must be an object") unless entry.is_a?(Hash)
+      source = entry[source_code]
+      target = entry[target_code]
+      unless source.is_a?(String) && source.present? && target.is_a?(String) && target.present?
+        fail!("entry #{ index + 1 }: missing or empty #{ source_code }/#{ target_code } name")
+      end
+      fail!("duplicate source name #{ source.inspect }") if sources.key?(source)
+      fail!("duplicate target name #{ target.inspect }") if targets.key?(target)
+
+      sources[source] = true
+      targets[target] = true
+      [source, target]
+    end
+  end
+
+  def resolve_tags entries, source_language
+    source_names = entries.map(&:first)
+    # SQL may use a case/accent-insensitive collation; the Ruby keys are exact.
+    matches = TagName.where(language_code: source_language, primary_flg: true,
+                            name: source_names).pluck(:name, :tag_id).group_by(&:first)
+    translations = {}
+    entries.each do |source, target|
+      rows = matches.fetch(source, [])
+      next if rows.empty?
+
+      unless rows.length == 1
+        fail!("source primary name #{ source.inspect } must match exactly one tag")
+      end
+      tag_id = rows.first.last
+      if translations.key?(tag_id)
+        fail!("multiple source primary names resolve to Tag##{ tag_id }: #{ source.inspect }")
+      end
+      translations[tag_id] = target
+    end
+    translations
+  end
+
+  def fail! message
+    raise Error, "Certified tag names #{@path.inspect}: #{ message }"
+  end
+end

backend/app/services/counts_axes.rb

@@ -0,0 +1,100 @@
+class CountsAxes
+  GRAINS = ['year', 'quarter', 'month', 'week', 'day', 'hour'].freeze
+
+  def self.time_expression(expression, grain, timezone)
+    raise ArgumentError unless GRAINS.include?(grain)
+
+    connection = ApplicationRecord.connection
+    # Use the installed TZInfo database, including DST, rather than MySQL timezone tables.
+    first = Time.utc(1900)
+    last = Time.utc(2101)
+    offset = timezone.period_for_utc(first).utc_total_offset
+    clauses = timezone.transitions_up_to(last, first).filter_map do |transition|
+      next if transition.offset.utc_total_offset == offset
+
+      previous = offset
+      offset = transition.offset.utc_total_offset
+      "WHEN #{ expression } < #{ connection.quote(transition.at.to_time) } THEN #{ previous }"
+    end
+    seconds = clauses.empty? ? offset.to_s : "CASE #{ clauses.join(' ') } ELSE #{ offset } END"
+    local = "DATE_ADD(#{ expression }, INTERVAL (#{ seconds }) SECOND)"
+    case grain
+    when 'year' then "DATE_FORMAT(#{ local }, '%Y')"
+    when 'quarter' then "CONCAT(YEAR(#{ local }), '-Q', QUARTER(#{ local }))"
+    when 'month' then "DATE_FORMAT(#{ local }, '%Y-%m')"
+    when 'week' then "DATE_FORMAT(#{ local }, '%x-W%v')"
+    when 'day' then "DATE_FORMAT(#{ local }, '%Y-%m-%d')"
+    else "DATE_FORMAT(#{ local }, '%Y-%m-%d %H:00')"
+    end
+  end
+
+  attr_reader :rows, :columns, :all
+
+  def initialize(population)
+    @population = population
+    @definition = population.definition
+    @rows = parse(population.params.fetch('rows', ''))
+    @columns = parse(population.params.fetch('cols', ''))
+    @all = rows + columns
+    raise ArgumentError if all.length > 8 || all.map { _1['field'] }.uniq.length != all.length
+  end
+
+  def expressions
+    @expressions ||= all.map do |axis|
+      field = axis.fetch('field')
+      case field
+      when 'tags'
+        "CASE WHEN tag_entry.tag_id IS NOT NULL " +
+            "THEN CONCAT('internal:', tag_entry.tag_id) " +
+            "ELSE CONCAT('external:', tag_entry.external_tag_id) END"
+      when 'changed_field' then 'changed.field_key'
+      else
+        expression = @definition.expression(field)
+        @definition.fields[field] == :datetime ? self.class.time_expression(
+            expression, axis.fetch('grain', 'month'), @population.timezone) : expression
+      end
+    end
+  end
+
+  def joins(fields = all.map { _1['field'] })
+    result = ''
+    if fields.include?('tags')
+      result += <<~SQL.squish
+         JOIN JSON_TABLE(CAST(v.tags_json AS JSON), '$[*]' COLUMNS (
+          tag_id BIGINT PATH '$.tag_id' NULL ON EMPTY,
+          external_tag_id BIGINT PATH '$.external_tag_id' NULL ON EMPTY)) tag_entry
+
+      SQL
+    end
+    if fields.include?('changed_field')
+      fields = @definition.changes.map do |field|
+        "SELECT #{ ApplicationRecord.connection.quote(field) } AS field_key"
+      end.join(' UNION ALL ')
+      condition = @definition.changes.map do |field|
+        "(changed.field_key = '#{ field }' AND #{ @population.changed_sql(field) })"
+      end.join(' OR ')
+      result += " JOIN (#{ fields }) changed ON (#{ condition })"
+    end
+    result
+  end
+
+  private
+
+  def parse(value)
+    axes = QueryState.axes(value)
+    raise ArgumentError unless axes.is_a?(Array)
+
+    axes.each do |axis|
+      raise ArgumentError unless axis.is_a?(Hash) && (axis.keys - ['field', 'grain']).empty?
+      raise ArgumentError unless @definition.axes(@population.history).include?(axis['field'])
+      if @definition.fields[axis['field']] == :datetime
+        raise ArgumentError unless GRAINS.include?(axis.fetch('grain', 'month'))
+
+        axis['grain'] ||= 'month'
+      elsif axis.key?('grain')
+        raise ArgumentError
+      end
+    end
+    axes
+  end
+end

backend/app/services/counts_defaults.rb

@@ -0,0 +1,53 @@
+# Only confirmed combinations belong here; all others retain the existing behaviour.
+class CountsDefaults
+  POST_COLUMNS = ['resource_id', 'version_no', 'is_latest', 'title', 'url',
+                  'thumbnail_base', 'tags', 'parents', 'original_created_from',
+                  'original_created_before', 'event_type', 'user', 'created_at'].freeze
+
+  def self.entries(resource)
+    result = [{ definition: 'history', mode: 'summary', params: {
+        'rows' => 'created_at:month',
+        'cols' => 'event_type',
+        'sort' => 'a0:desc', 'conditions' => '' } }]
+    result << { definition: 'latest', mode: 'summary', params: {
+        'rows' => '', 'cols' => '', 'conditions' => '', 'tags' => '' } }
+    if resource == 'posts'
+      result += [{ definition: 'latest', mode: 'details', params: {
+              'columns' => POST_COLUMNS.join(' '), 'order' => POST_COLUMNS.join(' '),
+              'children' => 'tags.name parents.id', 'nested_json' => '0',
+              'conditions' => '', 'tags' => '' } }]
+    end
+    if resource == 'tags'
+      columns = ['resource_id', 'version_no', 'is_latest', 'name', 'category',
+                 'aliases', 'parents', 'deprecated', 'event_type', 'user', 'created_at']
+      result << { definition: 'latest', mode: 'details', params: {
+          'columns' => columns.join(' '), 'order' => columns.join(' '),
+          'children' => 'aliases.name parents.name', 'nested_json' => '0' } }
+    end
+    result
+  end
+
+  def self.for(params)
+    settings = entries(params.fetch('resource', 'posts')).find do |entry|
+      entry[:definition] == params.fetch('definition', 'latest') &&
+          entry[:mode] == params.fetch('mode', 'summary')
+    end&.fetch(:params) || {}
+    if settings.key?('sort') && params.key?('rows')
+      actual = QueryState.axes(params['rows'])
+      expected = QueryState.axes(settings.fetch('rows'))
+      compatible = actual.is_a?(Array) && actual.first.is_a?(Hash) &&
+          actual.first['field'] == expected.first['field'] &&
+          actual.first.fetch('grain', 'month') == expected.first['grain']
+      unless compatible
+        settings = settings.except('sort')
+      end
+    end
+    settings
+  end
+
+  def self.metadata(resource)
+    entries(resource).map do |entry|
+      entry.merge(params: entry[:params].map { |key, value| { key:, value: } })
+    end
+  end
+end

backend/app/services/counts_export_download.rb

@@ -0,0 +1,106 @@
+require 'tempfile'
+require 'fileutils'
+
+# Finish generation before reporting readiness; downloads never hold the full file in RAM.
+class CountsExportDownload
+  EXPIRY = 10.minutes
+  MAX_CONCURRENT_EXPORTS = 2
+  MAX_FILE_BYTES = 128.megabytes
+  MAX_STORED_BYTES = 512.megabytes
+  MAX_EXPORT_SECONDS = 120
+  class Busy < StandardError; end
+  class TooLarge < StandardError; end
+  class TimedOut < StandardError; end
+  class StorageFull < StandardError; end
+
+  def self.directory
+    Rails.root.join(ENV.fetch('COUNTS_EXPORT_DIRECTORY', 'tmp/counts_exports'))
+  end
+  def self.verifier = Rails.application.message_verifier('counts_export_download')
+
+  def self.prepare(exporter, owner, progress: nil, check: nil, identifier: nil)
+    FileUtils.mkdir_p(directory, mode: 0o700)
+    authority = HistoryAccess.authority
+    slot = acquire_slot
+
+    begin
+      generate(exporter, owner, progress:, check:, identifier:, authority:)
+    ensure
+      slot.close
+    end
+  end
+
+  def self.acquire_slot
+    MAX_CONCURRENT_EXPORTS.times do |index|
+      path = directory.join(".slot-#{ index }.lock")
+      handle = File.open(path, File::RDWR | File::CREAT, 0o600)
+      return handle if handle.flock(File::LOCK_EX | File::LOCK_NB)
+
+      handle.close
+    end
+    raise Busy
+  end
+
+  def self.generate(exporter, owner, progress: nil, check: nil, identifier: nil, authority: HistoryAccess.authority)
+    FileUtils.mkdir_p(directory, mode: 0o700)
+    Dir.glob(directory.join('counts-*.export')).each do |path|
+      File.unlink(path) if File.mtime(path) < Time.current - EXPIRY
+    rescue Errno::ENOENT
+      # Another request may have removed an expired download.
+    end
+    stored = Dir.glob(directory.join('counts-*')).sum do |path|
+      File.size(path)
+    rescue Errno::ENOENT
+      0
+    end
+    reserved = MAX_CONCURRENT_EXPORTS * MAX_FILE_BYTES
+    raise StorageFull if stored + reserved > MAX_STORED_BYTES
+
+    prefix = identifier ? "counts-#{ identifier }-" : 'counts-'
+    file = Tempfile.create([prefix, '.part'], directory)
+    complete = false
+    bytes = 0
+    deadline = Process.clock_gettime(Process::CLOCK_MONOTONIC) + MAX_EXPORT_SECONDS
+    path = file.path
+    begin
+      file.binmode
+      exporter.body(progress:).each do |chunk|
+        check&.call
+        raise TimedOut if Process.clock_gettime(Process::CLOCK_MONOTONIC) > deadline
+
+        bytes += chunk.bytesize
+        raise TooLarge if bytes > MAX_FILE_BYTES
+
+        file.write(chunk)
+      end
+      file.flush
+      file.fsync
+      file.close
+      finished_path = path.sub(/\.part\z/, '.export')
+      File.rename(path, finished_path)
+      path = finished_path
+      token = verifier.generate({ 'file' => File.basename(path),
+                          'resource' => exporter.resource, 'format' => exporter.format,
+                          'owner' => owner, **authority }, expires_in: EXPIRY, purpose: 'counts_export')
+      complete = true
+      token
+    ensure
+      file.close unless file.closed?
+      File.unlink(path) if !complete && File.exist?(path)
+    end
+  end
+
+  def self.find(token, owner)
+    metadata = verifier.verified(token, purpose: 'counts_export')
+    return unless metadata.is_a?(Hash) && metadata['owner'] == owner
+    return unless HistoryAccess.authorised_artifact?(metadata)
+    return unless metadata['file']&.match?(/\Acounts-[a-zA-Z0-9_-]+\.export\z/)
+    return unless CountsResourceDefinition::DEFINITIONS.key?(metadata['resource'])
+    return unless ['json', 'csv'].include?(metadata['format'])
+
+    path = directory.join(metadata['file'])
+    return unless path.file?
+
+    metadata.merge('path' => path.to_s)
+  end
+end

backend/app/services/counts_export_jobs.rb

@@ -0,0 +1,255 @@
+require 'json'
+require 'fileutils'
+
+# A durable, bounded export spool. Web and supervised workers share the same POSIX volume.
+class CountsExportJobs
+  class Cancelled < StandardError; end
+  ACTIVE = ['queued', 'running', 'cancelling'].freeze
+
+  def self.directory = CountsExportDownload.directory.join('jobs')
+  def self.verifier = Rails.application.message_verifier('counts_export_job')
+
+  def self.synchronise
+    FileUtils.mkdir_p(directory, mode: 0o700)
+    File.open(directory.join('.registry.lock'), File::RDWR | File::CREAT, 0o600) do |lock|
+      lock.flock(File::LOCK_EX)
+      yield
+    end
+  end
+
+  def self.write(state)
+    path = directory.join("#{ state.fetch('id') }.json")
+    temporary = "#{ path }.#{ SecureRandom.hex(8) }.tmp"
+    File.open(temporary, 'w', 0o600) do |file|
+      file.write(JSON.generate(state))
+      file.flush
+      file.fsync
+    end
+    File.rename(temporary, path)
+  ensure
+    File.unlink(temporary) if temporary && File.exist?(temporary)
+  end
+
+  def self.read(id)
+    JSON.parse(File.read(directory.join("#{ id }.json")))
+  rescue Errno::ENOENT
+    nil
+  end
+
+  def self.remove_download(state)
+    name = state['download_file']
+    return unless name&.match?(/\Acounts-[a-zA-Z0-9_-]+\.export\z/)
+
+    File.unlink(CountsExportDownload.directory.join(name))
+  rescue Errno::ENOENT
+    nil
+  end
+
+  def self.remove_partial(id)
+    Dir.glob(CountsExportDownload.directory.join("counts-#{ id }-*")).each do |path|
+      File.unlink(path)
+    rescue Errno::ENOENT
+      nil
+    end
+  end
+
+  def self.clean
+    Dir.glob(directory.join('*.tmp')).each do |path|
+      File.unlink(path) if File.mtime(path) < Time.current - CountsExportDownload::EXPIRY
+    end
+    Dir.glob(directory.join('*.run')).each do |path|
+      id = File.basename(path, '.run')
+      next if directory.join("#{ id }.json").exist?
+
+      File.open(path, File::RDWR) do |lock|
+        File.unlink(path) if lock.flock(File::LOCK_EX | File::LOCK_NB)
+      end
+    end
+    Dir.glob(directory.join('*.json')).each do |path|
+      state = JSON.parse(File.read(path))
+      next if state.fetch('expires_at') > Time.current.to_f
+
+      run = directory.join("#{ state.fetch('id') }.run")
+      File.open(run, File::RDWR | File::CREAT, 0o600) do |lock|
+        unless lock.flock(File::LOCK_EX | File::LOCK_NB)
+          state['status'] = 'cancelling'
+          write(state)
+          next
+        end
+        remove_download(state)
+        remove_partial(state.fetch('id'))
+        File.unlink(path)
+        File.unlink(run)
+      end
+    end
+  end
+
+  def self.start(payload, owner, language)
+    payload = payload.dup
+    time_key = payload.key?('definition') ? 'execution_at' : 'at'
+    if payload[time_key].nil? || payload[time_key] == ''
+      payload[time_key] = Time.current.utc.iso8601(6)
+    end
+    exporter = CountsExporter.new(payload, language_code: language)
+    synchronise do
+      clean
+      active = Dir.glob(directory.join('*.json')).count do |path|
+        ACTIVE.include?(JSON.parse(File.read(path))['status'])
+      end
+      raise CountsExportDownload::Busy if active >= CountsExportDownload::MAX_CONCURRENT_EXPORTS
+
+      id = SecureRandom.hex(24)
+      state = { 'id' => id, 'owner' => owner, 'payload' => payload,
+          'language' => language, 'resource' => exporter.resource,
+          'format' => exporter.format, 'status' => 'queued',
+          'processed' => 0, 'total' => nil,
+          **HistoryAccess.authority,
+          'expires_at' => (Time.current + CountsExportDownload::EXPIRY).to_f }
+      write(state)
+      verifier.generate({ 'id' => id, 'owner' => owner },
+          expires_in: CountsExportDownload::EXPIRY, purpose: 'counts_export_job')
+    end
+  end
+
+  def self.authorised_id(token, owner)
+    value = verifier.verified(token, purpose: 'counts_export_job')
+    return unless value.is_a?(Hash) && value['owner'] == owner
+    return unless value['id']&.match?(/\A[a-f0-9]{48}\z/)
+
+    value['id']
+  end
+
+  def self.status(token, owner, cancel: false)
+    id = authorised_id(token, owner)
+    return unless id
+
+    synchronise do
+      state = read(id)
+      return unless state && state['owner'] == owner && state['expires_at'] > Time.current.to_f
+
+      return unless HistoryAccess.authorised_artifact?(state)
+
+      if cancel && ['queued', 'running', 'ready'].include?(state['status'])
+        state['status'] = state['status'] == 'running' ? 'cancelling' : 'cancelled'
+        remove_download(state)
+        state.delete('token')
+        write(state)
+      end
+      state.slice('status', 'processed', 'total', 'format', 'token', 'error')
+    end
+  end
+
+  def self.checkpoint(id, processed = nil, total = nil)
+    synchronise do
+      state = read(id)
+      unless state && state['status'] == 'running' && state['expires_at'] > Time.current.to_f
+        raise Cancelled
+      end
+      if processed
+        state.merge!('processed' => processed, 'total' => total)
+        write(state)
+      end
+    end
+  end
+
+  def self.work_one
+    FileUtils.mkdir_p(directory, mode: 0o700)
+    Dir.glob(directory.join('*.json')).sort.each do |path|
+      id = File.basename(path, '.json')
+      File.open(directory.join("#{ id }.run"), File::RDWR | File::CREAT, 0o600) do |lock|
+        next unless lock.flock(File::LOCK_EX | File::LOCK_NB)
+
+        state = synchronise do
+          candidate = read(id)
+          next unless candidate && ACTIVE.include?(candidate['status'])
+
+          if candidate['status'] != 'queued'
+            candidate['status'] = candidate['status'] == 'cancelling' ? 'cancelled' : 'failed'
+            remove_partial(id)
+            candidate['error'] = 'interrupted'
+            write(candidate)
+            next
+          end
+          candidate['status'] = 'running'
+          write(candidate)
+          candidate
+        end
+        next unless state
+
+        perform(state)
+        return true
+      end
+    end
+    synchronise { clean }
+    false
+  end
+
+  def self.perform(state)
+    actor = state['actor_id'] && User.find_by(id: state['actor_id'])
+    actor = nil unless state['admin'] && actor&.admin? && !actor.banned?
+    HistoryAccess.set(user: actor) do
+      unless HistoryAccess.authorised_artifact?(state)
+        synchronise do
+          current = read(state.fetch('id'))
+          current['status'] = 'failed'
+          write(current)
+        end
+        return
+      end
+      perform_authorised(state)
+    end
+  end
+
+  def self.perform_authorised(state)
+    id = state.fetch('id')
+    token = nil
+    begin
+      checkpoint(id)
+      exporter = CountsExporter.new(state.fetch('payload'), language_code: state.fetch('language'))
+      last = Process.clock_gettime(Process::CLOCK_MONOTONIC)
+      progress = lambda do |processed, total|
+        now = Process.clock_gettime(Process::CLOCK_MONOTONIC)
+        if processed.zero? || now - last >= 0.25 || processed == total
+          checkpoint(id, processed, total)
+          last = now
+        end
+      end
+      check = lambda do
+        now = Process.clock_gettime(Process::CLOCK_MONOTONIC)
+        if now - last >= 0.25
+          checkpoint(id)
+          last = now
+        end
+      end
+      token = CountsExportDownload.prepare(exporter, state.fetch('owner'),
+          progress:, check:, identifier: id)
+      synchronise do
+        current = read(id)
+        if (!current || current['status'] != 'running' ||
+            current['expires_at'] <= Time.current.to_f)
+          file = CountsExportDownload.find(token, state.fetch('owner'))
+          File.unlink(file.fetch('path')) if file
+          raise Cancelled
+        end
+        file = CountsExportDownload.find(token, state.fetch('owner'))
+        current.merge!('status' => 'ready', 'token' => token,
+            'download_file' => File.basename(file.fetch('path')))
+        current['total'] = current['processed']
+        write(current)
+      end
+    rescue StandardError => error
+      remove_partial(id)
+      synchronise do
+        current = read(id)
+        if current
+          current['status'] = error.is_a?(Cancelled) ? 'cancelled' : 'failed'
+          current['error'] = error.class.name
+          write(current)
+        end
+      end
+      unless error.is_a?(Cancelled)
+        Rails.logger.error("counts_export_job #{ id }: #{ error.class }")
+      end
+    end
+  end
+end

backend/app/services/counts_export_projection.rb

@@ -0,0 +1,265 @@
+require 'set'
+
+# Export identifiers are stable; UI translations are supplied separately as column labels.
+class CountsExportProjection
+  ROOT_KEYS = { 'event_type' => 'event_type', 'resource_id' => 'resource_id', 'version_no' => 'version_no',
+                'is_latest' => 'is_latest', 'thumbnail' => 'thumbnail_url',
+                'thumbnail_base' => 'thumbnail_base', 'title' => 'title', 'url' => 'url',
+                'tags' => 'tags', 'parents' => 'parents', 'video_ms' => 'video_ms',
+                'original_created_from' => 'original_created_from',
+                'original_created_before' => 'original_created_before',
+                'user' => 'created_by_user', 'created_at' => 'created_at',
+                'name' => 'name', 'names' => 'names', 'category' => 'category',
+                'aliases' => 'aliases', 'deprecated' => 'deprecated' }.freeze
+  CHILD_KEYS = { 'id' => 'id', 'versionNo' => 'version_no', 'kind' => 'kind',
+                 'name' => 'name', 'category' => 'category', 'sections' => 'sections',
+                 'names' => 'names', 'aliases' => 'aliases',
+                 'languageCode' => 'language', 'scriptCode' => 'script',
+                 'autoGenerated' => 'auto_generated' }.freeze
+  NAME_FIELDS = ['languageCode', 'scriptCode', 'name', 'aliases', 'autoGenerated'].freeze
+  CHILDREN = {
+      'tags' => ['id', 'versionNo', 'kind', 'name', 'category', 'sections', 'names'],
+      'names' => NAME_FIELDS,
+      'aliases' => ['languageCode', 'name', 'scriptCode'],
+      'parents' => ['id', 'name'] }.freeze
+
+  def self.default_columns(resource)
+    fields = { 'posts' => ['title', 'url', 'tags', 'parents', 'video_ms'],
+        'tags' => ['name', 'category', 'aliases', 'parents'],
+        'nico_tags' => ['name', 'linked_tags'],
+        'tag_descriptions' => ['language_code', 'body', 'source_language_code'],
+        'tag_groups' => ['name', 'description', 'members'],
+        'materials' => ['url', 'tag_id', 'source_kind'] }.fetch(resource)
+    ['resource_id', 'version_no'] + fields + ['event_type', 'user', 'created_at']
+  end
+
+  def self.default_children(resource)
+    { 'posts' => ['tags.name', 'parents.id'],
+        'tags' => ['aliases.name', 'parents.id'],
+        'tag_groups' => ['members.tagId'] }.fetch(resource, [])
+  end
+
+  attr_reader :columns, :headers
+
+  def initialize(params, resource:, summary:, tags:)
+    if params.key?('definition')
+      params = CountsDefaults.for(params.merge('resource' => resource)).merge(params)
+    end
+    @params = params
+    @labels = JSON.parse(params.fetch('labels', '{}'))
+    raise ArgumentError unless @labels.is_a?(Hash) && @labels.length <= 100
+    raise ArgumentError unless @labels.all? { |key, value| valid_label?(key, value) }
+    @summary = summary
+    allowed = if summary
+                tags ? ['kind', 'tag_id', 'name', 'deprecated', 'count'] : ['group', 'count']
+              else
+                CountsQuery::COLUMNS.fetch(resource)
+              end
+    defaults = summary ? allowed : self.class.default_columns(resource)
+    @columns = QueryState.list(params.fetch('columns', defaults.join(' ')))
+    raise ArgumentError if (@columns - allowed).any? || @columns.uniq != @columns
+
+    @headers = JSON.parse(params.fetch('headers', JSON.generate(@columns)))
+    unless (@headers.is_a?(Array) && @headers.length == @columns.length &&
+        @headers.all? { _1.is_a?(String) && _1.length <= 200 })
+      raise ArgumentError
+    end
+    @order = QueryState.list(params.fetch('order', ''))
+    @sorts = QueryState.list(params.fetch('child_sort', '')).map do |entry|
+      parts = entry.split(':', -1)
+      raise ArgumentError unless parts.length == 2
+
+      path, direction = parts
+      raise ArgumentError unless ['asc', 'desc'].include?(direction)
+
+      [path, direction]
+    end
+    @selected = params.key?('children') ? QueryState.list(params['children']).to_set :
+        self.class.default_children(resource).to_set
+    valid = CHILDREN.flat_map do |key, fields|
+      paths(fields, key)
+    end
+    raise ArgumentError if @selected && (@selected.to_a - valid).any?
+    prune = lambda do |fields, prefix|
+      fields.each do |key|
+        path = "#{ prefix }.#{ key }"
+        children = fields_for(key)
+        next if children.empty?
+
+        prune.call(children, path)
+        @selected.delete(path) unless children.any? { @selected.include?("#{ path }.#{ _1 }") }
+      end
+    end
+    CHILDREN.each { |key, fields| prune.call(fields, key) }
+    pairs = @columns.zip(@headers).select do |key, _header|
+      !CHILDREN.key?(key) || CHILDREN[key].any? { @selected.include?("#{ key }.#{ _1 }") }
+    end
+    @columns = pairs.map(&:first)
+    @headers = pairs.map(&:last)
+    allowed_sorts = valid.map do |path|
+      parts = path.split('.')
+      field = parts.pop
+      "#{ parts.join('.') }/#{ field }"
+    end
+    if ((@sorts.map(&:first) - allowed_sorts).any? ||
+        @sorts.map(&:first).uniq.length != @sorts.length)
+      raise ArgumentError
+    end
+  end
+
+  def metadata
+    @columns.each_with_index.map do |key, index|
+      { key: @summary ? key : ROOT_KEYS.fetch(key), label: @headers[index] }
+    end
+  end
+
+  def project(row)
+    @columns.to_h do |key|
+      if @summary
+        value = key == 'group' ? { key: row[:key], name: row[:name] } : row[key.to_sym]
+        [key, value]
+      else
+        value = case key
+                when 'thumbnail' then row[:thumbnail_base]
+                else row[ROOT_KEYS.fetch(key).to_sym]
+                end
+        value = nested(value || [], CHILDREN.fetch(key), key) if CHILDREN.key?(key)
+        [ROOT_KEYS.fetch(key), value]
+      end
+    end
+  end
+
+  def csv_record(record, row)
+    record.map do |key, value|
+      key == 'group' ? row[:name] || row[:key] : csv_display(value, key)
+    end
+  end
+
+  private
+
+  def valid_label?(key, value)
+    key.is_a?(String) && value.is_a?(String) && key.length <= 100 && value.length <= 300
+  end
+
+  def csv_display(value, key)
+    return @labels.fetch('bot', '') if key == 'created_by_user' && value.nil?
+    return @labels.fetch("event_type:#{ value }", value) if key == 'event_type'
+    return value ? @labels.fetch('latest', 'true') : '' if key == 'is_latest'
+    return @labels.fetch("kind:#{ value }", value) if key == 'kind'
+    return @labels.fetch("category:#{ value }", value) if key == 'category'
+    if ['language', 'script'].include?(key) && value.is_a?(Hash)
+      return @labels.fetch("#{ key }:#{ value[:code] }",
+                           value[:name] || @labels.fetch('notRegistered', ''))
+    end
+    return @labels.fetch("boolean:#{ value }", value) if value == true || value == false
+    return value.map { csv_display(_1, key) } if value.is_a?(Array)
+    if value.is_a?(Hash)
+      return value.to_h do |field, item|
+        [@labels.fetch(field.to_s, field), csv_display(item, field.to_s)]
+      end
+    end
+
+    value
+  end
+
+  def fields_for(key)
+    return NAME_FIELDS if key == 'names'
+    return ['name', 'scriptCode'] if key == 'aliases'
+
+    []
+  end
+
+  def paths(fields, prefix)
+    fields.flat_map do |key|
+      path = "#{ prefix }.#{ key }"
+      [path, *paths(fields_for(key), path)]
+    end
+  end
+
+  def fetch(row, key) = row.key?(key) ? row[key] : row[key.to_sym]
+
+  def named_code(row, kind)
+    code = fetch(row, "#{ kind }_code")
+    name = fetch(row, "#{ kind }_name")
+    { code:, name: name.present? && name.to_s.downcase != code.to_s.downcase ? name : nil }
+  end
+
+  def stable(value)
+    case value
+    when Hash
+      value.stringify_keys.sort.to_h.transform_values { stable(_1) }
+    when Array
+      value.map { stable(_1) }
+    else
+      value
+    end
+  end
+
+  def sort_value(row, key)
+    if key == 'languageCode'
+      language = named_code(row, 'language')
+      return @labels.fetch("language:#{ language[:code] }", language[:name])
+    end
+    return named_code(row, 'script')[:name] if key == 'scriptCode'
+
+    fetch(row, CHILD_KEYS.fetch(key))
+  end
+
+  def comparison(left, right)
+    if left.is_a?(Array) && right.is_a?(Array) && left.length != right.length
+      return left.length <=> right.length
+    end
+    return left <=> right if left.is_a?(Numeric) && right.is_a?(Numeric)
+    if [true, false].include?(left) && [true, false].include?(right)
+      return (left ? 1 : 0) <=> (right ? 1 : 0)
+    end
+
+    a = left.is_a?(Hash) || left.is_a?(Array) ? JSON.generate(stable(left)) : left.to_s
+    b = right.is_a?(Hash) || right.is_a?(Array) ? JSON.generate(stable(right)) : right.to_s
+    a <=> b
+  end
+
+  def sorted(rows, prefix, fields)
+    sorts = @sorts.filter_map do |path, direction|
+      parent, key = path.split('/', 2)
+      next unless parent == prefix
+      raise ArgumentError unless fields.include?(key)
+
+      [key, direction]
+    end
+    rows.each_with_index.sort do |(left, a), (right, b)|
+      result = 0
+      sorts.each do |key, direction|
+        first = sort_value(left, key)
+        last = sort_value(right, key)
+        result = if first.nil? || last.nil?
+                   first.nil? == last.nil? ? 0 : (first.nil? ? 1 : -1)
+                 else
+                   comparison(first, last) * (direction == 'asc' ? 1 : -1)
+                 end
+        break unless result.zero?
+      end
+      result.zero? ? a <=> b : result
+    end.map(&:first)
+  end
+
+  def nested(rows, fields, prefix)
+    keys = fields.select { !@selected || @selected.include?("#{ prefix }.#{ _1 }") }
+    ordered = (@order + keys.map { "#{ prefix }.#{ _1 }" }).uniq
+        .filter_map { |path| keys.find { "#{ prefix }.#{ _1 }" == path } }
+    sorted(rows, prefix, fields).map do |row|
+      ordered.to_h do |key|
+        stable = CHILD_KEYS.fetch(key)
+        value = case key
+                when 'languageCode' then named_code(row, 'language')
+                when 'scriptCode' then named_code(row, 'script')
+                else fetch(row, stable)
+                end
+        if fields_for(key).any?
+          value = nested(value || [], fields_for(key), "#{ prefix }.#{ key }")
+        end
+        [stable, value]
+      end
+    end
+  end
+end

backend/app/services/counts_exporter.rb

@@ -0,0 +1,152 @@
+require 'csv'
+
+# One read view and a fixed point for the entire download; only one page is held in memory.
+class CountsExporter
+  EXTRA_KEYS = ['basis', 'format', 'resource', 'headers', 'children', 'order',
+                'summary_columns', 'nested_json', 'child_sort', 'labels'].freeze
+  attr_reader :format, :resource
+
+  def initialize(params, language_code:)
+    if params.key?('definition')
+      @table_export = CountsTableExport.new(params)
+      @format = @table_export.format
+      @resource = @table_export.resource
+      return
+    end
+    allowed = (CountsQuery::KEYS + CountsStateQuery::KEYS + EXTRA_KEYS).uniq
+    QueryState.validate!(params, allowed)
+    raise ArgumentError unless params.values.all? { _1.is_a?(String) }
+
+    @params = params
+    @language = language_code
+    @format = params.fetch('format', 'json')
+    @resource = params.fetch('resource', 'posts')
+    @mode = params.fetch('mode', 'summary')
+    @history = params.fetch('basis', 'state') == 'history'
+    raise ArgumentError unless ['json', 'csv'].include?(format)
+    raise ArgumentError unless ['posts', 'tags'].include?(resource)
+    raise ArgumentError unless ['summary', 'details'].include?(@mode)
+    raise ArgumentError unless ['state', 'history'].include?(params.fetch('basis', 'state'))
+
+    @query_params = params.except(*EXTRA_KEYS, 'columns', 'page', 'limit')
+        .merge('mode' => @mode, 'page' => '1', 'limit' => '100')
+    point = CountsStateQuery.new(@query_params.slice('at').merge('resource' => resource),
+                                 language_code: @language).at
+    @query_params['at'] = point.iso8601(6)
+    @query_params['axis'] ||= !@history && resource == 'posts' ? 'tags' : 'month'
+    @projection = CountsExportProjection.new(params, resource:,
+                                             summary: @mode == 'summary',
+                                             tags: !@history && @query_params['axis'] == 'tags')
+    if @history
+      @query_params.delete('kind')
+      history_query
+    else
+      @query_params.delete('latest')
+      @state_query = CountsStateQuery.new(@query_params.merge('resource' => resource),
+                                          language_code: @language).validate_query!
+    end
+  end
+
+  def body(progress: nil)
+    return @table_export.body(progress:) if @table_export
+
+    Enumerator.new do |output|
+      ApplicationRecord.transaction(isolation: :repeatable_read) do
+        processed = 0
+        first_row = true
+        started = false
+        pages do |page|
+          total = @mode == 'details' ? page[:count] : nil
+          progress&.call(processed, total)
+          unless started
+            meta = { resource:, mode: @mode, basis: @history ? 'history' : 'state',
+                     at: @query_params['at'], boundary: 'inclusive',
+                     filters: @query_params.except('sort', 'page', 'limit', 'mode'),
+                     sort: export_sort,
+                     child_sort: sort_metadata(@params.fetch('child_sort', '')),
+                     columns: @projection.metadata,
+                     children: @params['children']&.split(' '),
+                     field_order: @QueryState.list(params.fetch('order', '')), timezone: 'UTC',
+                     count: @mode == 'summary' ? page[:group_count] : page[:count],
+                     totals: page[:totals], coverage: page[:coverage] }
+            output << (format == 'json' ? "{\"metadata\":#{ meta.to_json },\"data\":[" : "\uFEFF")
+            if format == 'csv'
+              output << CSV.generate_line(
+                  @projection.headers.map { csv_value(_1) }, row_sep: "\r\n")
+            end
+            started = true
+          end
+          (page[:versions] || page[:state_rows] || page[:groups] || []).each do |row|
+            record = @projection.project(row)
+            if format == 'json'
+              output << ',' unless first_row
+              output << record.to_json
+            else
+              values = @projection.csv_record(record, row).map { csv_value(_1) }
+              output << CSV.generate_line(values, row_sep: "\r\n")
+            end
+            first_row = false
+            processed += 1
+            progress&.call(processed, total)
+          end
+        end
+        progress&.call(processed, processed)
+        output << ']}' if format == 'json'
+      end
+    end
+  end
+
+  private
+
+  def export_sort
+    sort_metadata(@query_params.fetch('sort', default_sort))
+  end
+
+  def sort_metadata(value)
+    value.split(' ').map do |entry|
+      key, direction = entry.split(':', 2)
+      { key:, direction: }
+    end
+  end
+
+  def default_sort
+    return 'created_at:desc' if @mode == 'details'
+
+    !@history && @query_params['axis'] == 'tags' ? 'count:desc' : 'group:asc'
+  end
+
+  def history_query
+    scope = resource == 'posts' ? PostVersion.with_previous_version : TagVersion.all
+    CountsQuery.new(scope, resource, @query_params)
+  end
+
+  def pages
+    unless @history
+      @state_query.each_page { yield _1 }
+      return
+    end
+    query = history_query
+    total = @mode == 'summary' ? nil : query.count
+    loop do
+      result = if @mode == 'summary'
+                 query.summary(include_count: @query_params['page'] == '1')
+               else
+                 { count: total,
+                   versions: CountsVersionRepr.many(query.details, resource:,
+                                                   language_code: @language,
+                                                   at: Time.iso8601(@query_params['at'])) }
+               end
+      total ||= result[:group_count]
+      yield result
+      break if @query_params['page'].to_i * 100 >= total
+
+      @query_params['page'] = (@query_params['page'].to_i + 1).to_s
+      query = history_query
+    end
+  end
+
+  def csv_value(value)
+    text = value.is_a?(Hash) || value.is_a?(Array) ? JSON.generate(value) : value.to_s
+    text.match?(/\A\s*[=+@-]/) ? "'#{ text }" : text
+  end
+end

backend/app/services/counts_pivot_layout.rb

@@ -0,0 +1,28 @@
+# Coordinates carry their aggregation depth; SQL NULL remains an ordinary axis value.
+class CountsPivotLayout
+  def self.depths(keys, sorts, measures: true)
+    return [] if keys.length < 2 || sorts.first&.first == 'count' ||
+        sorts.first&.first&.match?(/\Am[0-9]+\z/)
+
+    relevant = sorts.map(&:first).select { keys.include?(_1) || (measures && (_1 == 'count' ||
+        _1.match?(/\Am[0-9]+\z/))) }
+    order = (relevant + keys).uniq
+    (1...keys.length).select { order.first(_1).sort == keys.first(_1).sort }
+  end
+
+  def self.headers(values, depths, next_values: nil, grand: false)
+    result = []
+    values.each_with_index do |tuple, index|
+      result << { values: tuple, kind: 'data' }
+      following = values[index + 1] || next_values
+      depths.reverse_each do |depth|
+        prefix = tuple.first(depth)
+        next if following && following.first(depth) == prefix
+
+        result << { values: prefix, kind: 'subtotal' }
+      end
+    end
+    result << { values: [], kind: 'grand_total' } if grand
+    result
+  end
+end

backend/app/services/counts_pivot_totals.rb

@@ -0,0 +1,59 @@
+# Re-aggregate the population, never add up already grouped cells (multi-value axes overlap).
+class CountsPivotTotals
+  def initialize(population, axes, row_headers, column_headers)
+    @population = population
+    @axes = axes
+    @rows = row_headers.group_by { _1[:values].length }
+    @columns = column_headers.group_by { _1[:values].length }
+  end
+
+  def call
+    queries = @rows.flat_map do |row_depth, headers|
+      @columns.filter_map do |column_depth, columns|
+        next if row_depth == @axes.rows.length && column_depth == @axes.columns.length
+
+        indexes = (0...row_depth).to_a +
+            (0...column_depth).map { @axes.rows.length + _1 }
+        expressions = indexes.map { @axes.expressions[_1] }
+        scope = @population.scope.except(:select, :order)
+        joins = @axes.joins(indexes.map { @axes.all[_1].fetch('field') })
+        scope = scope.joins(joins) unless joins.empty?
+        if row_depth.positive?
+          tuples = headers.map { _1[:values] }.uniq
+          predicate = tuples.map { Array.new(row_depth) { |i|
+            "(#{ expressions[i] }) <=> ?"
+          }.join(' AND ') }.map { "(#{ _1 })" }.join(' OR ')
+          scope = scope.where(predicate, *tuples.flatten)
+        end
+        if column_depth.positive?
+          tuples = columns.map { _1[:values] }.uniq
+          predicate = tuples.map { Array.new(column_depth) { |i|
+            "(#{ expressions[row_depth + i] }) <=> ?"
+          }.join(' AND ') }.map { "(#{ _1 })" }.join(' OR ')
+          scope = scope.where(predicate, *tuples.flatten)
+        end
+        grouped = expressions.each_with_index.map { |expression, index|
+          "#{ expression } AS v#{ index }"
+        }
+        # Fixed-width result coordinates, independent of NULL and of translated labels.
+        padding = (expressions.length...@axes.all.length).map { "NULL AS v#{ _1 }" }
+        fields = ["#{ row_depth } AS row_depth", "#{ column_depth } AS column_depth"] +
+            grouped + padding + ['COUNT(DISTINCT v.id) AS count']
+        scope = scope.group(*expressions.map { Arel.sql(_1) }) if expressions.any?
+        scope.optimizer_hints('MAX_EXECUTION_TIME(10000)')
+            .select(Arel.sql(fields.join(', '))).to_sql
+      end
+    end
+    return [] if queries.empty?
+
+    sql = queries.map { "(#{ _1 })" }.join(' UNION ALL ')
+    @population.definition.model.connection.select_all(
+        "SELECT /*+ MAX_EXECUTION_TIME(10000) */ * FROM (#{ sql }) totals").map do |record|
+      row_depth = record['row_depth'].to_i
+      column_depth = record['column_depth'].to_i
+      { row: (0...row_depth).map { record["v#{ _1 }"] },
+        column: (0...column_depth).map { record["v#{ row_depth + _1 }"] },
+        count: record['count'].to_i }
+    end
+  end
+end

backend/app/services/counts_population.rb

@@ -0,0 +1,210 @@
+# Population first, then filters. Previous versions are joined before period filtering.
+class CountsPopulation
+  attr_reader :definition, :params, :at, :history, :scope, :timezone
+
+  def initialize(params)
+    @params = params
+    @definition = CountsResourceDefinition.new(params.fetch('resource', 'posts'))
+    @history = params.fetch('definition', 'latest') == 'history'
+    raise ArgumentError unless ['latest', 'history'].include?(params.fetch('definition', 'latest'))
+
+    @timezone = TZInfo::Timezone.get(params.fetch('timezone', 'UTC'))
+    @at = timestamp(params['execution_at']) || Time.current.utc
+    raise ArgumentError if at > Time.current + 1.second
+
+    @scope = definition.model.all
+    make_population
+    @scope = HistoryAccess.visible(@scope, 'v')
+    apply_filters
+  rescue TZInfo::InvalidTimezoneIdentifier
+    raise ArgumentError
+  end
+
+  def timestamp(value, end_date: false)
+    return nil if value.nil? || value == ''
+    raise ArgumentError unless value.is_a?(String)
+
+    if value.match?(/\A\d{4}-\d{2}-\d{2}\z/)
+      date = Date.iso8601(value)
+      date += 1 if end_date
+      timezone.local_to_utc(Time.utc(date.year, date.month, date.day))
+    else
+      raise ArgumentError unless value.match?(/(?:Z|[+-]\d{2}:\d{2})\z/)
+
+      time = Time.iso8601(value).utc
+      raise ArgumentError unless time.year.between?(1900, 2100)
+
+      time
+    end
+  rescue TZInfo::AmbiguousTime, TZInfo::PeriodNotFound, Date::Error
+    raise ArgumentError
+  end
+
+  def latest_sql(row = 'v', cutoff = nil)
+    cutoff ||= history ? at : timestamp(params['at']) || at
+    keys = definition.keys.map { "newer.#{ _1 } = #{ row }.#{ _1 }" }.join(' AND ')
+    bound = definition.model.connection.quote(cutoff)
+    <<~SQL.squish
+      NOT EXISTS (SELECT 1 FROM #{ definition.table } newer
+        WHERE #{ keys } AND newer.version_no > #{ row }.version_no
+          AND newer.#{ definition.time_column } <= #{ bound })
+    SQL
+  end
+
+  def changed_sql(key)
+    column = definition.column(key)
+    "(previous.id IS NOT NULL AND NOT (v.#{ column } <=> previous.#{ column }))"
+  end
+
+  private
+
+  def make_population
+    table = definition.table
+    @scope = @scope.from("#{ table } v").select('v.*')
+    keys = definition.keys.map { "previous.#{ _1 } = v.#{ _1 }" }.join(' AND ')
+    @scope = @scope.joins("LEFT JOIN #{ table } previous ON #{ keys } " +
+        'AND previous.version_no = v.version_no - 1 AND ' +
+        HistoryAccess.sql(definition.model, 'previous'))
+    @scope = @scope.where("v.#{ definition.time_column } <= ?", at)
+    unless history
+      point = timestamp(params['at']) || at
+      raise ArgumentError if point > at
+
+      @scope = @scope.where("v.#{ definition.time_column } <= ?", point)
+          .where(Arel.sql(latest_sql('v', point))).where.not('v.event_type' => 'discard')
+      return
+    end
+    first = timestamp(params['from'])
+    last = timestamp(params['through'], end_date: true)
+    raise ArgumentError if first && last && first > last
+
+    @scope = @scope.where('v.created_at >= ?', first) if first
+    if last
+      operator = params['through'].length == 10 ? '<' : '<='
+      @scope = @scope.where("v.created_at #{ operator } ?", last)
+    end
+  end
+
+  def apply_filters
+    ['resource_id', 'user'].each do |key|
+      value = params[key == 'user' ? 'user_id' : key]
+      next if value.nil? || value == ''
+      raise ArgumentError unless value.match?(/\A[0-9]+\z/)
+
+      @scope = @scope.where("#{ definition.expression(key) } = ?", value.to_i)
+    end
+    apply_tags if params['tags'].present? || params['tag'].present?
+    conditions = QueryState.records(params, 'conditions')
+    raise ArgumentError unless conditions.is_a?(Array) && conditions.length <= 100
+
+    conditions.each { apply_condition(_1) }
+    drills = QueryState.records(params, 'drill')
+    raise ArgumentError unless drills.is_a?(Array) && drills.length <= 100
+
+    drills.each do |entry|
+      key = entry.fetch('field')
+      if key == 'tags'
+        @scope = @scope.where(snapshot_membership(entry.fetch('value')))
+      elsif key == 'changed_field'
+        raise ArgumentError unless history && definition.changes.include?(entry['value'])
+
+        @scope = @scope.where(Arel.sql(changed_sql(entry['value'])))
+      elsif definition.fields[key] == :datetime && entry['grain']
+        @scope = @scope.where(definition.expression(key) => nil) if entry['value'].nil?
+        if entry['value']
+          expression = CountsAxes.time_expression(definition.expression(key),
+                                                  entry['grain'], timezone)
+          @scope = @scope.where("#{ expression } = ?", entry['value'])
+        end
+      else
+        raise ArgumentError unless definition.axes(history).include?(key)
+
+        @scope = @scope.where(definition.expression(key) => entry['value'])
+      end
+    end
+  end
+
+  def apply_condition(entry)
+    raise ArgumentError unless entry.is_a?(Hash)
+    raise ArgumentError if (entry.keys - ['field', 'operator', 'value', 'end', 'blank']).any?
+
+    key = entry.fetch('field')
+    type = definition.fields.fetch(key) { raise ArgumentError }
+    operator = entry.fetch('operator')
+    raise ArgumentError unless CountsResourceDefinition::OPERATORS.fetch(type, []).include?(operator)
+    raise ArgumentError unless [true, false, nil].include?(entry['blank'])
+
+    expression = definition.expression(key)
+    blank = type == :string || type == :enum ?
+        "(#{ expression } IS NULL OR #{ expression } = '')" : "#{ expression } IS NULL"
+    if entry['blank']
+      @scope = @scope.where(Arel.sql(blank))
+      return
+    end
+    values = [entry['value'], entry['end']]
+    raise ArgumentError unless values.all? { _1.nil? || _1.is_a?(String) }
+    return if values.all? { _1.nil? || _1 == '' }
+
+    parsed = values.map do |value|
+      next nil if value.nil? || value == ''
+
+      case type
+      when :integer
+        raise ArgumentError unless value.match?(/\A-?\d+\z/)
+
+        value.to_i
+      when :datetime then timestamp(value)
+      else value
+      end
+    end
+    if operator == 'range'
+      @scope = @scope.where("#{ expression } >= ?", parsed[0]) if parsed[0]
+      @scope = @scope.where("#{ expression } <= ?", parsed[1]) if parsed[1]
+      return
+    end
+    return unless parsed[0]
+
+    sql_operator = { 'eq' => '=', 'ne' => '<>', 'gt' => '>', 'lt' => '<',
+                     'ge' => '>=', 'le' => '<=', 'like' => 'LIKE' }.fetch(operator)
+    sql = "#{ expression } #{ sql_operator } ?"
+    sql = "(#{ sql } OR #{ blank })" if operator == 'ne'
+    @scope = @scope.where(sql, parsed[0])
+  end
+
+  def snapshot_membership(value)
+    kind, identifier = value.to_s.split(':', 2)
+    raise ArgumentError unless ['internal', 'external'].include?(kind)
+    raise ArgumentError unless identifier&.match?(/\A[1-9]\d*\z/)
+
+    key = kind == 'internal' ? 'tag_id' : 'external_tag_id'
+    definition.model.sanitize_sql_array([
+        "JSON_CONTAINS(v.tags_json, JSON_OBJECT('#{ key }', CAST(? AS UNSIGNED)))",
+        identifier.to_i])
+  end
+
+  def apply_tags
+    raise ArgumentError unless definition.resource == 'posts'
+
+    if params['tag'].present?
+      @scope = @scope.where(snapshot_membership(
+          "#{ params.fetch('kind', 'internal') }:#{ params['tag'] }"))
+    end
+    return unless params['tags'].present?
+
+    locale = Locale.find_by!(code: params.fetch('locale', 'ja'))
+    refs = PostTagSearch.parse_references(params, params['tags'].split)
+    raw = params['tags']
+    literals = PostTagSearch.literals(locale, raw, references: refs)
+    predicates = literals.map do |literal|
+      internal, external = PostTagSearch.references(locale, literal)
+      parts = Array(internal).map { snapshot_membership("internal:#{ _1 }") } +
+          Array(external).map { snapshot_membership("external:#{ _1 }") }
+      expression = parts.empty? ? 'FALSE' : "(#{ parts.join(' OR ') })"
+      literal[:negative] ? "NOT #{ expression }" : expression
+    end
+    return if predicates.empty?
+
+    any = params['match'] == 'any' || raw.split.any? { _1.downcase == 'type:or' }
+    @scope = @scope.where(Arel.sql("(#{ predicates.join(any ? ' OR ' : ' AND ') })"))
+  end
+end

backend/app/services/counts_query.rb

@@ -0,0 +1,207 @@
+# Read-only history queries. SQL identifiers and expressions are server-owned.
+class CountsQuery
+  AXES = ['month', 'user'].freeze
+  COMMON_COLUMNS = ['resource_id', 'version_no', 'is_latest', 'event_type', 'user', 'created_at'].freeze
+  COLUMNS = {
+      'posts' => COMMON_COLUMNS + [
+          'thumbnail', 'thumbnail_base', 'title', 'url', 'tags', 'parents', 'video_ms',
+          'original_created_from', 'original_created_before'],
+      'tags' => COMMON_COLUMNS + [
+          'name', 'names', 'category', 'aliases', 'parents', 'deprecated'] }.freeze
+  KEYS = [
+      'locale', 'mode', 'axis', 'page', 'limit', 'from', 'before', 'latest',
+      'user_id', 'resource_id', 'tag', 'columns', 'sort', 'at'].freeze
+
+  attr_reader :page, :limit, :mode, :axis
+
+  def initialize(scope, resource, params, latest_expression: nil)
+    @table = scope.klass.table_name
+    @resource = resource
+    @id_column = resource == 'posts' ? 'post_id' : 'tag_id'
+    @params = params
+    @latest_expression = latest_expression
+    scope = HistoryAccess.visible(scope)
+    @scope = scope.except(:select, :order).joins(
+        "LEFT JOIN users counts_actor ON counts_actor.id = #{ @table }.created_by_user_id")
+    raise ArgumentError unless (params.keys - KEYS).empty?
+    raise ArgumentError unless params.values.all? { _1.is_a?(String) }
+
+    @mode = params.fetch('mode', 'summary')
+    @axis = params.fetch('axis', 'month')
+    raise ArgumentError unless ['summary', 'details'].include?(mode) && AXES.include?(axis)
+
+    @page = integer('page', 1)
+    @limit = integer('limit', 20)
+    raise ArgumentError if limit > 100 || page > 1_000_000
+
+    columns = QueryState.list(params.fetch('columns', COLUMNS.fetch(resource).join(' ')))
+    if columns.empty? || columns.uniq != columns || (columns - COLUMNS.fetch(resource)).any?
+      raise ArgumentError
+    end
+    @sort = parse_sort
+    apply_filters
+  end
+
+  def relation = @scope
+
+  def count = @scope.count
+
+  def latest_expression
+    return @latest_expression if @latest_expression
+
+    at = timestamp('at')
+    cutoff = at ? "AND newer.created_at <= #{ @scope.klass.connection.quote(at) }" : ''
+    <<~SQL.squish
+      NOT EXISTS (SELECT 1 FROM #{ @table } newer
+        WHERE newer.#{ @id_column } = #{ @table }.#{ @id_column }
+          AND newer.version_no > #{ @table }.version_no #{ cutoff })
+    SQL
+  end
+
+  def details
+    @scope.select("#{ @table }.*", Arel.sql("(#{ latest_expression }) AS counts_latest"))
+        .order(Arel.sql(order_sql(detail_expressions, ["#{ @table }.id DESC"])))
+        .limit(limit).offset((page - 1) * limit)
+  end
+
+  def summary(include_count: true)
+    expression = axis == 'month' ? "DATE_FORMAT(#{ @table }.created_at, '%Y-%m')" :
+                                  "#{ @table }.created_by_user_id"
+    group_order = axis == 'user' ? 'MAX(counts_actor.name)' : expression
+    sql = @scope.group(Arel.sql(expression)).select(Arel.sql(
+        "#{ expression } AS group_key, COUNT(*) AS record_count, " +
+        "#{ group_order } AS group_name")).to_sql
+    connection = @scope.klass.connection
+    source = "FROM (#{ sql }) counts_groups"
+    groups = include_count ? connection.select_value("SELECT COUNT(*) #{ source }") : nil
+    expressions = { 'group' => ['group_name'], 'count' => ['record_count'] }
+    order = order_sql(expressions, ['group_key IS NULL ASC', 'group_key ASC'])
+    rows = connection.select_all(
+        "SELECT group_key, group_name, record_count #{ source } " +
+        "ORDER BY #{ order } LIMIT #{ limit } OFFSET #{ (page - 1) * limit }")
+    { group_count: groups&.to_i,
+      groups: rows.map do |row|
+        { key: row['group_key'],
+          name: axis == 'user' ? row['group_name'] : nil,
+          count: row['record_count'].to_i }
+      end }
+  end
+
+  private
+
+  def parse_sort
+    value = @params.fetch('sort', mode == 'details' ? 'created_at:desc' : 'group:asc')
+    return [] if value.empty?
+
+    values = QueryState.list(value).map { _1.split(':', -1) }
+    allowed = mode == 'details' ? COLUMNS.fetch(@resource) - ['thumbnail'] : ['group', 'count']
+    invalid = values.any? do |key, direction, *extra|
+      extra.any? || !allowed.include?(key) || !['asc', 'desc'].include?(direction)
+    end
+    if (values.empty? || values.length > allowed.length || invalid ||
+        values.map(&:first).uniq.length != values.length)
+      raise ArgumentError
+    end
+    values
+  end
+
+  # NULL always sorts last, including descending keys. The final key is unique.
+  def order_sql(expressions, final)
+    (@sort.flat_map do |key, direction|
+      expressions.fetch(key).flat_map do |expression|
+        ["(#{ expression }) IS NULL ASC", "#{ expression } #{ direction.upcase }"]
+      end
+    end + final).join(', ')
+  end
+
+  def detail_expressions
+    common = { 'resource_id' => ["#{ @table }.#{ @id_column }"],
+               'version_no' => ["#{ @table }.version_no"],
+               'is_latest' => ["(#{ latest_expression })"],
+               'user' => ['counts_actor.name', "#{ @table }.created_by_user_id"],
+               'created_at' => ["#{ @table }.created_at"] }
+    if @resource == 'posts'
+      fields = [
+          'title', 'url', 'thumbnail_base', 'video_ms',
+          'original_created_from', 'original_created_before']
+      common.merge(fields.to_h { [_1, ["#{ @table }.#{ _1 }"]] }).merge(
+          'tags' => json_order('post_versions.tags_json'),
+          'parents' => list_order('post_versions.parent_post_ids'))
+    else
+      name = "JSON_UNQUOTE(JSON_EXTRACT(tag_versions.names, #{ locale_name_path }))"
+      common.merge(
+          'name' => ["COALESCE(#{ name }, tag_versions.name)"],
+          'names' => json_order('tag_versions.names'),
+          'category' => ['tag_versions.category'],
+          'aliases' => json_order(
+              "JSON_EXTRACT(tag_versions.names, '$.*.aliases[*]')"),
+          'parents' => list_order('tag_versions.parent_tag_ids'),
+          'deprecated' => ['tag_versions.deprecated_at'])
+    end
+  end
+
+  def locale_name_path
+    language = Locale.find_by(code: @params.fetch('locale', 'ja'))&.language_code || 'ja'
+    path = "$.#{ JSON.generate(language) }.name"
+    @scope.klass.connection.quote(path)
+  end
+
+  # Multi-valued parent cells sort by cardinality, then binary snapshot text.
+  def json_order(column)
+    [
+        "COALESCE(JSON_LENGTH(#{ column }), 0)",
+        "COALESCE(CAST(#{ column } AS CHAR), '[]') COLLATE utf8mb4_bin"]
+  end
+
+  def list_order(column)
+    [
+        "CASE WHEN #{ column } = '' THEN 0 ELSE " +
+            "1 + LENGTH(#{ column }) - LENGTH(REPLACE(#{ column }, ' ', '')) END",
+        "#{ column } COLLATE utf8mb4_bin"]
+  end
+
+  def integer(key, default = nil)
+    value = @params[key].presence
+    return default if value.nil? && !default.nil?
+    raise ArgumentError unless value&.match?(/\A[1-9][0-9]{0,9}\z/)
+
+    value.to_i
+  end
+
+  def timestamp(key)
+    value = @params[key].presence
+    return nil unless value
+    pattern = /\A\d{4}-\d{2}-\d{2}T\d{2}:\d{2}(?::\d{2}(?:\.\d{1,6})?)?Z\z/
+    raise ArgumentError unless value.match?(pattern)
+
+    Date.iso8601(value.first(10))
+    Time.iso8601(value.length == 17 ? value.sub(/Z\z/, ':00Z') : value)
+  end
+
+  def apply_filters
+    at = timestamp('at')
+    first = timestamp('from')
+    last = timestamp('before')
+    raise ArgumentError if first && last && first >= last
+    raise ArgumentError unless ['0', '1'].include?(@params.fetch('latest', '1'))
+
+    @scope = @scope.where("#{ @table }.created_at <= ?", at) if at
+    @scope = @scope.where(Arel.sql(latest_expression)) unless @params['latest'] == '0'
+    @scope = @scope.where("#{ @table }.created_at >= ?", first) if first
+    @scope = @scope.where("#{ @table }.created_at < ?", last) if last
+    if @params['resource_id'].present?
+      @scope = @scope.where(@table => { @id_column => integer('resource_id') })
+    end
+    if @params['user_id'].present?
+      user = @params['user_id'] == 'none' ? nil : integer('user_id')
+      @scope = @scope.where(@table => { created_by_user_id: user })
+    end
+    return unless @params['tag'].present?
+    raise ArgumentError unless @resource == 'posts'
+
+    id = integer('tag')
+    @scope = @scope.where(
+        "JSON_CONTAINS(post_versions.tags_json, JSON_OBJECT('tag_id', CAST(? AS UNSIGNED))) OR " +
+        "JSON_CONTAINS(prev.tags_json, JSON_OBJECT('tag_id', CAST(? AS UNSIGNED)))", id, id)
+  end
+end

backend/app/services/counts_resource_definition.rb

@@ -0,0 +1,65 @@
+# Server-owned version identities, attributes and field types. No client SQL is accepted.
+class CountsResourceDefinition
+  COMMON = { 'resource_id' => :integer, 'version_no' => :integer,
+             'event_type' => :enum, 'user' => :integer, 'created_at' => :datetime }.freeze
+  DEFINITIONS = {
+      'posts' => [PostVersion, ['post_id'], {
+          'title' => :string, 'url' => :string, 'thumbnail_base' => :string,
+          'tags_json' => :json, 'parent_post_ids' => :string, 'video_ms' => :integer,
+          'original_created_from' => :datetime, 'original_created_before' => :datetime }],
+      'tags' => [TagVersion, ['tag_id'], {
+          'name' => :string, 'names' => :json, 'category' => :enum, 'aliases' => :string,
+          'parent_tag_ids' => :string, 'deprecated_at' => :datetime }],
+      'nico_tags' => [NicoTagVersion, ['tag_id'], {
+          'name' => :string, 'linked_tags' => :string }],
+      'tag_descriptions' => [TagDescriptionVersion, ['tag_id', 'language_code'], {
+          'language_code' => :enum, 'body' => :string,
+          'source_language_code' => :enum, 'source_version_no' => :integer }],
+      'tag_groups' => [TagGroupVersion, ['tag_group_id'], {
+          'name' => :string, 'description' => :string,
+          'members' => :json, 'placements' => :json }],
+      'materials' => [MaterialVersion, ['material_id'], {
+          'url' => :string, 'parent_id' => :integer, 'tag_id' => :integer,
+          'tag_name' => :string, 'tag_category' => :enum,
+          'discarded_at' => :datetime, 'updated_at' => :datetime,
+          'updated_by_user_id' => :integer, 'export_paths_json' => :json,
+          'file_blob_id' => :integer, 'file_filename' => :string,
+          'file_content_type' => :string, 'file_byte_size' => :integer,
+          'file_checksum' => :string, 'file_sha256' => :string,
+          'source_kind' => :enum, 'source_uri' => :string, 'source_path' => :string,
+          'source_file_id' => :string, 'normalized_source_key' => :string }] }.freeze
+  OPERATORS = {
+      string: ['eq', 'ne', 'like'], enum: ['eq', 'ne'], boolean: ['eq', 'ne'],
+      integer: ['eq', 'ne', 'gt', 'lt', 'ge', 'le', 'range'],
+      datetime: ['eq', 'ne', 'gt', 'lt', 'ge', 'le', 'range'] }.freeze
+
+  attr_reader :model, :keys, :fields, :resource
+
+  def initialize(resource)
+    @resource = resource
+    @model, @keys, attributes = DEFINITIONS.fetch(resource) { raise ArgumentError }
+    @fields = COMMON.merge(attributes)
+  end
+
+  def table = model.table_name
+  def time_column = 'created_at'
+  def column(key)
+    raise ArgumentError unless fields.key?(key)
+
+    { 'resource_id' => keys.first, 'user' => 'created_by_user_id' }.fetch(key, key)
+  end
+
+  def expression(key, row = 'v') = "#{ row }.#{ column(key) }"
+  def changes = fields.keys - COMMON.keys - ['updated_at', 'updated_by_user_id']
+  def axes(history)
+    fields.reject { |_key, type| type == :json }.keys +
+        (resource == 'posts' ? ['tags'] : []) + (history ? ['changed_field'] : [])
+  end
+
+  def metadata
+    fields.map do |key, type|
+      { key:, type:, nullable: model.columns_hash.fetch(column(key)).null,
+        operators: OPERATORS.fetch(type, []) }
+    end
+  end
+end

backend/app/services/counts_state_query.rb

@@ -0,0 +1,370 @@
+# Restores one recorded version per resource before applying record filters.
+class CountsStateQuery
+  KEYS = ['locale', 'resource', 'mode', 'at', 'from', 'before', 'grain',
+          'kind', 'tag', 'page', 'limit', 'sort', 'axis', 'user_id', 'resource_id'].freeze
+  MAX_PERIODS = 366
+
+  attr_reader :at
+
+  def initialize(params, language_code:)
+    raise ArgumentError unless (params.keys - KEYS).empty?
+    raise ArgumentError unless params.values.all? { _1.is_a?(String) }
+
+    @params = params
+    @connection = ApplicationRecord.connection
+    @language = language_code
+    @resource = params.fetch('resource', 'posts')
+    @mode = params.fetch('mode', 'summary')
+    @inclusive = ['summary', 'details'].include?(@mode)
+    @axis = params.fetch('axis', @resource == 'posts' ? 'tags' : 'month')
+    @kind = params.fetch('kind', 'internal')
+    @grain = params.fetch('grain', 'month')
+    raise ArgumentError unless ['posts', 'tags'].include?(@resource)
+    raise ArgumentError unless ['summary', 'details', 'state', 'trend', 'snapshot'].include?(@mode)
+    raise ArgumentError unless ['internal', 'external', 'all'].include?(@kind)
+    raise ArgumentError unless ['day', 'month', 'year'].include?(@grain)
+
+    raise ArgumentError unless ['tags', 'month', 'user'].include?(@axis)
+    if @mode == 'summary' && @axis == 'tags' && @resource != 'posts'
+      raise ArgumentError
+    end
+
+    @page = integer('page', 1)
+    @limit = integer('limit', 20)
+    @tag = integer('tag') if params['tag'].present?
+    raise ArgumentError if @limit > 100 || @page > 1_000_000
+    raise ArgumentError if @tag && @kind == 'all'
+    raise ArgumentError if @mode == 'trend' && !@tag
+
+    @now = Time.current.utc
+    @at = timestamp('at') || @now
+    raise ArgumentError if @at > @now
+  end
+
+  def call(metadata: true)
+    @metadata = metadata
+    # Keep totals, rows, and detail pages on one repeatable-read database snapshot.
+    ApplicationRecord.transaction do
+      unless @history_loaded
+        @post_start = HistoryAccess.visible(PostVersion.all).minimum(:created_at)
+        @tag_start = HistoryAccess.visible(TagVersion.all).minimum(:created_at)
+        @history_loaded = true
+      end
+      result = { population_count: HistoryAccess.population_count(
+          @resource == 'posts' ? PostVersion : TagVersion), page: @page, limit: @limit, at: @at.iso8601(6),
+                 coverage: { basis: 'recorded_snapshots', exact: false,
+                             post_history_from: @post_start&.iso8601(6),
+                             tag_history_from: @tag_start&.iso8601(6) } }
+      case @mode
+      when 'trend'
+        result.merge(trend)
+      when 'summary'
+        result.merge(metadata ? { totals: totals } : {}).merge(summary)
+      when 'details', 'snapshot'
+        result.merge(metadata ? { totals: totals } : {}).merge(details)
+      else
+        result.merge(metadata ? { totals: totals } : {}).merge(listing)
+      end
+    end
+  end
+
+  def each_page
+    return enum_for(:each_page) unless block_given?
+
+    @page = 1
+    first = call
+    yield first
+    total = ['summary', 'state', 'trend'].include?(@mode) ? first[:group_count] : first[:count]
+    while @page * @limit < total
+      @page += 1
+      yield call(metadata: false)
+    end
+  end
+
+  def validate_query!
+    filtered_scope(@resource)
+    case @mode
+    when 'summary'
+      if @axis == 'tags'
+        order_sql({ 'name' => 'name', 'count' => 'post_count', 'tag_id' => 'tag_id',
+                    'kind' => 'kind', 'deprecated' => 'deprecated_at' }, 'count:desc', [])
+      else
+        record_query('summary')
+      end
+    when 'details', 'snapshot'
+      record_query('details')
+    end
+    self
+  end
+
+  private
+
+  def quote(value) = @connection.quote(value)
+
+  def integer(key, default = nil)
+    value = @params[key]
+    return default if value.blank? && default
+    raise ArgumentError unless value&.match?(/\A[1-9][0-9]{0,9}\z/)
+
+    value.to_i
+  end
+
+  def timestamp(key)
+    value = @params[key].presence
+    return nil unless value
+    pattern = /\A\d{4}-\d{2}-\d{2}T\d{2}:\d{2}(?::\d{2}(?:\.\d{1,6})?)?Z\z/
+    raise ArgumentError unless value.match?(pattern)
+
+    Date.iso8601(value.first(10))
+    Time.iso8601(value.length == 17 ? value.sub(/Z\z/, ':00Z') : value).utc
+  end
+
+  # Version numbers order equal-timestamp writes. Do not filter tags before this selection.
+  def latest_condition(table, row, boundary)
+    key = table == 'post_versions' ? 'post_id' : 'tag_id'
+    <<~SQL.squish
+      #{ row }.created_at #{ @inclusive ? '<=' : '<' } #{ boundary } AND NOT EXISTS (
+        SELECT 1 FROM #{ table } newer
+        WHERE newer.#{ key } = #{ row }.#{ key }
+          AND newer.created_at #{ @inclusive ? '<=' : '<' } #{ boundary }
+          AND newer.version_no > #{ row }.version_no)
+    SQL
+  end
+
+  def state_sql(table, boundary, live: false)
+    model = { 'post_versions' => PostVersion, 'tag_versions' => TagVersion,
+              'nico_tag_versions' => NicoTagVersion }.fetch(table)
+    sql = "SELECT v.* FROM #{ table } v WHERE #{ latest_condition(table, 'v', boundary) } " +
+        "AND (#{ HistoryAccess.sql(model, 'v') })"
+    live ? "#{ sql } AND v.event_type <> 'discard'" : sql
+  end
+
+  def membership(row)
+    key = @kind == 'external' ? 'external_tag_id' : 'tag_id'
+    "JSON_CONTAINS(#{ row }.tags_json, JSON_OBJECT('#{ key }', #{ @tag }))"
+  end
+
+  # Deprecation keeps a tag in existence. Only an explicit discard removes it.
+  def tag_visible(boundary)
+    return '1 = 1' if @kind == 'external'
+
+    <<~SQL.squish
+      NOT EXISTS (SELECT 1 FROM tag_versions t
+        WHERE t.tag_id = #{ @tag } AND t.event_type = 'discard'
+          AND #{ latest_condition('tag_versions', 't', boundary) })
+    SQL
+  end
+
+  def ctes
+    boundary = quote(@at)
+    <<~SQL
+      WITH post_at AS (#{ filtered_scope('posts').to_sql }),
+      tag_at AS (#{ state_sql('tag_versions', boundary) }),
+      nico_at AS (#{ state_sql('nico_tag_versions', boundary) }),
+      pairs AS (
+        SELECT DISTINCT p.post_id,
+          CASE WHEN entry.tag_id IS NULL THEN 'external' ELSE 'internal' END AS kind,
+          COALESCE(entry.tag_id, entry.external_tag_id) AS tag_id
+        FROM post_at p
+        JOIN JSON_TABLE(CAST(p.tags_json AS JSON), '$[*]' COLUMNS (
+          tag_id BIGINT PATH '$.tag_id' NULL ON EMPTY ERROR ON ERROR,
+          external_tag_id BIGINT PATH '$.external_tag_id' NULL ON EMPTY ERROR ON ERROR
+        )) entry
+        LEFT JOIN tag_at t ON t.tag_id = entry.tag_id
+        WHERE COALESCE(entry.tag_id, entry.external_tag_id) IS NOT NULL
+          AND (t.event_type IS NULL OR t.event_type <> 'discard')
+      )
+    SQL
+  end
+
+  def totals
+    row = @connection.select_one(<<~SQL)
+      #{ ctes }
+      SELECT (SELECT COUNT(*) FROM post_at) AS posts,
+        (SELECT COUNT(*) FROM (#{ filtered_scope('tags').to_sql }) counted_tags) AS tags,
+        (SELECT COUNT(*) FROM tag_at WHERE event_type <> 'discard'
+          AND deprecated_at IS NOT NULL) AS deprecated_tags,
+        (SELECT COUNT(*) FROM pairs) AS assignments,
+        (SELECT COUNT(*) FROM pairs WHERE kind = 'external') AS external_assignments
+    SQL
+    row.transform_values!(&:to_i)
+    row.merge('posts' => available?(@post_start, @at) ? row['posts'] : nil,
+              'assignments' => available?(@post_start, @at) ? row['assignments'] : nil,
+              'external_assignments' => available?(@post_start, @at) ?
+                  row['external_assignments'] : nil,
+              'tags' => available?(@tag_start, @at) ? row['tags'] : nil,
+              'deprecated_tags' => available?(@tag_start, @at) ? row['deprecated_tags'] : nil)
+  end
+
+  def available?(first, boundary)
+    first && (@inclusive ? first <= boundary : first < boundary)
+  end
+
+  def listing
+    path = quote("$.#{ JSON.generate(@language) }.name")
+    catalog = <<~SQL
+      , tag_counts AS (
+        SELECT membership.kind, membership.tag_id, COUNT(*) AS post_count
+        FROM pairs membership GROUP BY membership.kind, membership.tag_id
+      ), catalog AS (
+        SELECT 'internal' AS kind, tag_id,
+          COALESCE(JSON_UNQUOTE(JSON_EXTRACT(names, #{ path })), name) AS name,
+          deprecated_at FROM tag_at WHERE event_type <> 'discard'
+        UNION ALL
+        SELECT tc.kind, tc.tag_id, external_version.name, NULL AS deprecated_at
+        FROM tag_counts tc
+        LEFT JOIN tag_at internal_version
+          ON tc.kind = 'internal' AND internal_version.tag_id = tc.tag_id
+        LEFT JOIN nico_at external_version
+          ON tc.kind = 'external' AND external_version.tag_id = tc.tag_id
+        WHERE tc.kind = 'external' OR internal_version.tag_id IS NULL
+      )
+    SQL
+    filters = []
+    filters << "c.kind = #{ quote(@kind) }" unless @kind == 'all'
+    filters << "c.tag_id = #{ @tag }" if @tag
+    where = filters.any? ? "WHERE #{ filters.join(' AND ') }" : ''
+    source = 'FROM catalog c LEFT JOIN tag_counts n ON n.kind = c.kind AND n.tag_id = c.tag_id'
+    prefix = "#{ ctes } #{ catalog }"
+    total = if @metadata
+              @connection.select_value("#{ prefix } SELECT COUNT(*) #{ source } #{ where }").to_i
+            end
+    order = order_sql({ 'name' => 'c.name', 'count' => 'COALESCE(n.post_count, 0)',
+                        'tag_id' => 'c.tag_id', 'kind' => 'c.kind',
+                        'deprecated' => 'c.deprecated_at' }, 'count:desc',
+                      ['c.kind ASC', 'c.tag_id ASC'])
+    rows = @connection.select_all(<<~SQL)
+      #{ prefix }
+      SELECT c.kind, c.tag_id, c.name, c.deprecated_at,
+        COALESCE(n.post_count, 0) AS post_count #{ source } #{ where }
+      ORDER BY #{ order } LIMIT #{ @limit } OFFSET #{ (@page - 1) * @limit }
+    SQL
+    { count: total, group_count: total, state_rows: rows.map do |row|
+        { kind: row['kind'], tag_id: row['tag_id'], name: row['name'],
+          deprecated: row['deprecated_at']&.iso8601,
+          count: available?(@post_start, @at) ? row['post_count'].to_i : nil,
+          at: @at.iso8601(6) }
+      end }
+  end
+
+  # All record filters apply after selecting the latest recorded version at the point.
+  def restored_scope(resource)
+    model = resource == 'posts' ? PostVersion : TagVersion
+    table = model.table_name
+    scope = model.where(Arel.sql(latest_condition(table, table, quote(@at))))
+        .where.not(event_type: 'discard')
+    if @tag
+      scope = if resource == 'posts'
+                scope.where("#{ membership(table) } AND #{ tag_visible(quote(@at)) }")
+              elsif @kind == 'internal'
+                scope.where(tag_id: @tag)
+              else
+                scope.none
+              end
+    end
+    scope
+  end
+
+  def filtered_scope(resource)
+    params = @params.slice('from', 'before', 'user_id', 'resource_id')
+        .merge('mode' => 'details', 'latest' => '0', 'sort' => '')
+    CountsQuery.new(restored_scope(resource), resource, params,
+                    latest_expression: '1').relation
+  end
+
+  def record_query(mode)
+    params = @params.slice('locale', 'sort', 'page', 'limit')
+        .merge('mode' => mode, 'latest' => '0', 'axis' => @axis == 'tags' ? 'month' : @axis,
+               'page' => @page.to_s, 'limit' => @limit.to_s)
+    CountsQuery.new(filtered_scope(@resource), @resource, params, latest_expression: '1')
+  end
+
+  def summary
+    return listing if @axis == 'tags'
+
+    query = record_query('summary')
+    { count: @metadata ? query.count : nil }.merge(query.summary(include_count: @metadata))
+  end
+
+  def details
+    query = record_query('details')
+    { count: @metadata ? query.count : nil,
+      versions: CountsVersionRepr.many(query.details, resource: @resource,
+                                      language_code: @language, at: @at,
+                                      inclusive: @inclusive) }
+  end
+
+  def next_boundary(time)
+    case @grain
+    when 'day' then time.beginning_of_day + 1.day
+    when 'month' then time.beginning_of_month.next_month
+    else time.beginning_of_year.next_year
+    end
+  end
+
+  def periods
+    @before = timestamp('before') || @now.beginning_of_month
+    @from = timestamp('from') || @before.prev_year
+    raise ArgumentError if @from >= @before || @before > @now
+
+    rows = []
+    boundary = next_boundary(@from)
+    while boundary <= @before
+      raise ArgumentError if rows.length >= MAX_PERIODS
+
+      format = { 'day' => '%Y-%m-%d', 'month' => '%Y-%m', 'year' => '%Y' }.fetch(@grain)
+      rows << [boundary, (boundary - 1.second).strftime(format)]
+      boundary = next_boundary(boundary)
+    end
+    rows
+  end
+
+  def trend
+    points = periods
+    metadata = { count: points.length, group_count: points.length, state_rows: [],
+                 from: @from.iso8601, before: @before.iso8601, grain: @grain }
+    return metadata if points.empty?
+
+    values = points.map do |boundary, period|
+      "SELECT CAST(#{ quote(boundary) } AS DATETIME(6)) AS boundary, " +
+          "#{ quote(period) } AS period"
+    end.join(' UNION ALL ')
+    order = order_sql({ 'period' => 'boundary', 'count' => 'post_count' },
+                      'period:asc', ['boundary ASC'])
+    known = @post_start ? "periods.boundary > #{ quote(@post_start) }" : 'FALSE'
+    rows = @connection.select_all(<<~SQL)
+      WITH periods AS (#{ values }), series AS (
+        SELECT periods.boundary, periods.period,
+          CASE WHEN #{ known } THEN COUNT(DISTINCT p.post_id) ELSE NULL END AS post_count
+        FROM periods
+        LEFT JOIN post_versions p ON p.event_type <> 'discard'
+          AND #{ membership('p') }
+          AND #{ latest_condition('post_versions', 'p', 'periods.boundary') }
+          AND #{ tag_visible('periods.boundary') }
+        GROUP BY periods.boundary, periods.period
+      )
+      SELECT * FROM series ORDER BY #{ order }
+      LIMIT #{ @limit } OFFSET #{ (@page - 1) * @limit }
+    SQL
+    metadata.merge(state_rows: rows.map do |row|
+      { kind: @kind, tag_id: @tag, name: nil, period: row['period'],
+        at: row['boundary'].utc.iso8601(6), count: row['post_count']&.to_i }
+    end)
+  end
+
+  def order_sql(expressions, default, final)
+    raw = @params.fetch('sort', default)
+    sorts = raw.empty? ? [] : QueryState.list(raw).map { _1.split(':', -1) }
+    raise ArgumentError if sorts.map(&:first).uniq.length != sorts.length
+    raise ArgumentError if sorts.length > expressions.length
+
+    order = sorts.flat_map do |key, direction, *extra|
+      if extra.any? || !expressions.key?(key) || !['asc', 'desc'].include?(direction)
+        raise ArgumentError
+      end
+      expression = expressions.fetch(key)
+      ["(#{ expression }) IS NULL ASC", "#{ expression } #{ direction.upcase }"]
+    end
+    (order + final).join(', ')
+  end
+end

backend/app/services/counts_table_export.rb

@@ -0,0 +1,308 @@
+require 'csv'
+
+# Export uses the exact same population and grouped cells as the interactive table.
+class CountsTableExport
+  EXTRA = ['format', 'columns', 'headers', 'labels', 'children', 'child_sort',
+           'order', 'nested_json', 'summary_columns', 'basis', 'axis', 'latest'].freeze
+  attr_reader :format, :resource
+
+  def initialize(params)
+    QueryState.validate!(params, CountsTableQuery::KEYS + EXTRA + ['measures'])
+    raise ArgumentError unless params.values.all? { _1.is_a?(String) }
+
+    params = CountsDefaults.for(params).merge(params)
+    unless params.key?('children')
+      params = params.merge('children' => CountsExportProjection.default_children(
+          params.fetch('resource', 'posts')).join(' '))
+    end
+    @params = params
+    @labels = JSON.parse(params.fetch('labels', '{}'))
+    raise ArgumentError unless @labels.is_a?(Hash)
+    @selected = params.key?('children') ? QueryState.list(params['children']) : nil
+    @order = QueryState.list(params.fetch('order', ''))
+    @child_sort = QueryState.list(params.fetch('child_sort', '')).map { _1.split(':', -1) }
+    @child_sort.each do |path, direction, *extra|
+      raise ArgumentError if extra.any? || !['asc', 'desc'].include?(direction)
+      raise ArgumentError unless path&.match?(/\A[a-zA-Z_]+(?:\.[a-zA-Z]+)*\/[a-zA-Z]+\z/)
+    end
+    @format = params.fetch('format', 'json')
+    @resource = params.fetch('resource', 'posts')
+    raise ArgumentError unless ['json', 'csv'].include?(format)
+
+    @query = CountsTableQuery.new(params.select { |key, _value| CountsTableQuery::KEYS.include?(key) ||
+        key == 'measures' || QueryState.valid_key?(key) }
+        .merge('page' => '1', 'limit' => '100'))
+    @summary = @query.mode == 'summary'
+    default_columns = @summary ? @query.axes.all.map { _1.fetch('field') } + ['count'] :
+        CountsExportProjection.default_columns(resource)
+    @columns = QueryState.list(params.fetch('columns', default_columns.join(' ')))
+    default_headers = @columns.map { @labels.fetch(_1, _1) }
+    @headers = JSON.parse(params.fetch('headers', default_headers.to_json))
+    raise ArgumentError unless @headers.is_a?(Array) && @headers.all? { _1.is_a?(String) }
+    if @summary && @headers.any? && @headers.length != @query.axes.all.length + 1
+      raise ArgumentError
+    end
+
+    unless @summary
+      if ['posts', 'tags'].include?(resource)
+        @projection = CountsExportProjection.new(params, resource:, summary: false, tags: false)
+      else
+        allowed = @query.population.definition.fields.keys + ['is_latest']
+        raise ArgumentError if (@columns - allowed).any?
+        raise ArgumentError unless @columns.length == @headers.length && @columns.uniq == @columns
+      end
+    end
+  end
+
+  def body(progress: nil)
+    Enumerator.new do |output|
+      ApplicationRecord.transaction(isolation: :repeatable_read) do
+        @language_names = Language.pluck(:code, :name).to_h
+        processed = 0
+        started = false
+        first = true
+        @query.each_page do |page|
+          total = @summary ? nil : page[:count]
+          progress&.call(processed, total)
+          unless started
+            metadata = page.except(:versions, :cells, :axis_labels, :column_values,
+                :row_headers, :column_headers, :pivot_totals).merge(
+                resource:, filters: @params.select { |key, _value| CountsTableQuery::KEYS.include?(key) ||
+        key == 'measures' || QueryState.valid_key?(key) },
+                fields: @query.population.definition.metadata,
+                columns: @columns, column_labels: @headers,
+                order: @params['order'], children: @params['children'],
+                sort: @params.fetch('sort', ''), child_sort: @params['child_sort'])
+            output << (format == 'json' ? "{\"metadata\":#{ metadata.to_json },\"data\":[" : "\uFEFF")
+            if format == 'csv'
+              headings = @summary ? pivot_headers(page) : [@headers]
+              headings.each do |headers|
+                output << CSV.generate_line(headers.map { safe(_1) }, row_sep: "\r\n")
+              end
+            end
+            started = true
+          end
+          if @summary
+            pivot_rows(page).each do |header|
+              records = page.fetch(:column_headers).map { pivot_cell(header, _1, page) }
+              if format == 'csv'
+                labels = pivot_row_labels(header, page)
+                values = labels + records.map { _1[:count] }
+                output << CSV.generate_line(values.map { safe(_1) }, row_sep: "\r\n")
+                processed += records.length
+                progress&.call(processed, nil)
+              else
+                records.each do |record|
+                  output << ',' unless first
+                  output << record.to_json
+                  first = false
+                  processed += 1
+                  progress&.call(processed, nil)
+                end
+              end
+            end
+            next
+          end
+          page[:versions].each do |row|
+            record = project(row)
+            if format == 'json'
+              output << ',' unless first
+              output << record.to_json
+            else
+              values = if @summary
+                         row[:values].each_with_index.map do |value, index|
+                           summary_value(value, index, page)
+                         end + [row[:count]]
+                       elsif @projection
+                         @projection.csv_record(record, row)
+                       else
+                         record.map { |key, value| csv_logical(key, value) }
+                       end
+              output << CSV.generate_line(values.map { safe(_1) }, row_sep: "\r\n")
+            end
+            first = false
+            processed += 1
+            progress&.call(processed, total)
+          end
+        end
+        progress&.call(processed, processed)
+        output << ']}' if format == 'json'
+      end
+    end
+  end
+
+  private
+
+  def pivot_rows(page)
+    page.fetch(:row_headers).reject do |header|
+      header[:kind] == 'grand_total' && @query.page * @query.limit < page[:group_count]
+    end
+  end
+
+  def total_label(header)
+    key = header[:kind] == 'grand_total' ?
+        (@query.axes.all.length == 1 ? 'singleTotal' : 'grandTotal') : 'subtotal'
+    @labels.fetch(key, '')
+  end
+
+  def pivot_row_labels(header, page)
+    values = header[:values].each_with_index.map { |value, index|
+      summary_value(value, index, page)
+    }
+    if header[:kind] == 'data' && @previous_pivot_row&.fetch(:kind) == 'data'
+      values.each_index do |index|
+        if header[:values].first(index + 1) == @previous_pivot_row[:values].first(index + 1)
+          values[index] = ''
+        end
+      end
+    end
+    @previous_pivot_row = header
+    values << total_label(header) unless header[:kind] == 'data'
+    leading = [@query.axes.rows.length, @query.axes.columns.any? ? 1 : 0].max
+    Array.new(leading) { values[_1] || '' }
+  end
+
+  def pivot_headers(page)
+    row_count = @query.axes.rows.length
+    cols = @query.axes.columns
+    leading = [row_count, cols.any? ? 1 : 0].max
+    columns = page.fetch(:column_headers)
+    Array.new([cols.length, 1].max) do |level|
+      prefix = Array.new(leading) { level.zero? ? @headers[_1].to_s : '' }
+      prefix = [''] if row_count.zero? && cols.any?
+      prefix + columns.each_with_index.map do |header, index|
+        values = header[:values]
+        if cols.empty?
+          header[:kind] == 'data' ? @labels.fetch('count', '') : total_label(header)
+        elsif level == values.length && header[:kind] != 'data'
+          total_label(header)
+        elsif level < values.length
+          previous = index.positive? ? columns[index - 1][:values] : nil
+          if level < cols.length - 1 && previous && previous.first(level + 1) == values.first(level + 1)
+            ''
+          else
+            summary_value(values[level], row_count + level, page)
+          end
+        else
+          ''
+        end
+      end
+    end
+  end
+
+  def pivot_cell(row, column, page)
+    @pivot_lookup ||= {}
+    # Discard the previous page's lookup; memory remains bounded by one page and its rollups.
+    unless @pivot_page.equal?(page)
+      @pivot_page = page
+      @pivot_lookup = page.fetch(:cells).to_h do |cell|
+        values = cell[:values]
+        [[values.first(@query.axes.rows.length), values.drop(@query.axes.rows.length)], cell[:count]]
+      end
+      page.fetch(:pivot_totals).each { @pivot_lookup[[_1[:row], _1[:column]]] = _1[:count] }
+    end
+    kinds = [row[:kind], column[:kind]]
+    kind = if kinds.include?('grand_total')
+             'grand_total'
+           elsif kinds.include?('subtotal')
+             'subtotal'
+           else
+             'data'
+           end
+    { row: row[:values], column: column[:values], kind:,
+      row_kind: row[:kind], column_kind: column[:kind],
+      row_depth: row[:values].length, column_depth: column[:values].length,
+      count: @pivot_lookup.fetch([row[:values], column[:values]], 0) }
+  end
+
+  def summary_value(value, index, page)
+    field = @query.axes.all.fetch(index).fetch('field')
+    return @labels.fetch('bot', '') if field == 'user' && value.nil?
+
+    names = page[:axis_labels].fetch("a#{ index }", {})
+    name = field == 'user' ? names[value.to_s] : names.fetch(value.to_s, value)
+    if field == 'user'
+      name = @labels.fetch('notRegistered', '') if name.nil? || name == ''
+      pattern = @labels.fetch('actorFormat', '{{name}} (#{{id}})')
+      return pattern.gsub(/\{\{(name|id)\}\}/) do
+        Regexp.last_match(1) == 'name' ? name.to_s : value.to_s
+      end
+    end
+    @labels.fetch("#{ field }:#{ value }", name)
+  end
+
+  def project(row)
+    return { row: row[:values].first(@query.axes.rows.length),
+             column: row[:values].drop(@query.axes.rows.length), count: row[:count] } if @summary
+    return @projection.project(row) if @projection
+
+    @columns.to_h do |column|
+      key = column == 'user' ? 'created_by_user' : column
+      value = row[key]
+      value = project_nested(value, key) if ['members', 'placements', 'export_paths_json'].include?(key)
+      [key, value]
+    end
+  end
+
+  def camel(key)
+    key.gsub(/_([a-z])/) { Regexp.last_match(1).upcase }
+  end
+
+  def project_nested(value, path)
+    rows = path == 'export_paths_json' ? (value || {}).map { |profile, target|
+      { 'profile' => profile, 'path' => target }
+    } : Array(value)
+    sorts = @child_sort.filter_map do |key, direction|
+      parent, field = key.split('/', 2)
+      [field, direction] if parent == path
+    end
+    rows = rows.each_with_index.sort do |(first, a), (last, b)|
+      compared = 0
+      sorts.each do |field, direction|
+        left = first.find { |key, _| camel(key) == field }&.last
+        right = last.find { |key, _| camel(key) == field }&.last
+        compared = if left.nil? || right.nil?
+                     left.nil? == right.nil? ? 0 : (left.nil? ? 1 : -1)
+                   else
+                     (left.to_s <=> right.to_s) * (direction == 'asc' ? 1 : -1)
+                   end
+        break unless compared.zero?
+      end
+      compared.zero? ? a <=> b : compared
+    end.map(&:first)
+    rows.map do |row|
+      fields = row.keys.select { !@selected || @selected.include?("#{ path }.#{ camel(_1) }") }
+      ordered = (@order + fields.map { "#{ path }.#{ camel(_1) }" }).uniq
+          .filter_map { |item| fields.find { "#{ path }.#{ camel(_1) }" == item } }
+      ordered.to_h do |key|
+        item = row[key]
+        item = project_nested(item, "#{ path }.#{ camel(key) }") if item.is_a?(Array)
+        [key, item]
+      end
+    end
+  end
+
+  def csv_logical(key, value)
+    return @labels.fetch('bot', '') if key == 'created_by_user' && value.nil?
+    return value ? @labels.fetch('latest', '') : '' if key == 'is_latest'
+    if value.is_a?(Time) || value.is_a?(ActiveSupport::TimeWithZone)
+      return @query.population.timezone.utc_to_local(value.utc).iso8601
+    end
+    if key == 'language_code' || key == 'source_language_code'
+      return @language_names[value]
+    end
+    if value.is_a?(Hash)
+      return value.to_h { |field, item| [@labels.fetch(camel(field.to_s), ''), csv_logical(field, item)] }
+    end
+    return value.map { csv_logical(key, _1) } if value.is_a?(Array)
+
+    @labels.fetch("#{ key }:#{ value }", value)
+  end
+
+  def safe(value)
+    return value if value.is_a?(Numeric)
+
+    text = value.is_a?(Hash) || value.is_a?(Array) ? JSON.generate(value) : value.to_s
+    text.match?(/\A\s*[=+@-]/) ? "'#{ text }" : text
+  end
+end

backend/app/services/counts_table_query.rb

@@ -0,0 +1,316 @@
+# Common entrypoint for six version resources; legacy endpoints keep their contracts.
+class CountsTableQuery
+  MAX_COLUMNS = 100
+  class TooManyColumns < ArgumentError; end
+
+  KEYS = ['resource', 'definition', 'mode', 'at', 'execution_at', 'from', 'through',
+          'timezone', 'rows', 'cols', 'conditions', 'drill', 'tags', 'tag_refs', 'match',
+          'tag', 'kind', 'resource_id', 'user_id', 'page', 'limit', 'sort', 'locale', 'measures', 'column_order'].freeze
+  attr_reader :population, :axes, :page, :limit, :mode
+
+  def initialize(params)
+    QueryState.validate!(params, KEYS + ['measures'])
+    raise ArgumentError unless params.values.all? { _1.is_a?(String) }
+
+    params = CountsDefaults.for(params).slice(*KEYS).merge(params)
+    @params = params
+    @population = CountsPopulation.new(params)
+    @definition = population.definition
+    @connection = @definition.model.connection
+    @axes = CountsAxes.new(population)
+    @page = Integer(params.fetch('page', '1'), 10)
+    @limit = Integer(params.fetch('limit', '20'), 10)
+    @mode = params.fetch('mode', 'summary')
+    raise ArgumentError unless page.between?(1, 1_000_000) && limit.between?(1, 100)
+    raise ArgumentError unless ['summary', 'details'].include?(mode)
+
+    @sort = QueryState.list(params.fetch('sort', '')).map { _1.split(':', -1) }
+    allowed = mode == 'summary' ? axes.all.each_index.map { "a#{ _1 }" } + ['count'] :
+        @definition.fields.keys + ['is_latest'] +
+        (@definition.resource == 'posts' ? ['tags', 'parents'] : []) +
+        (@definition.resource == 'tags' ? ['parents', 'deprecated'] : [])
+    raise ArgumentError if @sort.map(&:first).uniq.length != @sort.length
+    @sort.each do |key, direction, *extra|
+      measure = mode == 'summary' && key&.match?(/\Am[0-9]{1,2}\z/)
+      measure_values(key) if measure
+      if extra.any? || (!allowed.include?(key) && !measure) || !['asc', 'desc'].include?(direction)
+        raise ArgumentError
+      end
+    end
+  end
+
+  def call
+    ApplicationRecord.transaction do
+      metadata.merge(mode == 'summary' ? summary : details)
+    end
+  end
+
+  def metadata
+    { population_count_scope: 'all_recorded_versions', population_count: HistoryAccess.population_count(@definition.model),
+      page:, limit:, execution_at: population.at.iso8601(6), at: @params['at'].presence,
+      definition: population.history ? 'history' : 'latest',
+      timezone: population.timezone.identifier, rows: axes.rows, cols: axes.columns,
+      fields: @definition.metadata,
+      count: population.scope.except(:select).optimizer_hints('MAX_EXECUTION_TIME(10000)').count }
+  end
+
+  def summary
+    expressions = axes.expressions
+    selected = expressions.each_with_index.map do |expression, index|
+      "#{ expression } AS a#{ index }"
+    end
+    source = population.scope
+    source = source.joins(axes.joins) unless axes.joins.empty?
+    sql = if expressions.empty?
+            source.except(:select).select('COUNT(DISTINCT v.id) AS count').to_sql
+          else
+            source.except(:select).group(*expressions.map { Arel.sql(_1) })
+                .select(Arel.sql((selected + ['COUNT(DISTINCT v.id) AS count']).join(', ')))
+                .to_sql
+          end
+    prefix = "WITH cells AS (#{ sql })"
+    column_values = limited_columns(prefix)
+    row_keys = axes.rows.each_index.map { "a#{ _1 }" }
+    # Page whole row tuples so a pivot row is never split across display pages.
+    row_selection = row_keys.empty? ? '1 AS singleton' : row_keys.join(', ')
+    row_group = row_keys.empty? ? 'GROUP BY singleton' : "GROUP BY #{ row_keys.join(', ') }"
+    measure_joins = []
+    extras = []
+    row_sort = @sort.each_with_index.filter_map do |(key, direction), index|
+      if key == 'count' || key.match?(/\Am[0-9]+\z/)
+        values = key == 'count' ? [] : measure_values(key)
+        relation = measure_relation(values)
+        matches = row_keys.map { "grouped.#{ _1 } <=> sort#{ index }.#{ _1 }" }
+        condition = matches.empty? ? 'TRUE' : matches.join(' AND ')
+        measure_joins << "LEFT JOIN (#{ relation }) sort#{ index } ON #{ condition }"
+        extras << "sort#{ index }.count AS sort#{ index }"
+        "sort#{ index } IS NULL ASC, sort#{ index } #{ direction.upcase }"
+      elsif row_keys.include?(key)
+        "#{ key } IS NULL ASC, #{ key } #{ direction.upcase }"
+      end
+    end
+    order_rows = row_sort + row_keys.map { "#{ _1 } IS NULL, #{ _1 }" }
+    order_rows = ['singleton ASC'] if order_rows.empty?
+    grouped = "SELECT #{ row_selection } FROM cells #{ row_group }"
+    selection = (['grouped.*'] + extras).join(', ')
+    group_sql = "SELECT #{ selection } FROM (#{ grouped }) grouped #{ measure_joins.join(' ') }"
+    total = @connection.select_value(<<~SQL).to_i
+      #{ prefix }
+      SELECT /*+ MAX_EXECUTION_TIME(10000) */ COUNT(*) FROM (#{ group_sql }) row_groups
+    SQL
+    join = row_keys.map { "cells.#{ _1 } <=> page_rows.#{ _1 }" }.join(' AND ')
+    join = 'TRUE' if row_keys.empty?
+    order_cols = @sort.filter_map do |key, direction|
+      next unless key.match?(/\Aa[0-9]+\z/) && !row_keys.include?(key)
+
+      "cells.#{ key } #{ direction.upcase }"
+    end
+    order = ['page_rows.position'] + order_cols +
+        axes.all.each_index.map { "cells.a#{ _1 }" }
+    records = @connection.select_all(<<~SQL)
+      #{ prefix }, row_groups AS (#{ group_sql }),
+      numbered AS (SELECT row_groups.*,
+        ROW_NUMBER() OVER (ORDER BY #{ order_rows.join(', ') }) AS position FROM row_groups),
+      page_rows AS (SELECT * FROM numbered
+        WHERE position > #{ (page - 1) * limit } AND position <= #{ page * limit })
+      SELECT /*+ MAX_EXECUTION_TIME(10000) */ cells.* FROM cells JOIN page_rows ON #{ join }
+      ORDER BY #{ order.join(', ') }
+    SQL
+    next_row = @connection.select_one(<<~SQL)
+      #{ prefix }, row_groups AS (#{ group_sql }),
+      numbered AS (SELECT row_groups.*,
+        ROW_NUMBER() OVER (ORDER BY #{ order_rows.join(', ') }) AS position FROM row_groups)
+      SELECT /*+ MAX_EXECUTION_TIME(10000) */ * FROM numbered
+      WHERE position = #{ page * limit + 1 }
+    SQL
+    row_values = records.map { |record| row_keys.map { record[_1] } }.uniq
+    row_depths = CountsPivotLayout.depths(row_keys, @sort)
+    column_keys = axes.columns.each_index.map { "a#{ axes.rows.length + _1 }" }
+    column_sort = @sort
+    if @params.key?('column_order') && column_keys.any? &&
+        !(@sort.first&.first == 'count' || @sort.first&.first&.match?(/\Am[0-9]+\z/))
+      column_sort = [[column_keys.first, 'asc']] + @sort
+    end
+    column_depths = CountsPivotLayout.depths(column_keys, column_sort, measures: false)
+    row_headers = CountsPivotLayout.headers(row_values, row_depths,
+        next_values: next_row && row_keys.map { next_row[_1] }, grand: axes.rows.any?)
+    if axes.rows.empty? && axes.columns.any?
+      row_headers = [{ values: [], kind: 'grand_total' }]
+    end
+    column_headers = CountsPivotLayout.headers(column_values, column_depths,
+        grand: axes.columns.any?)
+    if axes.columns.empty? && axes.rows.any?
+      column_headers = [{ values: [], kind: 'grand_total' }]
+    end
+    totals = CountsPivotTotals.new(population, axes, row_headers, column_headers).call
+    { group_count: total, column_values:, row_headers:, column_headers:, pivot_totals: totals,
+      cells: records.map do |row|
+        { values: axes.all.each_index.map { row["a#{ _1 }"] }, count: row['count'].to_i }
+      end, axis_labels: axis_labels(records, column_values) }
+  end
+
+  def details
+    query = population.scope.select(Arel.sql(
+        "(#{ population.latest_sql }) AS counts_latest"))
+    order = sort_sql { detail_expression(_1) }
+    query = query.order(Arel.sql((order + ['v.id DESC']).join(', ')))
+        .limit(limit).offset((page - 1) * limit)
+    rows = query.to_a
+    if ['posts', 'tags'].include?(@definition.resource)
+      point = population.history ? population.at :
+          population.timestamp(@params['at']) || population.at
+      values = CountsVersionRepr.many(rows, resource: @definition.resource,
+          language_code: locale.language_code,
+          at: point)
+      values.each_with_index { |value, index| value[:event_type] = rows[index].event_type }
+    else
+      users = User.where(id: rows.map(&:created_by_user_id)).pluck(:id, :name).to_h
+      values = rows.map do |row|
+        attributes = @definition.fields.keys.to_h do |key|
+          [key, row.public_send(@definition.column(key))]
+        end
+        attributes.merge('id' => row.id, 'resource_id' => row[@definition.keys.first],
+            'is_latest' => row.attributes['counts_latest'].to_i == 1,
+            'created_by_user' => row.created_by_user_id &&
+                { id: row.created_by_user_id, name: users[row.created_by_user_id] })
+      end
+    end
+    { versions: values }
+  end
+
+  def each_page
+    total = nil
+    loop do
+      result = call
+      total ||= mode == 'summary' ? result[:group_count] : result[:count]
+      yield result
+      break if page * limit >= total
+
+      @page += 1
+    end
+  end
+
+  private
+
+  def locale = @locale ||= Locale.find_by!(code: @params.fetch('locale', 'ja'))
+
+  def limited_columns(prefix)
+    keys = axes.columns.each_index.map { "a#{ axes.rows.length + _1 }" }
+    return [[]] if keys.empty?
+
+    configured = @sort.select { |key, _direction| keys.include?(key) }
+    directions = (configured + keys.map { [_1, 'asc'] }).uniq { _1.first }
+    order = directions.map { |key, direction|
+      "#{ key } IS NULL ASC, #{ key } #{ direction.upcase }"
+    }.join(', ')
+    custom = QueryState.records(@params, 'column_order')
+    if custom.any?
+      raise ArgumentError unless custom.map { _1['value'] }.uniq.length == custom.length
+
+      cases = custom.each_with_index.map do |record, index|
+        "WHEN #{ keys.first } <=> #{ @connection.quote(record['value']) } THEN #{ index }"
+      end.join(' ')
+      order = "CASE #{ cases } ELSE #{ custom.length } END, #{ order }"
+    end
+    values = @connection.select_rows(<<~SQL)
+      #{ prefix }
+      SELECT /*+ MAX_EXECUTION_TIME(10000) */ DISTINCT #{ keys.join(', ') } FROM cells
+      ORDER BY #{ order } LIMIT #{ MAX_COLUMNS + 1 }
+    SQL
+    raise TooManyColumns if values.length > MAX_COLUMNS
+
+    values
+  end
+
+  def measure_relation(values)
+    row_expressions = axes.expressions.first(axes.rows.length)
+    source = population.scope.except(:select, :order)
+    fields = axes.rows + axes.columns.first(values.length)
+    joins = axes.joins(fields.map { _1.fetch('field') })
+    source = source.joins(joins) unless joins.empty?
+    values.each_with_index do |value, index|
+      expression = axes.expressions[axes.rows.length + index]
+      source = source.where("(#{ expression }) <=> ?", value)
+    end
+    selected = row_expressions.each_with_index.map { |expression, index|
+      "#{ expression } AS a#{ index }"
+    } + ['COUNT(DISTINCT v.id) AS count']
+    source = source.group(*row_expressions.map { Arel.sql(_1) }) if row_expressions.any?
+    source.select(Arel.sql(selected.join(', '))).to_sql
+  end
+
+  def measure_values(key)
+    records = QueryState.records(@params, 'measures').select { _1['key'] == key }
+    raise ArgumentError if records.empty?
+    raise ArgumentError unless records.map { _1['axis'] }.sort == (0...records.length).to_a
+
+    value = records.sort_by { _1['axis'] }.map { _1['value'] }
+    unless (value.is_a?(Array) && value.length <= axes.columns.length &&
+        value.all? { _1.nil? || _1.is_a?(String) || _1.is_a?(Numeric) })
+      raise ArgumentError
+    end
+    value
+  end
+
+  def detail_expression(key)
+    return "(#{ population.latest_sql })" if key == 'is_latest'
+    aliases = { 'tags' => 'tags_json', 'parents' => @definition.resource == 'posts' ?
+        'parent_post_ids' : 'parent_tag_ids', 'deprecated' => 'deprecated_at' }
+    field = aliases.fetch(key, key)
+    expression = @definition.expression(field)
+    return expression unless @definition.fields[field] == :json
+
+    "CAST(#{ expression } AS CHAR) COLLATE utf8mb4_bin"
+  end
+
+  def sort_sql
+    @sort.flat_map do |key, direction|
+      expression = yield key
+      ["(#{ expression }) IS NULL ASC", "#{ expression } #{ direction.upcase }"]
+    end
+  end
+
+  def axis_labels(records, column_values)
+    result = {}
+    axes.all.each_with_index do |axis, index|
+      values = records.map { _1["a#{ index }"] }
+      if index >= axes.rows.length
+        values += column_values.map { _1[index - axes.rows.length] }
+      end
+      values = values.compact.uniq
+      labels = case axis['field']
+               when 'user'
+                 User.where(id: values).pluck(:id, :name).to_h.transform_keys(&:to_s)
+               when 'language_code', 'source_language_code'
+                 Language.where(code: values).pluck(:code, :name).to_h
+               when 'tags'
+                 tag_labels(values)
+               else {}
+               end
+      result["a#{ index }"] = labels
+    end
+    result
+  end
+
+  def tag_labels(values)
+    internal = values.grep(/\Ainternal:/).map { _1.split(':').last.to_i }
+    external = values.grep(/\Aexternal:/).map { _1.split(':').last.to_i }
+    point = population.timestamp(@params['at']) || population.at
+    labels = {}
+    sources = [[TagVersion, internal, 'internal'], [NicoTagVersion, external, 'external']]
+    sources.each do |model, ids, kind|
+      next if ids.empty?
+
+      table = model.table_name
+      model.where(tag_id: ids).where('created_at <= ?', point)
+          .where("NOT EXISTS (SELECT 1 FROM #{ table } newer " +
+              "WHERE newer.tag_id = #{ table }.tag_id AND newer.created_at <= ? " +
+              "AND newer.version_no > #{ table }.version_no)", point).each do |row|
+        names = row.respond_to?(:names) ? row.names : nil
+        name = names&.dig(locale.language_code, 'name') || row.name
+        labels["#{ kind }:#{ row.tag_id }"] = name
+      end
+    end
+    labels
+  end
+end

backend/app/services/material_version_recorder.rb

@@ -31,7 +31,7 @@ class MaterialVersionRecorder < VersionRecorder
     { url: @record.url,
       parent: @record.parent,
       tag: @record.tag,
-      tag_name: @record.tag&.name,
+      tag_name: @record.tag&.name('ja'),
       tag_category: @record.tag&.category,
       source_kind: @record.source_kind,
       source_uri: @record.source_uri,

backend/app/services/nico_tag_version_recorder.rb

@@ -1,19 +1,22 @@
 class NicoTagVersionRecorder < VersionRecorder
-  def self.record! tag:, event_type:, created_by_user:
+  def self.record! external_tag:, event_type:, created_by_user:
-    new(tag:, event_type:, created_by_user:).record!
+    new(external_tag:, event_type:, created_by_user:).record!
   end
 
-  def initialize tag:, event_type:, created_by_user:
+  def initialize external_tag:, event_type:, created_by_user:
-    super(record: tag, event_type:, created_by_user:)
+    super(record: external_tag, event_type:, created_by_user:)
   end
 
   private
 
   def version_class = NicoTagVersion
   def version_association = :nico_tag_versions
-  def record_key = :tag
+  def record_key = :external_tag
 
   def snapshot_attributes
-    { name: @record.name, linked_tags: @record.snapshot_linked_tag_names.join(' ') }
+    { name: "#{ @record.platform }:#{ @record.name }",
+      linked_tags: @record.snapshot_linked_tag_names.join(' ') }
   end
+
+  def tracks_version_no_on_record? = false
 end

backend/app/services/post_bulk_creator.rb

@@ -0,0 +1,202 @@
+class PostBulkCreator
+  def initialize actor:, posts:, thumbnails:, host: nil
+    @actor_id = actor.id
+    @posts = posts
+    @thumbnails = thumbnails
+    @host = host
+  end
+
+  def run locale
+    results = Array.new(@posts.length)
+    mutex = Mutex.new
+    next_index = 0
+
+    workers = Array.new(2) do
+      Thread.new do
+        Rails.application.executor.wrap do
+          ActiveRecord::Base.connection_pool.with_connection do
+            actor = User.find(@actor_id)
+            loop do
+              index = nil
+              begin
+                index = mutex.synchronize do
+                  current = next_index
+                  next_index += 1
+                  current
+                end
+                break if index >= @posts.length
+
+                attributes = @posts[index]
+                results[index] = create_row(locale, actor, attributes, index)
+              rescue StandardError => e
+                Rails.logger.error(
+                    "post_bulk_creator_worker_failure #{ { error: e.class.name,
+                                                          message: e.message,
+                                                          index: }.to_json }")
+                results[index] = {
+                    status: 'failed',
+                    recoverable: false,
+                    errors: { base: ['登録中にエラーが発生しました．'] },
+                    base_errors: [] }
+              end
+            end
+          end
+        end
+      end
+    end
+    workers.each(&:join)
+
+    results.each_index do |index|
+      next if results[index].present?
+
+      results[index] = {
+          status: 'failed',
+          recoverable: false,
+          errors: { base: ['登録中にエラーが発生しました．'] },
+          base_errors: [] }
+    end
+
+    { results: }
+  end
+
+  private
+
+  def create_row locale, actor, attributes, index
+    preflight =
+        PostCreatePreflight.new(
+            attributes: attributes,
+            thumbnail: thumbnail_for(index, attributes),
+            host: @host).run(locale)
+    if preflight[:existing_post_id].present?
+      return {
+          status: 'skipped',
+          existing_post_id: preflight[:existing_post_id],
+          existing_post: preflight[:existing_post] }
+    end
+
+    post = PostCreator.new(
+        actor: actor,
+        attributes: normalised_attributes(attributes, preflight, index)).create!(locale)
+    result = {
+        status: 'created',
+        post: { id: post.id } }
+    result[:field_warnings] = preflight[:field_warnings] if preflight[:field_warnings].present?
+    result[:base_warnings] = preflight[:base_warnings] if preflight[:base_warnings].present?
+    result
+  rescue PostCreatePreflight::ValidationFailed => e
+    {
+        status: 'failed',
+        recoverable: true,
+        errors: e.fields,
+        base_errors: e.base_errors }
+  rescue ActiveRecord::RecordInvalid => e
+    existing_post = existing_post_for_race(attributes, e.record)
+    if existing_post.present?
+      return {
+          status: 'skipped',
+          existing_post_id: existing_post[:id],
+          existing_post: existing_post }
+    end
+
+    {
+        status: 'failed',
+        recoverable: true,
+        errors: e.record.errors.to_hash,
+        base_errors: e.record.errors[:base] }
+  rescue ActiveRecord::RecordNotUnique => e
+    if e.message.include?('index_posts_on_url')
+      existing_post = existing_post_for_race(attributes)
+      return {
+          status: 'skipped',
+          existing_post_id: existing_post[:id],
+          existing_post: existing_post } if existing_post.present?
+    end
+
+    Rails.logger.error(
+        "post_bulk_creator_record_not_unique #{ { error: e.class.name,
+                                                 message: e.message }.to_json }")
+    {
+        status: 'failed',
+        recoverable: false,
+        errors: { base: ['登録中にエラーが発生しました．'] },
+        base_errors: [] }
+  rescue Tag::NicoTagNormalisationError
+    {
+        status: 'failed',
+        recoverable: true,
+        errors: { tags: ['ニコニコ・タグは直接指定できません．'] },
+        base_errors: [] }
+  rescue Tag::DeprecatedTagNormalisationError
+    {
+        status: 'failed',
+        recoverable: true,
+        errors: { tags: ['廃止済みタグは付与できません．'] },
+        base_errors: [] }
+  rescue PostCreator::VideoMsParseError
+    {
+        status: 'failed',
+        recoverable: true,
+        errors: { video_ms: ['動画時間の記法が不正です．'] },
+        base_errors: [] }
+  rescue Post::RemoteThumbnailFetchFailed
+    {
+        status: 'failed',
+        recoverable: true,
+        errors: { thumbnail_base: ['サムネイル画像の取得に失敗しました．'] },
+        base_errors: [] }
+  rescue ArgumentError => e
+    {
+        status: 'failed',
+        recoverable: true,
+        errors: { base: [e.message] },
+        base_errors: [] }
+  rescue StandardError => e
+    Rails.logger.error(
+        "post_bulk_creator_failure #{ { error: e.class.name,
+                                        message: e.message }.to_json }")
+    {
+        status: 'failed',
+        recoverable: false,
+        errors: { base: ['登録中にエラーが発生しました．'] },
+        base_errors: [] }
+  end
+
+  def normalised_attributes attributes, preflight, index
+    {
+        url: preflight[:url],
+        title: preflight[:title],
+        thumbnail_base: preflight[:thumbnail_base],
+        thumbnail: thumbnail_for(index, attributes),
+        tags: preflight[:tags],
+        parent_post_ids: preflight[:parent_post_ids],
+        original_created_from: preflight[:original_created_from],
+        original_created_before: preflight[:original_created_before],
+        duration: preflight[:duration],
+        video_ms: preflight[:video_ms],
+        direct_tag_specs: preflight[:direct_tag_specs],
+        default_tag_specs: preflight[:default_tag_specs],
+        snapshot_tag_specs: preflight[:snapshot_tag_specs],
+        post_tag_specs: preflight[:post_tag_specs],
+        tag_sections: preflight[:tag_sections],
+        normalised_parent_post_ids: preflight[:normalised_parent_post_ids] }
+  end
+
+  def thumbnail_for index, attributes
+    return nil if attributes['thumbnail_base'].present? || attributes[:thumbnail_base].present?
+
+    @thumbnails[index]
+  end
+
+  def existing_post_for_race attributes, record = nil
+    return nil if record.present? && !(record.errors.of_kind?(:url, :taken))
+
+    normal_url = PostUrlNormaliser.normalise(attributes['url'] || attributes[:url])
+    return nil if normal_url.blank?
+
+    compact_existing_post(Post.with_attached_thumbnail.find_by(url: normal_url))
+  end
+
+  def compact_existing_post post
+    PostCompactRepr.base(post, host: @host)
+  end
+end

backend/app/services/post_create_plan.rb

@@ -0,0 +1,277 @@
+class PostCreatePlan
+  VIDEO_TAG_NAME = '動画'.freeze
+  TAGME_TAG_NAME = 'タグ希望'.freeze
+  NO_DEERJIKIST_TAG_NAME = 'ニジラー情報不詳'.freeze
+
+  def initialize attributes:
+    @attributes = attributes.symbolize_keys
+    @existing_tags_by_name = nil
+  end
+
+  def build! locale
+    direct_tag_specs, tag_sections = parse_direct_tag_specs(locale)
+    default_tag_specs = build_default_tag_specs(locale, direct_tag_specs)
+    snapshot_tag_specs = merge_tag_specs(direct_tag_specs + default_tag_specs)
+    preload_existing_tags_by_name!(locale, snapshot_tag_specs.map { _1[:name] })
+    validate_new_tag_specs!(locale, snapshot_tag_specs)
+    post_tag_specs = expand_parent_tag_specs(locale, snapshot_tag_specs)
+    video_ms = normalise_video_ms(snapshot_tag_specs)
+    validate_video_sections!(video_ms, tag_sections)
+    parent_post_ids = normalise_parent_post_ids
+    validate_parent_post_ids!(parent_post_ids)
+
+    { url: @attributes[:url],
+      title: @attributes[:title].to_s,
+      thumbnail_base: @attributes[:thumbnail_base].presence,
+      original_created_from: @attributes[:original_created_from].presence,
+      original_created_before: @attributes[:original_created_before].presence,
+      tags: serialised_tags(direct_tag_specs, tag_sections),
+      display_tags: display_tags(direct_tag_specs, tag_sections),
+      duration: @attributes[:duration].to_s,
+      video_ms: video_ms,
+      parent_post_ids: parent_post_ids.join(' '),
+      direct_tag_specs: direct_tag_specs,
+      default_tag_specs: default_tag_specs,
+      snapshot_tag_specs: snapshot_tag_specs,
+      post_tag_specs: post_tag_specs,
+      tag_sections: tag_sections,
+      normalised_parent_post_ids: parent_post_ids }
+  end
+
+  private
+
+  def tag_names = @attributes[:tags].to_s.split
+
+  def parse_direct_tag_specs locale
+    tag_sections = { }
+    direct_tag_specs = []
+
+    tag_names.each do |raw_name|
+      tag_name, category, sections = parse_raw_tag_name(locale, raw_name)
+      existing_tag = existing_tags_by_name(locale)[tag_name]
+      if existing_tag&.deprecated?
+        raise Tag::DeprecatedTagNormalisationError, [existing_tag.name(locale.language_code)]
+      end
+
+      direct_tag_specs << {
+          name: tag_name,
+          category: (category || existing_tag&.category || 'general').to_sym }
+      if sections.present?
+        tag_sections[tag_name] ||= []
+        tag_sections[tag_name].concat(sections)
+        tag_sections[tag_name] = Tag.merge_section_ranges(tag_sections[tag_name])
+        tag_sections.delete(tag_name) if tag_sections[tag_name] == [[0, nil]]
+      end
+    end
+
+    [merge_tag_specs(direct_tag_specs), tag_sections]
+  end
+
+  def parse_raw_tag_name locale, raw_name
+    name = raw_name.to_s
+    prefix, category =
+        Tag::CATEGORY_PREFIXES.find {
+          name.downcase.start_with?(_1[0])
+        } || ['', nil]
+    name = name.sub(/\A#{ prefix }/i, '')
+
+    sections = []
+    while match = name.match(/\A(\S*?)\[([^\[\]\s]*)-([^\[\]\s]*)\](\S*)\z/)
+      name = "#{ match[1] }#{ match[4] }"
+      next if match[2].empty? && match[3].empty?
+
+      sections << Tag.normalise_section_range!(
+          begin_raw: match[2],
+          end_raw: match[3],
+          tag_name: name)
+    end
+    if name.include?('[') || name.include?(']')
+      raise Tag::SectionLiteralParseError.new(raw_name, raw_name)
+    end
+
+    [resolved_tag_name(locale, name), category&.to_sym, sections]
+  end
+
+  def build_default_tag_specs locale, direct_tag_specs
+    default_tag_specs = []
+    if direct_tag_specs.length < 10 && direct_tag_specs.none? { _1[:name] == TAGME_TAG_NAME }
+      default_tag_specs << { name: TAGME_TAG_NAME, category: :meta }
+    end
+    if direct_tag_specs.none? { deerjikist_tag_spec?(locale, _1) }
+      default_tag_specs << { name: NO_DEERJIKIST_TAG_NAME, category: :meta }
+    end
+
+    default_tag_specs
+  end
+
+  def validate_new_tag_specs! locale, specs
+    Array(specs).each do |spec|
+      next if existing_tags_by_name(locale).key?(spec[:name])
+
+      validate_new_tag_spec!(locale, spec)
+    end
+  end
+
+  def validate_new_tag_spec! locale, spec
+    tag = Tag.new(category: spec[:category])
+    tag_name = TagName.new(tag:, name: spec[:name], language_code: locale.language_code,
+                           script_code: locale.script_code, primary_flg: true)
+    return if tag_name.valid? && tag.valid?
+
+    post = Post.new
+    tag_name.errors[:name].each do |message|
+      post.errors.add :tags, "#{ spec[:name] }: #{ message }"
+    end
+    tag.errors.each do |error|
+      next if error.attribute == :tag_name
+
+      post.errors.add :tags, "#{ spec[:name] }: #{ error.message }"
+    end
+    raise ActiveRecord::RecordInvalid, post
+  end
+
+  def expand_parent_tag_specs locale, snapshot_tag_specs
+    existing_snapshot_tags = snapshot_tag_specs.filter_map do
+      existing_tags_by_name(locale)[_1[:name]]
+    end
+    expanded_parent_specs =
+        Tag.expand_parent_tags(existing_snapshot_tags)
+        .reject(&:deprecated?)
+        .map { |tag|
+          { name: tag.name(locale.language_code),
+            category: tag.category.to_sym }
+        }
+    merge_tag_specs(snapshot_tag_specs + expanded_parent_specs)
+  end
+
+  def merge_tag_specs specs
+    specs.each_with_object({ }) { |spec, merged|
+      merged[spec[:name]] =
+          if merged.key?(spec[:name]) && merged[spec[:name]][:category] != :general
+            merged[spec[:name]]
+          else
+            {
+                name: spec[:name],
+                category: spec[:category] }
+          end
+    }.values.sort_by { _1[:name] }
+  end
+
+  def existing_tags_by_name locale
+    @existing_tags_by_name ||= begin
+      names = tag_names.map { canonical_tag_name_without_sections(locale, _1) }.uniq
+      TagName
+          .where(language_code: locale.language_code, primary_flg: true, name: names)
+          .includes(:tag)
+          .each_with_object({ }) { |tag_name, tags| tags[tag_name.name] = tag_name.tag }
+    end
+  end
+
+  def preload_existing_tags_by_name! locale, names
+    wanted_names = Array(names).map { _1.to_s }.reject(&:blank?).uniq
+    missing_names = wanted_names - existing_tags_by_name(locale).keys
+    return if missing_names.empty?
+
+    existing_tags_by_name(locale).merge!(
+        TagName.where(language_code: locale.language_code,
+                      primary_flg: true,
+                      name: missing_names)
+               .includes(:tag)
+               .each_with_object({ }) { |tag_name, tags| tags[tag_name.name] = tag_name.tag })
+  end
+
+  def canonical_tag_name_without_sections locale, raw_name
+    name, = parse_raw_tag_name(locale, raw_name)
+    name
+  end
+
+  def deerjikist_tag_spec? locale, spec
+    return true if spec[:category] == :deerjikist
+
+    existing_tags_by_name(locale)[spec[:name]]&.deerjikist?
+  end
+
+  def normalise_parent_post_ids
+    Array(@attributes[:parent_post_ids]).flat_map { _1.to_s.split }.map { |token|
+      id = Integer(token, exception: false)
+      raise ArgumentError, "親投稿 Id. が不正です: #{ token }" if id.nil? || id <= 0
+
+      id
+    }.uniq.sort
+  end
+
+  def validate_parent_post_ids! ids
+    missing = ids - Post.where(id: ids).pluck(:id)
+    raise ArgumentError, "存在しない親投稿 Id. があります: #{ missing.join(' ') }" if missing.present?
+  end
+
+  def serialised_tags direct_tag_specs, tag_sections
+    direct_tag_specs.map { |spec|
+      "#{ spec[:name] }#{ tag_sections[spec[:name]].to_a.map { section_literal(_1) }.join }"
+    }.sort.join(' ')
+  end
+
+  def display_tags direct_tag_specs, tag_sections
+    direct_tag_specs.map { |spec|
+      {
+          name: spec[:name],
+          category: spec[:category].to_s,
+          section_literals: tag_sections[spec[:name]].to_a.map { section_literal(_1) } }
+    }.sort_by { _1[:name] }
+  end
+
+  def section_literal range
+    begin_ms, end_ms = range
+    "[#{ Post.ms_to_time(begin_ms) }-#{ end_ms ? Post.ms_to_time(end_ms) : '' }]"
+  end
+
+  def normalise_video_ms snapshot_tag_specs
+    return nil unless snapshot_tag_specs.any? { _1[:name] == VIDEO_TAG_NAME }
+
+    video_ms = @attributes[:video_ms]
+    if video_ms.present?
+      value = Integer(video_ms, exception: false)
+      raise PostCreator::VideoMsParseError unless value&.positive?
+
+      return value
+    end
+
+    duration = @attributes[:duration]
+    return nil if duration.blank?
+
+    value = Tag.time_to_ms!(duration.to_s, tag_name: '動画時間')
+    raise PostCreator::VideoMsParseError unless value.positive?
+
+    value
+  rescue Tag::SectionLiteralParseError
+    raise PostCreator::VideoMsParseError
+  end
+
+  def validate_video_sections! video_ms, tag_sections
+    return unless video_ms
+
+    tag_sections.each_value do |ranges|
+      ranges.each do |begin_ms, end_ms|
+        if begin_ms >= video_ms
+          post = Post.new
+          post.errors.add :video_ms, 'タグ区間の開始が動画時間以上です．'
+          raise ActiveRecord::RecordInvalid, post
+        end
+        if end_ms && end_ms > video_ms
+          post = Post.new
+          post.errors.add :video_ms, 'タグ区間の終端が動画時間を超えてゐます．'
+          raise ActiveRecord::RecordInvalid, post
+        end
+      end
+    end
+  end
+
+  def resolved_tag_name locale, name
+    tag_name = TagName.find_by(language_code: locale.language_code, name:)
+    return name if !(tag_name) || tag_name.primary?
+
+    TagName.find_by(language_code: locale.language_code,
+                    tag_id: tag_name.tag_id,
+                    primary_flg: true)&.name || name
+  end
+end

backend/app/services/post_create_preflight.rb

@@ -0,0 +1,142 @@
+class PostCreatePreflight
+  class ValidationFailed < StandardError
+    attr_reader :fields, :base_errors
+
+    def initialize fields: { }, base_errors: []
+      super('入力内容を確認してください．')
+      @fields = fields
+      @base_errors = base_errors
+    end
+  end
+
+  def initialize attributes:, thumbnail: nil, host: nil
+    @attributes = attributes.symbolize_keys
+    @thumbnail = thumbnail
+    @host = host
+  end
+
+  def run locale
+    preview = PostImportPreviewer.new.preview_rows(
+        rows: [preview_row],
+        locale:,
+        fetch_metadata: false).first
+    if preview[:existing_post_id].present?
+      return {
+          url: preview[:url],
+          title: preview[:attributes]['title'],
+          thumbnail_base: preview[:attributes]['thumbnail_base'],
+          tags: preview[:attributes]['tags'],
+          parent_post_ids: preview[:attributes]['parent_post_ids'],
+          original_created_from: preview[:attributes]['original_created_from'],
+          original_created_before: preview[:attributes]['original_created_before'],
+          duration: preview[:attributes]['duration'],
+          video_ms: preview[:attributes]['video_ms'],
+          display_tags: preview[:display_tags] || [],
+          field_warnings: final_field_warnings(preview[:field_warnings] || { }),
+          base_warnings: preview[:base_warnings],
+          existing_post_id: preview[:existing_post_id],
+          existing_post: existing_post_compact(preview[:existing_post_id]) }
+    end
+
+    if preview[:validation_errors].present?
+      raise ValidationFailed.new(fields: preview[:validation_errors])
+    end
+
+    validate_thumbnail_upload!
+
+    plan = PostCreatePlan.new(
+        attributes: {
+            url: preview[:url],
+            title: preview[:attributes]['title'],
+            thumbnail_base: preview[:attributes]['thumbnail_base'],
+            tags: preview[:attributes]['tags'],
+            parent_post_ids: preview[:attributes]['parent_post_ids'],
+            original_created_from: preview[:attributes]['original_created_from'],
+            original_created_before: preview[:attributes]['original_created_before'],
+            duration: preview[:attributes]['duration'],
+            video_ms: preview[:attributes]['video_ms'] }).build!(locale)
+
+    {
+        url: plan[:url],
+        title: plan[:title],
+        thumbnail_base: plan[:thumbnail_base],
+        tags: plan[:tags],
+        parent_post_ids: plan[:parent_post_ids],
+        original_created_from: plan[:original_created_from],
+        original_created_before: plan[:original_created_before],
+        duration: plan[:duration],
+        video_ms: plan[:video_ms],
+        display_tags: plan[:display_tags],
+        direct_tag_specs: plan[:direct_tag_specs],
+        default_tag_specs: plan[:default_tag_specs],
+        snapshot_tag_specs: plan[:snapshot_tag_specs],
+        post_tag_specs: plan[:post_tag_specs],
+        tag_sections: plan[:tag_sections],
+        normalised_parent_post_ids: plan[:normalised_parent_post_ids],
+        field_warnings: final_field_warnings(preview[:field_warnings] || { }),
+        base_warnings: preview[:base_warnings],
+        existing_post_id: preview[:existing_post_id],
+        existing_post: existing_post_compact(preview[:existing_post_id]) }
+  end
+
+  private
+
+  def preview_row
+    {
+        source_row: 1,
+        url: @attributes[:url].to_s,
+        attributes: {
+            'title' => @attributes[:title].to_s,
+            'thumbnail_base' => @attributes[:thumbnail_base].to_s,
+            'original_created_from' => @attributes[:original_created_from].to_s,
+            'original_created_before' => @attributes[:original_created_before].to_s,
+            'duration' => @attributes[:duration].to_s,
+            'video_ms' => @attributes[:video_ms],
+            'tags' => @attributes[:tags].to_s,
+            'parent_post_ids' => parent_post_ids_text },
+        provenance: {
+            'url' => 'manual',
+            'title' => 'manual',
+            'thumbnail_base' => 'manual',
+            'original_created_from' => 'manual',
+            'original_created_before' => 'manual',
+            'duration' => 'manual',
+            'video_ms' => 'manual',
+            'tags' => 'manual',
+            'parent_post_ids' => 'manual' },
+        tag_sources: {
+            'automatic' => '',
+            'manual' => @attributes[:tags].to_s } }
+  end
+
+  def parent_post_ids_text
+    Array(@attributes[:parent_post_ids]).flat_map { _1.to_s.split }.join(' ')
+  end
+
+  def validate_thumbnail_upload!
+    return if @attributes[:thumbnail_base].present?
+    return if @thumbnail.blank?
+
+    PostThumbnailUploadValidator.validate!(@thumbnail)
+  rescue PostThumbnailUploadValidator::InvalidUpload => e
+    raise ValidationFailed.new(fields: { thumbnail: [e.message] })
+  end
+
+  def existing_post_compact post_id
+    return nil if post_id.blank?
+
+    post = Post.with_attached_thumbnail.find_by(id: post_id)
+    PostCompactRepr.base(post, host: @host)
+  end
+
+  def final_field_warnings field_warnings
+    thumbnail_warnings = (field_warnings['thumbnail_base'] || []).reject { _1 == 'サムネールなし' }
+    if @attributes[:thumbnail_base].blank? && @thumbnail.blank?
+      thumbnail_warnings = (thumbnail_warnings + ['サムネールなし']).uniq
+    end
+
+    next_warnings = field_warnings.except('thumbnail_base')
+    next_warnings['thumbnail_base'] = thumbnail_warnings if thumbnail_warnings.present?
+    next_warnings
+  end
+end

backend/app/services/post_creator.rb

@@ -9,104 +9,130 @@ class PostCreator
     @field_warnings = { }
   end
 
-  def create!
+  def create! locale
+    thumbnail_attachment = prepare_thumbnail_attachment
     post = Post.new(title: @attributes[:title].presence,
                     url: @attributes[:url],
                     thumbnail_base: @attributes[:thumbnail_base].presence,
                     uploaded_user: @actor,
                     original_created_from: @attributes[:original_created_from].presence,
                     original_created_before: @attributes[:original_created_before].presence)
-    attach_thumbnail!(post)
 
     ApplicationRecord.transaction do
       post.save!
-      Tag.normalise_tags!(tag_names, deny_deprecated: true, with_sections: true) =>
+      post.thumbnail.attach(thumbnail_attachment) if thumbnail_attachment.present?
-          { tags:, sections: }
+      snapshot_tags = planned_snapshot_tags(locale)
-      TagVersioning.record_tag_snapshots!(tags, created_by_user: @actor)
+      post_tags = planned_post_tags(locale)
-      tags = Tag.expand_parent_tags(tags).reject(&:deprecated?)
+      sections = planned_sections(locale)
-      post.video_ms = normalise_video_ms(tags)
+      TagVersioning.record_tag_snapshots!(
-      validate_video_sections!(post.video_ms, sections)
+          snapshot_tags,
+          created_by_user: @actor,
+          language_code: locale.language_code)
+      post.video_ms = planned_video_ms(locale)
       post.save!
-      sync_post_tags!(post, tags, sections)
+      sync_post_tags!(post, post_tags, sections)
-      sync_parent_posts!(post, parent_post_ids)
+      sync_parent_posts!(post, planned_parent_post_ids(locale))
       PostVersionRecorder.record!(post:, event_type: :create, created_by_user: @actor)
     end
     post
+  rescue StandardError
+    post&.thumbnail&.purge if post&.thumbnail&.attached?
+    raise
   end
 
   private
 
-  def attach_thumbnail! post
+  def prepare_thumbnail_attachment
-    thumbnail = @attributes[:thumbnail]
+    PostThumbnailAttachmentBuilder.build(
-    if thumbnail.present?
+        thumbnail: @attributes[:thumbnail],
-      post.thumbnail.attach(Post.resized_thumbnail_attachment(thumbnail))
+        thumbnail_base: @attributes[:thumbnail_base].presence)
-      return
-    end
-
-    thumbnail_base = post.thumbnail_base
-    return if thumbnail_base.blank?
-
-    post.attach_thumbnail_from_url!(thumbnail_base)
-  rescue Post::RemoteThumbnailFetchFailed => e
-    @field_warnings[:thumbnail_base] = [e.message]
   end
 
-  def tag_names = @attributes[:tags].to_s.split
+  def planned_snapshot_tags(locale) = planned_create_attributes(locale)[:snapshot_tags]
 
-  def parent_post_ids
+  def planned_post_tags(locale) = planned_create_attributes(locale)[:post_tags]
-    Array(@attributes[:parent_post_ids]).flat_map { _1.to_s.split }.map { |token|
-      id = Integer(token, exception: false)
-      raise ArgumentError, "親投稿 Id. が不正です: #{ token }" if id.nil? || id <= 0
 
-      id
+  def planned_sections(locale) = planned_create_attributes(locale)[:tag_sections]
-    }.uniq
+
+  def planned_parent_post_ids locale
+    planned_create_attributes(locale)[:normalised_parent_post_ids]
   end
 
-  def normalise_video_ms tags
+  def planned_video_ms locale
-    return nil unless tags.any? { _1.id == Tag.video.id }
+    planned_create_attributes(locale)[:video_ms]
-
-    video_ms = @attributes[:video_ms]
-    if video_ms.present?
-      value = Integer(video_ms, exception: false)
-      raise VideoMsParseError unless value&.positive?
-
-      return value
-    end
-    duration = @attributes[:duration]
-    return nil if duration.blank?
-
-    value = Tag.time_to_ms!(duration.to_s, tag_name: '動画時間')
-    raise VideoMsParseError unless value.positive?
-
-    value
-  rescue Tag::SectionLiteralParseError
-    raise VideoMsParseError
   end
 
-  def validate_video_sections! video_ms, sections
+  def planned_create_attributes locale
-    return unless video_ms
+    @planned_create_attributes ||= begin
-
+      if @attributes.key?(:snapshot_tag_specs)
-    sections.each_value do |ranges|
+        snapshot_tags = materialise_tags(locale, @attributes[:snapshot_tag_specs] || [])
-      ranges.each do |begin_ms, end_ms|
+        post_tags = materialise_tags(locale, @attributes[:post_tag_specs] || [])
-        post = Post.new
+        {
-        if begin_ms >= video_ms
+            snapshot_tags: snapshot_tags,
-          post.errors.add :video_ms, 'タグ区間の開始が動画時間以上です．'
+            post_tags: post_tags,
-          raise ActiveRecord::RecordInvalid, post
+            tag_sections: materialise_sections(
-        end
+                @attributes[:tag_sections] || { },
-        if end_ms && end_ms > video_ms
+                snapshot_tags,
-          post.errors.add :video_ms, 'タグ区間の終端が動画時間を超えてゐます．'
+                post_tags,
-          raise ActiveRecord::RecordInvalid, post
+                locale),
-        end
+            normalised_parent_post_ids: @attributes[:normalised_parent_post_ids] || [],
+            video_ms: @attributes[:video_ms] }
+      else
+        build_materialised_plan(locale)
       end
     end
   end
 
+  def build_materialised_plan locale
+    plan = PostCreatePlan.new(attributes: @attributes).build!(locale)
+    snapshot_tags = materialise_tags(locale, plan[:snapshot_tag_specs] || [])
+    post_tags = materialise_tags(locale, plan[:post_tag_specs] || [])
+    {
+        snapshot_tags: snapshot_tags,
+        post_tags: post_tags,
+        tag_sections: materialise_sections(
+            plan[:tag_sections] || { },
+            snapshot_tags,
+            post_tags,
+            locale),
+        normalised_parent_post_ids: plan[:normalised_parent_post_ids] || [],
+        video_ms: plan[:video_ms] }
+  end
+
+  def materialise_tags locale, specs
+    Array(specs).each_with_object({ }) do |spec, tags|
+      name = spec[:name] || spec['name']
+      category = spec[:category] || spec['category']
+      next if name.blank? || category.blank?
+
+      tag = Tag.find_or_create_by_tag_name!(locale, name, category:, created_by_user: @actor)
+      tag.update!(category:) if tag.category.to_sym != category.to_sym
+      tags[name] ||= tag
+    end.values
+  end
+
+  def materialise_sections sections_by_name, snapshot_tags, post_tags, locale = Locale.nipponese
+    tags_by_name = post_tags.index_by { _1.name(locale.language_code) }
+    snapshot_tags.each do |tag|
+      tags_by_name[tag.name(locale.language_code)] ||= tag
+    end
+
+    sections_by_name.each_with_object({ }) do |(tag_name, ranges), sections|
+      tag = tags_by_name[tag_name.to_s]
+      next if tag.nil?
+
+      sections[tag.id] = Array(ranges).map { |range| [range[0], range[1]] }
+    end
+  end
+
   def sync_post_tags! post, desired_tags, sections
     desired_ids = desired_tags.map(&:id).to_set
     current_ids = post.tags.pluck(:id).to_set
+
     Tag.where(id: desired_ids - current_ids).find_each do |tag|
       PostTag.create_or_find_by!(post:, tag:, created_user: @actor)
     end
+
     PostTagSection.where(post_id: post.id).destroy_all
     sections.each do |tag_id, ranges|
       ranges.each do |begin_ms, end_ms|
@@ -116,10 +142,9 @@ class PostCreator
                                end_ms:)
       end
     end
+
     PostTag.where(post_id: post.id,
-                  tag_id: (current_ids - desired_ids).to_a).kept.find_each do |post_tag|
+                  tag_id: (current_ids - desired_ids).to_a).destroy_all
-      post_tag.discard_by!(@actor)
-    end
   end
 
   def sync_parent_posts! post, ids

backend/app/services/post_import_previewer.rb

@@ -3,19 +3,20 @@ require 'timeout'
 
 class PostImportPreviewer
   FIELDS = [
-    'title',
+      'title',
-    'thumbnail_base',
+      'thumbnail_base',
-    'original_created_from',
+      'original_created_from',
-    'original_created_before',
+      'original_created_before',
-    'duration',
+      'video_ms',
-    'tags',
+      'duration',
-    'parent_post_ids'].freeze
+      'tags',
+      'parent_post_ids'].freeze
   FETCH_WARNING_FIELDS = ['url', 'title', 'thumbnail_base'].freeze
   TITLE_FETCH_WARNING = 'タイトルを取得できませんでした．'.freeze
   THUMBNAIL_FETCH_WARNING = 'サムネールを取得できませんでした．'.freeze
   METADATA_FETCH_WARNING = '自動取得に失敗しました．'.freeze
 
-  def preview_rows rows:, fetch_metadata: true, metadata_cache: { }
+  def preview_rows rows:, locale: Locale.nipponese, fetch_metadata: true, metadata_cache: { }
     prepared_rows = rows.map { prepare_row(_1) }
     url_counts = prepared_rows.filter_map { _1[:normal_url] }.tally
     existing_posts =
@@ -29,13 +30,14 @@ class PostImportPreviewer
         url_counts)
     known_tags =
         preload_known_tags(
+            locale,
             prepared_rows,
             fetch_metadata,
             metadata_cache,
             existing_posts,
             url_counts)
     prepared_rows.map { |row|
-      preview_row(row,
+      preview_row(locale, row,
                   fetch_metadata:,
                   metadata_cache:,
                   existing_posts:,
@@ -58,7 +60,7 @@ class PostImportPreviewer
     source.merge(url_text: url, normal_url:, url_error: validate_url_safety(normal_url))
   end
 
-  def preview_row row,
+  def preview_row locale, row,
                   fetch_metadata:,
                   metadata_cache:,
                   existing_posts:,
@@ -123,9 +125,9 @@ class PostImportPreviewer
 
     attributes['url'] = url
     validate_basic_data(attributes, validation_errors)
-    validate_preview_tags(merged_tags(tag_sources, provenance['tags']),
+    validate_preview_tags(locale,
+                          merged_tags(tag_sources, provenance['tags']),
                           validation_errors,
-                          field_warnings,
                           known_tags)
     validate_parents(attributes['parent_post_ids'], validation_errors, existing_parent_ids)
     attributes.delete('url')
@@ -162,6 +164,8 @@ class PostImportPreviewer
       normalised =
           if field == 'duration'
             normalise_duration_attribute(value)
+          elsif field == 'video_ms'
+            value.nil? ? '' : value.to_s
           else
             value.to_s
           end
@@ -194,7 +198,7 @@ class PostImportPreviewer
 
   def clear_automatic_values! attributes, provenance, tag_sources
     ['title', 'thumbnail_base', 'original_created_from',
-     'original_created_before', 'duration'].each do |field|
+     'original_created_before', 'video_ms', 'duration'].each do |field|
       attributes[field] = '' if provenance[field] == 'automatic'
     end
     tag_sources['automatic'] = ''
@@ -223,15 +227,15 @@ class PostImportPreviewer
     end
     { data:, warnings:, validation_errors: { } }
   rescue Preview::UrlSafety::UnsafeUrl => e
+    payload = { error: e.class.name, message: e.message }
     Rails.logger.info(
-        "post_import_metadata_fetch_unsafe_url "\
+        "post_import_metadata_fetch_unsafe_url #{ payload.to_json }")
-        "#{ { error: e.class.name, message: e.message }.to_json }")
     { data: { }, warnings: { }, validation_errors: { url: [e.message] } }
   rescue Preview::HttpFetcher::FetchFailed,
          Preview::HttpFetcher::ResponseTooLarge => e
+    payload = { error: e.class.name, message: e.message }
     Rails.logger.info(
-        "post_import_metadata_fetch_failure "\
+        "post_import_metadata_fetch_failure #{ payload.to_json }")
-        "#{ { error: e.class.name, message: e.message }.to_json }")
     { data: { }, warnings: { 'url' => [METADATA_FETCH_WARNING] }, validation_errors: { } }
   end
 
@@ -252,7 +256,15 @@ class PostImportPreviewer
   def sanitise_metadata_url value
     return nil unless value.is_a?(String)
 
-    PostUrlNormaliser.normalise(value)
+    stripped = value.strip
+    return nil if stripped.blank?
+
+    uri = URI.parse(stripped)
+    return nil unless uri.is_a?(URI::HTTP) && uri.host.present?
+
+    stripped
+  rescue URI::InvalidURIError
+    nil
   end
 
   def sanitise_metadata_time value
@@ -325,18 +337,18 @@ class PostImportPreviewer
   def safe_fetch_metadata url
     fetch_metadata(url)
   rescue Preview::UrlSafety::UnsafeUrl => e
+    payload = { error: e.class.name, message: e.message }
     Rails.logger.info(
-        "post_import_metadata_fetch_unsafe_url "\
+        "post_import_metadata_fetch_unsafe_url #{ payload.to_json }")
-        "#{ { error: e.class.name, message: e.message }.to_json }")
     { data: { }, warnings: { }, validation_errors: { url: [e.message] } }
   rescue StandardError => e
+    payload = { error: e.class.name, message: e.message }
     Rails.logger.error(
-        "post_import_metadata_fetch_unexpected_failure "\
+        "post_import_metadata_fetch_unexpected_failure #{ payload.to_json }")
-        "#{ { error: e.class.name, message: e.message }.to_json }")
     { data: { }, warnings: { 'url' => [METADATA_FETCH_WARNING] }, validation_errors: { } }
   end
 
-  def preload_known_tags prepared_rows, fetch_metadata, metadata_cache, existing_posts, url_counts
+  def preload_known_tags locale, prepared_rows, fetch_metadata, metadata_cache, existing_posts, url_counts
     names = prepared_rows.flat_map { |row|
       attributes = initial_attributes(row)
       provenance = initial_provenance(row)
@@ -348,15 +360,13 @@ class PostImportPreviewer
         metadata = metadata_for(row[:normal_url], metadata_cache)
         apply_metadata!(attributes, provenance, tag_sources, metadata[:data])
       end
-      preview_tag_names(merged_tags(tag_sources, provenance['tags']))
+      preview_tag_names(locale, merged_tags(tag_sources, provenance['tags']))
     }.compact.uniq
     return { } if names.empty?
 
-    Tag.joins(:tag_name)
+    TagName.where(language_code: locale.language_code, primary_flg: true, name: names)
-      .where(tag_names: { name: names })
+           .includes(:tag)
-      .includes(:tag_name)
+           .each_with_object({ }) { |tag_name, tags| tags[tag_name.name] = tag_name.tag }
-      .to_a
-      .index_by(&:name)
   end
 
   def preload_parent_ids prepared_rows
@@ -427,19 +437,21 @@ class PostImportPreviewer
     sources['automatic'].to_s
   end
 
-  def preview_tag_names raw
+  def preview_tag_names locale, raw
     names = raw.to_s.split
     return [] if names.empty?
     if names.any? { _1.downcase.start_with?('nico:') }
       return []
     end
 
-    names.map { |name| TagName.canonicalise(name.sub(/\[.*\]\z/, '')).first }
+    names.map { |name|
+      TagName.canonicalise(locale, name.sub(/\[.*\]\z/, '')).first
+    }
   rescue Tag::SectionLiteralParseError
     []
   end
 
-  def validate_preview_tags raw, errors, field_warnings, known_tags
+  def validate_preview_tags locale, raw, errors, known_tags
     names = raw.to_s.split
     return if names.empty?
     if names.any? { _1.downcase.start_with?('nico:') }
@@ -447,15 +459,12 @@ class PostImportPreviewer
       return
     end
 
-    parsed = names.map { |name| TagName.canonicalise(name.sub(/\[.*\]\z/, '')).first }
+    parsed = names.map { |name|
+      TagName.canonicalise(locale, name.sub(/\[.*\]\z/, '')).first
+    }
     existing = parsed.filter_map { known_tags[_1] }
-    deprecated = existing.select(&:deprecated?).map(&:name)
+    deprecated = existing.select(&:deprecated?).map { _1.name(locale.language_code) }
     errors[:tags] = ["廃止済みタグがあります: #{ deprecated.join(' ') }"] if deprecated.present?
-    known = existing.reject(&:deprecated?).map(&:name)
-    new_tags = parsed.uniq - known
-    if new_tags.present?
-      add_field_warning!(field_warnings, 'tags', "新規タグを作成します: #{ new_tags.join(' ') }")
-    end
   rescue Tag::SectionLiteralParseError
     errors[:tags] = ['タグ区間の記法が不正です．']
   end
@@ -466,7 +475,7 @@ class PostImportPreviewer
                     thumbnail_base: attributes['thumbnail_base'].presence,
                     original_created_from: attributes['original_created_from'].presence,
                     original_created_before: attributes['original_created_before'].presence,
-                    video_ms: parse_duration(attributes['duration'], errors))
+                    video_ms: parse_video_ms(attributes, errors))
     post.valid?
     post.errors.each do |error|
       next if error.attribute == :url && error.type == :taken
@@ -475,7 +484,21 @@ class PostImportPreviewer
     end
   end
 
-  def parse_duration value, errors
+  def parse_video_ms attributes, errors
+    return nil unless attributes['tags'].to_s.split.include?('動画')
+
+    video_ms = attributes['video_ms']
+    if video_ms.present?
+      value = Integer(video_ms, exception: false)
+      if value&.positive?
+        return value
+      end
+
+      errors[:video_ms] = ['動画時間の記法が不正です．']
+      return nil
+    end
+
+    value = attributes['duration']
     return nil if value.blank?
 
     Tag.time_to_ms!(value.to_s, tag_name: '動画時間')

backend/app/services/post_import_row_normaliser.rb

@@ -1,190 +0,0 @@
-class PostImportRowNormaliser
-  ORIGINS = ['automatic', 'manual'].freeze
-  STRING_FIELDS = [
-    'title',
-    'thumbnail_base',
-    'original_created_from',
-    'original_created_before',
-    'duration',
-    'tags',
-    'parent_post_ids'].freeze
-  FLEXIBLE_FIELDS = ['video_ms'].freeze
-  ATTRIBUTE_FIELDS = (STRING_FIELDS + FLEXIBLE_FIELDS).freeze
-
-  def self.normalise! rows, allow_warning_fields: false
-    raise ArgumentError, '取込行の形式が不正です．' unless rows.is_a?(Array)
-    raise ArgumentError, '取込件数が多すぎます．' if rows.length > PostImportUrlListParser::MAX_ROWS
-
-    normalised_rows = rows.map { normalise_row!(_1, allow_warning_fields:) }
-    source_rows = normalised_rows.map { _1['source_row'] }
-    raise ArgumentError, '元行番号が重複しています．' if source_rows.uniq.length != source_rows.length
-    if normalised_rows.sum { row_bytesize(_1) } > PostImportUrlListParser::MAX_BYTES
-      raise ArgumentError, '取込データが大きすぎます．'
-    end
-
-    normalised_rows
-  end
-
-  def self.normalise_row! row, allow_warning_fields:
-    unless row.is_a?(Hash) || row.is_a?(ActionController::Parameters)
-      raise ArgumentError, '取込行の形式が不正です．'
-    end
-
-    parameters =
-        row.is_a?(ActionController::Parameters) ? row : ActionController::Parameters.new(row)
-    permitted = parameters.permit(*permitted_keys(allow_warning_fields))
-    normalised = permitted.to_h.deep_transform_keys { _1.to_s.underscore }
-
-    normalised['source_row'] = normalise_source_row!(normalised['source_row'])
-    normalise_url!(normalised['url'])
-    normalise_metadata_url!(normalised['metadata_url'])
-    normalise_attributes!(normalised.fetch('attributes', { }))
-    normalise_provenance!(normalised.fetch('provenance', { }))
-    normalise_tag_sources!(normalised['tag_sources'])
-    normalise_warning_values!(normalised, allow_warning_fields:)
-    if row_bytesize(normalised) > PostImportUrlListParser::MAX_BYTES
-      raise ArgumentError, '取込行が大きすぎます．'
-    end
-
-    normalised
-  end
-
-  def self.permitted_keys allow_warning_fields
-    keys = [
-      :source_row,
-      :sourceRow,
-      :url,
-      :metadata_url,
-      :metadataUrl,
-      { attributes: {} },
-      { provenance: {} },
-      { tag_sources: {} },
-      { tagSources: {} }]
-    return keys unless allow_warning_fields
-
-    keys + [
-      { field_warnings: {} },
-      { fieldWarnings: {} },
-      { base_warnings: [] },
-      { baseWarnings: [] }]
-  end
-  private_class_method :permitted_keys
-
-  def self.normalise_source_row! value
-    source_row = Integer(value, exception: false)
-    raise ArgumentError, '元行番号が不正です．' if source_row.nil? || source_row <= 0
-
-    source_row
-  end
-  private_class_method :normalise_source_row!
-
-  def self.normalise_url! value
-    unless value.is_a?(String)
-      raise ArgumentError, 'URL の形式が不正です．'
-    end
-    if value.bytesize > PostImportUrlListParser::MAX_URL_BYTES
-      raise ArgumentError, 'URL が長すぎます．'
-    end
-  end
-  private_class_method :normalise_url!
-
-  def self.normalise_metadata_url! value
-    raise ArgumentError, 'metadata_url の形式が不正です．' unless value.nil? || value.is_a?(String)
-    if value.to_s.bytesize > PostImportUrlListParser::MAX_URL_BYTES
-      raise ArgumentError, 'metadata_url が長すぎます．'
-    end
-  end
-  private_class_method :normalise_metadata_url!
-
-  def self.normalise_attributes! attributes
-    raise ArgumentError, 'attributes の形式が不正です．' unless attributes.is_a?(Hash)
-    raise ArgumentError, '取込項目が不正です．' unless (attributes.keys - ATTRIBUTE_FIELDS).empty?
-
-    attributes.each do |key, value|
-      case key
-      when *STRING_FIELDS
-        raise ArgumentError, '取込項目の型が不正です．' unless value.nil? || value.is_a?(String)
-      when *FLEXIBLE_FIELDS
-        unless value.nil? || value.is_a?(String) || value.is_a?(Numeric)
-          raise ArgumentError, '取込項目の型が不正です．'
-        end
-      end
-      if value.to_s.bytesize > PostImportUrlListParser::MAX_URL_BYTES
-        raise ArgumentError, '取込項目が大きすぎます．'
-      end
-    end
-  end
-  private_class_method :normalise_attributes!
-
-  def self.normalise_provenance! provenance
-    unless provenance.is_a?(Hash)
-      raise ArgumentError, 'provenance の形式が不正です．'
-    end
-
-    allowed = ATTRIBUTE_FIELDS + ['url']
-    unless (provenance.keys - allowed).empty?
-      raise ArgumentError, '値の由来が不正です．'
-    end
-    unless provenance.values.all? { ORIGINS.include?(_1) }
-      raise ArgumentError, '値の由来が不正です．'
-    end
-  end
-  private_class_method :normalise_provenance!
-
-  def self.normalise_tag_sources! tag_sources
-    return if tag_sources.nil?
-
-    unless tag_sources.is_a?(Hash)
-      raise ArgumentError, 'タグ由来の形式が不正です．'
-    end
-    unless (tag_sources.keys - ORIGINS).empty?
-      raise ArgumentError, 'タグ由来の形式が不正です．'
-    end
-    unless tag_sources.values.all? { _1.is_a?(String) }
-      raise ArgumentError, 'タグ由来の形式が不正です．'
-    end
-    if tag_sources.values.any? { _1.bytesize > PostImportUrlListParser::MAX_URL_BYTES }
-      raise ArgumentError, 'タグ由来が大きすぎます．'
-    end
-    if tag_sources.values.sum(&:bytesize) > PostImportUrlListParser::MAX_URL_BYTES
-      raise ArgumentError, 'タグ由来が大きすぎます．'
-    end
-  end
-  private_class_method :normalise_tag_sources!
-
-  def self.normalise_warning_values! normalised, allow_warning_fields:
-    return unless allow_warning_fields
-
-    field_warnings = normalised['field_warnings']
-    unless field_warnings.nil? || field_warnings.is_a?(Hash)
-      raise ArgumentError, '警告の形式が不正です．'
-    end
-    field_warnings&.each do |key, values|
-      unless ATTRIBUTE_FIELDS.include?(key) || key == 'url'
-        raise ArgumentError, '警告の形式が不正です．'
-      end
-      unless values.is_a?(Array) && values.all? { _1.is_a?(String) }
-        raise ArgumentError, '警告の形式が不正です．'
-      end
-      if values.any? { _1.bytesize > PostImportUrlListParser::MAX_URL_BYTES }
-        raise ArgumentError, '警告が大きすぎます．'
-      end
-    end
-
-    base_warnings = normalised['base_warnings']
-    return if base_warnings.nil?
-
-    unless base_warnings.is_a?(Array) && base_warnings.all? { _1.is_a?(String) }
-      raise ArgumentError, '警告の形式が不正です．'
-    end
-    if base_warnings.any? { _1.bytesize > PostImportUrlListParser::MAX_URL_BYTES }
-      raise ArgumentError, '警告が大きすぎます．'
-    end
-  end
-  private_class_method :normalise_warning_values!
-
-  def self.row_bytesize row
-    row.to_json.bytesize
-  end
-  private_class_method :row_bytesize
-end

backend/app/services/post_import_runner.rb

@@ -1,109 +0,0 @@
-class PostImportRunner
-  def initialize actor:, rows:
-    @actor = actor
-    @rows = rows
-  end
-
-  def run
-    normalised_rows = PostImportRowNormaliser.normalise!(@rows)
-    previews = PostImportPreviewer.new.preview_rows(rows: normalised_rows,
-                                                    fetch_metadata: false)
-    preview_map = previews.index_by { _1[:source_row] }
-
-    results = normalised_rows.map do |row|
-      run_row(row, preview_map.fetch(row['source_row']))
-    end
-
-    { created: results.count { _1[:status] == 'created' },
-      skipped: results.count { _1[:status] == 'skipped' },
-      failed: results.count { _1[:status] == 'failed' },
-      rows: results }
-  end
-
-  private
-
-  def run_row row, preview
-    attributes = row.fetch('attributes', { }).transform_keys { _1.to_s.underscore }
-    return { source_row: row['source_row'],
-             status: 'failed',
-             errors: preview[:validation_errors],
-             recoverable: true } if preview[:validation_errors].present?
-    if preview[:skip_reason] == 'existing'
-      return { source_row: row['source_row'],
-               status: 'skipped',
-               existing_post_id: preview[:existing_post_id] }
-    end
-
-    attributes['tags'] = preview[:attributes]['tags']
-    attributes['url'] = row['url']
-    creator = PostCreator.new(actor: @actor, attributes:)
-    post = creator.create!
-    result = { source_row: row['source_row'], status: 'created', post: PostRepr.base(post) }
-    if creator.field_warnings.present?
-      result[:field_warnings] = creator.field_warnings
-    end
-    result
-  rescue ActiveRecord::RecordInvalid => e
-    existing_post = existing_post_for_race(row, e.record)
-    if existing_post
-      return { source_row: row['source_row'],
-               status: 'skipped',
-               existing_post_id: existing_post.id }
-    end
-
-    { source_row: row['source_row'],
-      status: 'failed',
-      errors: e.record.errors.to_hash,
-      recoverable: true }
-  rescue ActiveRecord::RecordNotUnique => e
-    raise unless url_record_not_unique?(e)
-
-    existing_post = existing_post_for_race(row)
-    raise unless existing_post
-
-    { source_row: row['source_row'],
-      status: 'skipped',
-      existing_post_id: existing_post.id }
-  rescue Tag::NicoTagNormalisationError
-    { source_row: row['source_row'],
-      status: 'failed',
-      errors: { tags: ['ニコニコ・タグは直接指定できません．'] },
-      recoverable: true }
-  rescue Tag::DeprecatedTagNormalisationError
-    { source_row: row['source_row'],
-      status: 'failed',
-      errors: { tags: ['廃止済みタグは付与できません．'] },
-      recoverable: true }
-  rescue PostCreator::VideoMsParseError
-    { source_row: row['source_row'],
-      status: 'failed',
-      errors: { duration: ['動画時間の記法が不正です．'] },
-      recoverable: true }
-  rescue ArgumentError
-    { source_row: row['source_row'],
-      status: 'failed',
-      errors: { base: ['入力値が不正です．'] },
-      recoverable: true }
-  rescue StandardError => e
-    Rails.logger.error("post_import_runner_failure #{ { error: e.class.name,
-                                                        message: e.message }.to_json }")
-    { source_row: row['source_row'],
-      status: 'failed',
-      errors: { base: ['登録中にエラーが発生しました．'] } }
-  end
-
-  def existing_post_for_race row, record = nil
-    if record && !(record.errors.of_kind?(:url, :taken))
-      return nil
-    end
-
-    normal_url = PostUrlNormaliser.normalise(row['url'])
-    return nil if normal_url.blank?
-
-    Post.find_by(url: normal_url)
-  end
-
-  def url_record_not_unique? error
-    error.message.include?('index_posts_on_url')
-  end
-end

backend/app/services/post_import_url_list_parser.rb

@@ -1,25 +0,0 @@
-class PostImportUrlListParser
-  MAX_ROWS = 100
-  MAX_BYTES = 1.megabyte
-  MAX_URL_BYTES = 20.kilobytes
-
-  def self.parse source
-    raw = source.to_s
-    raise ArgumentError, '入力が大きすぎます．' if raw.bytesize > MAX_BYTES
-
-    rows = raw.split(/\r\n|\n|\r/).each_with_index.filter_map { |line, index|
-      url = line.strip
-      next if url.blank?
-
-      if url.bytesize > MAX_URL_BYTES
-        raise ArgumentError, "#{ index + 1 } 行目: URL が長すぎます．"
-      end
-
-      { source_row: index + 1, url: }
-    }
-    raise ArgumentError, 'URL を入力してください．' if rows.empty?
-    raise ArgumentError, "取込件数は #{ MAX_ROWS } 件までです．" if rows.length > MAX_ROWS
-
-    rows
-  end
-end

backend/app/services/post_metadata_fetcher.rb

@@ -3,10 +3,9 @@ require 'date'
 
 class PostMetadataFetcher
   TIMESTAMP_PATTERN =
-      Regexp.new(
+      /\A(\d{4})-(\d{2})-(\d{2})T(\d{2})
-          '\A(\d{4})-(\d{2})-(\d{2})T(\d{2})' \
+        (?::(\d{2})(?::(\d{2})(?:\.(\d+))?)?)?
-          '(?::(\d{2})(?::(\d{2})(?:\.(\d+))?)?)?' \
+        (Z|[+-]\d{2}:?\d{2})?\z/x
-          '(Z|[+-]\d{2}:?\d{2})?\z')
 
   def self.fetch raw_url
     uri, = Preview::UrlSafety.validate(raw_url)
@@ -32,6 +31,7 @@ class PostMetadataFetcher
       original_created_from: serialise_time(created_range&.first),
       original_created_before: serialise_time(created_range&.last),
       duration: serialise_duration(duration),
+      display_tags: display_tags(platform_tags),
       tags: platform_tags.join(' ') }
   end
 
@@ -42,6 +42,24 @@ class PostMetadataFetcher
     []
   end
 
+  def self.display_tags names
+    return [] if names.blank?
+
+    existing_tags =
+        Tag
+        .joins(:tag_names)
+        .where(tag_names: { language_code: 'ja', primary_flg: true, name: names })
+        .index_by { _1.name('ja') }
+
+    names.map { |name|
+      tag = existing_tags[name]
+      {
+          name: name,
+          category: (tag&.category || 'meta'),
+          section_literals: [] }
+    }
+  end
+
   def self.original_created_range value
     return nil if value.blank?
 
@@ -168,6 +186,7 @@ class PostMetadataFetcher
   end
 
   private_class_method :platform_tags,
+                       :display_tags,
                        :original_created_range,
                        :parse_timestamp_range,
                        :parse_nanoseconds,

backend/app/services/post_tag_search.rb

@@ -0,0 +1,42 @@
+# Shared literal parsing and reference resolution for current and snapshot post search.
+class PostTagSearch
+  def self.parse_references(raw, literals)
+    return nil unless raw.key?('tag_refs')
+
+    references = QueryState.records(raw, 'tag_refs')
+    valid = references.is_a?(Array) && references.all? do |reference|
+      next false unless reference.is_a?(Hash) && reference['name'].is_a?(String)
+
+      ids = [reference['tag_id'], reference['external_tag_id']].compact
+      ((ids.any? || reference['missing'] == true) &&
+          ids.all? { _1.is_a?(Integer) && _1.positive? } &&
+          (!reference.key?('missing') || (reference['missing'] == true && ids.empty?)) &&
+          (!reference.key?('negative') || [true, false].include?(reference['negative'])))
+    end
+    raise ArgumentError unless valid
+
+    references.map { _1['name'] } == literals ? references : nil
+  end
+
+  def self.literals(locale, raw, references: nil)
+    raw.split.each_with_index.filter_map do |token, index|
+      next if token.downcase == 'type:or'
+
+      reference = references&.[](index)
+      name = token.sub(/\Anot:/i, '')
+      name = TagName.canonicalise(locale, name).first unless reference
+      negative = reference&.key?('negative') ? reference['negative'] :
+          token.downcase.start_with?('not:')
+      { name:, negative:, reference: }
+    end
+  end
+
+  def self.references(locale, literal)
+    reference = literal[:reference]
+    return [reference['tag_id'], reference['external_tag_id']] if reference
+
+    [TagName.where(language_code: locale.language_code, name: literal[:name])
+         .distinct.pluck(:tag_id),
+     ExternalTag.where("CONCAT(platform, ':', name) = ?", literal[:name]).pluck(:id)]
+  end
+end

backend/app/services/post_thumbnail_attachment_builder.rb

@@ -0,0 +1,11 @@
+class PostThumbnailAttachmentBuilder
+  def self.build thumbnail:, thumbnail_base:
+    if thumbnail_base.present?
+      return Post.remote_thumbnail_attachment(thumbnail_base)
+    end
+
+    return nil if thumbnail.blank?
+
+    Post.resized_thumbnail_attachment(thumbnail)
+  end
+end

backend/app/services/post_thumbnail_upload_validator.rb

@@ -0,0 +1,34 @@
+class PostThumbnailUploadValidator
+  MAX_THUMBNAIL_BYTES = 20 * 1024 * 1024
+  ALLOWED_CONTENT_TYPES = Preview::ThumbnailFetcher::RASTER_IMAGE_CONTENT_TYPES.freeze
+
+  class InvalidUpload < StandardError; end
+
+  def self.validate! thumbnail
+    return if thumbnail.blank?
+    unless thumbnail.is_a?(ActionDispatch::Http::UploadedFile)
+      raise InvalidUpload, 'thumbnail upload が不正です．'
+    end
+    raise InvalidUpload, 'thumbnail file size が大きすぎます．' if thumbnail.size > MAX_THUMBNAIL_BYTES
+    raise InvalidUpload, 'サムネイル画像の形式が不正です．' unless allowed_content_type?(thumbnail.content_type)
+    bytes = thumbnail.read
+    raise InvalidUpload, 'サムネイル画像の形式が不正です．' if Post.svg_document_bytes?(bytes)
+    unless Post.raster_thumbnail_bytes?(bytes)
+      raise InvalidUpload, 'サムネイル画像の形式が不正です．'
+    end
+
+    attachment = Post.resized_thumbnail_attachment(
+        StringIO.new(bytes),
+        content_type: thumbnail.content_type)
+    attachment[:io].close if attachment[:io].respond_to?(:close)
+  rescue MiniMagick::Error, Timeout::Error
+    raise InvalidUpload, 'サムネイル画像の変換に失敗しました．'
+  ensure
+    thumbnail&.rewind if thumbnail.respond_to?(:rewind)
+  end
+
+  def self.allowed_content_type? content_type
+    mime_type = content_type.to_s.split(';', 2).first.to_s.downcase.strip
+    ALLOWED_CONTENT_TYPES.include?(mime_type)
+  end
+end