#55 マイグレーション・ファイルのみ

AGENTS.md

@@ -1,18 +1,5 @@
 # AGENTS.md
 
-## Mandatory security and URL contracts
-
-- URL クエリへの JSON 格納を禁止する。Base64・16進数等による迂回も禁止。
-  単純な配列は半角スペース区切り、構造化条件は番号付きの個別キーで表す。
-  未指定と明示的な空選択を区別し、自由入力を区切り文字で分割しない。
-  Frontend / Backend で同じ文法・上限・検証を使い、新しい URL 状態の追加前に確認する。
-- 一度除票されたリソースの除票以前（除票版を含む）の履歴は admin 以外へ公開しない。
-  恢復しても解除しない。すべての後続除票を確定版番号と正しい複合キーで判定する。
-  本文・属性・ID・版番号・関連参照・旧値を API / SSR / キャッシュ / 出力から漏らさない。
-  サーバーが確定した current_user の role のみを認可に使用する。
-  秘匿版も安全な全体件数には含めるが、公開明細・内訳・検索件数とは分離する。
-  秘匿属性を条件・軸・ソート・差分件数から推測できる実装を禁止する。
-
 ## Project overview
 
 BTRC Hub / タグ広場 is a split Rails API and React frontend repository.
@@ -586,10 +573,10 @@ and layout reuse, follow `frontend/AGENTS.md`.
 - When new user-facing wording appears necessary, ask the user for the exact
   wording and placement before implementing it.
 - Do not invent replacement copy when removing unrequested wording.
-- 明示的な個別指示がない限り、テストの追加・修正・削除・実行を一切行わない。
+- Do not create, modify, or run tests unless the user explicitly asks for
-  fixture、mock、snapshot、テスト補助コードも対象とする。
+  test work. When the user asks for tests, keep working within the permitted
-  既存のテスト差分を勝手に修正・差戻ししない。
+  test-file scope and rerun them until they pass or the remaining failure is
-  この規約はディレクトリ固有の自動的なテスト実行推奨より優先する。
+  clearly blocked.
 - Test-only work includes adding, updating, deleting, reorganising, or fixing
   SyntaxError in tests. During test-only work, do not modify production code.
 - During test-only work, do not change production constants, behaviour, API
@@ -644,19 +631,6 @@ and layout reuse, follow `frontend/AGENTS.md`.
 
 ## Frontend rules
 
-- 原則として、ユーザ向け UI 文言を TSX / TypeScript に直接記述しない。
-  新しい文言は既存の i18n システムで管理し、適切な既存キーを先に確認する。
-  不必要な重複定義を避け、新規機能固有の文言は適切な namespace にまとめる。
-- 対応 locale の辞書を整合させる。未翻訳項目には既存の日本語 fallback と
-  `_untranslated` 規約を使い、language metadata を維持する。
-  翻訳を捏造せず、未翻訳文字列を正式な翻訳として登録しない。
-- UI 文言の追加・変更時はビルド、静的検査、実画面の確認等で、辞書と
-  SSR / hydration の整合及び翻訳キーの非露出を検証する。
-  i18n の検証を理由にテスト関連作業を開始しない。
-- API の内部識別子、パラメータ、ユーザ入力データ、外部 URL は機械的に
-  翻訳しない。ボタン、項目名、CSV ヘッダ、エラー、空状態、ツールチップ、
-  アクセシビリティ用ラベルも UI 文言として扱う。
-
 - Use `frontend/src/lib/api.ts` for API calls so headers and camelCase conversion stay consistent.
 - Add or reuse TanStack Query keys through `frontend/src/lib/queryKeys.ts`;
   avoid ad hoc query key arrays.

backend/AGENTS.md

@@ -1,20 +1,5 @@
 # backend/AGENTS.md
 
-## Mandatory security and URL contracts
-
-- URL クエリへの JSON 格納を禁止する。Base64・16進数等による迂回も禁止。
-  単純な配列は半角スペース区切り、構造化条件は番号付きの個別キーで表す。
-  未指定と明示的な空選択を区別し、自由入力を区切り文字で分割しない。
-  Frontend / Backend で同じ文法・上限・検証を使い、新しい URL 状態の追加前に確認する。
-- 一度除票されたリソースの除票以前（除票版を含む）の履歴は admin 以外へ公開しない。
-  恢復しても解除しない。すべての後続除票を確定版番号と正しい複合キーで判定する。
-  本文・属性・ID・版番号・関連参照・旧値を API / SSR / キャッシュ / 出力から漏らさない。
-  サーバーが確定した current_user の role のみを認可に使用する。
-  秘匿版も安全な全体件数には含めるが、公開明細・内訳・検索件数とは分離する。
-  秘匿属性を条件・軸・ソート・差分件数から推測できる実装を禁止する。
-- 履歴読み取りは HistoryAccess の共通認可を必須とする。書込用 relation を公開しない。
-  export は生成時と取得時に認可し、権限変更・除票後に古いファイルを再利用しない。
-
 ## Scope
 
 These rules apply to work under `backend/`.

backend/app/controllers/application_controller.rb

@@ -1,63 +1,20 @@
 class ApplicationController < ActionController::API
-  include RequestAccess
-  include ActionController::Cookies
-  include ActionController::RequestForgeryProtection
-
-  class UnsupportedLocale < StandardError
-    ;
-  end
-
   rescue_from ActiveRecord::RecordInvalid, with: :render_record_invalid
   rescue_from ActiveRecord::RecordNotUnique, with: :render_record_not_unique
-  rescue_from(ActiveRecord::RecordNotFound) { head :not_found }
-  rescue_from(UnsupportedLocale) { head :not_found }
 
   before_action :reject_banned_ip_address!
   before_action :authenticate_user
   before_action :reject_banned_user!
-  before_action :validate_explicit_locale!
+
-  before_action :enforce_query_contract!
+  def current_user = @current_user
-  around_action :with_history_authority
-  rescue_from(HistoryAccess::Forbidden) { head :not_found }
-  protect_from_forgery with: :exception
-  self.forgery_protection_origin_check = true
-  rescue_from ActionController::InvalidAuthenticityToken, with: :reject_invalid_csrf
-  after_action :prevent_authenticated_caching
 
   private
 
-  def enforce_query_contract!
+  def authenticate_user
-    pairs = URI.decode_www_form(request.query_string)
+    code = request.headers['X-Transfer-Code'] || request.headers['HTTP_X_TRANSFER_CODE']
-    return head :bad_request if pairs.length > 1000 || pairs.map(&:first).uniq.length != pairs.length
+    return if code.blank?
-    return head :bad_request if request.query_parameters.values.any? { !_1.is_a?(String) }
-    if (controller_name.start_with?('counts_') &&
-        (request.query_parameters.keys & ['headers', 'labels']).any?)
-      return head :bad_request
-    end
-  rescue ArgumentError
-    head :bad_request
-  end
 
-  def with_history_authority(&action)
+    @current_user = User.find_by(inheritance_code: code)
-    response.headers['Cache-Control'] = 'private, no-store'
-    HistoryAccess.set(user: current_user, &action)
-  end
-
-  def reject_invalid_csrf
-    response.headers['X-CSRF-Invalid'] = '1'
-    response.headers['Cache-Control'] = 'private, no-store'
-    head :forbidden
-  end
-
-  def prevent_authenticated_caching
-    response.headers['Cache-Control'] = 'private, no-store' if current_user
-  end
-
-  def establish_user_session user
-    reset_session
-    session[:inheritance_code] = user.inheritance_code
-    @current_user = user
-    response.headers['Cache-Control'] = 'private, no-store'
   end
 
   def bool? key, default: false
@@ -92,6 +49,19 @@ class ApplicationController < ActionController::API
     render_validation_error base: ['すでに存在してゐます．']
   end
 
+  def reject_banned_ip_address!
+    ip_address = IpAddress.find_by(ip_address: IPAddr.new(request.remote_ip).hton)
+    return unless ip_address&.banned?
+
+    head :forbidden
+  end
+
+  def reject_banned_user!
+    return unless current_user&.banned?
+
+    head :forbidden
+  end
+
   def render_validation_error record = nil, fields: { }, base: [], status: :unprocessable_entity
     errors = { }
 
@@ -115,29 +85,4 @@ class ApplicationController < ActionController::API
                    base_errors: },
            status:
   end
-
-  def normalise_json value
-    return nil if value.nil?
-    return JSON.parse(value) if value.is_a?(String)
-
-    value
-  end
-
-  def resolve_locale!
-    return @resolved_locale if defined?(@resolved_locale)
-
-    code = params.fetch(:locale, 'ja')
-    unless code.is_a?(String) && Locale.codes.value?(code)
-      raise UnsupportedLocale
-    end
-
-    locale = Locale.find_by(code:)
-    raise UnsupportedLocale unless locale && locale.deprecated_at.nil?
-
-    @resolved_locale = locale
-  end
-
-  def validate_explicit_locale!
-    resolve_locale! if params.key?(:locale)
-  end
 end

backend/app/controllers/concerns/counts_listing.rb

@@ -1,21 +0,0 @@
-module CountsListing
-  extend ActiveSupport::Concern
-
-  def index
-    locale = resolve_locale!
-    resource = is_a?(CountsPostsController) ? 'posts' : 'tags'
-    query = CountsQuery.new(version_scope, resource, request.query_parameters.except('locale')
-        .merge('locale' => locale.code))
-    result = { population_count_scope: 'all_recorded_versions', population_count: HistoryAccess.population_count(query.relation.klass), count: query.count, page: query.page, limit: query.limit,
-               at: request.query_parameters['at'].presence || Time.current.utc.iso8601(6) }
-    if query.mode == 'summary'
-      result.merge!(query.summary)
-    else
-      result[:versions] = CountsVersionRepr.many(
-          query.details, resource:, language_code: locale.language_code)
-    end
-    render json: result
-  rescue ArgumentError, TypeError
-    head :bad_request
-  end
-end

backend/app/controllers/concerns/history_storage_access.rb

@@ -1,52 +0,0 @@
-# Permanent storage URLs must not bypass a resource's later discard boundary.
-module HistoryStorageAccess
-  extend ActiveSupport::Concern
-
-  included do
-    include RequestAccess
-    prepend_before_action :authorise_history_storage!
-  end
-
-  private
-
-  def authorise_history_storage!
-    response.headers['Cache-Control'] = 'private, no-store'
-    return unless request.get? || request.head?
-
-    reject_banned_ip_address!
-    return if performed?
-
-    authenticate_user
-    reject_banned_user!
-    return if performed?
-
-    signed = params[:signed_blob_id] || params[:signed_id]
-    blob = if signed
-             ActiveStorage::Blob.find_signed(signed)
-           elsif params[:encoded_key]
-             payload = ActiveStorage.verifier.verified(params[:encoded_key], purpose: :blob_key)
-             key = payload && (payload[:key] || payload['key'])
-             if key
-               original = key.start_with?('variants/') ? key.split('/')[1] : key
-               ActiveStorage::Blob.find_by(key: original)
-             end
-           end
-    return head :not_found unless blob
-
-    allowed = HistoryAccess.set(user: current_user) { HistoryAccess.blob_visible?(blob) }
-    head :not_found unless allowed
-  rescue ActiveSupport::MessageVerifier::InvalidSignature
-    head :not_found
-  end
-
-  # Override the engine's public-forever caching before a streaming response commits.
-  def http_cache_forever(**_options)
-    response.headers['Cache-Control'] = 'private, no-store'
-    yield
-  end
-
-  def expires_in(*arguments, **options)
-    super
-    response.headers['Cache-Control'] = 'private, no-store'
-  end
-end

backend/app/controllers/concerns/request_access.rb

@@ -1,28 +0,0 @@
-# Canonical read-side session and BAN boundary, shared by API and storage controllers.
-module RequestAccess
-  extend ActiveSupport::Concern
-
-  def current_user = @current_user
-
-  private
-
-  def authenticate_user
-    code = session[:inheritance_code]
-    return if code.blank?
-
-    @current_user = User.find_by(inheritance_code: code)
-  end
-
-  def reject_banned_ip_address!
-    ip_address = IpAddress.find_by(ip_address: IPAddr.new(request.remote_ip).hton)
-    return unless ip_address&.banned?
-
-    head :forbidden
-  end
-
-  def reject_banned_user!
-    return unless current_user&.banned?
-
-    head :forbidden
-  end
-end

backend/app/controllers/counts_exports_controller.rb

@@ -1,83 +0,0 @@
-class CountsExportsController < ApplicationController
-  wrap_parameters false
-
-  def create_job
-    locale = resolve_locale!
-    payload = request.request_parameters.merge(request.query_parameters)
-    session[:counts_export_owner] ||= SecureRandom.hex(32)
-    job = CountsExportJobs.start(payload, session[:counts_export_owner], locale.language_code)
-    response.headers['Cache-Control'] = 'private, no-store'
-    render json: { job: }, status: :accepted
-  rescue CountsExportDownload::Busy
-    head :too_many_requests
-  rescue ArgumentError, TypeError, KeyError, JSON::ParserError
-    head :bad_request
-  end
-
-  def job_status
-    job_state(cancel: false)
-  end
-
-  def cancel_job
-    job_state(cancel: true)
-  end
-
-  def prepare
-    locale = resolve_locale!
-    payload = request.request_parameters.merge(request.query_parameters)
-    exporter = CountsExporter.new(payload, language_code: locale.language_code)
-    session[:counts_export_owner] ||= SecureRandom.hex(32)
-    token = CountsExportDownload.prepare(exporter, session[:counts_export_owner])
-    response.headers['Cache-Control'] = 'private, no-store'
-    render json: { token: }
-  rescue CountsExportDownload::Busy
-    head :too_many_requests
-  rescue CountsExportDownload::TooLarge
-    head 413
-  rescue CountsExportDownload::TimedOut
-    head :gateway_timeout
-  rescue CountsExportDownload::StorageFull, Errno::ENOSPC
-    head :insufficient_storage
-  rescue ArgumentError, TypeError, KeyError, JSON::ParserError
-    head :bad_request
-  end
-
-  def download
-    file = CountsExportDownload.find(params[:token], session[:counts_export_owner])
-    return head :not_found unless file
-
-    response.headers['Cache-Control'] = 'private, no-store'
-    send_file file.fetch('path'),
-              filename: "counts-#{ file.fetch('resource') }.#{ file.fetch('format') }",
-              type: file['format'] == 'json' ? 'application/json' : 'text/csv',
-              disposition: 'attachment'
-  end
-
-  def job_state(cancel:)
-    result = CountsExportJobs.status(params[:token], session[:counts_export_owner], cancel:)
-    return head :not_found unless result
-
-    response.headers['Cache-Control'] = 'private, no-store'
-    render json: result
-  end
-  private :job_state
-
-  def index
-    locale = resolve_locale!
-    payload = request.request_parameters.except('authenticity_token')
-        .merge(request.query_parameters)
-    exporter = CountsExporter.new(payload,
-                                  language_code: locale.language_code)
-    session[:counts_export_owner] ||= SecureRandom.hex(32)
-    token = CountsExportDownload.prepare(exporter, session[:counts_export_owner])
-    file = CountsExportDownload.find(token, session[:counts_export_owner])
-    return head :not_found unless file
-
-    send_file file.fetch('path'),
-        filename: "counts-#{ exporter.resource }.#{ exporter.format }",
-        type: exporter.format == 'json' ? 'application/json' : 'text/csv',
-        disposition: 'attachment'
-  rescue ArgumentError, TypeError, KeyError, JSON::ParserError
-    head :bad_request
-  end
-end

backend/app/controllers/counts_posts_controller.rb

@@ -1,3 +0,0 @@
-class CountsPostsController < PostVersionsController
-  include CountsListing
-end

backend/app/controllers/counts_states_controller.rb

@@ -1,10 +0,0 @@
-class CountsStatesController < ApplicationController
-  def index
-    locale = resolve_locale!
-    result = CountsStateQuery.new(request.query_parameters,
-                                  language_code: locale.language_code).call
-    render json: result
-  rescue ArgumentError, TypeError
-    head :bad_request
-  end
-end

backend/app/controllers/counts_tables_controller.rb

@@ -1,23 +0,0 @@
-class CountsTablesController < ApplicationController
-  def definition
-    resolve_locale!
-    definition = CountsResourceDefinition.new(params[:resource])
-    render json: { fields: definition.metadata,
-                   output_fields: definition.output_fields,
-                   filter_fields: definition.filter_metadata,
-                   axes: { latest: definition.axes(false), history: definition.axes(true) },
-                   grains: CountsAxes::GRAINS, defaults: CountsDefaults.metadata(params[:resource]) }
-  rescue ArgumentError
-    head :bad_request
-  end
-
-  def index
-    resolve_locale!
-    render json: CountsTableQuery.new(request.query_parameters).call
-  rescue CountsTableQuery::TooManyColumns
-    render json: { error: 'too_many_columns', max_columns: CountsTableQuery::MAX_COLUMNS },
-           status: :unprocessable_entity
-  rescue ArgumentError, TypeError, KeyError, JSON::ParserError
-    head :bad_request
-  end
-end

backend/app/controllers/counts_tags_controller.rb

@@ -1,3 +0,0 @@
-class CountsTagsController < TagVersionsController
-  include CountsListing
-end

backend/app/controllers/deerjikists_controller.rb

@@ -1,7 +1,5 @@
 class DeerjikistsController < ApplicationController
   def show
-    locale = resolve_locale!
-
     platform = params[:platform].to_s.strip
     code = params[:code].to_s.strip
     return render_bad_request('platform は必須です．') if platform.blank?
@@ -9,10 +7,10 @@ class DeerjikistsController < ApplicationController
 
     deerjikist = Deerjikist
                  .joins(:tag)
-                 .includes(tag: :tag_names)
+                 .includes(tag: :tag_name)
                  .find_by(platform:, code:)
     if deerjikist
-      render json: DeerjikistRepr.base(deerjikist, language_code: locale.language_code)
+      render json: DeerjikistRepr.base(deerjikist)
     else
       head :not_found
     end
@@ -22,7 +20,6 @@ class DeerjikistsController < ApplicationController
     return head :unauthorized unless current_user
     return head :forbidden unless current_user.gte_member?
 
-    locale = resolve_locale!
     platform = params[:platform].to_s.strip
     code = params[:code].to_s.strip
     tag_id = params[:tag_id].to_i
@@ -35,7 +32,7 @@ class DeerjikistsController < ApplicationController
       d.save!
     end
 
-    render json: DeerjikistRepr.base(deerjikist, language_code: locale.language_code)
+    render json: DeerjikistRepr.base(deerjikist)
   end
 
   def destroy

backend/app/controllers/gekanator_posts_controller.rb

@@ -2,7 +2,7 @@ class GekanatorPostsController < ApplicationController
   def index
     posts =
         Post
-        .preload(:post_similarities, tags: :tag_names)
+        .preload(:post_similarities, tags: :tag_name)
         .with_attached_thumbnail
         .order(Arel.sql(
             'COALESCE(posts.original_created_before - INTERVAL 1 MINUTE, ' \
@@ -56,7 +56,7 @@ class GekanatorPostsController < ApplicationController
   def tag_json tag
     {
       id: tag.id,
-      name: tag.name('ja'),
+      name: tag.name,
       category: tag.category
     }
   end

backend/app/controllers/gekanator_questions_controller.rb

@@ -126,9 +126,9 @@ class GekanatorQuestionsController < ApplicationController
     end
 
     Tag
-        .joins(:tag_names)
+        .joins(:tag_name)
         .where(category: categories.uniq)
-        .where(tag_names: { language_code: 'ja', primary_flg: true, name: names.uniq })
+        .where(tag_names: { name: names.uniq })
         .where.not(deprecated_at: nil)
         .pluck('tags.category', 'tag_names.name')
         .each_with_object({ }) do |(category, name), h|

backend/app/controllers/locales_controller.rb

@@ -1,10 +0,0 @@
-class LocalesController < ApplicationController
-  def index
-    locales = Locale.where(deprecated_at: nil).order(:created_at).map do |locale|
-      { code: locale.code_before_type_cast, name: locale.name }
-    end
-
-    response.headers['Cache-Control'] = 'no-store'
-    render json: { locales: }
-  end
-end

backend/app/controllers/material_versions_controller.rb

@@ -8,7 +8,7 @@ class MaterialVersionsController < ApplicationController
 
     offset = (page - 1) * limit
 
-    q = HistoryAccess.visible(MaterialVersion.includes(:created_by_user))
+    q = MaterialVersion.includes(:created_by_user)
     q = q.where(material_id: params[:material_id]) if params[:material_id].present?
     q = q.where(tag_name: params[:tag].to_s.strip) if params[:tag].present?
     q = q.where(event_type: params[:event_type]) if params[:event_type].present?
@@ -20,7 +20,7 @@ class MaterialVersionsController < ApplicationController
         .limit(limit)
         .offset(offset)
 
-    render json: { versions: serialise_versions(versions), count:, population_count: HistoryAccess.population_count(MaterialVersion) }
+    render json: { versions: serialise_versions(versions), count: }
   end
 
   private

backend/app/controllers/materials_controller.rb

@@ -7,8 +7,6 @@ class MaterialsController < ApplicationController
   rescue_from MaterialZipExporter::MissingFileError, with: :render_zip_missing_file
 
   def index
-    locale = resolve_locale!
-
     page = (params[:page].presence || 1).to_i
     limit = (params[:limit].presence || 20).to_i
 
@@ -22,7 +20,7 @@ class MaterialsController < ApplicationController
     q = Material.includes(:material_export_items,
                           thumbnail_attachment: :blob,
                           file_attachment: :blob,
-                          tag: :tag_names)
+                          tag: :tag_name)
     q = q.where(tag_id: nil) if filters[:tag_state] == 'untagged'
     q = q.where.not(tag_id: nil) if filters[:tag_state] == 'tagged'
     q = apply_material_tag_filter(q, filters, tag_graph)
@@ -44,10 +42,7 @@ class MaterialsController < ApplicationController
         .offset(offset)
         .to_a
 
-    response = { materials: MaterialRepr.list_many(
+    response = { materials: MaterialRepr.list_many(materials, host: request.base_url),
-                   materials,
-                   host: request.base_url,
-                   language_code: locale.language_code),
                  count: }
     if filters[:tag_id]
       response[:tag_scope] = material_index_tag_scope(filters, tag_graph)
@@ -60,8 +55,6 @@ class MaterialsController < ApplicationController
   end
 
   def show
-    locale = resolve_locale!
-
     material =
         Material
         .includes(:tag, :material_export_items)
@@ -70,20 +63,15 @@ class MaterialsController < ApplicationController
         .find_by(id: params[:id])
     return head :not_found unless material
 
-    wiki_page_body =
+    wiki_page_body = material.tag&.tag_name&.wiki_page&.current_revision&.body
-        TagDescription
-        .find_by(tag_id: material.tag_id, language_code: locale.language_code)
-        &.body
 
-    render json: MaterialRepr.base(material, host: request.base_url,
+    render json: MaterialRepr.base(material, host: request.base_url).merge(wiki_page_body:)
-                                   language_code: locale.language_code).merge(wiki_page_body:)
   end
 
   def create
     return head :unauthorized unless current_user
     return head :forbidden unless current_user.gte_member?
 
-    locale = resolve_locale!
     tag_name_raw = params[:tag].to_s.strip
     file = params[:file]
     file_sha256 = MaterialFileSha256.from_upload(file)
@@ -101,14 +89,13 @@ class MaterialsController < ApplicationController
 
     begin
       Material.transaction do
-        tag = resolve_material_tag!(locale, tag_name_raw)
+        tag = resolve_material_tag!(tag_name_raw)
         material = Material.new(tag:, url:,
                                 created_by_user: current_user,
                                 updated_by_user: current_user)
         material.file.attach(uploaded_blob) if uploaded_blob
         material.save!
-        TagVersioning.record_tag_snapshot!(tag, created_by_user: current_user,
+        TagVersioning.record_tag_snapshot!(tag, created_by_user: current_user)
-                                           language_code: locale.language_code)
         upsert_export_paths!(material)
         MaterialVersionRecorder.record!(material:, event_type: :create,
                                         created_by_user: current_user)
@@ -120,8 +107,7 @@ class MaterialsController < ApplicationController
 
     if material
       log_thumbnail_generation(material, MaterialThumbnailGenerator.generate!(material))
-      render json: MaterialRepr.base(material, host: request.base_url,
+      render json: MaterialRepr.base(material, host: request.base_url), status: :created
-                                     language_code: locale.language_code), status: :created
     else
       render_validation_error material
     end
@@ -146,14 +132,12 @@ class MaterialsController < ApplicationController
     block = MaterialImportBlockMatcher.match_for_sha256(file_sha256)
     return render_material_import_block(block) if block
 
-    locale = resolve_locale!
-
     uploaded_blob = build_uploaded_material_blob!(file, file_sha256)
 
     begin
       Material.transaction do
         MaterialVersionRecorder.ensure_snapshot!(material, created_by_user: current_user)
-        tag = resolve_material_tag!(locale, tag_name_raw)
+        tag = resolve_material_tag!(tag_name_raw)
         material.assign_attributes(tag:, url:, updated_by_user: current_user)
         if uploaded_blob
           material.file.attach(uploaded_blob)
@@ -161,8 +145,7 @@ class MaterialsController < ApplicationController
           material.file.detach
         end
         material.save!
-        TagVersioning.record_tag_snapshot!(tag, created_by_user: current_user,
+        TagVersioning.record_tag_snapshot!(tag, created_by_user: current_user)
-                                           language_code: locale.language_code)
         upsert_export_paths!(material)
         MaterialVersionRecorder.record!(material:, event_type: :update,
                                         created_by_user: current_user)
@@ -174,8 +157,7 @@ class MaterialsController < ApplicationController
 
     log_thumbnail_generation(material, MaterialThumbnailGenerator.generate!(material))
 
-    render json: MaterialRepr.base(material, host: request.base_url,
+    render json: MaterialRepr.base(material, host: request.base_url)
-                                   language_code: locale.language_code)
   end
 
   def destroy
@@ -254,9 +236,10 @@ class MaterialsController < ApplicationController
     nil
   end
 
-  def resolve_material_tag! locale, tag_name_raw
+  def resolve_material_tag! tag_name_raw
-    Tag.find_or_create_by_tag_name!(locale, tag_name_raw, category: :material,
+    tag_name = TagName.find_undiscard_or_create_by!(name: tag_name_raw)
-                                   created_by_user: current_user)
+    tag = tag_name.tag
+    tag || Tag.create!(tag_name:, category: :material)
   end
 
   def material_index_needs_tag_name? filters
@@ -270,12 +253,7 @@ class MaterialsController < ApplicationController
   end
 
   def material_index_join_tag_name q
-    q.left_joins(:tag).joins(<<~SQL.squish)
+    q.left_joins(tag: :tag_name)
-      LEFT JOIN tag_names
-        ON tag_names.tag_id = tags.id
-       AND tag_names.language_code = 'ja'
-       AND tag_names.primary_flg = TRUE
-    SQL
   end
 
   def material_index_join_file_blob q
@@ -291,11 +269,9 @@ class MaterialsController < ApplicationController
 
   def apply_material_query q, term
     like = "%#{ ActiveRecord::Base.sanitize_sql_like(term) }%"
-    q.where(<<~SQL.squish, q: like)
+    q.where('tag_names.name LIKE :q OR materials.url LIKE :q OR ' \
-      tag_names.name LIKE :q
+            'material_file_blobs.filename LIKE :q',
-      OR materials.url LIKE :q
+            q: like)
-      OR material_file_blobs.filename LIKE :q
-    SQL
   end
 
   def apply_material_media_kind q, media_kind
@@ -347,9 +323,8 @@ class MaterialsController < ApplicationController
         end
     tags_by_id =
         Tag
-        .joins(:tag_names)
+        .joins(:tag_name)
-        .where(id: scope_tag_ids,
+        .where(id: scope_tag_ids)
-               tag_names: { language_code: 'ja', primary_flg: true })
         .pluck('tags.id', 'tag_names.name', 'tags.category', 'tags.deprecated_at')
         .each_with_object({ }) do |(id, name, category, deprecated_at), hash|
           hash[id] = { id:, name:, category:, deprecated: deprecated_at.present? }

backend/app/controllers/nico_tags_controller.rb

@@ -1,7 +1,5 @@
 class NicoTagsController < ApplicationController
   def index
-    locale = resolve_locale!
-
     name = params[:name].presence
     linked_tag = params[:linked_tag].presence
     link_status = params[:link_status].presence
@@ -17,36 +15,30 @@ class NicoTagsController < ApplicationController
     limit = 1 if limit < 1
 
     post_tag_max_sql =
-        PostExternalTag
+        PostTag
-        .select('external_tag_id, MAX(created_at) AS max_created_at')
+        .select('tag_id, MAX(created_at) AS max_created_at')
-        .group('external_tag_id')
+        .group('tag_id')
         .to_sql
 
-    q =
+    q = Tag.nico_tags
-        ExternalTag
+           .joins(:tag_name)
-        .joins("LEFT JOIN (#{ post_tag_max_sql }) post_tag_max " +
+           .joins("LEFT JOIN (#{ post_tag_max_sql }) post_tag_max " \
-               'ON post_tag_max.external_tag_id = external_tags.id')
+                  'ON post_tag_max.tag_id = tags.id')
-        .includes(linked_tags: [:tag_names, :tag_descriptions])
+           .includes(:tag_name, tag_name: :wiki_page, linked_tags: { tag_name: :wiki_page })
-    if name
+    q = q.where('tag_names.name LIKE ?', "%#{ name }%") if name
-      q = q.where(('external_tags.name LIKE ? ' +
-                   "OR CONCAT(external_tags.platform, ':', external_tags.name) LIKE ?"),
-                  "%#{ name }%", "%#{ name }")
-    end
-
     if linked_tag
       linked_tag_ids =
-          Tag
+        Tag
-          .joins(:tag_names)
+        .joins(:tag_name)
-          .where(tag_names: { language_code: locale.language_code, primary_flg: true })
+        .where('tag_names.name LIKE ?', "%#{ linked_tag }%")
-          .where('tag_names.name LIKE ?', "%#{ linked_tag }%")
+        .pluck(:id)
-          .pluck(:id)
       linked_nico_tag_ids = NicoTagRelation.where(tag_id: linked_tag_ids).pluck(:nico_tag_id)
       q = q.where(id: linked_nico_tag_ids)
     end
     if link_status.in?(['linked', 'unlinked'])
       exists_sql =
-          'EXISTS (SELECT 1 FROM nico_tag_relations ' +
+          'EXISTS (SELECT 1 FROM nico_tag_relations ' \
-          'WHERE nico_tag_relations.nico_tag_id = external_tags.id)'
+          'WHERE nico_tag_relations.nico_tag_id = tags.id)'
       q = link_status == 'linked' ? q.where(exists_sql) : q.where("NOT #{ exists_sql }")
     end
 
@@ -54,69 +46,54 @@ class NicoTagsController < ApplicationController
     sort_sql =
         case order[0]
         when 'name'
-          'external_tags.name'
+          'tag_names.name'
         when 'updated_at'
           'post_tag_max.max_created_at'
         else
-          "external_tags.#{ order[0] }"
+          "tags.#{ order[0] }"
         end
-    tags = q.reselect('external_tags.*',
+    tags = q.reselect('tags.*',
                       Arel.sql('post_tag_max.max_created_at AS recent_post_tag_created_at'))
-            .order(Arel.sql("#{ sort_sql } #{ order[1] }, external_tags.id #{ order[1] }"))
+            .order(Arel.sql("#{ sort_sql } #{ order[1] }, tags.id #{ order[1] }"))
             .limit(limit)
             .offset((page - 1) * limit)
             .to_a
 
     render json: { tags: tags.map { |tag|
-      external_tag_json(tag).merge(
+      TagRepr.base(tag).merge(
           recent_post_tag_created_at: tag.recent_post_tag_created_at,
-          linked_tags: tag.linked_tags.map { |lt|
+          linked_tags: tag.linked_tags.map { |lt| TagRepr.base(lt) })
-            TagRepr.base(lt, language_code: locale.language_code)
-          })
     }, count: }
   end
 
-  def show
-    tag = ExternalTag.find(params[:id])
-
-    render json: external_tag_json(tag)
-  end
-
   def update
     return head :unauthorized unless current_user
     return head :forbidden unless current_user.gte_member?
 
-    locale = resolve_locale!
-
     id = params[:id].to_i
 
-    tag = ExternalTag.find(id)
+    tag = Tag.find(id)
+    return render_bad_request('ニコニコ・タグを指定してください．') unless tag.nico?
 
     linked_tag_names = params[:tags].to_s.split
     linked_tags = nil
 
     ApplicationRecord.transaction do
-      linked_tags = Tag.normalise_tags!(locale, linked_tag_names,
+      linked_tags = Tag.normalise_tags!(linked_tag_names, with_tagme: false,
-                                        with_tagme: false,
+                                                          with_no_deerjikist: false)
-                                        with_no_deerjikist: false,
+      if linked_tags.any? { |t| t.nico? }
-                                        created_by_user: current_user)
+        raise Tag::NicoTagNormalisationError
+      end
 
-      TagVersioning.record_tag_snapshots!(
+      TagVersioning.record_tag_snapshots!(linked_tags, created_by_user: current_user)
-          linked_tags,
-          created_by_user: current_user,
-          language_code: locale.language_code)
 
       tag.linked_tags = linked_tags
       tag.save!
 
-      NicoTagVersionRecorder.record!(external_tag: tag,
+      NicoTagVersionRecorder.record!(tag:, event_type: :update, created_by_user: current_user)
-                                     event_type: :update,
-                                     created_by_user: current_user)
     end
 
-    render json: tag.linked_tags.map { |t|
+    render json: tag.linked_tags.map { |t| TagRepr.base(t) }, status: :ok
-      TagRepr.base(t, language_code: locale.language_code)
-    }, status: :ok
   rescue Tag::NicoTagNormalisationError
     render_validation_error fields: { tags: ['ニコニコ・タグ同士は連携できません．'] }
   rescue ActiveRecord::RecordInvalid => e
@@ -125,39 +102,11 @@ class NicoTagsController < ApplicationController
 
   private
 
-  def external_tag_json tag
-    { id: tag.id,
-      name: "#{ tag.platform }:#{ tag.name }",
-      category: 'nico',
-      post_count: tag.post_count,
-      created_at: tag.created_at,
-      updated_at: tag.created_at,
-      deprecated_at: nil,
-      aliases: [],
-      parents: [],
-      has_wiki: false,
-      material_id: nil,
-      has_deerjikists: false }
-  end
-
   def render_nico_tag_form_record_invalid record
-    names =
+    if record.is_a?(TagName) || record.is_a?(Tag)
-        case record
+      render_validation_error fields: { tags: record.errors.full_messages.map { |message|
-        when TagName
+        "タグ名 “#{ record.name }”: #{ message }"
-          [record]
+      } }
-        when Tag
-          record.tag_names.to_a
-        else
-          []
-        end
-    messages = names.flat_map do |tag_name|
-      tag_name.errors.full_messages.map do |message|
-        "タグ名 “#{ tag_name.name }”: #{ message }"
-      end
-    end
-
-    if messages.any?
-      render_validation_error fields: { tags: messages }
     else
       render_validation_error record
     end

backend/app/controllers/post_versions_controller.rb

@@ -1,11 +1,7 @@
 class PostVersionsController < ApplicationController
   def index
-    locale = resolve_locale!
     post_id = params[:post].presence
-    tag_id = params[:tag].presence&.to_i
+    tag_id = params[:tag].presence
-    external_tag_id = params[:external_tag].presence&.to_i
-    return head :bad_request if tag_id && external_tag_id
-
     page = (params[:page].presence || 1).to_i
     limit = (params[:limit].presence || 20).to_i
 
@@ -14,16 +10,30 @@ class PostVersionsController < ApplicationController
 
     offset = (page - 1) * limit
 
-    q = HistoryAccess.visible(version_scope)
+    tag_name =
+        if tag_id
+          TagName.joins(:tag).find_by(tag: { id: tag_id })
+        end
+    return render json: { versions: [], count: 0 } if tag_id && tag_name.blank?
+
+    q = PostVersion.joins(<<~SQL.squish)
+          LEFT JOIN
+            post_versions prev
+            ON
+              prev.post_id = post_versions.post_id
+              AND prev.version_no = post_versions.version_no - 1
+        SQL
+        .select('post_versions.*', 'prev.title AS prev_title', 'prev.url AS prev_url',
+                'prev.thumbnail_base AS prev_thumbnail_base', 'prev.tags AS prev_tags',
+                'prev.video_ms AS prev_video_ms',
+                'prev.original_created_from AS prev_original_created_from',
+                'prev.original_created_before AS prev_original_created_before')
     q = q.where('post_versions.post_id = ?', post_id) if post_id
-    if external_tag_id || (tag_id && !(Tag.exists?(id: tag_id)))
+    if tag_name
-      q = q.where('JSON_CONTAINS(post_versions.tags_json,' +
+      escaped = ActiveRecord::Base.sanitize_sql_like(tag_name.name)
-                  "JSON_OBJECT('external_tag_id', #{ external_tag_id || tag_id })) " +
+      q = q.where(("CONCAT(' ', post_versions.tags, ' ') LIKE :kw " +
-                  'OR JSON_CONTAINS(prev.tags_json,' +
+                   "OR CONCAT(' ', prev.tags, ' ') LIKE :kw"),
-                  "JSON_OBJECT('external_tag_id', #{ external_tag_id || tag_id }))")
+                  kw: "% #{ escaped } %")
-    elsif tag_id
-      q = q.where("JSON_CONTAINS(post_versions.tags_json, JSON_OBJECT('tag_id', #{ tag_id })) " +
-                  "OR JSON_CONTAINS(prev.tags_json, JSON_OBJECT('tag_id', #{ tag_id }))")
     end
 
     count = q.except(:select, :order, :limit, :offset).count
@@ -32,129 +42,83 @@ class PostVersionsController < ApplicationController
                 .limit(limit)
                 .offset(offset)
 
-    render json: { versions: serialise_versions(versions, locale.language_code), count:, population_count: HistoryAccess.population_count(PostVersion) }
+    render json: { versions: serialise_versions(versions), count: }
   end
 
   private
 
-  def version_scope
+  def serialise_versions rows
-    PostVersion.with_previous_version
-        .select('post_versions.*',
-                'prev.title AS prev_title',
-                'prev.url AS prev_url',
-                'prev.thumbnail_base AS prev_thumbnail_base',
-                'prev.tags_json AS prev_tags_json',
-                'prev.video_ms AS prev_video_ms',
-                'prev.original_created_from AS prev_original_created_from',
-                'prev.original_created_before AS prev_original_created_before')
-  end
-
-  def serialise_versions rows, language_code
-    rows = rows.to_a
     user_ids = rows.map(&:created_by_user_id).compact.uniq
     users_by_id = User.where(id: user_ids).pluck(:id, :name).to_h
-    snapshots = rows.flat_map { |row|
-      [normalise_json(row.tags_json),
-       normalise_json(row.attributes['prev_tags_json']) || []]
-    }
-    tag_ids = snapshots.flatten.filter_map { _1['tag_id'] }.uniq
-    tag_names = TagName.where(tag_id: tag_ids, language_code:, primary_flg: true)
-                       .pluck(:tag_id, :name).to_h
-    external_tag_names = external_tag_names_for(snapshots)
 
     rows.map do |row|
-      cur_tags = snapshot_tag_literals(
+      cur_tags = split_tags(row.tags)
-          normalise_json(row.tags_json), tag_names, external_tag_names)
+      prev_tags = split_tags(row.attributes['prev_tags'])
-      prev_tags = snapshot_tag_literals(
-          normalise_json(row.attributes['prev_tags_json']) || [], tag_names, external_tag_names)
 
-      { post_id: row.post_id,
+      {
+        post_id: row.post_id,
         version_no: row.version_no,
         event_type: row.event_type,
-        title: { current: row.title, prev: row.attributes['prev_title'] },
+        title: {
-        url: { current: row.url, prev: row.attributes['prev_url'] },
+          current: row.title,
-        thumbnail: { current: nil, prev: nil },
+          prev: row.attributes['prev_title']
-        thumbnail_base: { current: row.thumbnail_base,
+        },
-                          prev: row.attributes['prev_thumbnail_base'] },
+        url: {
-        video_ms: { current: row.video_ms, prev: row.attributes['prev_video_ms'] },
+          current: row.url,
+          prev: row.attributes['prev_url']
+        },
+        thumbnail: {
+          current: nil,
+          prev: nil
+        },
+        thumbnail_base: {
+          current: row.thumbnail_base,
+          prev: row.attributes['prev_thumbnail_base']
+        },
+        video_ms: {
+          current: row.video_ms,
+          prev: row.attributes['prev_video_ms']
+        },
         tags: build_version_tags(cur_tags, prev_tags),
-        parent_post_ids: row.parent_post_ids,
         original_created_from: {
-            current: row.original_created_from&.iso8601,
+          current: row.original_created_from&.iso8601,
-            prev: row.attributes['prev_original_created_from']&.iso8601 },
+          prev: row.attributes['prev_original_created_from']&.iso8601
+        },
         original_created_before: {
-            current: row.original_created_before&.iso8601,
+          current: row.original_created_before&.iso8601,
-            prev: row.attributes['prev_original_created_before']&.iso8601 },
+          prev: row.attributes['prev_original_created_before']&.iso8601
+        },
         created_at: row.created_at.iso8601,
         created_by_user:
-            if row.created_by_user_id
+          if row.created_by_user_id
-              { id: row.created_by_user_id,
+            {
-                name: users_by_id[row.created_by_user_id] }
+              id: row.created_by_user_id,
-            end }
+              name: users_by_id[row.created_by_user_id]
+            }
+          end
+      }
     end
   end
 
-  def external_tag_names_for snapshots
+  def build_version_tags(cur_tags, prev_tags)
-    ids = snapshots.flatten.filter_map { _1['external_tag_id'] }.uniq
+    (cur_tags | prev_tags).map do |name|
-    names = ExternalTag.where(id: ids).pluck(:id, :platform, :name).to_h { |id, platform, name|
-      [id, "#{ platform }:#{ name }"]
-    }
-    missing_ids = ids - names.keys
-
-    HistoryAccess.visible(NicoTagVersion.all)
-        .where(tag_id: missing_ids)
-        .order(:tag_id, version_no: :desc)
-        .pluck(:tag_id, :name)
-        .each { |id, name| names[id] ||= name }
-
-    names
-  end
-
-  def snapshot_tag_literals snapshots, tag_names, external_tag_names
-    snapshots
-        .filter_map { |snapshot|
-          if snapshot.key?('tag_id')
-            sections = snapshot.fetch('sections', []).map { |section|
-              [section.fetch('begin_ms'), section['end_ms']]
-            }.sort_by { |begin_ms, end_ms| [begin_ms, end_ms || Float::INFINITY] }
-            name = tag_names[snapshot['tag_id']] || snapshot.fetch('name')
-            [tag_category_order(snapshot['category']),
-             { identity: [:tag, snapshot['tag_id'], sections],
-               name: Post.tag_snapshot_literal(snapshot.merge('name' => name)),
-               restore_name: tag_names[snapshot['tag_id']] &&
-                   Post.tag_snapshot_literal(snapshot.merge('name' => name)),
-               external: false }]
-          elsif external_tag_names[snapshot['external_tag_id']]
-            [6, { identity: [:external_tag, snapshot['external_tag_id']],
-                  name: external_tag_names[snapshot['external_tag_id']],
-                  restore_name: nil, external: true }]
-          end
-        }
-        .sort_by { |order, tag| [order, tag[:name].downcase] }
-        .map(&:second)
-  end
-
-  def tag_category_order category
-    ['deerjikist', 'meme', 'character', 'general', 'material', 'meta'].index(category) || 6
-  end
-
-  def build_version_tags cur_tags, prev_tags
-    current = cur_tags.index_by { _1[:identity] }
-    previous = prev_tags.index_by { _1[:identity] }
-
-    (current.keys | previous.keys).map do |identity|
       type =
-          if current.key?(identity) && previous.key?(identity)
+        if cur_tags.include?(name) && prev_tags.include?(name)
-            'context'
+          'context'
-          elsif current.key?(identity)
+        elsif cur_tags.include?(name)
-            'added'
+          'added'
-          else
+        else
-            'removed'
+          'removed'
-          end
+        end
 
-      entry = current[identity] || previous[identity]
+      {
-      { name: entry[:name], restore_name: entry[:restore_name],
+        name:,
-        external: entry[:external], type: }
+        type:
+      }
     end
   end
+
+  def split_tags(tags)
+    tags.to_s.split(/\s+/).reject(&:blank?)
+  end
 end

backend/app/controllers/posts_controller.rb

@@ -7,8 +7,6 @@ class PostsController < ApplicationController
   end
 
   def index
-    locale = resolve_locale!
-
     url = params[:url].presence
     title = params[:title].presence
     original_created_from = params[:original_created_from].presence
@@ -38,8 +36,8 @@ class PostsController < ApplicationController
     offset = (page - 1) * limit
 
     pt_max_sql =
-        HistoryAccess.visible(PostVersion.all)
+        PostTag
-        .select('post_id, MAX(created_at) AS max_updated_at')
+        .select('post_id, MAX(updated_at) AS max_updated_at')
         .group('post_id')
         .to_sql
 
@@ -48,12 +46,13 @@ class PostsController < ApplicationController
         'COALESCE(pt_max.max_updated_at, posts.updated_at))'
 
     q =
-        filtered_posts(locale)
+        filtered_posts
         .joins("LEFT JOIN (#{ pt_max_sql }) pt_max ON pt_max.post_id = posts.id")
         .reselect('posts.*', Arel.sql("#{ updated_at_all_sql } AS updated_at_all"))
-        .preload(:external_tags, :uploaded_user, :parents, :children,
+        .preload(:uploaded_user, :parents, :children,
-                 post_tags: [:sections, { tag: [:deerjikists, :materials,
+                 active_post_tags: [:sections,
-                                                :tag_names, :tag_descriptions] }])
+                                    { tag: [:deerjikists, :materials,
+                                            { tag_name: :wiki_page }] }])
         .with_attached_thumbnail
 
     q = q.where('posts.url LIKE ?', "%#{ url }%") if url
@@ -92,8 +91,7 @@ class PostsController < ApplicationController
     q = q.except(:select, :order)
 
     render json: { posts: posts.map { |post|
-      PostRepr.base(post, language_code: locale.language_code)
+      PostRepr.base(post).merge(updated_at: post.updated_at_all).tap do |json|
-              .merge(updated_at: post.updated_at_all).tap do |json|
         json['thumbnail'] =
             if post.thumbnail.attached?
               rails_storage_proxy_url(post.thumbnail, only_path: false)
@@ -105,19 +103,16 @@ class PostsController < ApplicationController
   end
 
   def random
-    locale = resolve_locale!
+    post = filtered_posts.preload(:uploaded_user, :parents, :children,
-
+                                  active_post_tags: [:sections,
-    post =
+                                                     { tag: [:deerjikists, :materials,
-        filtered_posts(locale)
+                                                             { tag_name: :wiki_page }] }])
-        .preload(:uploaded_user, :parents, :children,
+                         .with_attached_thumbnail
-                 post_tags: [:sections, { tag: [:deerjikists, :materials,
+                         .order('RAND()')
-                                                :tag_names, :tag_descriptions] }])
+                         .first
-        .with_attached_thumbnail
-        .order('RAND()')
-        .first
     return head :not_found unless post
 
-    render json: PostRepr.base(post, current_user, language_code: locale.language_code)
+    render json: PostRepr.base(post, current_user)
   end
 
   def metadata
@@ -192,13 +187,12 @@ class PostsController < ApplicationController
   end
 
   def show
-    locale = resolve_locale!
-
     post =
         Post
         .includes(:uploaded_user, :parents, :children,
-                  post_tags: [:sections, { tag: [:deerjikists, :materials,
+                  active_post_tags: [:sections,
-                                                 :tag_names, :tag_descriptions] }])
+                                     { tag: [:deerjikists, :materials,
+                                             { tag_name: :wiki_page }] }])
         .with_attached_thumbnail
         .find_by(id: params[:id])
     return head :not_found unless post
@@ -212,21 +206,18 @@ class PostsController < ApplicationController
                                  parent_posts:,
                                  child_posts:,
                                  sibling_posts:,
-                                 related:,
+                                 related:)
-                                 language_code: locale.language_code)
+                         .merge(tags: build_tag_tree_for(post))
-                         .merge(tags: build_tag_tree_for(post, locale.language_code))
   end
 
   def create
     return head :unauthorized unless current_user
     return head :forbidden unless current_user.gte_member?
 
-    locale = resolve_locale!
-
     preflight = PostCreatePreflight.new(
         attributes: post_create_attributes,
         thumbnail: params[:thumbnail],
-        host: request.base_url).run(locale)
+        host: request.base_url).run
     return render json: dry_run_json(preflight) if bool?(:dry)
     if preflight[:existing_post_id].present?
       post = Post.new(url: preflight[:url])
@@ -252,10 +243,10 @@ class PostsController < ApplicationController
                                    :post_tag_specs,
                                    :tag_sections,
                                    :normalised_parent_post_ids).symbolize_keys).merge(
-                               thumbnail: params[:thumbnail])).create!(locale)
+                               thumbnail: params[:thumbnail])).create!
 
     post.reload
-    render json: PostRepr.base(post, language_code: locale.language_code), status: :created
+    render json: PostRepr.base(post), status: :created
   rescue PostCreatePreflight::ValidationFailed => e
     render_validation_error fields: e.fields, base: e.base_errors
   rescue Tag::NicoTagNormalisationError
@@ -281,18 +272,13 @@ class PostsController < ApplicationController
     return head :forbidden unless current_user.gte_member?
     return head :unsupported_media_type unless request.content_mime_type == Mime[:multipart_form]
     return head :payload_too_large if request.content_length.to_i > MAX_BULK_REQUEST_BYTES
-
-    locale = resolve_locale!
-
     posts = parse_bulk_posts_manifest
     thumbnails = parse_bulk_thumbnails(posts.length)
-
     result = PostBulkCreator.new(
         actor: current_user,
         posts:,
         thumbnails:,
-        host: request.base_url).run(locale)
+        host: request.base_url).run
-
     render json: result
   rescue JSON::ParserError
     render_bad_request 'posts manifest の JSON が不正です．'
@@ -325,7 +311,6 @@ class PostsController < ApplicationController
     base_version_no = parse_base_version_no
     return render_bad_request('base_version_no は必須です．') if !(force) && !(base_version_no)
 
-    locale = Locale.find_by(code: params[:locale].presence) || Locale.nipponese
     title = params[:title].presence
     tag_names = params[:tags].to_s.split
     original_created_from = params[:original_created_from]
@@ -344,12 +329,10 @@ class PostsController < ApplicationController
       unless force
         base_version = post.post_versions.find_by!(version_no: base_version_no)
 
-        HistoryAccess.require!(base_version)
         base_snapshot = post_snapshot_from_version(base_version)
-        current_snapshot = post_snapshot_from_record(post, locale)
+        current_snapshot = post_snapshot_from_record(post)
       end
-      incoming_snapshot = post_incoming_snapshot(locale:,
+      incoming_snapshot = post_incoming_snapshot(title:,
-                                                 title:,
                                                  original_created_from:,
                                                  original_created_before:,
                                                  tag_names:,
@@ -378,14 +361,14 @@ class PostsController < ApplicationController
             end
           end
 
-      apply_post_snapshot!(locale, post, snapshot_to_apply)
+      apply_post_snapshot!(post, snapshot_to_apply)
     end
 
     return render json: conflict_json, status: :conflict if conflict_json
 
     post.reload
-    json = PostRepr.base(post, current_user, language_code: locale.language_code)
+    json = PostRepr.base(post, current_user)
-    json['tags'] = build_tag_tree_for(post, locale.language_code)
+    json['tags'] = build_tag_tree_for(post)
     render json:, status: :ok
   rescue Tag::NicoTagNormalisationError
     render_validation_error fields: { tags: ['ニコニコ・タグは直接指定できません．'] }
@@ -401,84 +384,77 @@ class PostsController < ApplicationController
     render_post_form_record_invalid e.record
   end
 
-  def tag_filters
+  def changes
-    source = resolve_locale!
+    id = params[:id].presence
-    code = params[:target_locale]
+    tag_id = params[:tag].presence
-    return head :not_found unless code.is_a?(String) && Locale.codes.value?(code)
+    page = (params[:page].presence || 1).to_i
+    limit = (params[:limit].presence || 20).to_i
 
-    target = Locale.find_by(code:, deprecated_at: nil)
+    page = 1 if page < 1
-    return head :not_found unless target
+    limit = 1 if limit < 1
 
-    literals = params[:tags].to_s.split
+    offset = (page - 1) * limit
-    references = tag_filter_references(literals)
+
-    names = literals.each_with_index.map do |literal, index|
+    pts = PostTag.with_discarded
-      reference = references&.[](index)
+    pts = pts.where(post_id: id) if id.present?
-      (reference && reference['negative'] == false) ? literal : literal.sub(/\Anot:/i, '')
+    pts = pts.where(tag_id:) if tag_id.present?
+    pts = pts.includes(:post, :created_user, :deleted_user,
+                       tag: [:deerjikists, :materials, { tag_name: :wiki_page }])
+
+    events = []
+    pts.each do |pt|
+      tag = TagRepr.base(pt.tag)
+      post = pt.post
+
+      events << Event.new(
+          post:,
+          tag:,
+          user: pt.created_user && { id: pt.created_user.id, name: pt.created_user.name },
+          change_type: 'add',
+          timestamp: pt.created_at)
+
+      if pt.discarded_at
+        events << Event.new(
+            post:,
+            tag:,
+            user: pt.deleted_user && { id: pt.deleted_user.id, name: pt.deleted_user.name },
+            change_type: 'remove',
+            timestamp: pt.discarded_at)
+      end
     end
-    internal = references ? {} : TagName.where(
+    events.sort_by!(&:timestamp)
-        language_code: source.language_code, name: names).pluck(:name, :tag_id).to_h
+    events.reverse!
-    canonical = references ? {} : TagName.where(
-        language_code: source.language_code, primary_flg: true,
-        tag_id: internal.values).pluck(:tag_id, :name).to_h
-    external_names = names.map { canonical[internal[_1]] || _1 }
-    external = references ? {} : ExternalTag.where(
-        "CONCAT(platform, ':', name) IN (?)", external_names).pluck(
-        Arel.sql("CONCAT(platform, ':', name)"), :id).to_h
-    ids = references ? references.filter_map { _1['tag_id'] } : internal.values
-    translated = TagName.where(language_code: target.language_code,
-                               primary_flg: true, tag_id: ids).pluck(:tag_id, :name).to_h
 
-    filters = literals.each_with_index.map do |literal, index|
+    render json: { changes: (events.slice(offset, limit) || []).as_json, count: events.size }
-      name = names[index]
-      reference = if references
-                    references[index].except('name')
-                  else
-                    matches = {}
-                    matches['tag_id'] = internal[name] if internal[name]
-                    external_id = external[external_names[index]]
-                    matches['external_tag_id'] = external_id if external_id
-                    matches.empty? ? { 'missing' => true } : matches
-                  end
-      name = translated[reference['tag_id']] || name
-      negative = reference.fetch('negative', literal.downcase.start_with?('not:'))
-      prefix = negative ? literal[/\Anot:/i] || 'not:' : ''
-      reference.merge('name' => "#{ prefix }#{ name }", 'negative' => negative)
-    end
-    render json: { references: filters }
   end
 
   private
 
-  def tag_filter_references literals
+  def filtered_posts
-    PostTagSearch.parse_references(params.to_unsafe_h, literals)
-  rescue ArgumentError, JSON::ParserError, TypeError
-    raise ActionController::BadRequest
-  end
-
-  def filtered_posts locale
     tag_names = params[:tags].to_s.split
     match_type = params[:match]
     if tag_names.present?
-      filter_posts_by_tags(locale, tag_names, match_type)
+      filter_posts_by_tags(tag_names, match_type)
     else
       Post.all
     end
   end
 
-  def filter_posts_by_tags locale, tag_names, match_type
+  def filter_posts_by_tags tag_names, match_type
-    references = tag_filter_references(tag_names)
+    literals = tag_names.map do |raw_name|
-    literals = PostTagSearch.literals(locale, tag_names.join(' '), references:)
+      { name: TagName.canonicalise(raw_name.sub(/\Anot:/i, '')).first,
+        negative: raw_name.downcase.start_with?('not:') }
+    end
 
     return Post.all if literals.empty?
 
     if match_type == 'any'
       literals.reduce(Post.none) do |posts, literal|
-        posts.or(tag_literal_relation(locale, literal[:name], negative: literal[:negative],
+        posts.or(tag_literal_relation(literal[:name], negative: literal[:negative]))
-                                      reference: literal[:reference]))
       end
     else
       literals.reduce(Post.all) do |posts, literal|
-        ids = tagged_post_ids_for(locale, literal[:name], reference: literal[:reference])
+        ids = tagged_post_ids_for(literal[:name])
         if literal[:negative]
           posts.where.not(id: ids)
         else
@@ -488,8 +464,8 @@ class PostsController < ApplicationController
     end
   end
 
-  def tag_literal_relation locale, name, negative:, reference: nil
+  def tag_literal_relation name, negative:
-    ids = tagged_post_ids_for(locale, name, reference:)
+    ids = tagged_post_ids_for(name)
     if negative
       Post.where.not(id: ids)
     else
@@ -497,12 +473,8 @@ class PostsController < ApplicationController
     end
   end
 
-  def tagged_post_ids_for locale, name, reference: nil
+  def tagged_post_ids_for(name) =
-    internal, external = PostTagSearch.references(locale, { name:, reference: })
+      Post.joins(tags: :tag_name).where(tag_names: { name: }).select(:id)
-    Post.where(id: PostTag.where(tag_id: Array(internal)).select(:post_id))
-        .or(Post.where(id: PostExternalTag.where(external_tag_id: Array(external))
-            .select(:post_id))).select(:id)
-  end
 
   def sync_post_tags! post, desired_tags, sections
     desired_tags.each do |t|
@@ -530,13 +502,13 @@ class PostsController < ApplicationController
       end
     end
 
-    PostTag.where(post_id: post.id, tag_id: to_remove.to_a).find_each do |pt|
+    PostTag.where(post_id: post.id, tag_id: to_remove.to_a).kept.find_each do |pt|
-      pt.destroy!
+      pt.discard_by!(current_user)
     end
   end
 
-  def build_tag_tree_for post, language_code = 'ja'
+  def build_tag_tree_for post
-    post_tags = post.post_tags.reject { |post_tag| post_tag.tag.deprecated? }
+    post_tags = post.active_post_tags.reject { |post_tag| post_tag.tag.deprecated? }
     tags = post_tags.map(&:tag)
     tag_ids = tags.map(&:id)
 
@@ -565,7 +537,7 @@ class PostsController < ApplicationController
       sections = sections_by_tag_id.fetch(tag_id, [])
 
       if path.include?(tag_id)
-        return TagRepr.inline(tag, language_code:).merge(children: [], sections:)
+        return TagRepr.inline(tag).merge(children: [], sections:)
       end
 
       if memo.key?(tag_id)
@@ -577,13 +549,10 @@ class PostsController < ApplicationController
 
       children = child_ids.filter_map { |cid| build_node.(cid, new_path) }
 
-      memo[tag_id] = TagRepr.inline(tag, language_code:).merge(children:, sections:)
+      memo[tag_id] = TagRepr.inline(tag).merge(children:, sections:)
     end
 
-    internal_tags = root_ids.filter_map { |id| build_node.call(id, []) }
+    root_ids.filter_map { |id| build_node.call(id, []) }
-    external_tags =
-        post.external_tags.map { ExternalTagRepr.inline(_1).merge(children: [], sections: []) }
-    internal_tags + external_tags
   end
 
   def sibling_posts_by_parent parent_post_ids
@@ -730,50 +699,43 @@ class PostsController < ApplicationController
   end
 
   def editable_tag_names_from_version version
-    version.tags_json
+    version.tags.to_s.split.reject { |name| name.downcase.start_with?('nico:') }.sort
-        .select { _1.key?('tag_id') }
-        .map { Post.tag_snapshot_literal(_1) }
-        .compact
-        .sort
   end
 
-  def post_snapshot_from_record post, locale
+  def post_snapshot_from_record post
     { title: post.title,
       video_ms: post.video_ms,
       original_created_from: snapshot_time(post.original_created_from),
       original_created_before: snapshot_time(post.original_created_before),
-      tag_names: editable_tag_names_from_post(post, locale.language_code),
+      tag_names: editable_tag_names_from_post(post),
       parent_post_ids: post.parent_posts.order(:id).pluck(:id) }
   end
 
-  def editable_tag_names_from_post post, language_code
+  def editable_tag_names_from_post post
     post
         .post_tags
-        .joins(tag: :tag_names)
+        .kept
+        .joins(tag: :tag_name)
+        .merge(Tag.not_nico)
         .merge(Tag.where(deprecated_at: nil))
-        .where(tag_names: { language_code:, primary_flg: true })
+        .includes(:sections, tag: :tag_name)
-        .includes(:sections, tag: :tag_names)
         .order('tag_names.name')
         .map do |post_tag|
-      name = post_tag.tag.name(language_code)
+      name = post_tag.tag.tag_name.name
       sections = post_tag.sections.sort_by(&:begin_ms)
 
       next name if sections.empty?
 
       "#{ name }#{ sections.map { Post.section_literal(_1) }.join }"
-    end.compact.sort
+    end
   end
 
-  def post_incoming_snapshot locale:, title:, original_created_from:, original_created_before:,
+  def post_incoming_snapshot title:, original_created_from:, original_created_before:,
                              tag_names:, video_ms_param:, duration_param:, parent_post_ids:
     validate_original_created_values!(original_created_from, original_created_before)
-
+    Tag.normalise_tags!(tag_names, with_tagme: false, deny_deprecated: true,
-    Tag.normalise_tags!(locale, tag_names,
+                                        with_sections: true) =>
-                        with_tagme: false,
+        { tags:, sections: }
-                        with_no_deerjikist: false,
-                        deny_deprecated: true,
-                        with_sections: true,
-                        created_by_user: current_user) => { tags:, sections: }
 
     tags = Tag.expand_parent_tags(tags).reject(&:deprecated?)
     video_ms = normalise_video_ms(tags, video_ms_param:, duration_param:)
@@ -784,7 +746,7 @@ class PostsController < ApplicationController
       original_created_from: snapshot_time(original_created_from),
       original_created_before: snapshot_time(original_created_before),
       tag_names: tags.uniq(&:id).map { |tag|
-        "#{ tag.name(locale.language_code) }#{ sections[tag.id].to_a.map { section_literal(_1) }.join }"
+        "#{ tag.name }#{ sections[tag.id].to_a.map { section_literal(_1) }.join }"
       }.sort,
       parent_post_ids: parent_post_ids.sort }
   end
@@ -908,7 +870,7 @@ class PostsController < ApplicationController
     (added_by_current & removed_by_me).present? || (removed_by_current & added_by_me).present?
   end
 
-  def apply_post_snapshot! locale, post, snapshot
+  def apply_post_snapshot! post, snapshot
     PostVersionRecorder.ensure_snapshot!(post, created_by_user: current_user)
 
     post.update!(title: snapshot[:title],
@@ -916,14 +878,15 @@ class PostsController < ApplicationController
                  original_created_from: snapshot[:original_created_from],
                  original_created_before: snapshot[:original_created_before])
 
-    Tag.normalise_tags!(locale, snapshot[:tag_names],
+    Tag.normalise_tags!(snapshot[:tag_names], with_tagme: false,
-                        with_tagme: false,
+                                               deny_deprecated: true,
-                        deny_deprecated: true,
+                                               with_sections: true) =>
-                        with_sections: true,
+        { tags: editable_tags, sections: }
-                        created_by_user: current_user) => { tags:, sections: }
+    TagVersioning.record_tag_snapshots!(editable_tags, created_by_user: current_user)
-    TagVersioning.record_tag_snapshots!(tags, created_by_user: current_user,
-                                        language_code: locale.language_code)
 
+    readonly_tags = post.tags.nico.to_a
+
+    tags = readonly_tags + editable_tags
     tags = Tag.expand_parent_tags(tags).reject(&:deprecated?)
 
     post.video_ms = tags.any? { _1.id == Tag.video.id } ? snapshot[:video_ms] : nil

backend/app/controllers/sitemap_entries_controller.rb

@@ -1,58 +0,0 @@
-class SitemapEntriesController < ApplicationController
-  PAGE_SIZE = 1_000
-  RESOURCES = [
-      'posts', 'tags', 'deerjikists', 'materials',
-      'descriptions', 'tag_groups', 'theatres'].freeze
-
-  def index
-    locale = resolve_locale!
-    resources = RESOURCES.map { |resource| { resource:, count: entries(resource, locale).count } }
-    render json: { resources:, page_size: PAGE_SIZE }
-  end
-
-  def show
-    locale = resolve_locale!
-    resource = params[:resource]
-    return head :not_found unless RESOURCES.include?(resource)
-
-    page = Integer(params.fetch(:page, '1'), exception: false)
-    return head :bad_request unless page && page.positive?
-
-    scope = entries(resource, locale)
-    return head :not_found if page > [(scope.count.to_f / PAGE_SIZE).ceil, 1].max
-
-    ids = scope.order(:id).offset((page - 1) * PAGE_SIZE).limit(PAGE_SIZE).pluck(:id)
-    render json: { ids: }
-  end
-
-  private
-
-  def entries resource, locale
-    named_tags = TagName.where(language_code: locale.language_code, primary_flg: true)
-        .select(:tag_id)
-    case resource
-    when 'posts'
-      Post.all
-    when 'tags'
-      Tag.where(id: named_tags)
-    when 'deerjikists'
-      Tag.where(id: named_tags).where(id: Deerjikist.select(:tag_id))
-    when 'materials'
-      Material.all
-    when 'descriptions'
-      TagDescription.where(language_code: locale.language_code, tag_id: named_tags)
-          .where.not(body: [nil, ''])
-          .where(HistoryAccess.visible(TagDescriptionVersion.all)
-              .where('tag_description_versions.tag_id = tag_descriptions.tag_id')
-              .where('tag_description_versions.language_code = tag_descriptions.language_code')
-              .where('tag_description_versions.version_no = tag_descriptions.version_no')
-              .arel.exists)
-    when 'tag_groups'
-      TagGroup.joins(:versions)
-          .where('tag_group_versions.version_no = tag_groups.version_no')
-          .where(Arel.sql(HistoryAccess.sql(TagGroupVersion)))
-    when 'theatres'
-      Theatre.kept
-    end
-  end
-end

backend/app/controllers/tag_children_controller.rb

@@ -10,6 +10,7 @@ class TagChildrenController < ApplicationController
 
     parent = Tag.find(parent_id)
     child = Tag.find(child_id)
+    return render_bad_request('ニコニコ・タグの階層は変更できません．') if parent.nico? || child.nico?
 
     ApplicationRecord.transaction do
       TagVersioning.ensure_snapshot!(child, created_by_user: current_user)
@@ -32,6 +33,7 @@ class TagChildrenController < ApplicationController
 
     parent = Tag.find(parent_id)
     child = Tag.find(child_id)
+    return render_bad_request('ニコニコ・タグの階層は変更できません．') if parent.nico? || child.nico?
 
     ApplicationRecord.transaction do
       TagVersioning.ensure_snapshot!(child, created_by_user: current_user)

backend/app/controllers/tag_descriptions_controller.rb

@@ -1,63 +0,0 @@
-class TagDescriptionsController < ApplicationController
-  def show
-    locale = resolve_locale!
-    description = TagDescription.find_by(tag_id: params[:id], language_code: locale.language_code)
-    return head :not_found unless description
-
-    render json: representation(description)
-  end
-
-  def show_by_title
-    locale = resolve_locale!
-    tag_name = TagName.find_by(
-        language_code: locale.language_code, name: params[:name].to_s.strip)
-    description = tag_name&.tag && TagDescription.find_by(
-        tag: tag_name.tag, language_code: locale.language_code)
-    return head :not_found unless description
-
-    render json: representation(description)
-  end
-
-  def update
-    return head :unauthorized unless current_user
-    return head :forbidden unless current_user.gte_member?
-
-    body = params[:body].to_s
-    return render_unprocessable_entity('本文は必須です．', field: :body) if body.blank?
-
-    locale = resolve_locale!
-    tag = Tag.find_by(id: params[:id])
-    return head :not_found unless tag
-
-    description = TagDescription.find_by(tag:, language_code: locale.language_code)
-
-    if description
-      TagDescriptionOperation.update!(locale, tag:, body:, updated_by_user: current_user)
-      description.reload
-      render json: representation(description)
-    else
-      TagDescriptionOperation.create!(locale, tag:, body:, created_by_user: current_user)
-      description = TagDescription.find_by!(tag:, language_code: locale.language_code)
-      render json: representation(description), status: :created
-    end
-  rescue ActiveRecord::RecordInvalid => e
-    render_validation_error e.record
-  rescue ActiveRecord::RecordNotUnique
-    render_record_not_unique
-  end
-
-  private
-
-  def representation description
-    HistoryAccess.require_current!(description, TagDescriptionVersion)
-    description.as_json(only: [
-        :tag_id,
-        :language_code,
-        :body,
-        :version_no,
-        :created_by_user_id,
-        :updated_by_user_id,
-        :created_at,
-        :updated_at])
-  end
-end

backend/app/controllers/tag_groups_controller.rb

@@ -1,43 +0,0 @@
-class TagGroupsController < ApplicationController
-  def index
-    locale = resolve_locale!
-    groups = HistoryAccess.current(TagGroup.all, TagGroupVersion).includes(:created_by_user, :updated_by_user).order(:id)
-    render json: TagGroupRepr.many(locale, groups.to_a)
-  end
-
-  def show
-    locale = resolve_locale!
-    group = TagGroup.includes(:created_by_user, :updated_by_user).find_by(id: params[:id])
-    return head :not_found unless group
-
-    render json: TagGroupRepr.one(locale, group)
-  end
-
-  def create
-    return head :unauthorized unless current_user
-    return head :forbidden unless current_user.gte_member?
-
-    locale = resolve_locale!
-    group = TagGroupOperation.create!(locale, **write_attributes, actor: current_user)
-    render json: TagGroupRepr.one(locale, group), status: :created
-  end
-
-  def update
-    return head :unauthorized unless current_user
-    return head :forbidden unless current_user.gte_member?
-
-    locale = resolve_locale!
-    group = TagGroup.find_by(id: params[:id])
-    return head :not_found unless group
-
-    TagGroupOperation.update!(locale, group:, **write_attributes, actor: current_user)
-    render json: TagGroupRepr.one(locale, group.reload)
-  end
-
-  private
-
-  def write_attributes
-    { name: params[:name], description: params[:description],
-      members: params[:members], placements: params[:placements] }
-  end
-end

backend/app/controllers/tag_versions_controller.rb

@@ -1,6 +1,5 @@
 class TagVersionsController < ApplicationController
   def index
-    locale = resolve_locale!
     tag_id = params[:id].presence
     page = (params[:page].presence || 1).to_i
     limit = (params[:limit].presence || 20).to_i
@@ -10,7 +9,16 @@ class TagVersionsController < ApplicationController
 
     offset = (page - 1) * limit
 
-    q = HistoryAccess.visible(version_scope)
+    q = TagVersion.joins(<<~SQL.squish)
+          LEFT JOIN
+            tag_versions prev
+            ON
+              prev.tag_id = tag_versions.tag_id
+              AND prev.version_no = tag_versions.version_no - 1
+        SQL
+      .select('tag_versions.*', 'prev.name AS prev_name', 'prev.category AS prev_category',
+              'prev.deprecated_at AS prev_deprecated_at',
+              'prev.aliases AS prev_aliases', 'prev.parent_tag_ids AS prev_parent_tag_ids')
     q = q.where('tag_versions.tag_id = ?', tag_id) if tag_id
 
     count = q.except(:select, :order, :limit, :offset).count
@@ -19,68 +27,41 @@ class TagVersionsController < ApplicationController
                 .limit(limit)
                 .offset(offset)
 
-    render json: { versions: serialise_versions(versions, locale.language_code), count:, population_count: HistoryAccess.population_count(TagVersion) }
+    render json: { versions: serialise_versions(versions), count: }
   end
 
   private
 
-  def version_scope
+  def serialise_versions rows
-    TagVersion.joins(<<~SQL.squish)
-          LEFT JOIN
-            tag_versions prev
-            ON
-              prev.tag_id = tag_versions.tag_id
-              AND prev.version_no = tag_versions.version_no - 1
-              AND #{ HistoryAccess.sql(TagVersion, 'prev') }
-        SQL
-      .select('tag_versions.*', 'prev.name AS prev_name', 'prev.category AS prev_category',
-              'prev.deprecated_at AS prev_deprecated_at',
-              'prev.aliases AS prev_aliases', 'prev.names AS prev_names',
-              'prev.parent_tag_ids AS prev_parent_tag_ids')
-  end
-
-  def serialise_versions rows, language_code
-    rows = rows.to_a
-    tag_ids = rows.map(&:tag_id).uniq
-    current_names_by_tag_id = TagName.where(tag_id: tag_ids, language_code:, primary_flg: true)
-        .pluck(:tag_id, :name, :auto_generated).to_h do |id, name, automatic|
-      [id, { name:, auto_generated: automatic }]
-    end
     user_ids = rows.map(&:created_by_user_id).compact.uniq
     users_by_id = User.where(id: user_ids).pluck(:id, :name).to_h
-    parent_ids = rows.flat_map do |row|
-      split_parent_tag_ids(row.parent_tag_ids) +
-          split_parent_tag_ids(row.attributes['prev_parent_tag_ids'])
-    end.uniq
-    parents_by_id = Tag.includes(
-        :tag_names, :materials, :deerjikists, :tag_descriptions,
-        parents: [:tag_names, :materials, :deerjikists, :tag_descriptions])
-        .where(id: parent_ids).index_by(&:id)
-    parent_json = parents_by_id.transform_values { TagRepr.base(_1, language_code:) }
 
     rows.map do |row|
-      current_names = names_for_locale(row.names, language_code)
+      cur_aliases = split_values(row.aliases)
-      previous_names = names_for_locale(row.attributes['prev_names'], language_code)
+      prev_aliases = split_values(row.attributes['prev_aliases'])
-      cur_aliases = current_names.fetch('aliases', []).map { _1.fetch('name') }
-      prev_aliases = previous_names.fetch('aliases', []).map { _1.fetch('name') }
 
       cur_parent_tag_ids = split_parent_tag_ids(row.parent_tag_ids)
       prev_parent_tag_ids = split_parent_tag_ids(row.attributes['prev_parent_tag_ids'])
 
+      all_parent_tag_ids = (cur_parent_tag_ids | prev_parent_tag_ids)
+
+      tags_by_id =
+          Tag
+          .includes(:tag_name, :materials, { tag_name: :wiki_page })
+          .where(id: all_parent_tag_ids)
+          .index_by(&:id)
+
       parent_tags =
           build_version_values(cur_parent_tag_ids, prev_parent_tag_ids, key: :tag_id)
           .map do |h|
-            { tag: parent_json.fetch(h[:tag_id]),
+            { tag: TagRepr.base(tags_by_id[h[:tag_id]]),
               type: h[:type] }
           end
 
       { tag_id: row.tag_id,
         version_no: row.version_no,
         event_type: row.event_type,
-        names: row.names,
+        name: { current: row.name, prev: row.attributes['prev_name'] },
-        name_changes: name_changes(row.names, row.attributes['prev_names']),
-        current_name_reference: current_names_by_tag_id[row.tag_id],
-        name: { current: current_names['name'], prev: previous_names['name'] },
         category: { current: row.category, prev: row.attributes['prev_category'] },
         deprecated_at: { current: row.deprecated_at&.iso8601,
                          prev: row.attributes['prev_deprecated_at']&.iso8601 },
@@ -110,45 +91,5 @@ class TagVersionsController < ApplicationController
 
   def split_values(values) = values.to_s.split(/\s+/).reject(&:blank?)
 
-  def names_for_locale names, language_code
-    values = snapshot_names(names)[language_code]
-    values.is_a?(Hash) ? values : {}
-  end
-
-  def snapshot_names names
-    values = names.is_a?(String) ? JSON.parse(names) : names
-    values.is_a?(Hash) ? values : {}
-  rescue JSON::ParserError
-    {}
-  end
-
-  def name_changes current_names, previous_names
-    current = snapshot_names(current_names)
-    previous = snapshot_names(previous_names)
-    (current.keys | previous.keys).sort.filter_map do |language|
-      current_name = language_snapshot(current[language])
-      previous_name = language_snapshot(previous[language])
-      next if current_name == previous_name
-
-      { language_code: language, current: current_name, prev: previous_name }
-    end
-  end
-
-  def language_snapshot entry
-    return nil unless entry.is_a?(Hash)
-
-    aliases = Array(entry['aliases']).filter_map do |value|
-      value = { 'name' => value } if value.is_a?(String)
-      next unless value.is_a?(Hash) && value['name'].is_a?(String)
-
-      { name: value['name'], script_code: value['script_code'] }
-    end
-    aliases = aliases.uniq.sort_by { [_1[:name], _1[:script_code].to_s] }
-    { name: entry['name'],
-      script_code: entry['script_code'],
-      auto_generated: entry['auto_generated'],
-      aliases: }
-  end
-
   def split_parent_tag_ids(values) = split_values(values).map(&:to_i)
 end

backend/app/controllers/tags_controller.rb

@@ -5,8 +5,6 @@ require 'set'
 
 class TagsController < ApplicationController
   def index
-    locale = resolve_locale!
-
     post_id = params[:post]
 
     name = params[:name].presence
@@ -36,162 +34,49 @@ class TagsController < ApplicationController
 
     offset = (page - 1) * limit
 
-    tags =
+    q =
         if post_id.present?
-          Tag
+          Tag.joins(:posts, :tag_name)
-              .joins(:posts, :tag_names)
-              .where(tag_names: { language_code: locale.language_code, primary_flg: true },
-                     posts: { id: post_id })
         else
-          Tag
+          Tag.joins(:tag_name)
-              .joins(:tag_names)
-              .where(tag_names: { language_code: locale.language_code, primary_flg: true })
         end
+        .includes(:tag_name, :materials, tag_name: :wiki_page)
+    q = q.where(posts: { id: post_id }) if post_id.present?
 
-    external_tags =
+    q = q.where('tag_names.name LIKE ?', "%#{ name }%") if name
-        if post_id.present?
+    q = q.where(category:) if category
-          ExternalTag.joins(:posts).where(posts: { id: post_id })
+    q = q.where('tags.post_count >= ?', post_count_between[0]) if post_count_between[0]
-        else
+    q = q.where('tags.post_count <= ?', post_count_between[1]) if post_count_between[1]
-          ExternalTag.all
+    q = q.where('tags.created_at >= ?', created_between[0]) if created_between[0]
-        end
+    q = q.where('tags.created_at <= ?', created_between[1]) if created_between[1]
-
+    q = q.where('tags.updated_at >= ?', updated_between[0]) if updated_between[0]
-    if name
+    q = q.where('tags.updated_at <= ?', updated_between[1]) if updated_between[1]
-      tags = tags.where('tag_names.name LIKE ?', "%#{ name }%")
-      external_tags =
-          external_tags.where("CONCAT(external_tags.platform, ':', external_tags.name) LIKE ?",
-                              "%#{ name }%")
-    end
-
-    if category == 'nico'
-      tags = tags.none
-    elsif category
-      tags = tags.where(category:)
-      external_tags = external_tags.none
-    end
-
-    if post_count_between[0]
-      tags = tags.where('tags.post_count >= ?', post_count_between[0])
-      external_tags = external_tags.where('external_tags.post_count >= ?', post_count_between[0])
-    end
-
-    if post_count_between[1]
-      tags = tags.where('tags.post_count <= ?', post_count_between[1])
-      external_tags = external_tags.where('external_tags.post_count <= ?', post_count_between[1])
-    end
-
-    if created_between[0]
-      tags = tags.where('tags.created_at >= ?', created_between[0])
-      external_tags = external_tags.where('external_tags.created_at >= ?', created_between[0])
-    end
-
-    if created_between[1]
-      tags = tags.where('tags.created_at <= ?', created_between[1])
-      external_tags = external_tags.where('external_tags.created_at <= ?', created_between[1])
-    end
-
-    if updated_between[0]
-      tags = tags.where('tags.updated_at >= ?', updated_between[0])
-      external_tags = external_tags.where('external_tags.created_at >= ?', updated_between[0])
-    end
-
-    if updated_between[1]
-      tags = tags.where('tags.updated_at <= ?', updated_between[1])
-      external_tags = external_tags.where('external_tags.created_at <= ?', updated_between[1])
-    end
-
     if deprecated_given
-      if deprecated
+      q = deprecated ? q.where.not(deprecated_at: nil) : q.where(deprecated_at: nil)
-        tags = tags.where.not(deprecated_at: nil)
-        external_tags = external_tags.none
-      else
-        tags = tags.where(deprecated_at: nil)
-      end
     end
 
-    tag_sql = tags.select("'tag' AS source",
-                          'tags.id',
-                          'tag_names.name',
-                          'tags.category',
-                          'tags.post_count',
-                          'tags.created_at',
-                          'tags.updated_at',
-                          'tags.deprecated_at').to_sql
-
-    external_tag_sql = external_tags.select(
-        "'external_tag' AS source",
-        'external_tags.id',
-        "CONCAT(external_tags.platform, ':', external_tags.name) AS name",
-        "'nico' AS category",
-        'external_tags.post_count',
-        'external_tags.created_at',
-        'external_tags.created_at AS updated_at',
-        'NULL AS deprecated_at').to_sql
-
-    union_sql = "#{ tag_sql } UNION ALL #{ external_tag_sql }"
-
     sort_sql =
-        if order[0] == 'category'
+        case order[0]
-          'CASE category ' +
+        when 'name'
+          'tag_names.name'
+        when 'category'
+          'CASE tags.category ' +
           "WHEN 'deerjikist' THEN 0 " +
           "WHEN 'meme' THEN 1 " +
           "WHEN 'character' THEN 2 " +
           "WHEN 'general' THEN 3 " +
           "WHEN 'material' THEN 4 " +
           "WHEN 'meta' THEN 5 " +
-          "WHEN 'nico' THEN 6 " +
+          "WHEN 'nico' THEN 6 END"
-          'END'
         else
-          order[0]
+          "tags.#{ order[0] }"
         end
+    tags = q.order(Arel.sql("#{ sort_sql } #{ order[1] }, tags.id #{ order[1] }"))
+            .limit(limit)
+            .offset(offset)
+            .to_a
 
-    connection = ApplicationRecord.connection
+    render json: { tags: TagRepr.many(tags), count: q.size }
-
-    count = connection.select_value(<<~SQL)
-      SELECT
-        COUNT(0)
-      FROM
-        (#{ union_sql }) legacy_tags
-    SQL
-
-    rows = connection.select_all(<<~SQL).to_a
-      SELECT
-        source
-      , id
-      FROM
-        (#{ union_sql }) legacy_tags
-      ORDER BY
-        #{ sort_sql } #{ order[1] }
-      , id #{ order[1] }
-      , source #{ order[1] }
-      LIMIT
-        #{ limit }
-      OFFSET
-        #{ offset }
-    SQL
-
-    tag_ids = rows.filter { _1['source'] == 'tag' }.map { _1['id'] }
-    external_tag_ids = rows.filter { _1['source'] == 'external_tag' }.map { _1['id'] }
-
-    tags_by_id =
-        Tag
-        .joins(:tag_names)
-        .preload(:tag_names, :materials, :deerjikists, :tag_descriptions,
-                 parents: [:tag_names, :materials, :deerjikists, :tag_descriptions])
-        .where(tag_names: { language_code: locale.language_code, primary_flg: true },
-               id: tag_ids)
-        .index_by(&:id)
-    external_tags_by_id =
-        ExternalTag
-        .where(id: external_tag_ids)
-        .index_by(&:id)
-
-    render json: { tags: rows.map { |row|
-      if row['source'] == 'tag'
-        TagRepr.base(tags_by_id.fetch(row['id']), language_code: locale.language_code)
-      else
-        external_tag_json(external_tags_by_id.fetch(row['id']))
-      end
-    }, count: }
   end
 
   def with_depth
@@ -199,8 +84,7 @@ class TagsController < ApplicationController
     parent_tag_id = nil if parent_tag_id <= 0
     material_filter = material_filter_param(default: 'any')
 
-    locale = resolve_locale!
+    graph = build_with_depth_graph(material_filter)
-    graph = build_with_depth_graph(material_filter, locale.language_code)
 
     tag_ids =
         if parent_tag_id
@@ -215,231 +99,137 @@ class TagsController < ApplicationController
   end
 
   def autocomplete
-    locale = resolve_locale!
-
     q = params[:q].to_s.strip.sub(/\Anot:/i, '')
-    prefix = "#{ ActiveRecord::Base.sanitize_sql_like(q) }%"
 
     with_nico = bool?(:nico, default: true)
     present_only = bool?(:present, default: true)
 
     alias_rows =
         TagName
-        .where(language_code: locale.language_code, primary_flg: false)
+        .where('name LIKE ?', "#{ q }%")
-        .where('name LIKE ?', prefix)
+        .where.not(canonical_id: nil)
-        .pluck(:tag_id, :name)
+        .pluck(:canonical_id, :name)
 
-    matched_alias_by_tag_id = { }
+    matched_alias_by_tag_name_id = { }
-    tag_ids = []
+    canonical_ids = []
-    alias_rows.each do |tag_id, alias_name|
+
-      tag_ids << tag_id
+    alias_rows.each do |canonical_id, alias_name|
-      matched_alias_by_tag_id[tag_id] ||= alias_name
+      canonical_ids << canonical_id
+      matched_alias_by_tag_name_id[canonical_id] ||= alias_name
     end
 
-    base =
+    base = Tag.joins(:tag_name)
-        Tag
+              .includes(:tag_name, :materials, tag_name: :wiki_page)
-        .joins(:tag_names)
+              .where(deprecated_at: nil)
-        .preload(:tag_names, :materials, :deerjikists, :tag_descriptions,
-                 parents: [:tag_names, :materials, :deerjikists, :tag_descriptions])
-        .where(tag_names: { language_code: locale.language_code, primary_flg: true },
-               deprecated_at: nil)
     base = base.where('tags.post_count > 0') if present_only
 
-    canonical_hit = base.where('tag_names.name LIKE ?', prefix)
+    canonical_hit =
+        base
+        .where(((with_nico ? '(tags.category = ? AND tag_names.name LIKE ?) OR ' : '') +
+                'tag_names.name LIKE ?'),
+               *(with_nico ? ['nico', "nico:#{ q }%"] : []), "#{ q }%")
 
-    internal_tags = canonical_hit.or(base.where(id: tag_ids.uniq))
+    tags =
+        if canonical_ids.present?
+          canonical_hit.or(base.where(tag_name_id: canonical_ids.uniq))
+        else
+          canonical_hit
+        end
 
-    internal_rows =
+    tags = tags.order(Arel.sql('post_count DESC, tag_names.name')).limit(20).to_a
-        internal_tags
-        .order(Arel.sql('tags.post_count DESC, tag_names.name'))
-        .limit(20)
-        .map { |tag|
-          TagRepr.base(tag, language_code: locale.language_code)
-                 .merge(matched_alias: matched_alias_by_tag_id[tag.id])
-        }
 
-    return render json: internal_rows unless with_nico
+    render json: tags.map { |tag|
-
+      TagRepr.base(tag).merge(matched_alias: matched_alias_by_tag_name_id[tag.tag_name_id])
-    external_base = ExternalTag.all
+    }
-    external_base = external_base.where('post_count > 0') if present_only
-
-    external_rows =
-        external_base
-        .where("CONCAT(platform, ':', name) LIKE ? OR name LIKE ?", prefix, prefix)
-        .order(post_count: :desc, name: :asc)
-        .limit(20)
-        .map { external_tag_json(_1) }
-
-    rows =
-    (internal_rows + external_rows)
-        .sort_by { |row| [-row['post_count'], row['name'] || row[:name]] }
-        .first(20)
-
-    render json: rows
   end
 
   def show
-    locale = resolve_locale!
+    tag = Tag.joins(:tag_name)
-
+             .includes(:tag_name, :materials, tag_name: :wiki_page)
-    tag =
+             .find_by(id: params[:id])
-        Tag
+    if tag
-        .joins(:tag_names)
+      render json: TagRepr.base(tag)
-        .preload(:tag_names, :materials, :deerjikists, :tag_descriptions,
+    else
-                 parents: [:tag_names, :materials, :deerjikists, :tag_descriptions])
+      head :not_found
-        .find_by(id: params[:id],
+    end
-                 tag_names: { language_code: locale.language_code, primary_flg: true })
-    return render json: TagRepr.base(tag, language_code: locale.language_code) if tag
-
-    external_tag = ExternalTag.find_by(id: params[:id])
-    return render json: ExternalTagRepr.base(external_tag) if external_tag
-
-    head :not_found
   end
 
   def show_by_name
     name = params[:name].to_s.strip
     return render_bad_request('name は必須です．') if name.blank?
 
-    locale = resolve_locale!
+    tag = Tag.joins(:tag_name)
-
+             .includes(:tag_name, :materials, tag_name: :wiki_page)
-    tag =
+             .find_by(tag_names: { name: })
-        Tag
+    if tag
-        .joins(:tag_names)
+      render json: TagRepr.base(tag)
-        .preload(:tag_names, :materials, :deerjikists, :tag_descriptions,
+    else
-                 parents: [:tag_names, :materials, :deerjikists, :tag_descriptions])
+      head :not_found
-        .find_by(tag_names: { name:, language_code: locale.language_code })
+    end
-    return render json: TagRepr.base(tag, language_code: locale.language_code) if tag
-
-    platform, external_name = name.split(':', 2)
-    return head :not_found unless external_name
-
-    external_tag = ExternalTag.find_by(platform:, name: external_name)
-    return head :not_found unless external_tag
-
-    render json: ExternalTagRepr.base(external_tag)
   end
 
   def deerjikists
-    locale = resolve_locale!
+    tag = Tag.joins(:tag_name)
-
+             .includes(:tag_name, tag_name: :wiki_page)
-    tag = 
+             .find_by(id: params[:id])
-        Tag.joins(:tag_names)
-        .preload(:tag_names, :materials, :deerjikists, :tag_descriptions,
-                 parents: [:tag_names, :materials, :deerjikists, :tag_descriptions])
-        .find_by(id: params[:id],
-                 tag_names: { language_code: locale.language_code, primary_flg: true })
     return head :not_found unless tag
 
-    render json: { tag: TagRepr.base(tag, language_code: locale.language_code),
+    render json: { tag: TagRepr.base(tag),
-                   deerjikists: DeerjikistRepr.many(
+                   deerjikists: DeerjikistRepr.many(tag.deerjikists) }
-                       tag.deerjikists,
-                       language_code: locale.language_code) }
   end
 
   def deerjikists_by_name
     name = params[:name].to_s.strip
     return render_bad_request('name は必須です．') if name.blank?
 
-    locale = resolve_locale!
+    tag = Tag.joins(:tag_name)
-
+             .includes(:tag_name, tag_name: :wiki_page)
-    tag =
+             .find_by(tag_names: { name: })
-        Tag
-        .joins(:tag_names)
-        .preload(:tag_names, :materials, :deerjikists, :tag_descriptions,
-                 parents: [:tag_names, :materials, :deerjikists, :tag_descriptions])
-        .find_by(tag_names: { name:, language_code: locale.language_code, primary_flg: true })
     return head :not_found unless tag
 
-    render json: { tag: TagRepr.base(tag, language_code: locale.language_code),
+    render json: { tag: TagRepr.base(tag),
-                   deerjikists: DeerjikistRepr.many(
+                   deerjikists: DeerjikistRepr.many(tag.deerjikists) }
-                       tag.deerjikists,
-                       language_code: locale.language_code) }
   end
 
   def update_deerjikists
     return head :unauthorized unless current_user
     return head :forbidden unless current_user.gte_member?
 
-    locale = resolve_locale!
+    tag = Tag.joins(:tag_name)
-
+             .includes(:tag_name, tag_name: :wiki_page)
-    tag =
+             .find_by(id: params[:id])
-        Tag
-        .joins(:tag_names)
-        .preload(:tag_names, :materials, :deerjikists, :tag_descriptions,
-                 parents: [:tag_names, :materials, :deerjikists, :tag_descriptions])
-        .find_by(id: params[:id],
-                 tag_names: { language_code: locale.language_code, primary_flg: true })
     return head :not_found unless tag
 
-    rows = normalise_deerjikist_rows(tag)
-    return if performed?
-
     ApplicationRecord.transaction do
-      SharedResourceIdAllocator.lock_space!(Tag, 'tags')
+      tag.deerjikists = []
-      tag.lock!
+      params[:_json].each.with_index do |item, i|
-
+        platform = item[:platform]
-      requested_keys = rows.map { |row| [row[:platform], row[:code]] }.uniq
+        code = normalise_deerjikist_code(platform, item[:code])
-      row_indexes_by_key = rows_by_key(rows)
+        deerjikist = Deerjikist.find_or_initialize_by(platform:, code:)
-      locked_deerjikists = lock_deerjikists_for_tag_update(tag.id, requested_keys)
+        deerjikist.tag = tag
-      current_deerjikists = locked_deerjikists.filter { |deerjikist|
+        render_deerjikist_form_record_invalid(deerjikist, i) unless deerjikist.save
-        deerjikist.tag_id == tag.id
-      }
-      requested_deerjikists = locked_deerjikists.filter { |deerjikist|
-        row_indexes_by_key.key?([deerjikist.platform, deerjikist.code])
-      }
-
-      render_deerjikist_conflicts(locale, requested_deerjikists, row_indexes_by_key, tag)
-      raise ActiveRecord::Rollback if performed?
-
-      requested_keys_set = requested_keys.to_set
-      current_deerjikists.each do |deerjikist|
-        key = [deerjikist.platform, deerjikist.code]
-        deerjikist.destroy! unless requested_keys_set.include?(key)
-      end
-
-      existing_keys = requested_deerjikists.to_h { |deerjikist|
-        [[deerjikist.platform, deerjikist.code], true]
-      }
-      requested_keys.each do |platform, code|
-        next if existing_keys[[platform, code]]
-
-        deerjikist = Deerjikist.new(platform:, code:, tag:)
-        row_index = row_indexes_by_key[[platform, code]].first
-        begin
-          render_deerjikist_form_record_invalid(deerjikist, row_index) unless deerjikist.save
-        rescue ActiveRecord::RecordNotUnique
-          conflicts = lock_deerjikists_for_tag_update(tag.id, [[platform, code]])
-          render_deerjikist_conflicts(locale, conflicts, row_indexes_by_key, tag)
-        end
         raise ActiveRecord::Rollback if performed?
       end
     end
 
     return if performed?
 
-    render json: DeerjikistRepr.many(tag.reload.deerjikists,
+    render json: DeerjikistRepr.many(tag.reload.deerjikists)
-                                     language_code: locale.language_code)
   end
 
   def materials_by_name
     name = params[:name].to_s.strip
     return render_bad_request('name は必須です．') if name.blank?
-
-    locale = resolve_locale!
-
     material_filter = material_filter_param(default: 'any')
 
-    tag =
+    tag = Tag.joins(:tag_name)
-        Tag
+             .includes(:tag_name, :materials, tag_name: :wiki_page)
-        .joins(:tag_names)
+             .find_by(tag_names: { name: })
-        .preload(:tag_names, :materials, :deerjikists, :tag_descriptions,
-                 parents: [:tag_names, :materials, :deerjikists, :tag_descriptions])
-        .find_by(tag_names: { name:, language_code: locale.language_code, primary_flg: true })
     return head :not_found unless tag
 
-    graph = build_with_depth_graph(material_filter, locale.language_code)
+    graph = build_with_depth_graph(material_filter)
 
-    render json: build_tag_children(tag, graph:, language_code: locale.language_code)
+    render json: build_tag_children(tag, graph:)
   end
 
   def update_all
@@ -455,37 +245,39 @@ class TagsController < ApplicationController
     return render_unprocessable_entity('カテゴリは必須です．', field: :category) if category.blank?
     return render_unprocessable_entity '廃止状態は必須です．', field: :deprecated unless params.key?(:deprecated)
 
-    locale = resolve_locale!
+    return unless validate_tag_rename(tag, name)
-
-    return unless validate_tag_rename(locale, tag, name)
 
     alias_names = params[:aliases].to_s.split.uniq
     parent_names = params[:parent_tags].to_s.split.uniq
     deprecated = bool?(:deprecated)
 
-    if category == 'nico'
+    if tag.nico? && deprecated
+      return render_unprocessable_entity 'ニコタグは廃止できません．', field: :deprecated
+    end
+
+    if tag.nico? || category == 'nico'
       return render_unprocessable_entity 'ニコタグは変更できません．', field: :category
     end
 
     ApplicationRecord.transaction do
-      TagVersioning.ensure_snapshot!(tag, created_by_user: current_user,
+      TagVersioning.ensure_snapshot!(tag, created_by_user: current_user)
-                                     language_code: locale.language_code)
 
-      old_name = tag.name(locale.language_code)
+      old_name = tag.name
       name_changed = name != old_name
+      wiki_page = tag.tag_name.wiki_page if name_changed
 
       if tag.deprecated? == deprecated
         tag.update!(category:)
       else
         tag.update!(category:, deprecated_at: deprecated ? Time.current : nil)
       end
-      rename_tag_name!(locale, tag, name) if name_changed
+      rename_tag_name!(tag, name) if name_changed
 
-      alias_names << old_name if name_changed && old_name.present?
+      alias_names << old_name if name_changed
       alias_names.delete(name)
 
-      update_aliases!(locale, tag, alias_names)
+      update_aliases!(tag, alias_names)
-      update_parent_tags!(locale, tag, parent_names)
+      update_parent_tags!(tag, parent_names)
 
       tag.reload
 
@@ -493,18 +285,17 @@ class TagsController < ApplicationController
           tag,
           event_type: :update,
           created_by_user: current_user,
-          language_code: locale.language_code)
+          name_changed:,
+          wiki_page:)
     end
 
-    render json: TagRepr.base(tag.reload, language_code: locale.language_code)
+    render json: TagRepr.base(tag.reload)
   end
 
   def update
     return head :unauthorized unless current_user
     return head :forbidden unless current_user.gte_member?
 
-    locale = resolve_locale!
-
     name = params[:name].presence
     category = params[:category].presence
     deprecated_given = params.key?(:deprecated)
@@ -512,20 +303,24 @@ class TagsController < ApplicationController
 
     tag = Tag.find(params[:id])
 
-    return unless validate_tag_rename(locale, tag, name)
+    if tag.nico? && deprecated_given && deprecated
+      return render_unprocessable_entity 'ニコタグは廃止できません．', field: :deprecated
+    end
 
-    if category.present? && category == 'nico'
+    return unless validate_tag_rename(tag, name)
+
+    if tag.nico? || (category.present? && category == 'nico')
       return render_unprocessable_entity 'ニコタグは変更できません．', field: :category
     end
 
     ApplicationRecord.transaction do
-      TagVersioning.ensure_snapshot!(tag, created_by_user: current_user,
+      TagVersioning.ensure_snapshot!(tag, created_by_user: current_user)
-                                     language_code: locale.language_code)
 
-      old_name = tag.name(locale.language_code)
+      old_name = tag.name
       name_changed = name.present? && name != old_name
+      wiki_page = tag.tag_name.wiki_page if name_changed
 
-      rename_tag_name!(locale, tag, name) if name_changed
+      rename_tag_name!(tag, name) if name_changed
       tag.update!(category:) if category.present?
       if deprecated_given && tag.deprecated? != deprecated
         tag.update!(deprecated_at: deprecated ? Time.current : nil)
@@ -537,10 +332,11 @@ class TagsController < ApplicationController
           tag,
           event_type: :update,
           created_by_user: current_user,
-          language_code: locale.language_code)
+          name_changed:,
+          wiki_page:)
     end
 
-    render json: TagRepr.base(tag.reload, language_code: locale.language_code)
+    render json: TagRepr.base(tag.reload)
   end
 
   private
@@ -552,7 +348,7 @@ class TagsController < ApplicationController
     value
   end
 
-  def build_with_depth_graph material_filter, language_code
+  def build_with_depth_graph material_filter
     children_by_parent_id = Hash.new { |h, k| h[k] = [] }
     parent_ids_by_child_id = Hash.new { |h, k| h[k] = [] }
 
@@ -567,13 +363,10 @@ class TagsController < ApplicationController
 
     material_tag_ids = Material.unscoped.kept.where.not(tag_id: nil).distinct.pluck(:tag_id).to_set
 
-    tags_by_id =
+    tags_by_id = Tag.joins(:tag_name)
-        Tag
+                    .where(id: tag_ids)
-        .joins(:tag_names)
+                    .pluck('tags.id', 'tag_names.name', 'tags.category', 'tags.deprecated_at')
-        .where(id: tag_ids,
+                    .each_with_object({ }) do |(id, name, category, deprecated_at), h|
-               tag_names: { language_code:, primary_flg: true })
-        .pluck('tags.id', 'tag_names.name', 'tags.category', 'tags.deprecated_at')
-        .each_with_object({ }) do |(id, name, category, deprecated_at), h|
       h[id] = { name:, category:, deprecated: deprecated_at.present?,
                 has_material: material_tag_ids.include?(id) }
     end
@@ -705,147 +498,145 @@ class TagsController < ApplicationController
       children: [] }
   end
 
-  def build_tag_children tag, graph: nil, language_code: 'ja'
+  def build_tag_children tag, graph: nil
     material = tag.materials.first
     tag_graph = graph && graph[:tags_by_id][tag.id]
     material =
         nil if tag_graph && !with_depth_visible_tag?(tag_graph, graph[:material_filter])
 
-    children = tag.children.sort_by { _1.name(language_code) }
+    children = tag.children.sort_by(&:name)
     if graph
       children = children.filter { |child_tag| visible_subtree?(child_tag.id, graph) }
     end
 
-    TagRepr.base(tag, language_code:).merge(
+    TagRepr.base(tag).merge(
-        children: children.map { build_tag_children(_1, graph:, language_code:) },
+        children: children.map { build_tag_children(_1, graph:) },
         has_material: tag_graph ? tag_graph[:has_material] : material.present?,
-        material: material && MaterialRepr.base(
+        material: material && MaterialRepr.base(material, host: request.base_url))
-            material,
-            host: request.base_url,
-            language_code:))
   end
 
-  def record_tag_version! tag, event_type:, created_by_user:, language_code: 'ja'
+  def record_tag_version! tag, event_type:, created_by_user:, name_changed: false, wiki_page: nil
-    TagVersionRecorder.record!(tag:, event_type:, created_by_user:, language_code:)
+    if tag.nico?
+      NicoTagVersionRecorder.record!(tag:, event_type:, created_by_user:)
+      return
+    end
+
+    TagVersionRecorder.record!(tag:, event_type:, created_by_user:)
+
+    return unless name_changed
+
+    wiki_page ||= tag.tag_name.wiki_page
+    return unless wiki_page&.wiki_versions&.exists?
+
+    WikiVersionRecorder.record!(
+        page: wiki_page,
+        event_type: :update,
+        created_by_user:)
   end
 
-  def validate_tag_rename locale, tag, name
+  def validate_tag_rename tag, name
-    return true if name.blank? || name == tag.name(locale.language_code)
+    return true if name.blank? || name == tag.name
 
-    system_tag_names = ['タグ希望', 'bot操作', 'ニジラー情報不詳', '動画', 'ニコニコ']
+    if tag.in?([Tag.tagme, Tag.bot, Tag.no_deerjikist, Tag.video, Tag.niconico])
-    if TagName.exists?(tag_id: tag.id,
-                       language_code: 'ja',
-                       primary_flg: true,
-                       name: system_tag_names)
       render_unprocessable_entity 'システム・タグの名称は変更できません．', field: :name
       return false
     end
 
-    target_tag_name = TagName.find_by(language_code: locale.language_code, name:)
+    target_tag_name = TagName.with_discarded.find_by(name:)
-    return true unless target_tag_name&.primary_flg
+    return true if target_tag_name.nil?
+    return true if target_tag_name.canonical_id?
 
     render_unprocessable_entity 'その名前は既に使はれてゐます．', field: :name
     false
   end
 
-  def rename_tag_name! locale, tag, name
+  def rename_tag_name! tag, name
-    return if name == tag.name(locale.language_code)
+    return if name == tag.name
 
-    current_tag_name = tag.tag_name(locale.language_code)
+    current_tag_name = tag.tag_name
-    target_tag_name = TagName.find_by(language_code: locale.language_code, name:)
+    target_tag_name = TagName.with_discarded.find_by(name:)
 
-    unless target_tag_name
+    if target_tag_name.nil?
-      if current_tag_name
+      current_tag_name.update!(name:)
-        current_tag_name.update!(name:, auto_generated: false)
-      else
-        current_tag_name = TagName.create!(
-            tag:,
-            language_code: locale.language_code,
-            script_code: locale.script_code,
-            name:,
-            primary_flg: true)
-        tag.association(:tag_names).reset
-      end
       return
     end
 
-    promote_tag_alias!(tag, current_tag_name:, promoted_tag_name: target_tag_name)
+    promote_tag_alias!(
+        tag,
+        current_tag_name:,
+        promoted_tag_name: target_tag_name)
   end
 
   def promote_tag_alias! tag, current_tag_name:, promoted_tag_name:
-    old_owner_tag = promoted_tag_name.tag
+    old_owner_tag = promoted_tag_name.canonical&.tag
 
     if old_owner_tag && old_owner_tag != tag
       TagVersioning.ensure_snapshot!(old_owner_tag, created_by_user: current_user)
     end
 
-    current_tag_name&.update!(primary_flg: false, auto_generated: false)
+    promoted_tag_name.undiscard! if promoted_tag_name.discarded?
-    promoted_tag_name.update!(tag:, primary_flg: true, auto_generated: false)
+    promoted_tag_name.update!(canonical: nil)
-    tag.association(:tag_names).reset
+
+    TagName.with_discarded
+           .where(canonical_id: current_tag_name.id)
+           .where.not(id: promoted_tag_name.id)
+           .find_each do |alias_tag_name|
+      alias_tag_name.update!(canonical: promoted_tag_name)
+    end
+
+    current_tag_name.wiki_page&.update!(tag_name: promoted_tag_name)
+    tag.update!(tag_name: promoted_tag_name)
+    current_tag_name.association(:wiki_page).reset
+    current_tag_name.association(:tag).reset
+    current_tag_name.reload.update!(canonical: promoted_tag_name)
 
     return unless old_owner_tag && old_owner_tag != tag
 
-    record_tag_version!(old_owner_tag.reload, event_type: :update,
+    record_tag_version!(old_owner_tag.reload, event_type: :update, created_by_user: current_user)
-                        created_by_user: current_user,
-                        language_code: promoted_tag_name.language_code)
   end
 
-  def update_aliases! locale, tag, alias_names
+  def update_aliases! tag, alias_names
     alias_names = alias_names.uniq
 
     affected_tags = [tag]
 
-    current_aliases = tag.tag_names.where(language_code: locale.language_code,
+    current_aliases = tag.tag_name.aliases.to_a
-                                          primary_flg: false).to_a
 
-    requested_aliases = TagName.where(
+    current_aliases.each do |alias_tag_name|
-        language_code: locale.language_code, name: alias_names)
+      next if alias_names.include?(alias_tag_name.name)
-        .includes(:tag).index_by(&:name)
 
-    alias_names.each do |alias_name|
+      affected_tags << alias_tag_name.canonical&.tag
-      alias_tag_name = requested_aliases[alias_name]
-      if alias_tag_name&.primary? && alias_tag_name.tag_id && alias_tag_name.tag_id != tag.id
-        alias_tag_name.errors.add(:name, 'その名前は既に使はれてゐます．')
-        raise ActiveRecord::RecordInvalid, alias_tag_name
-      end
-      affected_tags << alias_tag_name&.tag
     end
 
-    affected_tags = affected_tags.compact.uniq(&:id)
+    alias_names.each do |alias_name|
-    affected_tags.each do |affected_tag|
+      alias_tag_name = TagName.find_undiscard_or_create_by!(name: alias_name)
-      TagVersioning.ensure_snapshot!(affected_tag, created_by_user: current_user,
+      affected_tags << alias_tag_name.canonical&.tag
-                                     language_code: locale.language_code)
+    end
+
+    affected_tags.compact.uniq.each do |affected_tag|
+      TagVersioning.ensure_snapshot!(affected_tag, created_by_user: current_user)
     end
 
     current_aliases.each do |alias_tag_name|
       next if alias_names.include?(alias_tag_name.name)
 
-      alias_tag_name.destroy!
+      alias_tag_name.update!(canonical: nil)
     end
 
     alias_names.each do |alias_name|
-      alias_tag_name = requested_aliases[alias_name] || TagName.new(
+      alias_tag_name = TagName.find_undiscard_or_create_by!(name: alias_name)
-          language_code: locale.language_code,
+      alias_tag_name.update!(canonical: tag.tag_name)
-          name: alias_name)
-      alias_tag_name.script_code = locale.script_code
-      alias_tag_name.update!(tag:, primary_flg: false, auto_generated: false)
     end
 
-    affected_tags.each do |affected_tag|
+    affected_tags.compact.uniq.each do |affected_tag|
-      # update_all records this Tag after its parent relationships are updated.
+      record_tag_version!(affected_tag, event_type: :update, created_by_user: current_user)
-      next if affected_tag.id == tag.id
-
-      record_tag_version!(affected_tag, event_type: :update,
-                          created_by_user: current_user,
-                          language_code: locale.language_code)
     end
   end
 
-  def update_parent_tags! locale, tag, parent_names
+  def update_parent_tags! tag, parent_names
-    parent_tags = Tag.normalise_tags!(locale, parent_names,
+    parent_tags = Tag.normalise_tags!(parent_names, with_tagme: false,
-                                      with_tagme: false,
+                                                    with_no_deerjikist: false,
-                                      with_no_deerjikist: false,
+                                                    deny_nico: true)
-                                      created_by_user: current_user)
 
     old_parent_tags = tag.parents.to_a
 
@@ -862,7 +653,6 @@ class TagsController < ApplicationController
   end
 
   def normalise_deerjikist_code platform, code
-    code = code.to_s
     return code if platform != 'youtube' || code[0] != '@'
 
     url = "https://www.youtube.com/#{ code }"
@@ -879,84 +669,6 @@ class TagsController < ApplicationController
     nil
   end
 
-  def normalise_deerjikist_rows tag
-    rows = []
-
-    params[:_json].each.with_index do |item, index|
-      platform = item[:platform]
-      unless Deerjikist.platforms.key?(platform)
-        render_deerjikist_platform_invalid(index)
-        return rows
-      end
-
-      code = normalise_deerjikist_code(platform, item[:code])
-      deerjikist = Deerjikist.new(platform:, code:, tag:)
-      unless deerjikist.valid?
-        render_deerjikist_form_record_invalid(deerjikist, index)
-        return rows
-      end
-
-      rows << { index:, platform:, code: }
-    end
-
-    rows
-  end
-
-  def lock_deerjikists_for_tag_update tag_id, keys
-    clauses = ['tag_id = ?']
-    values = [tag_id]
-
-    keys.each do |platform, code|
-      clauses << '(platform = ? AND code = ?)'
-      values << platform << code
-    end
-
-    Deerjikist
-      .where(clauses.join(' OR '), *values)
-      .order(:platform, :code)
-      .lock
-      .to_a
-  end
-
-  def render_deerjikist_conflicts locale, deerjikists, row_indexes_by_key, tag
-    conflicts = deerjikists.filter { |deerjikist| deerjikist.tag_id != tag.id }
-    return if conflicts.empty?
-
-    tag_names_by_id =
-        Tag
-        .joins(:tag_names)
-        .where(tag_names: { language_code: locale.language_code, primary_flg: true },
-               id: conflicts.map(&:tag_id).uniq)
-        .pluck('tags.id', 'tag_names.name')
-        .to_h
-    fields = { }
-
-    conflicts.each do |deerjikist|
-      message = "この情報は既に「#{ tag_names_by_id[deerjikist.tag_id] }」に紐づいてゐます．"
-      row_indexes_by_key[[deerjikist.platform, deerjikist.code]].each do |index|
-        field = :"deerjikists.#{ index }.code"
-        fields[field] ||= []
-        fields[field] << message
-      end
-    end
-
-    render_validation_error fields:
-  end
-
-  def render_deerjikist_platform_invalid index
-    render_validation_error fields: {
-      :"deerjikists.#{ index }.platform" => ['値が不正です．'],
-    }
-  end
-
-  def rows_by_key rows
-    rows.each_with_object({ }) do |row, result|
-      key = [row[:platform], row[:code]]
-      result[key] ||= []
-      result[key] << row[:index]
-    end
-  end
-
   def render_deerjikist_form_record_invalid deerjikist, index
     fields = { }
 
@@ -975,20 +687,4 @@ class TagsController < ApplicationController
 
     render_validation_error fields:
   end
-
-  def external_tag_json tag
-    { 'id' => tag.id,
-      'name' => "#{ tag.platform }:#{ tag.name }",
-      'category' => 'nico',
-      'deprecated_at' => nil,
-      'created_at' => tag.created_at,
-      'updated_at' => tag.created_at,
-      'post_count' => tag.post_count,
-      'matched_alias' => nil,
-      'aliases' => [],
-      'parents' => [],
-      'has_wiki' => false,
-      'material_id' => nil,
-      'has_deerjikists' => false }
-  end
 end

backend/app/controllers/theatre_skip_events_controller.rb

@@ -6,7 +6,7 @@ class TheatreSkipEventsController < ApplicationController
     events =
         TheatreSkipEvent
         .where(theatre_id: params[:theatre_id])
-        .includes(:post, tags: :tag_names)
+        .includes(:post, tags: :tag_name)
         .order(created_at: :desc)
         .limit(limit)
 
@@ -14,7 +14,7 @@ class TheatreSkipEventsController < ApplicationController
       { id: event.id,
         theatre_id: event.theatre_id,
         post: { id: event.post.id, title: event.post.title, url: event.post.url },
-        tags: event.tags.map { |tag| { id: tag.id, name: tag.name('ja') } },
+        tags: event.tags.map { |tag| { id: tag.id, name: tag.name } },
         programme_position: event.programme_position,
         created_at: event.created_at }
     }

backend/app/controllers/users_controller.rb

@@ -1,69 +1,39 @@
 class UsersController < ApplicationController
   def create
-    if current_user
-      return render json: { code: current_user.inheritance_code,
-                            user: current_user.slice(:id, :name, :inheritance_code, :role) }
-    end
-
     user = nil
     User.transaction do
       user = User.create!(inheritance_code: SecureRandom.uuid, role: :guest)
       attach_ip_address!(user)
     end
 
-    establish_user_session(user)
     render json: { code: user.inheritance_code,
                    user: user.slice(:id, :name, :inheritance_code, :role) },
            status: :created
   end
 
   def verify
-    user = if bool?(:restore) && current_user
+    user = User.find_by(inheritance_code: params[:code])
-             current_user
-           else
-             User.find_by(inheritance_code: params[:code])
-           end
     return render json: { valid: false } unless user
     return head :forbidden if user.banned?
 
     attach_ip_address!(user)
-    establish_user_session(user)
 
     render json: { valid: true, user: user.slice(:id, :name, :inheritance_code, :role) }
   end
 
-  def show_session
-    return head :unauthorized unless current_user
-
-    render json: current_user.slice(:id, :name, :role)
-  end
-
-  def csrf
-    response.headers['Cache-Control'] = 'private, no-store'
-    render json: { token: form_authenticity_token }
-  end
-
-  def destroy_session
-    reset_session
-    response.headers['Cache-Control'] = 'private, no-store'
-    head :no_content
-  end
-
   def renew
     return head :unauthorized unless current_user
 
     user = current_user
     user.inheritance_code = SecureRandom.uuid
     user.save!
-    establish_user_session(user)
     render json: { code: user.inheritance_code }
   end
 
   def me
-    user = current_user
+    user = User.find_by(inheritance_code: params[:code])
-    return head :unauthorized unless user
+    return head :not_found unless user
 
-    attach_ip_address!(user)
     render json: user.slice(:id, :name, :inheritance_code, :role)
   end
 

backend/app/controllers/wiki_pages_controller.rb

@@ -1,168 +1,86 @@
 class WikiPagesController < ApplicationController
-  # 互換性のための Facade API
+  rescue_from Wiki::Commit::Conflict, with: :render_wiki_conflict
-
-  rescue_from TagDescriptionOperation::Conflict, with: :render_wiki_conflict
 
   def index
-    locale = resolve_locale!
-
     title = params[:title].to_s.strip
-    descriptions =
+    if title.blank?
-        HistoryAccess.current(TagDescription.all, TagDescriptionVersion)
+      return render json: WikiPageRepr.base(
-        .where(language_code: locale.language_code)
+          WikiPage.joins(:tag_name).includes(tag_name: :tag))
-        .includes(tag: :tag_names)
-
-    descriptions = descriptions.order(updated_at: :desc, id: :asc)
-    tag_groups = HistoryAccess.current(TagGroup.all, TagGroupVersion).order(updated_at: :desc, id: :asc)
-    if title.present?
-      pattern = "%#{ ApplicationRecord.sanitize_sql_like(title) }%"
-      descriptions =
-          descriptions.joins(tag: :tag_names)
-          .where(tag_names: { language_code: locale.language_code, primary_flg: true })
-          .where('tag_names.name LIKE ?', pattern).limit(20)
-      tag_groups = tag_groups.where(
-          'CONCAT(?, name) LIKE ? OR CONCAT(?, name) LIKE ?',
-          TagGroupRepr::PREFIX, pattern, 'tag_group:', pattern).limit(20)
     end
 
-    json = descriptions.map { |description| WikiPageRepr.tag_description(locale, description) }
+    q = WikiPage.joins(:tag_name).includes(tag_name: :tag)
-    json.concat(tag_groups.map { WikiPageRepr.tag_group(_1) })
+        .where('tag_names.name LIKE ?', "%#{ WikiPage.sanitize_sql_like(title) }%")
-    json.sort_by! do |row|
+    render json: WikiPageRepr.base(q.limit(20))
-      [-row['updated_at'].to_f, row.fetch('resource_type'), row.fetch('id')]
-    end
-    json = json.first(20) if title.present?
-    render json:
-  end
-
-  def autolink_candidates
-    locale = resolve_locale!
-    language = params.fetch(:language, locale.language_code)
-    language = 'ja' if language == 'und'
-    return head :not_found unless language.is_a?(String)
-    return head :not_found unless Language.exists?(code: language)
-
-    names = HistoryAccess.current(TagDescription.all, TagDescriptionVersion).joins(tag: :tag_names)
-        .where(language_code: locale.language_code, tag_names: { language_code: language })
-        .pluck('tag_descriptions.tag_id', 'tag_descriptions.id', 'tag_names.name')
-    candidates = names.map do |tag_id, description_id, name|
-      WikiPageRepr.autolink_tag(locale, tag_id:, description_id:, name:, language:)
-    end
-    candidates.concat(HistoryAccess.current(TagGroup.all, TagGroupVersion).order(:id).pluck(:id, :name).map do |id, name|
-      WikiPageRepr.autolink_group(locale, id:, name:, language:)
-    end)
-    render json: candidates
   end
 
   def show
-    locale = resolve_locale!
+    page = WikiPage.joins(:tag_name)
-    reference = TagDescription.find_by(id: params[:id])
+                   .includes(tag_name: :tag)
-    tag_description = reference && TagDescription.find_by(
+                   .find_by(id: params[:id])
-        tag_id: reference.tag_id, language_code: locale.language_code)
+    render_wiki_page_or_404 page
-
-    return render_wiki_page_by_tag_description locale, tag_description if tag_description
-    return head :not_found if reference
-
-    tag_group = TagGroup.find_by(id: params[:id])
-    return render_tag_group locale, tag_group if tag_group
-
-    head :not_found
   end
 
   def show_by_title
-    locale = resolve_locale!
     title = params[:title].to_s.strip
-
+    page = WikiPage.joins(:tag_name)
-    if (name = TagGroupRepr.name_from_title(title))
+                   .includes(tag_name: :tag)
-      tag_group = TagGroup.find_by(name:)
+                   .find_by(tag_name: { name: title })
-      return render_tag_group locale, tag_group if tag_group
+    render_wiki_page_or_404 page
-      return head :not_found
-    end
-
-    tag_description = tag_description_by_title(locale, title)
-
-    return render_wiki_page_by_tag_description locale, tag_description if tag_description
-
-    head :not_found
-  end
-
-  def tag_reference
-    locale = resolve_locale!
-    description = TagDescription.find_by(id: params[:id])
-    return head :not_found unless description
-
-    tag = Tag.preload(:tag_names, :materials, :deerjikists, :tag_descriptions,
-                      parents: [:tag_names, :materials, :deerjikists, :tag_descriptions])
-        .find(description.tag_id)
-    render json: TagRepr.base(tag, language_code: locale.language_code)
   end
 
   def exists
-    locale = resolve_locale!
+    if WikiPage.exists?(id: params[:id])
-    if TagDescription.exists?(id: params[:id], language_code: locale.language_code)
+      head :no_content
-      return head :no_content
+    else
+      head :not_found
     end
-    return head :no_content if TagGroup.exists?(id: params[:id])
-
-    head :not_found
   end
 
   def exists_by_title
-    locale = resolve_locale!
     title = params[:title].to_s.strip
-    if (name = TagGroupRepr.name_from_title(title))
+    if WikiPage.joins(:tag_name).exists?(tag_names: { name: title })
-      exists = TagGroup.exists?(name:)
+      head :no_content
-      return head exists ? :no_content : :not_found
+    else
+      head :not_found
     end
-
-    return head :no_content if tag_description_by_title(locale, title)
-
-    head :not_found
   end
 
   def diff
     id = params[:id]
     return render_bad_request('id は必須です．') if id.blank?
 
-    locale = resolve_locale!
-
     from = params[:from].presence
     to = params[:to].presence
 
-    description = TagDescription.find_by(id:, language_code: locale.language_code)
+    page = WikiPage.joins(:tag_name).includes(tag_name: :tag).find(id)
-    if description
-      versions = description_versions(description)
-      from_rev = from && versions.find(from)
-      to_rev = to ? versions.find(to) : versions.find_by!(version_no: description.version_no)
-      resource = WikiPageRepr.tag_description(locale, description)
-    else
-      tag_group = TagGroup.find_by(id:)
-      if tag_group
-        from_rev = from && group_version(tag_group, from)
-        to_rev = to ? group_version(tag_group, to) : current_group_version(tag_group)
-        return head :not_found if (from && !from_rev) || !to_rev
 
-        from_body = from_rev && render_group_version(locale, from_rev)
+    from_rev = from && page.wiki_revisions.find(from)
-        to_body = render_group_version(locale, to_rev)
+    to_rev = to ? page.wiki_revisions.find(to) : page.current_revision
-        resource = { 'id' => tag_group.id,
+    if ((from_rev && !(from_rev.content?)) || !(to_rev&.content?))
-                     'title' => "#{ TagGroupRepr::PREFIX }#{ tag_group.name }",
+      return render_unprocessable_entity('差分を表示できない版です．')
-                     'deprecated_at' => nil }
-      else
-        return head :not_found
-      end
     end
 
-    from_body ||= from_rev&.body
+    diffs = Diff::LCS.sdiff(from_rev&.body&.lines || [], to_rev.body.lines)
-    to_body ||= to_rev.body
+    diff_json = diffs.map { |change|
-    diff_json = wiki_diff(from_body, to_body)
+      case change.action
+      when ?=
+        { type: 'context', content: change.old_element }
+      when ?!
+        [{ type: 'removed', content: change.old_element },
+         { type: 'added', content: change.new_element }]
+      when ?+
+        { type: 'added', content: change.new_element }
+      when ?-
+        { type: 'removed', content: change.old_element }
+      end
+    }.flatten.compact
 
-    render json: { wiki_page_id:      resource.fetch('id'),
+    render json: { wiki_page_id:      page.id,
-                   title:             resource.fetch('title'),
+                   title:             page.title,
-                   deprecated_at:     resource.fetch('deprecated_at'),
+                   deprecated_at:     page.deprecated_at,
                    older_revision_id: from_rev&.id,
                    newer_revision_id: to_rev.id,
                    diff:              diff_json }
-  rescue TagGroupMarkdownRenderer::UnsupportedStructure => error
-    render_unprocessable_entity(error.message)
   end
 
   def create
@@ -176,37 +94,20 @@ class WikiPagesController < ApplicationController
     return render_unprocessable_entity('タイトルは必須です．', field: :title) if title.blank?
     return render_unprocessable_entity('本文は必須です．', field: :body) if body.blank?
 
-    locale = resolve_locale!
+    tag_name = TagName.find_undiscard_or_create_by!(name: title)
 
-    if (name = TagGroupRepr.name_from_title(title))
+    page =
-      parsed = TagGroupMarkdownParser.parse(locale, body:, group: nil)
+        Wiki::Commit.create_content!(
-      group = TagGroupOperation.create!(
+            tag_name:,
-          locale, name:, **parsed.except(:compatibility_body), actor: current_user,
+            body:,
-          require_compatibility: true)
+            created_by_user: current_user,
-      version = current_group_version(group)
+            message:)
-      return render_group_representation(TagGroupRepr.one(locale, group), version,
-                                         status: :created)
-    end
 
-    description =
+    render json: WikiPageRepr.base(page), status: :created
-        if title.include?(':')
-          return render_unprocessable_entity field: :title
-        else
-          tag_name = TagName.find_by(language_code: locale.language_code, name: title)
-          return render_unprocessable_entity field: :title unless tag_name&.tag
-
-          TagDescriptionOperation.create!(
-              locale,
-              tag: tag_name.tag, body:, created_by_user: current_user, message:)
-        end
-
-    render json: WikiPageRepr.tag_description(locale, description), status: :created
   rescue ActiveRecord::RecordInvalid => e
     render_validation_error e.record
   rescue ActiveRecord::RecordNotUnique
     render_record_not_unique
-  rescue TagGroupMarkdownParser::InvalidDocument => error
-    render_unprocessable_entity(error.message)
   end
 
   def update
@@ -219,229 +120,91 @@ class WikiPagesController < ApplicationController
     return render_unprocessable_entity('タイトルは必須です．', field: :title) if title.blank?
     return render_unprocessable_entity('本文は必須です．', field: :body) if body.blank?
 
-    locale = resolve_locale!
+    page = WikiPage.find(params[:id])
-
-    group = TagGroup.find_by(id: params[:id])
-    if (name = TagGroupRepr.name_from_title(title))
-      return head :not_found unless group
-
-      parsed = TagGroupMarkdownParser.parse(locale, body:, group:)
-      TagGroupOperation.update!(
-          locale, group:, name:, **parsed.except(:compatibility_body), actor: current_user,
-          require_compatibility: true)
-      return head :ok
-    end
-
-    return render_unprocessable_entity field: :title if title.include?(':')
-    description = TagDescription.find_by(id: params[:id], language_code: locale.language_code)
-    unless description
-      return render_unprocessable_entity field: :title if group
-      return head :not_found
-    end
-
-    tag = description.tag
-    tag_name = tag.tag_name(locale.language_code)
-    return render_unprocessable_entity field: :title unless tag_name
-    return render_unprocessable_entity field: :title if tag_name.name.include?(':')
-
     base_revision_id = params[:base_revision_id].presence
 
     ApplicationRecord.transaction do
-      SharedResourceIdAllocator.lock_space!(Tag, 'tags')
+      page.lock!
-      tag.lock!
+
-      old_title = tag_name.name
+      old_title = page.title
-      if title != old_title
+
+      tag = Tag.find_by(tag_name_id: page.tag_name_id)
+
+      if tag && title != old_title
         TagVersioning.ensure_snapshot!(tag, created_by_user: current_user)
       end
 
-      tag_name.update!(name: title) if title != old_title
+      page.tag_name.update!(name: title) if title != old_title
 
       message = params[:message].presence
-      TagDescriptionOperation.update!(locale, tag:, body:, updated_by_user: current_user,
+      Wiki::Commit.content!(page:,
-                                      message:, base_revision_id:)
+                            body:,
+                            created_user: current_user,
+                            message:,
+                            base_revision_id:)
 
-      if title != old_title
+      if tag && title != old_title
         tag.reload
         TagVersionRecorder.record!(tag:, event_type: :update, created_by_user: current_user)
       end
     end
 
     head :ok
-  rescue ActiveRecord::RecordInvalid => e
+  end
-    render_validation_error e.record
+
-  rescue ActiveRecord::RecordNotUnique
+  def search
-    render_record_not_unique
+    index
-  rescue TagGroupMarkdownParser::InvalidDocument => error
-    render_unprocessable_entity(error.message)
   end
 
   def changes
-    locale = resolve_locale!
-
     id = params[:id].presence
-    description = id && TagDescription.find_by(id:, language_code: locale.language_code)
+    q = WikiRevision.joins(wiki_page: :tag_name)
-    if description
+                    .includes(:created_user, wiki_page: { tag_name: :tag })
-      return render json: description_changes(locale, TagDescription.where(id: description.id))
+                    .order(id: :desc)
-    end
+    q = q.where(wiki_page_id: id) if id
 
-    tag_group = id && TagGroup.find_by(id:)
+    render json: q.limit(200).map { |rev|
-    if tag_group
+      { revision_id:  rev.id,
-      return render json: tag_group_changes(HistoryAccess.visible(TagGroupVersion.all).where(tag_group_id: tag_group.id))
+        pred:         rev.base_revision_id,
-    end
+        succ:         nil,
-
+        wiki_page:    { id: rev.wiki_page_id,
-    json = []
+                        title: rev.wiki_page.title,
-    unless id
+                        deprecated_at: rev.wiki_page.deprecated_at },
-      json.concat(tag_group_changes(HistoryAccess.visible(TagGroupVersion.all)))
+        user:         rev.created_user && { id: rev.created_user.id, name: rev.created_user.name },
-      json.concat(description_changes(locale,
+        kind:         rev.kind,
-                                      TagDescription.where(language_code: locale.language_code)))
+        message:      rev.message,
-      json = json.each_with_index.sort_by do |row, index|
+        timestamp:    rev.created_at }
-        [-row.fetch(:timestamp).to_f, index]
+    }.compact
-      end.map(&:first)
-    end
-    render json: json.first(200)
   end
 
   private
 
-  def render_tag_group locale, group
+  def render_wiki_page_or_404 page
-    version = if params[:version].present?
+    return head :not_found unless page
-      group_version(group, params[:version])
-    else
-      current_group_version(group)
-    end
-    return head :not_found unless version
 
-    representation =
+    rev = find_revision(page)
-        if params[:version].present?
-          TagGroupRepr.historical(locale, version)
-        else
-          TagGroupRepr.one(locale, group)
-        end
-    render_group_representation(representation, version)
-  end
-
-  def render_group_version locale, version
-    representation = TagGroupRepr.historical(locale, version)
-    TagGroupMarkdownRenderer.render(representation)
-  end
-
-  def render_group_representation representation, version, body: nil, status: :ok
-    body ||= TagGroupMarkdownRenderer.render(representation)
-    versions = HistoryAccess.visible(TagGroupVersion.all).where(tag_group_id: version.tag_group_id)
-    pred = versions.find_by(version_no: version.version_no - 1)&.id
-    succ = versions.find_by(version_no: version.version_no + 1)&.id
-    updated_at = version.created_at
-    render json: {
-        'id' => version.tag_group_id,
-        'resource_type' => 'tag_group',
-        'title' => "#{ TagGroupRepr::PREFIX }#{ representation.fetch('name') }",
-        'deprecated_at' => nil,
-        'body' => body,
-        'revision_id' => version.id,
-        'pred' => pred,
-        'succ' => succ,
-        'updated_at' => updated_at }, status:
-  rescue TagGroupMarkdownRenderer::UnsupportedStructure => error
-    render_unprocessable_entity(error.message)
-  end
-
-  def current_group_version group
-    HistoryAccess.visible(TagGroupVersion.all).find_by(tag_group_id: group.id, version_no: group.version_no)
-  end
-
-  def group_version group, version_id
-    HistoryAccess.visible(TagGroupVersion.all).find_by(tag_group_id: group.id, id: version_id)
-  end
-
-  def wiki_diff from_body, to_body
-    diffs = Diff::LCS.sdiff(from_body.to_s.lines, to_body.to_s.lines)
-    diffs.map { |change|
-      case change.action
-      when ?=
-        { type: 'context', content: change.old_element }
-      when ?!
-        [{ type: 'removed', content: change.old_element },
-         { type: 'added', content: change.new_element }]
-      when ?+
-        { type: 'added', content: change.new_element }
-      when ?-
-        { type: 'removed', content: change.old_element }
-      end
-    }.flatten.compact
-  end
-
-  def tag_group_changes versions
-    rows = versions.includes(:created_by_user).order(id: :desc).limit(200).to_a
-    groups = TagGroup.where(id: rows.map(&:tag_group_id).uniq).index_by(&:id)
-    predecessors = HistoryAccess.visible(TagGroupVersion.all).where(tag_group_id: groups.keys,
-                                         version_no: rows.map { _1.version_no - 1 })
-        .pluck(:tag_group_id, :version_no, :id)
-        .to_h { |group_id, number, version_id| [[group_id, number], version_id] }
-    rows.map do |version|
-      group = groups.fetch(version.tag_group_id)
-      user = version.created_by_user
-      { revision_id: version.id,
-        pred: predecessors[[group.id, version.version_no - 1]],
-        succ: nil,
-        wiki_page: { id: group.id, title: "#{ TagGroupRepr::PREFIX }#{ version.name }",
-                     deprecated_at: nil },
-        user: user && { id: user.id, name: user.name },
-        kind: 'content', message: nil, timestamp: version.created_at }
-    end
-  end
-
-  def render_wiki_page_by_tag_description locale, description
-    versions = description_versions(description)
-    historical = params[:version].present?
-    rev = historical ? versions.find_by(id: params[:version]) :
-        versions.find_by(version_no: description.version_no)
     return head :not_found unless rev
 
-    pred = versions.find_by(version_no: rev.version_no - 1)&.id
+    if rev.redirect?
-    succ = versions.find_by(version_no: rev.version_no + 1)&.id
+      return (
-    body = historical ? rev.body : description.body
+          redirect_to wiki_page_by_title_path(title: rev.redirect_page.title),
-    updated_at = historical ? rev.created_at : description.updated_at
+                      status: :moved_permanently)
-    render json: WikiPageRepr.tag_description(locale, description).merge(
+    end
-        body:, revision_id: rev.id, pred:, succ:, updated_at:)
+
+    body = rev.body
+    revision_id = rev.id
+    pred = page.pred_revision_id(revision_id)
+    succ = page.succ_revision_id(revision_id)
+    updated_at = rev.created_at
+
+    render json: WikiPageRepr.base(page).merge(body:, revision_id:, pred:, succ:, updated_at:)
   end
 
-  def description_versions description
+  def find_revision page
-    HistoryAccess.visible(TagDescriptionVersion.all).where(tag_id: description.tag_id,
+    if params[:version].present?
-                                language_code: description.language_code)
+      page.wiki_revisions.find_by(id: params[:version])
-  end
+    else
-
+      page.current_revision
-  def tag_description_by_title locale, title
-    HistoryAccess.current(TagDescription.all, TagDescriptionVersion).joins(tag: :tag_names)
-        .find_by(language_code: locale.language_code,
-                 tag_names: { language_code: locale.language_code, name: title })
-  end
-
-  def description_changes locale, descriptions
-    versions = HistoryAccess.visible(TagDescriptionVersion.all).where(language_code: locale.language_code,
-                                           tag_id: descriptions.select(:tag_id))
-        .order(id: :desc).limit(200).to_a
-    return [] if versions.empty?
-
-    by_tag = descriptions.where(tag_id: versions.map(&:tag_id))
-        .includes(tag: :tag_names).index_by(&:tag_id)
-    users = User.where(id: versions.map(&:created_by_user_id)).index_by(&:id)
-    predecessors = HistoryAccess.visible(TagDescriptionVersion.all).where(
-        language_code: locale.language_code, tag_id: by_tag.keys,
-        version_no: versions.map { |version| version.version_no - 1 })
-        .pluck(:tag_id, :version_no, :id)
-        .to_h { |tag_id, version_no, id| [[tag_id, version_no], id] }
-    versions.map do |version|
-      description = by_tag.fetch(version.tag_id)
-      user = users[version.created_by_user_id]
-      { revision_id: version.id,
-        pred: predecessors[[version.tag_id, version.version_no - 1]],
-        succ: nil,
-        wiki_page: { id: description.id, title: description.tag.name(locale.language_code),
-                     deprecated_at: description.tag.deprecated_at },
-        user: user && { id: user.id, name: user.name },
-        kind: 'content',
-        message: nil,
-        timestamp: version.created_at }
     end
   end
 

backend/app/models/external_tag.rb

@@ -1,26 +0,0 @@
-class ExternalTag < ApplicationRecord
-  before_create { SharedResourceIdAllocator.assign!(self) }
-
-  enum :platform, nico: 'nico'
-  validates :platform, presence: true, inclusion: { in: ExternalTag.platforms.keys }
-
-  has_many :post_external_tags, dependent: :delete_all
-  has_many :posts, through: :post_external_tags
-
-  has_many :nico_tag_versions, foreign_key: :tag_id, inverse_of: :external_tag
-
-  has_many :nico_tag_relations,
-           foreign_key: :nico_tag_id,
-           inverse_of: :nico_tag,
-           dependent: :destroy
-
-  has_many :linked_tags, through: :nico_tag_relations, source: :tag
-
-  def snapshot_linked_tag_names language_code = 'ja'
-    linked_tags
-        .joins(:tag_names)
-        .where(tag_names: { language_code:, primary_flg: true })
-        .order('tag_names.name')
-        .pluck('tag_names.name')
-  end
-end

backend/app/models/history_access.rb

@@ -1,160 +0,0 @@
-require 'digest'
-
-# Request/worker authority comes only from a server-resolved user. Default is public.
-class HistoryAccess < ActiveSupport::CurrentAttributes
-  attribute :user
-  class Forbidden < StandardError; end
-
-  KEYS = { 'post_versions' => ['post_id'], 'tag_versions' => ['tag_id'],
-           'nico_tag_versions' => ['tag_id'],
-           'tag_description_versions' => ['tag_id', 'language_code'],
-           'tag_group_versions' => ['tag_group_id'], 'material_versions' => ['material_id'] }.freeze
-
-  def self.revision
-    values = KEYS.keys.map do |table|
-      table.classify.constantize.unscoped.where(event_type: 'discard').maximum(:id)
-    end
-    Digest::SHA256.hexdigest(values.to_json)
-  end
-
-  def self.authority
-    { 'policy_revision' => revision, 'admin' => admin?, 'actor_id' => user&.id }
-  end
-
-  def self.authorised_artifact?(metadata)
-    return false unless metadata['policy_revision'].is_a?(String)
-    if metadata['admin']
-      admin? && user.id == metadata['actor_id']
-    else
-      metadata['policy_revision'] == revision
-    end
-  end
-
-  def self.admin? = user&.admin? == true
-
-  def self.sql(model, row = model.table_name, references: true)
-    return 'TRUE' if admin?
-
-    table = model.table_name
-    keys = KEYS.fetch(table)
-    matches = keys.map { "boundary.#{ _1 } = #{ row }.#{ _1 }" }.join(' AND ')
-    clauses = ["NOT EXISTS (SELECT 1 FROM #{ table } boundary WHERE #{ matches } " +
-        "AND boundary.event_type = 'discard' AND boundary.version_no >= #{ row }.version_no)"]
-    clauses += reference_sql(table, row) if references
-    clauses.map { "(#{ _1 })" }.join(' AND ')
-  end
-
-  def self.discard_reference(table, key, expression)
-    "NOT EXISTS (SELECT 1 FROM #{ table } blocked WHERE blocked.event_type = 'discard' " +
-        "AND #{ expression.call("blocked.#{ key }") })"
-  end
-
-  def self.reference_sql(table, row)
-    case table
-    when 'post_versions'
-      [['tag_versions', 'tag_id', 'tag_id'],
-       ['nico_tag_versions', 'tag_id', 'external_tag_id']].map do |target, key, field|
-        discard_reference(target, key, ->(id) {
-          "JSON_CONTAINS(#{ row }.tags_json, JSON_OBJECT('#{ field }', #{ id }))"
-        })
-      end + [discard_reference('post_versions', 'post_id', ->(id) {
-        "FIND_IN_SET(#{ id }, REPLACE(#{ row }.parent_post_ids, ' ', ',')) > 0"
-      })]
-    when 'tag_versions'
-      [discard_reference('tag_versions', 'tag_id', ->(id) {
-        "FIND_IN_SET(#{ id }, REPLACE(#{ row }.parent_tag_ids, ' ', ',')) > 0"
-      })]
-    when 'tag_description_versions'
-      [discard_reference('tag_versions', 'tag_id', ->(id) { "#{ row }.tag_id = #{ id }" })]
-    when 'material_versions'
-      [discard_reference('tag_versions', 'tag_id', ->(id) { "#{ row }.tag_id = #{ id }" }),
-       discard_reference('material_versions', 'material_id', ->(id) {
-         "#{ row }.parent_id = #{ id }"
-       })]
-    when 'tag_group_versions'
-      [['tag_versions', 'tag_id'], ['tag_group_versions', 'tag_group_id']].map do |target, field|
-        discard_reference(target, field, ->(id) {
-          ['members', 'placements'].map do |column|
-            "JSON_CONTAINS(COALESCE(JSON_EXTRACT(#{ row }.#{ column }, " +
-                "'$**.#{ field }'), JSON_ARRAY()), CAST(#{ id } AS CHAR))"
-          end.join(' OR ')
-        })
-      end
-    else []
-    end
-  end
-
-  def self.visible(scope, row = scope.klass.table_name)
-    scope.where(Arel.sql(sql(scope.klass, row)))
-  end
-
-  def self.current(scope, version_model)
-    return scope if admin?
-
-    table = scope.klass.table_name
-    keys = KEYS.fetch(version_model.table_name)
-    matches = keys.map do |key|
-      column = scope.klass.column_names.include?(key) ? key : 'id'
-      "current_snapshot.#{ key } = #{ table }.#{ column }"
-    end.join(' AND ')
-    boundary = "#{ matches } AND current_snapshot.version_no >= #{ table }.version_no " +
-        "AND current_snapshot.event_type = 'discard'"
-    scope = scope.where(Arel.sql("NOT EXISTS (SELECT 1 FROM #{ version_model.table_name } " +
-        "current_snapshot WHERE #{ boundary })"))
-    condition = "#{ matches } AND current_snapshot.version_no = #{ table }.version_no"
-    scope.where(Arel.sql("NOT EXISTS (SELECT 1 FROM #{ version_model.table_name } " +
-        "current_snapshot WHERE #{ condition } AND NOT (#{ sql(version_model, 'current_snapshot') }))"))
-  end
-
-  def self.require_current!(record, version_model)
-    return if admin?
-    raise Forbidden unless current(record.class.where(id: record.id), version_model).exists?
-  end
-
-  def self.blob_visible?(blob, seen = [])
-    return true if admin?
-    return false if seen.include?(blob.id)
-    seen = seen + [blob.id]
-    private_versions = MaterialVersion.where(file_blob_id: blob.id)
-        .where(Arel.sql("NOT (#{ sql(MaterialVersion) })"))
-    return false if private_versions.exists?
-
-    blob.attachments.each do |attachment|
-      model, key = { 'Post' => [PostVersion, 'post_id'],
-                     'Material' => [MaterialVersion, 'material_id'] }[attachment.record_type]
-      if model && model.where(key => attachment.record_id, event_type: 'discard').exists?
-        return false
-      end
-      if attachment.record_type == 'ActiveStorage::VariantRecord'
-        original = ActiveStorage::VariantRecord.find_by(id: attachment.record_id)&.blob
-        return false unless original && blob_visible?(original, seen)
-      end
-    end
-    true
-  end
-
-  def self.require_many!(records)
-    return if admin? || records.empty?
-
-    ids = records.map(&:id).uniq
-    raise Forbidden unless visible(records.first.class.where(id: ids)).count == ids.length
-  end
-
-  def self.require!(record)
-    return record if admin? || visible(record.class.where(id: record.id)).exists?
-
-    raise Forbidden
-  end
-
-  # This number deliberately ignores time, identity and content filters to prevent count probes.
-  def self.population_count(model) = model.unscoped.count
-
-  # Only the resource-wide denied count is public, never a condition-dependent denied count.
-  def self.population_counts(model)
-    total, excluded = model.unscoped.optimizer_hints('MAX_EXECUTION_TIME(10000)').pick(
-        Arel.sql('COUNT(*)'),
-        Arel.sql("COALESCE(SUM(CASE WHEN #{ sql(model) } THEN 0 ELSE 1 END), 0)"))
-    { population_count: total.to_i, excluded_count: excluded.to_i,
-      excluded_count_scope: 'all_recorded_versions' }
-  end
-end

backend/app/models/language.rb

@@ -1,5 +0,0 @@
-class Language < ApplicationRecord
-  has_many :locales, foreign_key: :language_code, primary_key: :code
-
-  enum :code, ja: 'ja', ain: 'ain'
-end

backend/app/models/locale.rb

@@ -1,38 +0,0 @@
-class Locale < ApplicationRecord
-  attr_accessor :certified_tag_names_path
-
-  around_save :save_with_tag_names!, if: :will_generate_tag_names?
-
-  belongs_to :language, foreign_key: :language_code, primary_key: :code
-  belongs_to :script, foreign_key: :script_code, primary_key: :code
-
-  # Enum deserialisation uses the first key for each stored value.
-  # Keep the legacy key for existing predicates, scopes and query arguments.
-  enum :code, { ja: 'ja', 'ain-Kana' => 'ain-Kana', ain_kana: 'ain-Kana' }
-
-  def code= value
-    super(self.class.codes.fetch(value.to_s, value))
-  end
-
-  def self.nipponese = Locale.find('ja')
-
-  private
-
-  def will_generate_tag_names?
-    deprecated_at.nil? && (will_save_change_to_code? || will_save_change_to_deprecated_at?)
-  end
-
-  def save_with_tag_names!
-    completed = self.class.transaction(requires_new: true) do
-      SharedResourceIdAllocator.lock_space!(Tag, 'tags')
-      saved = yield
-      raise ActiveRecord::RecordNotSaved, 'Locale save was halted' unless saved
-
-      TagNameCompletion.for_locale!(self)
-      true
-    end
-    unless completed
-      raise ActiveRecord::RecordNotSaved, 'Locale tag completion was rolled back'
-    end
-  end
-end

backend/app/models/nico_tag_relation.rb

@@ -1,10 +1,24 @@
 class NicoTagRelation < ApplicationRecord
-  belongs_to :nico_tag,
+  belongs_to :nico_tag, class_name: 'Tag'
-             class_name: 'ExternalTag',
+  belongs_to :tag, class_name: 'Tag'
-             foreign_key: :nico_tag_id,
-             inverse_of: :nico_tag_relations
-  belongs_to :tag, class_name: 'Tag', foreign_key: :tag_id
 
   validates :nico_tag_id, presence: true
   validates :tag_id, presence: true
+
+  validate :nico_tag_must_be_nico
+  validate :tag_mustnt_be_nico
+
+  private
+
+  def nico_tag_must_be_nico
+    if nico_tag && nico_tag.category != 'nico'
+      errors.add :nico_tag_id, 'タグのカテゴリがニコニコである必要があります．'
+    end
+  end
+
+  def tag_mustnt_be_nico
+    if tag && tag.category == 'nico'
+      errors.add :tag_id, '連携先タグのカテゴリはニコニコであってはなりません．'
+    end
+  end
 end

backend/app/models/nico_tag_version.rb

@@ -1,7 +1,7 @@
 class NicoTagVersion < ApplicationRecord
   include VersionRecord
 
-  belongs_to :external_tag, foreign_key: :tag_id, inverse_of: :nico_tag_versions
+  belongs_to :tag
 
   validates :name, presence: true
 end

backend/app/models/post.rb

@@ -55,27 +55,25 @@ class Post < ApplicationRecord
   belongs_to :uploaded_user, class_name: 'User', optional: true
 
   has_many :post_tags, dependent: :destroy, inverse_of: :post
-  has_many :tags, through: :post_tags
+  has_many :active_post_tags, -> { kept }, class_name: 'PostTag', inverse_of: :post
+  has_many :post_tags_with_discarded, -> { with_discarded }, class_name: 'PostTag'
+  has_many :tags, through: :active_post_tags
   has_many :active_tags, -> { where(tags: { deprecated_at: nil }) },
-           through: :post_tags,
+           through: :active_post_tags, source: :tag
-           source: :tag
 
   has_many :user_post_views, dependent: :delete_all
   has_many :post_similarities, dependent: :delete_all
   has_many :post_versions
-
   has_many :gekanator_guessed_games,
            class_name: 'GekanatorGame',
            foreign_key: :guessed_post_id,
            dependent: :delete_all,
            inverse_of: :guessed_post
-
   has_many :gekanator_correct_games,
            class_name: 'GekanatorGame',
            foreign_key: :correct_post_id,
            dependent: :delete_all,
            inverse_of: :correct_post
-
   has_many :gekanator_question_examples, dependent: :delete_all
 
   has_many :parent_post_implications,
@@ -92,9 +90,6 @@ class Post < ApplicationRecord
            inverse_of: :parent_post
   has_many :children, through: :child_post_implications, source: :post
 
-  has_many :post_external_tags, dependent: :destroy
-  has_many :external_tags, through: :post_external_tags
-
   has_one_attached :thumbnail
 
   attribute :version_no, :integer, default: 1
@@ -126,14 +121,14 @@ class Post < ApplicationRecord
     super(options).merge(thumbnail: nil)
   end
 
-  def snapshot_tag_names language_code = 'ja'
+  def snapshot_tag_names
     post_tags
-        .joins(tag: :tag_names)
+        .kept
-        .where(tag_names: { language_code:, primary_flg: true })
+        .joins(tag: :tag_name)
-        .includes(:sections, tag: :tag_names)
+        .includes(:sections, tag: :tag_name)
         .order('tag_names.name')
         .map do |post_tag|
-      name = post_tag.tag.name(language_code)
+      name = post_tag.tag.tag_name.name
       sections = post_tag.sections.sort_by(&:begin_ms)
 
       next name if sections.empty?
@@ -142,42 +137,6 @@ class Post < ApplicationRecord
     end
   end
 
-  def self.tag_snapshot_literal tag
-    sections = tag.fetch('sections', []).map do |sec|
-      begin_ms = sec.fetch('begin_ms')
-      end_ms = sec['end_ms']
-
-      "[#{ Post.ms_to_time(begin_ms) }-#{ end_ms ? Post.ms_to_time(end_ms) : '' }]"
-    end
-
-    "#{ tag.fetch('name') }#{ sections.join }"
-  end
-
-  def snapshot_tags_json language_code = 'ja'
-    tag_snapshots =
-        post_tags
-            .joins(tag: :tag_names)
-            .where(tag_names: { language_code:, primary_flg: true })
-            .includes(:sections, tag: :tag_names)
-            .order('tags.id')
-            .map { |pt|
-          { 'tag_id' => pt.tag.id,
-            'version_no' => pt.tag.version_no,
-            'name' => pt.tag.name(language_code),
-            'category' => pt.tag.category,
-            'sections' => pt.sections.sort_by(&:begin_ms).map {
-                            { 'begin_ms' => _1.begin_ms, 'end_ms' => _1.end_ms }
-                          } }
-        }
-
-    external_tag_snapshots =
-        post_external_tags.order(:external_tag_id).map {
-          { 'external_tag_id' => _1.external_tag_id }
-        }
-
-    tag_snapshots + external_tag_snapshots
-  end
-
   def self.section_literal section
     end_ms =
         section.end_ms ? Post.ms_to_time(section.end_ms) : ''

backend/app/models/post_external_tag.rb

@@ -1,4 +0,0 @@
-class PostExternalTag < ApplicationRecord
-  belongs_to :post
-  belongs_to :external_tag, counter_cache: :post_count
-end

backend/app/models/post_tag.rb

@@ -1,7 +1,14 @@
 class PostTag < ApplicationRecord
+  include Discard::Model
+
+  before_destroy do
+    raise ActiveRecord::ReadOnlyRecord, '消さないでください．'
+  end
+
   belongs_to :post
   belongs_to :tag, counter_cache: :post_count
   belongs_to :created_user, class_name: 'User', optional: true
+  belongs_to :deleted_user, class_name: 'User', optional: true
 
   has_many :sections, -> { order(:begin_ms) }, class_name: 'PostTagSection',
                                                foreign_key: [:post_id, :tag_id],
@@ -11,5 +18,18 @@ class PostTag < ApplicationRecord
 
   validates :post_id, presence: true
   validates :tag_id, presence: true
-  validates :post_id, uniqueness: { scope: :tag_id }
+  validates :post_id, uniqueness: {
+      scope: :tag_id,
+      conditions: -> { where(discarded_at: nil) } }
+
+  def discard_by! deleted_user
+    return self if discarded?
+
+    transaction do
+      update!(discarded_at: Time.current, deleted_user:)
+      Tag.where(id: tag_id).update_all('post_count = GREATEST(post_count - 1, 0)')
+    end
+
+    self
+  end
 end

backend/app/models/post_tag_section.rb

@@ -4,10 +4,10 @@ class PostTagSection < ApplicationRecord
   belongs_to :post
   belongs_to :tag
 
-  belongs_to :post_tag, foreign_key: [:post_id, :tag_id],
+  belongs_to :post_tag, -> { kept }, foreign_key: [:post_id, :tag_id],
-                        primary_key: [:post_id, :tag_id],
+                                     primary_key: [:post_id, :tag_id],
-                        inverse_of: :sections,
+                                     inverse_of: :sections,
-                        optional: true
+                                     optional: true
 
   validates :post_id, presence: true
   validates :tag_id, presence: true

backend/app/models/post_version.rb

@@ -1,12 +1,6 @@
 class PostVersion < ApplicationRecord
   include VersionRecord
 
-  scope :with_previous_version, -> {
-    joins('LEFT JOIN post_versions prev ON prev.post_id = post_versions.post_id ' +
-          'AND prev.version_no = post_versions.version_no - 1 AND ' +
-          HistoryAccess.sql(PostVersion, 'prev'))
-  }
-
   belongs_to :post
   belongs_to :parent, class_name: 'Post', optional: true
 

backend/app/models/region.rb

@@ -1,3 +0,0 @@
-class Region < ApplicationRecord
-  ;
-end

backend/app/models/script.rb

@@ -1,11 +0,0 @@
-class Script < ApplicationRecord
-  has_many :locales, foreign_key: :script_code, primary_key: :code
-
-  # Canonical keys come first so reads and plucks retain ISO 15924 spelling.
-  # Legacy keys preserve the existing enum predicates, scopes and queries.
-  enum :code, { 'Jpan' => 'Jpan', 'Kana' => 'Kana', jpan: 'Jpan', kana: 'Kana' }
-
-  def code= value
-    super(self.class.codes.fetch(value.to_s, value))
-  end
-end

backend/app/models/tag.rb

@@ -2,7 +2,7 @@ require 'set'
 
 
 class Tag < ApplicationRecord
-  before_create { SharedResourceIdAllocator.assign!(self) }
+  include MyDiscard
 
   class NicoTagNormalisationError < ArgumentError
     ;
@@ -28,12 +28,15 @@ class Tag < ApplicationRecord
   end
 
   has_many :post_tags, inverse_of: :tag
-  has_many :posts, through: :post_tags
+  has_many :active_post_tags, -> { kept }, class_name: 'PostTag', inverse_of: :tag
+  has_many :post_tags_with_discarded, -> { with_discarded }, class_name: 'PostTag'
+  has_many :posts, through: :active_post_tags
+
+  has_many :nico_tag_relations, foreign_key: :nico_tag_id, dependent: :destroy
+  has_many :linked_tags, through: :nico_tag_relations, source: :tag
 
   has_many :reversed_nico_tag_relations,
-           class_name: 'NicoTagRelation',
+           class_name: 'NicoTagRelation', foreign_key: :tag_id, dependent: :destroy
-           foreign_key: :tag_id,
-           dependent: :destroy
   has_many :linked_nico_tags, through: :reversed_nico_tag_relations, source: :nico_tag
 
   has_many :tag_implications, foreign_key: :parent_tag_id, dependent: :destroy
@@ -51,23 +54,32 @@ class Tag < ApplicationRecord
   has_many :materials
 
   has_many :tag_versions
+  has_many :nico_tag_versions
 
-  has_many :tag_names
+  belongs_to :tag_name
-  has_many :tag_descriptions
+  delegate :wiki_page, to: :tag_name
 
   attribute :version_no, :integer, default: 1
 
+  delegate :name, to: :tag_name, allow_nil: true
+  validates :tag_name, presence: true
+
   enum :category, deerjikist:  'deerjikist',
                   meme:        'meme',
                   character:   'character',
                   general:     'general',
                   material:    'material',
+                  nico:        'nico',
                   meta:        'meta'
 
   validates :category, presence: true, inclusion: { in: Tag.categories.keys }
 
+  validate :nico_tag_name_must_start_with_nico
+  validate :tag_name_must_be_canonical
   validate :category_must_be_deerjikist_with_deerjikists
-  validate :tag_names_must_be_valid
+  validate :nico_tags_cannot_be_deprecated
+
+  scope :nico_tags, -> { nico }
 
   CATEGORY_PREFIXES = {
       'general:'    => :general,
@@ -81,296 +93,233 @@ class Tag < ApplicationRecord
       'mtr:'        => :material,
       'meta:'       => :meta }.freeze
 
-  SYSTEM_TAG_NAMES = ['タグ希望', 'bot操作', 'ニジラー情報不詳', '動画', 'ニコニコ', 'YouTube'].freeze
+  def name= val
-
+    (self.tag_name ||= build_tag_name).name = val
-  def tag_name language_code
-    if tag_names.loaded?
-      tag_names.find { _1.language_code == language_code && _1.primary_flg }
-    else
-      tag_names.find_by(language_code:, primary_flg: true)
-    end
-  end
-  def name(language_code) = tag_name(language_code)&.name
-
-  def system?
-    tag_name('ja')&.name.in?(SYSTEM_TAG_NAMES)
   end
 
   def deprecated? = deprecated_at?
 
-  def has_description? language_code:
+  def has_wiki = wiki_page.present?
-    if tag_descriptions.loaded?
-      tag_descriptions.any? { _1.language_code == language_code }
-    else
-      tag_descriptions.exists?(language_code:)
-    end
-  end
 
   def material_id = materials.first&.id
 
   def has_deerjikists = deerjikists.loaded? ? deerjikists.any? : deerjikists.exists?
 
-  def snapshot_aliases language_code
+  def self.tagme = find_or_create_by_tag_name!('タグ希望', category: :meta)
-    if tag_names.loaded?
+  def self.bot = find_or_create_by_tag_name!('bot操作', category: :meta)
-      return tag_names.select { _1.language_code == language_code && !_1.primary? }
+  def self.no_deerjikist = find_or_create_by_tag_name!('ニジラー情報不詳', category: :meta)
-          .sort_by(&:name).map(&:name)
+  def self.video = find_or_create_by_tag_name!('動画', category: :meta)
+  def self.niconico = find_or_create_by_tag_name!('ニコニコ', category: :meta)
+  def self.youtube = find_or_create_by_tag_name!('YouTube', category: :meta)
+
+  def self.normalise_tags! tag_names, with_tagme: true,
+                                      with_no_deerjikist: true,
+                                      deny_nico: true,
+                                      deny_deprecated: false,
+                                      with_sections: false
+    if deny_nico && tag_names.any? { |n| n.downcase.start_with?('nico:') }
+      raise NicoTagNormalisationError
     end
 
-    tag_names.where(language_code:, primary_flg: false).order(:name).pluck(:name)
+    sections = { }
+    tags = tag_names.map do |name|
+      raw_name = name
+      pf, cat = CATEGORY_PREFIXES.find { |p, _| name.downcase.start_with?(p) } || ['', nil]
+
+      name = name.sub(/\A#{ pf }/i, '')
+
+      sections_by_tag = []
+      while (match = name.match(/\A(\S*?)\[([^\[\]\s]*)-([^\[\]\s]*)\](\S*)\z/))
+        name = "#{ match[1] }#{ match[4] }"
+        next if match[2].empty? && match[3].empty?
+
+        sections_by_tag << normalise_section_range!(
+            begin_raw: match[2],
+            end_raw: match[3],
+            tag_name: name)
+      end
+
+      if name.include?('[') || name.include?(']')
+        raise SectionLiteralParseError.new(raw_name, raw_name)
+      end
+
+      name = TagName.canonicalise(name).first
+
+      find_or_create_by_tag_name!(name, category: (cat || :general)).tap do |tag|
+        if deny_deprecated && tag.deprecated?
+          raise DeprecatedTagNormalisationError, [tag.name]
+        end
+
+        tag.update!(category: cat) if cat && tag.category != cat
+        next if sections_by_tag.empty?
+
+        sections[tag.id] ||= []
+        sections[tag.id].concat(sections_by_tag)
+        sections[tag.id] = merge_section_ranges(sections[tag.id])
+        sections.delete(tag.id) if sections[tag.id] == [[0, nil]]
+      end
+    end
+
+    tags << Tag.tagme if with_tagme && tags.size < 10 && tags.none?(Tag.tagme)
+    tags << Tag.no_deerjikist if with_no_deerjikist && tags.all? { |t| !(t.deerjikist?) }
+    tags.uniq!(&:id)
+
+    if with_sections
+      { tags:, sections: }
+    else
+      tags
+    end
   end
 
+  def self.expand_parent_tags tags
+    return [] if tags.blank?
+
+    seen = Set.new
+    result = []
+    stack = tags.compact.dup
+
+    until stack.empty?
+      tag = stack.pop
+      next unless tag
+
+      tag.parents.each do |parent|
+        next if seen.include?(parent.id)
+
+        seen << parent.id
+        result << parent
+        stack << parent
+      end
+    end
+
+    (result + tags).uniq { |t| t.id }
+  end
+
+  def self.normalise_section_range! begin_raw:, end_raw:, tag_name:
+    begin_ms = begin_raw.empty? ? 0 : time_to_ms!(begin_raw, tag_name:)
+    end_ms = end_raw.empty? ? nil : time_to_ms!(end_raw, tag_name:)
+
+    if end_ms
+      begin_ms, end_ms = end_ms, begin_ms if begin_ms > end_ms
+      end_ms = begin_ms + 1 if begin_ms == end_ms
+    end
+
+    [begin_ms, end_ms]
+  end
+
+  def self.merge_section_ranges ranges
+    sorted_ranges = ranges.sort_by { |begin_ms, end_ms| [begin_ms, end_ms || Float::INFINITY] }
+    merged = []
+
+    sorted_ranges.each do |begin_ms, end_ms|
+      if merged.empty?
+        merged << [begin_ms, end_ms]
+        next
+      end
+
+      last_begin_ms, last_end_ms = merged[-1]
+      if last_end_ms.nil? || begin_ms <= last_end_ms
+        merged[-1] = [last_begin_ms, merge_section_end(last_end_ms, end_ms)]
+      else
+        merged << [begin_ms, end_ms]
+      end
+    end
+
+    merged
+  end
+
+  def self.merge_section_end left_end_ms, right_end_ms
+    return nil if left_end_ms.nil? || right_end_ms.nil?
+
+    [left_end_ms, right_end_ms].max
+  end
+
+  def self.find_or_create_by_tag_name! name, category:
+    tn = TagName.find_undiscard_or_create_by!(name: name.to_s.strip)
+    tn = tn.canonical if tn.canonical_id?
+
+    Tag.find_undiscard_or_create_by!(tag_name_id: tn.id) do |t|
+      t.category = category
+    end
+  rescue ActiveRecord::RecordNotUnique
+    retry
+  end
+
+  def self.merge_tags! target_tag, source_tags, created_by_user: nil
+    target_tag => Tag
+
+    affected_post_ids = Set.new
+
+    Tag.transaction do
+      TagVersioning.ensure_snapshot!(target_tag, created_by_user:)
+
+      Array(source_tags).compact.uniq.each do |source_tag|
+        source_tag => Tag
+
+        next if source_tag == target_tag
+
+        TagVersioning.ensure_snapshot!(source_tag, created_by_user:)
+
+        source_tag.post_tags.kept.find_each do |source_pt|
+          post_id = source_pt.post_id
+          affected_post_ids << post_id
+          source_pt.discard_by!(created_by_user)
+          unless PostTag.kept.exists?(post_id:, tag: target_tag)
+            PostTag.create!(post_id:, tag: target_tag)
+          end
+        end
+
+        source_tag_name = source_tag.tag_name
+
+        if source_tag_name.wiki_page.present?
+          raise ActiveRecord::RecordInvalid.new(source_tag_name)
+        end
+
+        TagVersioning.record!(source_tag, event_type: :discard, created_by_user:)
+        source_tag.discard!
+
+        if source_tag.nico?
+          source_tag_name.discard!
+        else
+          source_tag_name.update_columns(canonical_id: target_tag.tag_name_id,
+                                         updated_at: Time.current)
+        end
+
+        TagVersioning.record!(target_tag, event_type: :update, created_by_user:)
+      end
+
+      Post.where(id: affected_post_ids.to_a).find_each do |post|
+        PostVersionRecorder.ensure_snapshot!(post, created_by_user:)
+        PostVersionRecorder.record!(post:, event_type: :update, created_by_user:)
+      end
+
+      # 投稿件数を再集計
+      target_tag.update_columns(post_count: PostTag.kept.where(tag: target_tag).count)
+    end
+
+    target_tag.reload
+  end
+
+  def snapshot_aliases = tag_name.aliases.kept.order(:name).pluck(:name)
+
   def snapshot_parent_tag_ids = parents.order(:id).pluck(:id)
 
-  def snapshot_linked_tag_names language_code
+  def snapshot_linked_tag_names
-    linked_tags
+    linked_tags.joins(:tag_name).order('tag_names.name').pluck('tag_names.name')
-        .joins(:tag_names)
-        .where(tag_names: { language_code:, primary_flg: true })
-        .order('tag_names.name')
-        .pluck('tag_names.name')
-  end
-
-  class << self
-    def tagme = find_or_create_by_tag_name!(Locale.nipponese, 'タグ希望', category: :meta)
-    def bot = find_or_create_by_tag_name!(Locale.nipponese, 'bot操作', category: :meta)
-    def no_deerjikist =
-        find_or_create_by_tag_name!(Locale.nipponese, 'ニジラー情報不詳', category: :meta)
-    def video = find_or_create_by_tag_name!(Locale.nipponese, '動画', category: :meta)
-    def niconico = find_or_create_by_tag_name!(Locale.nipponese, 'ニコニコ', category: :meta)
-    def youtube = find_or_create_by_tag_name!(Locale.nipponese, 'YouTube', category: :meta)
-
-    def normalise_tags! locale, tag_names,
-                             with_tagme: true,
-                             with_no_deerjikist: true,
-                             deny_deprecated: false,
-                             with_sections: false,
-                             created_by_user: nil
-      if tag_names.any? { |n| n.downcase.start_with?('nico:') }
-        raise NicoTagNormalisationError
-      end
-
-      sections = { }
-      tags = tag_names.map do |name|
-        raw_name = name
-        pf, cat = CATEGORY_PREFIXES.find { |p, _| name.downcase.start_with?(p) } || ['', nil]
-
-        name = name.sub(/\A#{ pf }/i, '')
-
-        sections_by_tag = []
-        while (match = name.match(/\A(\S*?)\[([^\[\]\s]*)-([^\[\]\s]*)\](\S*)\z/))
-          name = "#{ match[1] }#{ match[4] }"
-          next if match[2].empty? && match[3].empty?
-
-          sections_by_tag << normalise_section_range!(
-              begin_raw: match[2],
-              end_raw: match[3],
-              tag_name: name)
-        end
-
-        if name.include?('[') || name.include?(']')
-          raise SectionLiteralParseError.new(raw_name, raw_name)
-        end
-
-        name = TagName.canonicalise(locale, name).first
-
-        find_or_create_by_tag_name!(locale, name, category: (cat || :general),
-                                   created_by_user:).tap do |tag|
-          if deny_deprecated && tag.deprecated?
-            raise DeprecatedTagNormalisationError, [tag.name(locale.language_code)]
-          end
-
-          tag.update!(category: cat) if cat && tag.category != cat
-          next if sections_by_tag.empty?
-
-          sections[tag.id] ||= []
-          sections[tag.id].concat(sections_by_tag)
-          sections[tag.id] = merge_section_ranges(sections[tag.id])
-          sections.delete(tag.id) if sections[tag.id] == [[0, nil]]
-        end
-      end
-
-      tags << Tag.tagme if with_tagme && tags.size < 10 && tags.none?(Tag.tagme)
-      tags << Tag.no_deerjikist if with_no_deerjikist && tags.all? { |t| !(t.deerjikist?) }
-      tags.uniq!(&:id)
-
-      if with_sections
-        { tags:, sections: }
-      else
-        tags
-      end
-    end
-
-    def expand_parent_tags tags
-      return [] if tags.blank?
-
-      seen = Set.new
-      result = []
-      stack = tags.compact.dup
-
-      until stack.empty?
-        tag = stack.pop
-        next unless tag
-
-        tag.parents.each do |parent|
-          next if seen.include?(parent.id)
-
-          seen << parent.id
-          result << parent
-          stack << parent
-        end
-      end
-
-      (result + tags).uniq { |t| t.id }
-    end
-
-    def normalise_section_range! begin_raw:, end_raw:, tag_name:
-      begin_ms = begin_raw.empty? ? 0 : time_to_ms!(begin_raw, tag_name:)
-      end_ms = end_raw.empty? ? nil : time_to_ms!(end_raw, tag_name:)
-
-      if end_ms
-        begin_ms, end_ms = end_ms, begin_ms if begin_ms > end_ms
-        end_ms = begin_ms + 1 if begin_ms == end_ms
-      end
-
-      [begin_ms, end_ms]
-    end
-
-    def merge_section_ranges ranges
-      sorted_ranges = ranges.sort_by { |begin_ms, end_ms| [begin_ms, end_ms || Float::INFINITY] }
-      merged = []
-
-      sorted_ranges.each do |begin_ms, end_ms|
-        if merged.empty?
-          merged << [begin_ms, end_ms]
-          next
-        end
-
-        last_begin_ms, last_end_ms = merged[-1]
-        if last_end_ms.nil? || begin_ms <= last_end_ms
-          merged[-1] = [last_begin_ms, merge_section_end(last_end_ms, end_ms)]
-        else
-          merged << [begin_ms, end_ms]
-        end
-      end
-
-      merged
-    end
-
-    def merge_section_end left_end_ms, right_end_ms
-      return nil if left_end_ms.nil? || right_end_ms.nil?
-
-      [left_end_ms, right_end_ms].max
-    end
-
-    def find_or_create_by_tag_name! locale, name, category:, created_by_user: nil
-      language_code = locale.language_code
-      name = name.to_s.strip
-
-      ApplicationRecord.transaction do
-        SharedResourceIdAllocator.lock_space!(Tag, 'tags')
-        tn = TagName.lock.find_by(language_code:, name:)
-        next tn.tag if tn
-
-        tag = Tag.new(category:)
-        tag.tag_names.build(language_code:, name:, script_code: locale.script_code,
-                            primary_flg: true, auto_generated: false)
-        TagNameCompletion.for_tag!(tag, created_by_user:)
-        tag.association(:tag_names).reset
-
-        tag
-      end
-    rescue ActiveRecord::RecordNotUnique
-      existing = TagName.find_by(language_code:, name:)
-      raise unless existing
-
-      existing.tag
-    end
-
-    def merge_tags! target_tag, source_tags, created_by_user: nil
-      target_tag => Tag
-
-      affected_post_ids = Set.new
-
-      Tag.transaction do
-        TagVersioning.ensure_snapshot!(target_tag, created_by_user:)
-
-        Array(source_tags).compact.uniq.each do |source_tag|
-          source_tag => Tag
-
-          next if source_tag == target_tag
-
-          TagVersioning.ensure_snapshot!(source_tag, created_by_user:)
-
-          source_tag.post_tags.find_each do |source_pt|
-            post_id = source_pt.post_id
-            affected_post_ids << post_id
-            source_pt.destroy!
-            unless PostTag.exists?(post_id:, tag: target_tag)
-              PostTag.create!(post_id:, tag: target_tag)
-            end
-          end
-
-          if source_tag.tag_descriptions.exists?
-            source_tag.errors.add(:base, :invalid)
-            raise ActiveRecord::RecordInvalid, source_tag
-          end
-
-          TagVersioning.record!(source_tag, event_type: :discard, created_by_user:)
-          target_languages = target_tag.tag_names.where(primary_flg: true).pluck(:language_code)
-          source_tag.tag_names.where(language_code: target_languages)
-              .update_all(primary_flg: false, auto_generated: false, updated_at: Time.current)
-          source_tag.tag_names.update_all(tag_id: target_tag.id, updated_at: Time.current)
-          source_tag.destroy!
-
-          TagVersioning.record!(target_tag, event_type: :update, created_by_user:)
-        end
-
-        Post.where(id: affected_post_ids.to_a).find_each do |post|
-          PostVersionRecorder.ensure_snapshot!(post, created_by_user:)
-          PostVersionRecorder.record!(post:, event_type: :update, created_by_user:)
-        end
-
-        # 投稿件数を再集計
-        target_tag.update_columns(post_count: PostTag.where(tag: target_tag).count)
-      end
-
-      target_tag.reload
-    end
-
-    def time_to_ms! str, tag_name:
-      match =
-          case str
-          when /\A(?<seconds>\d+)(?:\.(?<ms>\d{1,3}))?\z/
-            { hours: nil, minutes: nil, seconds: Regexp.last_match[:seconds],
-              ms: Regexp.last_match[:ms] }
-          when /\A(?<minutes>\d+):(?<seconds>[0-5]?\d)(?:\.(?<ms>\d{1,3}))?\z/
-            { hours: nil, minutes: Regexp.last_match[:minutes],
-              seconds: Regexp.last_match[:seconds],
-              ms: Regexp.last_match[:ms] }
-          when /\A(?<hours>\d+):(?<minutes>[0-5]?\d):(?<seconds>[0-5]?\d)(?:\.(?<ms>\d{1,3}))?\z/
-            { hours: Regexp.last_match[:hours],
-              minutes: Regexp.last_match[:minutes],
-              seconds: Regexp.last_match[:seconds],
-              ms: Regexp.last_match[:ms] }
-          end
-
-      raise SectionLiteralParseError.new(tag_name, str) unless match
-
-      total_s = match[:seconds].to_i
-      total_s += match[:minutes].to_i * 60 if match[:minutes]
-      total_s += match[:hours].to_i * 3_600 if match[:hours]
-
-      total_s * 1_000 + match[:ms].to_s.ljust(3, '0')[0, 3].to_i
-    end
-
   end
 
   private
 
-  def tag_names_must_be_valid
+  def nico_tag_name_must_start_with_nico
-    tag_names.each do |tag_name|
+    n = name.to_s
-      next if tag_name.valid?
+    if ((nico? && !(n.downcase.start_with?('nico:'))) || 
+        (!(nico?) && n.downcase.start_with?('nico:')))
+      errors.add :name, 'ニコニコ・タグの命名規則に反してゐます．'
+    end
+  end
 
-      tag_name.errors[:name].each { errors.add :name, _1 }
+  def tag_name_must_be_canonical
+    if tag_name&.canonical_id?
+      errors.add :tag_name, 'tag_names へは実体を示す必要があります．'
     end
   end
 
@@ -379,4 +328,36 @@ class Tag < ApplicationRecord
       errors.add :category, 'ニジラーと紐づいてゐるタグはニジラー・カテゴリである必要があります．'
     end
   end
+
+  def self.time_to_ms! str, tag_name:
+    match =
+        case str
+        when /\A(?<seconds>\d+)(?:\.(?<ms>\d{1,3}))?\z/
+          { hours: nil, minutes: nil, seconds: Regexp.last_match[:seconds],
+            ms: Regexp.last_match[:ms] }
+        when /\A(?<minutes>\d+):(?<seconds>[0-5]?\d)(?:\.(?<ms>\d{1,3}))?\z/
+          { hours: nil, minutes: Regexp.last_match[:minutes],
+            seconds: Regexp.last_match[:seconds],
+            ms: Regexp.last_match[:ms] }
+        when /\A(?<hours>\d+):(?<minutes>[0-5]?\d):(?<seconds>[0-5]?\d)(?:\.(?<ms>\d{1,3}))?\z/
+          { hours: Regexp.last_match[:hours],
+            minutes: Regexp.last_match[:minutes],
+            seconds: Regexp.last_match[:seconds],
+            ms: Regexp.last_match[:ms] }
+        end
+
+    raise SectionLiteralParseError.new(tag_name, str) unless match
+
+    total_s = match[:seconds].to_i
+    total_s += match[:minutes].to_i * 60 if match[:minutes]
+    total_s += match[:hours].to_i * 3_600 if match[:hours]
+
+    total_s * 1_000 + match[:ms].to_s.ljust(3, '0')[0, 3].to_i
+  end
+
+  def nico_tags_cannot_be_deprecated
+    if nico? && deprecated_at.present?
+      errors.add :deprecated_at, 'ニコタグは廃止できません．'
+    end
+  end
 end

backend/app/models/tag_description.rb

@@ -1,5 +0,0 @@
-class TagDescription < ApplicationRecord
-  before_create { SharedResourceIdAllocator.assign!(self) }
-
-  belongs_to :tag
-end

backend/app/models/tag_description_version.rb

@@ -1,5 +0,0 @@
-class TagDescriptionVersion < ApplicationRecord
-  include VersionRecord
-
-  before_create { SharedResourceIdAllocator.assign!(self) }
-end

backend/app/models/tag_group.rb

@@ -1,27 +0,0 @@
-class TagGroup < ApplicationRecord
-  before_create { SharedResourceIdAllocator.assign!(self) }
-  before_destroy :delete_current_structure!
-
-  belongs_to :created_by_user, class_name: 'User', optional: true
-  belongs_to :updated_by_user, class_name: 'User', optional: true
-
-  has_many :tag_group_tags, dependent: :delete_all
-  has_many :tags, through: :tag_group_tags
-  has_many :tag_group_groups, dependent: :delete_all
-  has_many :child_tag_groups, through: :tag_group_groups
-  has_many :placements, -> { order(:position) },
-           class_name: 'TagGroupPlacement', dependent: :delete_all
-  has_many :versions, -> { order(:version_no) },
-           class_name: 'TagGroupVersion'
-
-  validates :name, presence: true, uniqueness: true, length: { maximum: 255 }
-  validates :version_no, numericality: { only_integer: true, greater_than: 0 }
-
-  private
-
-  def delete_current_structure!
-    placements.reorder(position: :desc).each(&:delete)
-    tag_group_tags.delete_all
-    tag_group_groups.delete_all
-  end
-end

backend/app/models/tag_group_group.rb

@@ -1,7 +0,0 @@
-class TagGroupGroup < ApplicationRecord
-  self.primary_key = [:tag_group_id, :child_tag_group_id]
-
-  belongs_to :tag_group
-  belongs_to :child_tag_group, class_name: 'TagGroup'
-  belongs_to :created_by_user, class_name: 'User', optional: true
-end

backend/app/models/tag_group_placement.rb

@@ -1,9 +0,0 @@
-class TagGroupPlacement < ApplicationRecord
-  self.primary_key = [:tag_group_id, :position]
-
-  belongs_to :tag_group
-  belongs_to :tag, optional: true
-  belongs_to :child_tag_group, class_name: 'TagGroup', optional: true
-  belongs_to :created_by_user, class_name: 'User', optional: true
-  belongs_to :updated_by_user, class_name: 'User', optional: true
-end

backend/app/models/tag_group_tag.rb

@@ -1,7 +0,0 @@
-class TagGroupTag < ApplicationRecord
-  self.primary_key = [:tag_group_id, :tag_id]
-
-  belongs_to :tag_group
-  belongs_to :tag
-  belongs_to :created_by_user, class_name: 'User', optional: true
-end

backend/app/models/tag_group_version.rb

@@ -1,9 +0,0 @@
-class TagGroupVersion < ApplicationRecord
-  include VersionRecord
-
-  before_create { SharedResourceIdAllocator.assign!(self) }
-
-  belongs_to :tag_group
-
-  scope :ordered, -> { order(:version_no) }
-end

backend/app/models/tag_name.rb

@@ -1,82 +1,51 @@
 class TagName < ApplicationRecord
-  belongs_to :language, foreign_key: :language_code, primary_key: :code
+  include MyDiscard
-  belongs_to :tag
 
-  validates :name, presence: true,
+  has_one :tag
-                   length: { maximum: 255 },
+  has_one :wiki_page
-                   uniqueness: { scope: :language_code }
 
+  belongs_to :canonical, class_name: 'TagName', optional: true
+  has_many :aliases, class_name: 'TagName', foreign_key: :canonical_id
+
+  validates :name, presence: true, length: { maximum: 255 }, uniqueness: true
+
+  validate :canonical_must_be_canonical
   validate :alias_name_must_not_have_prefix
+  validate :canonical_must_not_be_present_with_tag_or_wiki_page
   validate :name_must_be_sanitised
-  validate :name_mustnt_start_with_nico
-  validate :generated_name_must_be_reserved
 
-  def primary? = primary_flg
+  def self.canonicalise names
-
-  def self.canonicalise locale, names
     names = Array(names).map { |n| n.to_s.strip }.reject(&:blank?)
     return [] if names.blank?
 
-    tns = TagName.where(language_code: locale.language_code, name: names).index_by(&:name)
+    tns = TagName.includes(:canonical).where(name: names).index_by(&:name)
 
-    primary_names = TagName.where(
+    names.map { |name| tns[name]&.canonical&.name || name }.uniq
-        language_code: locale.language_code, tag_id: tns.values.map(&:tag_id),
-        primary_flg: true).pluck(:tag_id, :name).to_h
-
-    names.map { |name|
-      tag_name = tns[name]
-      if !tag_name || tag_name.primary?
-        name
-      else
-        primary_names[tag_name.tag_id] || name
-      end
-    }.uniq
-  end
-
-  def self.generate_name locale, tag, base_tag_name
-    fallback = "Tag_##{ tag.id }"
-    name =
-        case [locale.language_code, base_tag_name&.language_code]
-        when ['ain', 'ja']
-          TagNameTranscription.ainu(base_tag_name.name)
-        when ['ja', 'ain']
-          base_tag_name.name
-        else
-          return fallback
-        end
-    candidate = new(tag:, name:, language_code: locale.language_code,
-                    script_code: locale.script_code,
-                    primary_flg: true, auto_generated: true)
-    candidate.valid? ? name : fallback
-  rescue TagNameTranscription::Unconvertible
-    fallback
   end
 
   private
 
-  def generated_name_must_be_reserved
+  def canonical_must_be_canonical
-    if !auto_generated && name.to_s.start_with?('Tag_#')
+    if canonical&.canonical_id?
-      errors.add :name, 'タグの命名規則に反してゐます．'
+      errors.add :canonical, 'canonical は実体を示す必要があります．'
     end
   end
 
   def alias_name_must_not_have_prefix
-    if !(primary?) && name.to_s.include?(':')
+    if canonical_id? && name.to_s.include?(':')
       errors.add :name, 'エーリアス名にプレフィクスを含むことはできません．'
     end
   end
 
-  def name_must_be_sanitised
+  def canonical_must_not_be_present_with_tag_or_wiki_page
-    return if auto_generated && name == "Tag_##{ tag_id }"
+    if canonical_id? && (tag || wiki_page)
+      errors.add :canonical, 'タグもしくは Wiki の参照がある名前はエーリアスになれません．'
+    end
+  end
 
+  def name_must_be_sanitised
     if name? && name != TagNameSanitisationRule.sanitise(name)
       errors.add :name, '名前に使用できない文字が含まれてゐます．'
     end
   end
-
-  def name_mustnt_start_with_nico
-    if name.to_s.downcase.start_with?('nico:')
-      errors.add :name, 'タグの命名規則に反してゐます．'
-    end
-  end
 end

backend/app/models/tag_name_sanitisation_rule.rb

@@ -10,43 +10,35 @@ class TagNameSanitisationRule < ApplicationRecord
   validate :source_pattern_must_be_regexp
 
   class << self
-    def sanitise(name)
+    def sanitise(name) =
-      rules.reduce(name.dup) { |name, (pattern, replacement)| name.gsub(pattern, replacement) }
+        rules.reduce(name.dup) { |name, (pattern, replacement)| name.gsub(pattern, replacement) }
-    end
 
     def apply!
-      Language.find_each do |language|
+      TagName.find_each do |tn|
-        TagName.where(language:).find_each do |candidate|
+        name = sanitise(tn.name)
-          TagName.transaction do
+        next if name == tn.name
-            tn = TagName.find_by(id: candidate.id)
-            next unless tn
 
-            name = sanitise(tn.name)
+        TagName.transaction do
-            next if name == tn.name
+          existing_tn = TagName.find_by(name:)
+          if existing_tn
+            existing_tn = existing_tn.canonical || existing_tn
+            next if existing_tn.id == tn.id
 
-            existing_tn = TagName.find_by(language:, name:)
+            existing_tag = Tag.find_by(tag_name_id: existing_tn.id)
-            source_tag = tn.tag
+            source_tag = Tag.find_by(tag_name_id: tn.id)
-            target_tag = existing_tn ? existing_tn.tag : source_tag
-            [source_tag, target_tag].uniq(&:id).sort_by(&:id).each(&:lock!)
-            TagVersioning.ensure_snapshot!(source_tag, created_by_user: nil)
 
-            if existing_tn && target_tag != source_tag
+            if existing_tag
-              Tag.merge_tags!(target_tag, source_tag)
+              Tag.merge_tags!(existing_tag, source_tag) if tn.tag
-              tn.reload.destroy!
+            elsif source_tag
-            elsif existing_tn
+              source_tag.update_columns(tag_name_id: existing_tn.id, updated_at: Time.current)
-              if tn.primary?
-                existing_tn.destroy!
-                tn.update_columns(name:, updated_at: Time.current)
-              else
-                tn.destroy!
-              end
-            else
-              # TagName 側の自動サニタイズを回避
-              tn.update_columns(name:, updated_at: Time.current)
             end
+            tn.discard!
 
-            TagVersioning.record!(target_tag, event_type: :update, created_by_user: nil)
+            next
           end
+
+          # TagName 側の自動サニタイズを回避
+          tn.update_columns(name:, updated_at: Time.current)
         end
       end
     end

backend/app/models/tag_version.rb

@@ -11,6 +11,5 @@ class TagVersion < ApplicationRecord
                     meta: 'meta' }, validate: true
 
   validates :name, presence: true
-  validates :names, presence: true
   validates :category, presence: true
 end

backend/app/models/user.rb

@@ -13,10 +13,10 @@ class User < ApplicationRecord
   has_many :ip_addresses, through: :user_ips
   has_many :user_post_views, dependent: :destroy
   has_many :viewed_posts, through: :user_post_views, source: :post
-  has_many :created_tag_descriptions, class_name: 'TagDescription',
+  has_many :created_wiki_pages,
-           foreign_key: :created_by_user_id, dependent: :nullify
+           class_name: 'WikiPage', foreign_key: :created_user_id, dependent: :nullify
-  has_many :updated_tag_descriptions, class_name: 'TagDescription',
+  has_many :updated_wiki_pages,
-           foreign_key: :updated_by_user_id, dependent: :nullify
+           class_name: 'WikiPage', foreign_key: :updated_user_id, dependent: :nullify
 
   def viewed?(post) = user_post_views.exists?(post_id: post.id)
 

backend/app/models/version_record.rb

@@ -6,8 +6,6 @@ module VersionRecord
                              discard: 'discard',
                              restore: 'restore' }.freeze
 
-  def require_history_access! = HistoryAccess.require!(self)
-
   def readonly? = persisted?
 
   included do

backend/app/models/wiki_line.rb

@@ -0,0 +1,15 @@
+class WikiLine < ApplicationRecord
+  has_many :wiki_revision_lines, dependent: :restrict_with_exception
+
+  validates :sha256, presence: true, uniqueness: true, length: { is: 64 }
+  validates :body, presence: true
+
+  def self.upsert_by_body! body
+    sha = Digest::SHA256.hexdigest(body)
+    now = Time.current
+
+    upsert(sha256: sha, body:, created_at: now, updated_at: now)
+
+    find_by!(sha256: sha)
+  end
+end

backend/app/models/wiki_page.rb

@@ -0,0 +1,55 @@
+require 'set'
+
+
+class WikiPage < ApplicationRecord
+  include MyDiscard
+
+  has_many :wiki_revisions, dependent: :destroy
+  belongs_to :created_user, class_name: 'User'
+  belongs_to :updated_user, class_name: 'User'
+
+  has_many :redirected_from_revisions,
+           class_name: 'WikiRevision',
+           foreign_key: :redirect_page_id,
+           dependent: :nullify
+
+  has_many :wiki_versions
+
+  attribute :version_no, :integer, default: 1
+
+  belongs_to :tag_name
+  validates :tag_name, presence: true
+  validates :body, presence: true
+
+  def title = tag_name.name
+  def deprecated_at = tag_name.tag&.deprecated_at
+
+  def title= val
+    (self.tag_name ||= build_tag_name).name = val
+  end
+
+  def current_revision = wiki_revisions.order(id: :desc).first
+
+  def resolve_redirect limit: 10
+    page = self
+    visited = Set.new
+
+    limit.times do
+      return page if visited.include?(page.id)
+
+      visited.add(page.id)
+
+      rev = page.current_revision
+      return page if !(rev&.redirect?) || !(rev.redirect_page)
+
+      page = rev.redirect_page
+    end
+
+    page
+  end
+
+  def pred_revision_id(revision_id) =
+      wiki_revisions.where('id < ?', revision_id).order(id: :desc).limit(1).pick(:id)
+  def succ_revision_id(revision_id) =
+      wiki_revisions.where('id > ?', revision_id).order(id: :asc).limit(1).pick(:id)
+end

backend/app/models/wiki_revision.rb

@@ -0,0 +1,55 @@
+class WikiRevision < ApplicationRecord
+  belongs_to :wiki_page
+  belongs_to :base_revision, class_name: 'WikiRevision', optional: true
+  belongs_to :created_user, class_name: 'User'
+  belongs_to :redirect_page, class_name: 'WikiPage', optional: true
+
+  has_many :wiki_revision_lines, dependent: :delete_all
+  has_many :wiki_lines, through: :wiki_revision_lines
+
+  enum :kind, content: 0, redirect: 1
+
+  validates :kind, presence: true
+  validates :lines_count, numericality: { only_integer: true, greater_than_or_equal_to: 0 }
+  validates :tree_sha256, length: { is: 64 }, allow_nil: true
+
+  validate :kind_consistency
+
+  def body
+    return unless content?
+
+    wiki_revision_lines
+        .includes(:wiki_line)
+        .order(:position)
+        .map { |rev| rev.wiki_line.body }
+        .join("\n")
+  end
+
+  private
+
+  def kind_consistency
+    if content?
+      if tree_sha256.blank?
+        errors.add(:tree_sha256, '種類がページの場合は必須です．') 
+      end
+
+      if redirect_page_id.present?
+        errors.add(:redirect_page_id, '種類がページの場合は空である必要があります．')
+      end
+    end
+
+    if redirect?
+      if redirect_page_id.blank?
+        errors.add(:redirect_page_id, '種類がリダイレクトの場合は必須です．')
+      end
+
+      if tree_sha256.present?
+        errors.add(:tree_sha256, '種類がリダイレクトの場合は空である必要があります．')
+      end
+
+      if lines_count.to_i > 0
+        errors.add(:lines_count, '種類がリダイレクトの場合は 0 である必要があります．')
+      end
+    end
+  end
+end

backend/app/models/wiki_revision_line.rb

@@ -0,0 +1,8 @@
+class WikiRevisionLine < ApplicationRecord
+  belongs_to :wiki_revision
+  belongs_to :wiki_line
+
+  validates :position, presence: true,
+                       numericality: { only_integer: true, greater_than_or_equal_to: 0 }
+  validates :position, uniqueness: { scope: :wiki_revision_id }
+end

backend/app/models/wiki_version.rb

@@ -0,0 +1,8 @@
+class WikiVersion < ApplicationRecord
+  include VersionRecord
+
+  belongs_to :wiki_page
+
+  validates :title, presence: true
+  validates :body, presence: true
+end

backend/app/representations/counts_version_repr.rb

@@ -1,135 +0,0 @@
-# Counts uses recorded values only; history/diff endpoints retain their own contract.
-class CountsVersionRepr
-  def self.many(rows, resource:, language_code:, at: nil, inclusive: true)
-    new(rows.to_a, resource:, language_code:, at:, inclusive:).serialise
-  end
-
-  def initialize(rows, resource:, language_code:, at:, inclusive:)
-    @rows = rows
-    HistoryAccess.require_many!(rows)
-    @at = at
-    @inclusive = inclusive
-    @resource = resource
-    @language = language_code
-    @users = User.where(id: rows.map(&:created_by_user_id).compact).pluck(:id, :name).to_h
-    @languages = master_names(Language)
-    @scripts = master_names(Script)
-    @tag_versions = {}
-    load_tag_versions if resource == 'posts'
-    load_parent_versions if resource == 'tags'
-  end
-
-  def serialise
-    @rows.map do |row|
-      resource_id = row.public_send(@resource == 'posts' ? :post_id : :tag_id)
-      common = { id: row.id, resource_id:,
-                 version_no: row.version_no, is_latest: row.attributes['counts_latest'].to_i == 1,
-                 created_at: row.created_at.iso8601,
-                 created_by_user: row.created_by_user_id &&
-                     { id: row.created_by_user_id, name: @users[row.created_by_user_id] } }
-      common.merge(@resource == 'posts' ? post(row) : tag(row))
-    end
-  end
-
-  private
-
-  def master_names(model)
-    model.select('code AS counts_code', :name).to_h do |record|
-      [record.attributes['counts_code'], record.name]
-    end
-  end
-
-  def json(value)
-    value.is_a?(String) ? JSON.parse(value) : value
-  end
-
-  def load_tag_versions
-    pairs = @rows.flat_map { Array(json(_1.tags_json)) }
-        .filter_map { |entry| [entry['tag_id'], entry['version_no']] if entry['tag_id'] }
-        .uniq
-    pairs.each_slice(500) do |batch|
-      placeholders = batch.map { '(?, ?)' }.join(', ')
-      scope = HistoryAccess.visible(TagVersion.all)
-      if @at
-        scope = scope.where("created_at #{ @inclusive ? '<=' : '<' } ?", @at)
-      end
-      scope.where(["(tag_id, version_no) IN (#{ placeholders })", *batch.flatten])
-          .select(:tag_id, :version_no, :name, :category, :names).each do |version|
-        @tag_versions[[version.tag_id, version.version_no]] = version
-      end
-    end
-  end
-
-  def names(value)
-    (json(value) || {}).sort.map do |language, entry|
-      aliases = entry.fetch('aliases', []).map do |alias_entry|
-        alias_entry.merge('script_name' => @scripts[alias_entry['script_code']])
-      end
-      { language_code: language, language_name: @languages[language],
-        script_code: entry['script_code'], script_name: @scripts[entry['script_code']],
-        name: entry['name'], auto_generated: entry['auto_generated'], aliases: }
-    end
-  end
-
-  def post(row)
-    snapshots = Array(json(row.tags_json)).map do |entry|
-      version = @tag_versions[[entry['tag_id'], entry['version_no']]]
-      { id: entry['tag_id'] || entry['external_tag_id'],
-        kind: entry['tag_id'] ? 'internal' : 'external',
-        tag_id: entry['tag_id'], external_tag_id: entry['external_tag_id'],
-        version_no: entry['version_no'], name: entry['name'] || version&.name,
-        category: entry['category'] || version&.category,
-        sections: entry.fetch('sections', []), names: version ? names(version.names) : [] }
-    end
-    { title: row.title, url: row.url, thumbnail_base: row.thumbnail_base,
-      tags: snapshots, parents: row.parent_post_ids.to_s.split.map { { id: _1.to_i } },
-      video_ms: row.video_ms,
-      original_created_from: row.original_created_from&.iso8601,
-      original_created_before: row.original_created_before&.iso8601 }
-  end
-
-  def parent_point(row) = @at || row.created_at
-
-  def load_parent_versions
-    @parent_versions = {}
-    requests = @rows.flat_map do |row|
-      row.parent_tag_ids.to_s.split.map { [_1.to_i, parent_point(row)] }
-    end.uniq
-    requests.each_slice(100) do |batch|
-      queries = batch.map do |id, point|
-        scope = HistoryAccess.visible(TagVersion.where(tag_id: id))
-            .where('created_at <= ?', point)
-            .where('NOT EXISTS (SELECT 1 FROM tag_versions newer ' +
-                'WHERE newer.tag_id = tag_versions.tag_id AND newer.created_at <= ? ' +
-                'AND newer.version_no > tag_versions.version_no)', point)
-        quoted = TagVersion.connection.quote(point.iso8601(6))
-        scope.select("tag_id, name, names, #{ quoted } AS reference_point").to_sql
-      end
-      next if queries.empty?
-
-      TagVersion.connection.select_all(queries.map { "(#{ _1 })" }.join(' UNION ALL ')).each do |value|
-        @parent_versions[[value['tag_id'], value['reference_point']]] = value
-      end
-    end
-  end
-
-  def tag(row)
-    recorded_names = names(row.names)
-    selected = recorded_names.find { _1[:language_code] == @language }
-    aliases = recorded_names.flat_map do |entry|
-      entry[:aliases].map {
-        _1.merge('language_code' => entry[:language_code],
-                 'language_name' => entry[:language_name])
-      }
-    end
-    { name: selected&.fetch(:name) || row.name, names: recorded_names,
-      category: row.category, aliases:,
-      parents: row.parent_tag_ids.to_s.split.map do |id|
-        value = @parent_versions[[id.to_i, parent_point(row).iso8601(6)]]
-        snapshots = value && json(value['names'])
-        name = snapshots&.dig(@language, 'name') || value&.fetch('name')
-        { id: id.to_i, name: }
-      end,
-      deprecated: row.deprecated_at&.iso8601 }
-  end
-end

backend/app/representations/deerjikist_repr.rb

@@ -6,12 +6,11 @@ module DeerjikistRepr
 
   module_function
 
-  def base deerjikist, language_code: 'ja'
+  def base deerjikist
-    deerjikist.as_json(BASE).merge(
+    deerjikist.as_json(BASE)
-        tag: deerjikist.tag && TagRepr.base(deerjikist.tag, language_code:))
   end
 
-  def many deerjikists, language_code: 'ja'
+  def many deerjikists
-    deerjikists.map { |d| base(d, language_code:) }
+    deerjikists.map { |d| base(d) }
   end
 end

backend/app/representations/external_tag_repr.rb

@@ -1,20 +0,0 @@
-module ExternalTagRepr
-  module_function
-
-  def base tag
-    { 'id' => tag.id,
-      'name' => "#{ tag.platform }:#{ tag.name }",
-      'category' => 'nico',
-      'post_count' => tag.post_count,
-      'created_at' => tag.created_at,
-      'updated_at' => tag.created_at,
-      'deprecated_at' => nil,
-      'aliases' => [],
-      'parents' => [],
-      'has_wiki' => false,
-      'material_id' => nil,
-      'has_deerjikists' => false }
-  end
-
-  def inline(tag) = base(tag)
-end

backend/app/representations/material_repr.rb

@@ -12,15 +12,14 @@ module MaterialRepr
 
   module_function
 
-  def base material, host:, language_code: 'ja'
+  def base material, host:
     material.as_json(BASE).merge(
-        tag: material.tag && TagRepr.base(material.tag, language_code:),
         file: if material.file.attached?
                 Rails.application.routes.url_helpers.rails_storage_proxy_url(
                     material.file, host:)
               end,
         thumbnail: thumbnail_url(material, host:),
-        thumbnail_fallback_text: thumbnail_fallback_text(material, language_code:),
+        thumbnail_fallback_text: thumbnail_fallback_text(material),
         thumbnail_fallback_kind: thumbnail_fallback_kind(material),
         media_kind: media_kind(material),
         file_byte_size: material.file_byte_size,
@@ -28,11 +27,11 @@ module MaterialRepr
         export_items: export_items(material))
   end
 
-  def many materials, host:, language_code: 'ja'
+  def many materials, host:
-    materials.map { |m| base(m, host:, language_code:) }
+    materials.map { |m| base(m, host:) }
   end
 
-  def list material, host:, language_code: 'ja'
+  def list material, host:
     { id: material.id,
       version_no: material.version_no,
       url: material.url,
@@ -41,9 +40,9 @@ module MaterialRepr
       source_path: material.source_path,
       source_file_id: material.source_file_id,
       normalized_source_key: material.normalized_source_key,
-      tag: compact_tag(material.tag, language_code:),
+      tag: compact_tag(material.tag),
       thumbnail: thumbnail_url(material, host:),
-      thumbnail_fallback_text: thumbnail_fallback_text(material, language_code:),
+      thumbnail_fallback_text: thumbnail_fallback_text(material),
       thumbnail_fallback_kind: thumbnail_fallback_kind(material),
       media_kind: media_kind(material),
       content_type: material.content_type,
@@ -54,8 +53,8 @@ module MaterialRepr
       export_items: export_items(material) }
   end
 
-  def list_many materials, host:, language_code: 'ja'
+  def list_many materials, host:
-    materials.map { |m| list(m, host:, language_code:) }
+    materials.map { |m| list(m, host:) }
   end
 
   def export_paths material
@@ -80,8 +79,8 @@ module MaterialRepr
         material.thumbnail, host:)
   end
 
-  def thumbnail_fallback_text material, language_code:
+  def thumbnail_fallback_text material
-    material.tag&.name(language_code) || material.created_at&.strftime('%Y-%m-%d')
+    material.tag&.name || material.created_at&.strftime('%Y-%m-%d')
   end
 
   def thumbnail_fallback_kind material
@@ -99,11 +98,11 @@ module MaterialRepr
     'file_other'
   end
 
-  def compact_tag tag, language_code:
+  def compact_tag tag
     return nil unless tag
 
     { id: tag.id,
-      name: tag.name(language_code),
+      name: tag.name,
       category: tag.category,
       deprecated_at: tag.deprecated_at }
   end

backend/app/representations/post_repr.rb

@@ -17,23 +17,23 @@ module PostRepr
 
   module_function
 
-  def base post, current_user = nil, host: nil, language_code: 'ja'
+  def base post, current_user = nil, host: nil
     json =
         if host.present?
           common(post, host:)
         else
           common(post)
         end
-    json['tags'] = tag_json(post, language_code:)
+    json['tags'] = tag_json(post)
     json['uploaded_user'] = post.uploaded_user && UserRepr.base(post.uploaded_user)
     json['viewed'] = current_user ? current_user.viewed?(post) : false
     json
   end
 
   def detail post, current_user = nil, parent_posts: [], child_posts: [],
-             sibling_posts: { }, related: [], host: nil, language_code: 'ja'
+             sibling_posts: { }, related: [], host: nil
     if host.present?
-      base(post, current_user, host:, language_code:).merge(
+      base(post, current_user, host:).merge(
           'parent_posts' => cards(parent_posts, host:),
           'child_posts' => cards(child_posts, host:),
           'sibling_posts' => sibling_posts.transform_keys(&:to_s).transform_values { |posts|
@@ -41,7 +41,7 @@ module PostRepr
           },
           'related' => cards(related, host:))
     else
-      base(post, current_user, language_code:).merge(
+      base(post, current_user).merge(
           'parent_posts' => cards(parent_posts),
           'child_posts' => cards(child_posts),
           'sibling_posts' => sibling_posts.transform_keys(&:to_s).transform_values { |posts|
@@ -86,25 +86,16 @@ module PostRepr
                        end)
   end
 
-  def tag_json post, language_code:
+  def tag_json post
-    internal_tags =
+    post
-        post
+        .active_post_tags
-        .post_tags
         .reject { _1.tag.deprecated? }
-        .sort_by { TagRepr.name_for(_1.tag, language_code) }
+        .sort_by { _1.tag.name }
         .map { |post_tag|
-          TagRepr.inline(post_tag.tag, language_code:).merge(
+      TagRepr.inline(post_tag.tag).merge(
-              'children' => [],
+          'children' => [],
-              'sections' => post_tag.sections.as_json(only: [:begin_ms, :end_ms]))
+          'sections' => post_tag.sections.as_json(only: [:begin_ms, :end_ms]))
-        }
+    }
-
-    external_tags =
-        post
-        .external_tags
-        .sort_by { _1.name }
-        .map { ExternalTagRepr.base(_1).merge('children' => [], 'sections' => []) }
-
-    internal_tags + external_tags
   end
 
   def thumbnail_url post, host: nil

backend/app/representations/tag_group_repr.rb

@@ -1,207 +0,0 @@
-module TagGroupRepr
-  PREFIX = 'タググループ:'
-
-  module_function
-
-  def name_from_title title
-    prefix = [PREFIX, 'tag_group:'].find { title.downcase.start_with?(_1) }
-    title[prefix.length..] if prefix
-  end
-
-  def many locale, groups
-    ids = groups.map(&:id)
-    return [] if ids.empty?
-
-    tag_members = TagGroupTag.where(tag_group_id: ids)
-        .group_by(&:tag_group_id)
-    group_members = TagGroupGroup.where(tag_group_id: ids)
-        .group_by(&:tag_group_id)
-    placements = TagGroupPlacement.where(tag_group_id: ids).order(:position)
-        .group_by(&:tag_group_id)
-
-    target_ids =
-        tag_members.values.flatten.map(&:tag_id) +
-        placements.values.flatten.filter_map(&:tag_id)
-    child_ids =
-        group_members.values.flatten.map(&:child_tag_group_id) +
-        placements.values.flatten.filter_map(&:child_tag_group_id)
-    tags = Tag.select(:id, :category).where(id: target_ids.uniq).index_by(&:id)
-    names =
-        TagName
-        .where(tag_id: target_ids.uniq,
-               language_code: locale.language_code,
-               primary_flg: true)
-        .pluck(:tag_id, :name)
-        .to_h
-    child_groups = TagGroup.where(id: child_ids.uniq).index_by(&:id)
-
-    groups.map do |group|
-      group_json(group,
-                 tag_members.fetch(group.id, []),
-                 group_members.fetch(group.id, []),
-                 placements.fetch(group.id, []), tags, names, child_groups)
-    end
-  end
-
-  def one locale, group
-    HistoryAccess.require_current!(group, TagGroupVersion)
-    many(locale, [group]).first
-  end
-
-  def tag_group wiki_page
-    { 'id' => wiki_page.id,
-      'title' => "#{ PREFIX }#{ wiki_page.name }",
-      'deprecated_at' => nil,
-      'created_at' => wiki_page.created_at,
-      'updated_at' => wiki_page.updated_at }
-  end
-
-  def current_version group
-    HistoryAccess.visible(TagGroupVersion.all).find_by(tag_group_id: group.id, version_no: group.version_no)
-  end
-
-  def historical locale, version
-    HistoryAccess.require!(version)
-    members = Array(version.members)
-    placements = Array(version.placements)
-    tag_references = members.filter_map { |member| member['tag_id'] } +
-        collect_ids(placements, 'tag_id')
-    group_references = members.filter_map { |member| member['tag_group_id'] } +
-        collect_ids(placements, 'tag_group_id')
-    tag_versions = historical_tag_versions(tag_references)
-    group_versions = historical_group_versions(group_references)
-    tag_version_numbers = members.filter_map do |member|
-      next unless member['tag_id']
-
-      [member.fetch('tag_id'), member.fetch('version_no')]
-    end.to_h
-    group_version_numbers = members.filter_map do |member|
-      next unless member['tag_group_id']
-
-      [member.fetch('tag_group_id'), member.fetch('version_no')]
-    end.to_h
-    historical_members = members.map do |member|
-      historical_member(locale, member, tag_versions, group_versions)
-    end
-    historical_placements = placements.map do |node|
-      historical_placement(locale, node, tag_versions, group_versions,
-                           tag_version_numbers, group_version_numbers)
-    end
-    { 'id' => version.tag_group_id,
-      'name' => version.name,
-      'description' => version.description,
-      'version_no' => version.version_no,
-      'members' => historical_members,
-      'placements' => historical_placements,
-      'created_at' => version.created_at,
-      'updated_at' => version.created_at,
-      'created_by_user' => user_json(version.created_by_user),
-      'updated_by_user' => user_json(version.created_by_user) }
-  end
-
-  def group_json group, tag_members, group_members, rows, tags, names, child_groups
-    tree = placement_tree(rows, tags, names, child_groups)
-    members = tag_members.sort_by(&:tag_id).map do |member|
-      { 'tag' => tag_json(tags.fetch(member.tag_id), names[member.tag_id]) }
-    end
-    members.concat(group_members.sort_by(&:child_tag_group_id).map do |member|
-      { 'tag_group' => group_json_reference(child_groups.fetch(member.child_tag_group_id)) }
-    end)
-    { 'id' => group.id,
-      'name' => group.name,
-      'description' => group.description,
-      'version_no' => group.version_no,
-      'members' => members,
-      'placements' => tree,
-      'created_at' => group.created_at,
-      'updated_at' => group.updated_at,
-      'created_by_user' => user_json(group.created_by_user),
-      'updated_by_user' => user_json(group.updated_by_user) }
-  end
-
-  def user_json user
-    UserRepr.base(user) if user
-  end
-
-  def placement_tree rows, tags, names, child_groups
-    by_parent = rows.group_by(&:parent_position)
-    build = lambda do |parent|
-      by_parent.fetch(parent, []).sort_by(&:position).map do |row|
-        node = if row.tag_id
-          { 'tag' => tag_json(tags.fetch(row.tag_id), names[row.tag_id]) }
-        elsif row.child_tag_group_id
-          { 'tag_group' => group_json_reference(child_groups.fetch(row.child_tag_group_id)) }
-        else
-          { 'section_name' => row.section_name, 'description' => row.description }
-        end
-        node.merge('children' => build.call(row.position))
-      end
-    end
-    build.call(nil)
-  end
-
-  def tag_json tag, name
-    { 'id' => tag.id, 'name' => name, 'category' => tag.category }
-  end
-
-  def group_json_reference group
-    { 'id' => group.id, 'name' => group.name }
-  end
-
-  def collect_ids nodes, key
-    nodes.flat_map do |node|
-      [node[key], *collect_ids(Array(node['children']), key)].compact
-    end
-  end
-
-  def historical_tag_versions ids
-    HistoryAccess.visible(TagVersion.all).where(tag_id: ids.uniq).to_a.index_by do |version|
-      [version.tag_id, version.version_no]
-    end
-  end
-
-  def historical_group_versions ids
-    HistoryAccess.visible(TagGroupVersion.all).where(tag_group_id: ids.uniq).to_a.index_by do |version|
-      [version.tag_group_id, version.version_no]
-    end
-  end
-
-  def historical_member locale, member, tag_versions, group_versions
-    if member['tag_id']
-      id = member.fetch('tag_id')
-      version = tag_versions.fetch([id, member.fetch('version_no')])
-      { 'tag' => { 'id' => id, 'name' => version_name(locale, version),
-                   'category' => version.category } }
-    else
-      id = member.fetch('tag_group_id')
-      version = group_versions.fetch([id, member.fetch('version_no')])
-      { 'tag_group' => { 'id' => id, 'name' => version.name } }
-    end
-  end
-
-  def historical_placement locale, node, tag_versions, group_versions,
-                           tag_version_numbers, group_version_numbers
-    result = if node['tag_id']
-      id = node.fetch('tag_id')
-      version_no = tag_version_numbers.fetch(id)
-      version = tag_versions.fetch([id, version_no])
-      { 'tag' => { 'id' => id, 'name' => version_name(locale, version),
-                   'category' => version.category } }
-    elsif node['tag_group_id']
-      id = node.fetch('tag_group_id')
-      version_no = group_version_numbers.fetch(id)
-      version = group_versions.fetch([id, version_no])
-      { 'tag_group' => { 'id' => id, 'name' => version.name } }
-    else
-      { 'section_name' => node.fetch('section_name'), 'description' => node['description'] }
-    end
-    result.merge('children' => Array(node['children']).map do |child|
-      historical_placement(locale, child, tag_versions, group_versions,
-                           tag_version_numbers, group_version_numbers)
-    end)
-  end
-
-  def version_name locale, version
-    version.names[locale.language_code]&.fetch('name')
-  end
-end

backend/app/representations/tag_repr.rb

@@ -3,30 +3,18 @@
 
 module TagRepr
   BASE = { only: [:id, :category, :post_count, :created_at, :updated_at, :deprecated_at],
-           methods: [:material_id, :has_deerjikists] }.freeze
+           methods: [:name, :has_wiki, :material_id, :has_deerjikists] }.freeze
 
   module_function
 
-  def base tag, language_code: 'ja'
+  def base tag
-    tag.as_json(BASE).merge('has_wiki' => tag.has_description?(language_code:),
+    tag.as_json(BASE).merge(aliases: tag.snapshot_aliases,
-                            name: name_for(tag, language_code),
+                            parents: tag.parents.map { _1.as_json(BASE) })
-                            aliases: tag.snapshot_aliases(language_code),
-                            parents: tag.parents.map { inline(_1, language_code:) })
   end
 
-  def inline tag, language_code: 'ja'
+  def inline tag
-    tag.as_json(BASE).merge('has_wiki' => tag.has_description?(language_code:),
+    tag.as_json(BASE).merge(aliases: [], parents: [])
-                            name: name_for(tag, language_code), aliases: [], parents: [])
   end
 
-  def many tags, language_code: 'ja'
+  def many(tags) = tags.map { |t| base(t) }
-    records = tags.to_a
-    ActiveRecord::Associations::Preloader.new(
-        records:, associations: [:tag_descriptions, { parents: :tag_descriptions }]).call
-    records.map { |tag| base(tag, language_code:) }
-  end
-
-  def name_for tag, language_code
-    tag.name(language_code) || (tag.system? ? tag.name('ja') : nil)
-  end
 end

backend/app/representations/wiki_page_repr.rb

@@ -2,49 +2,15 @@
 
 
 module WikiPageRepr
+  BASE = { methods: [:title, :deprecated_at] }.freeze
+
   module_function
 
-  def tag_description locale, description
+  def base wiki_page
-    HistoryAccess.require_current!(description, TagDescriptionVersion)
+    wiki_page.as_json(BASE)
-    tag = description.tag
-
-    { 'id' => description.id,
-      'resource_type' => 'tag_description',
-      'tag_id' => tag.id,
-      'language_code' => description.language_code,
-      'tag_name_id' => tag.tag_name(locale.language_code)&.id,
-      'body' => description.body,
-      'created_user_id' => description.created_by_user_id,
-      'updated_user_id' => description.updated_by_user_id,
-      'created_at' => description.created_at,
-      'updated_at' => description.updated_at,
-      'discarded_at' => nil,
-      'version_no' => description.version_no,
-      'title' => tag.name(locale.language_code),
-      'deprecated_at' => tag.deprecated_at }
   end
 
-  def tag_group tag_group
+  def many wiki_pages
-    TagGroupRepr.tag_group(tag_group).merge('resource_type' => 'tag_group')
+    wiki_pages.map { |p| base(p) }
-  end
-
-  def autolink_tag locale, tag_id:, description_id:, name:, language:
-    { resource_type: 'tag_description',
-      resource_id: description_id,
-      tag_id:,
-      tag_description_id: description_id,
-      name:,
-      language_code: language,
-      locale_code: locale.code_before_type_cast }
-  end
-
-  def autolink_group locale, id:, name:, language:
-    { resource_type: 'tag_group',
-      resource_id: id,
-      tag_id: nil,
-      tag_description_id: nil,
-      name: "#{ TagGroupRepr::PREFIX }#{ name }",
-      language_code: language,
-      locale_code: locale.code_before_type_cast }
   end
 end

backend/app/services/certified_tag_name_translations.rb

@@ -1,105 +0,0 @@
-require 'json'
-require 'pathname'
-
-
-class CertifiedTagNameTranslations
-  FORMAT = 'btrc-hub.certified-tag-name-translations.v1'.freeze
-
-  class Error < ArgumentError
-    ;
-  end
-
-  def self.load! path, locale:
-    new(path, locale:).load!
-  end
-
-  def initialize path, locale:
-    @path = path
-    @locale = locale
-  end
-
-  def load!
-    document = read_document
-    unless document.is_a?(Hash) && document['format'] == FORMAT
-      fail!('invalid format or version')
-    end
-
-    source_code = document['source_locale']
-    target_code = document['target_locale']
-    unless target_code.is_a?(String) && target_code == Locale.codes.fetch(@locale.code)
-      fail!("target_locale #{ target_code.inspect } does not match the new Locale")
-    end
-    unless source_code.is_a?(String) && Locale.codes.value?(source_code)
-      fail!("invalid source_locale #{ source_code.inspect }")
-    end
-    source_locale = Locale.find_by(code: source_code)
-    fail!("source Locale #{ source_code.inspect } does not exist") unless source_locale
-    fail!('source_locale and target_locale must differ') if source_code == target_code
-
-    entries = validate_entries(document['entries'], source_code, target_code)
-    resolve_tags(entries, source_locale.language_code)
-  end
-
-  private
-
-  def read_document
-    unless (@path.is_a?(String) && !@path.include?("\0") &&
-            Pathname.new(@path).absolute?)
-      fail!('certified_tag_names_path must be an absolute file path')
-    end
-    fail!('JSON file does not exist or is not a regular file') unless File.file?(@path)
-
-    text = File.read(@path, encoding: Encoding::UTF_8)
-    fail!('JSON file is not valid UTF-8') unless text.valid_encoding?
-    JSON.parse(text)
-  rescue JSON::ParserError, SystemCallError, IOError, EncodingError => error
-    fail!("cannot read JSON: #{ error.message }")
-  end
-
-  def validate_entries entries, source_code, target_code
-    fail!('entries must be an array') unless entries.is_a?(Array)
-
-    sources = {}
-    targets = {}
-    entries.each_with_index.map do |entry, index|
-      fail!("entry #{ index + 1 } must be an object") unless entry.is_a?(Hash)
-      source = entry[source_code]
-      target = entry[target_code]
-      unless source.is_a?(String) && source.present? && target.is_a?(String) && target.present?
-        fail!("entry #{ index + 1 }: missing or empty #{ source_code }/#{ target_code } name")
-      end
-      fail!("duplicate source name #{ source.inspect }") if sources.key?(source)
-      fail!("duplicate target name #{ target.inspect }") if targets.key?(target)
-
-      sources[source] = true
-      targets[target] = true
-      [source, target]
-    end
-  end
-
-  def resolve_tags entries, source_language
-    source_names = entries.map(&:first)
-    # SQL may use a case/accent-insensitive collation; the Ruby keys are exact.
-    matches = TagName.where(language_code: source_language, primary_flg: true,
-                            name: source_names).pluck(:name, :tag_id).group_by(&:first)
-    translations = {}
-    entries.each do |source, target|
-      rows = matches.fetch(source, [])
-      next if rows.empty?
-
-      unless rows.length == 1
-        fail!("source primary name #{ source.inspect } must match exactly one tag")
-      end
-      tag_id = rows.first.last
-      if translations.key?(tag_id)
-        fail!("multiple source primary names resolve to Tag##{ tag_id }: #{ source.inspect }")
-      end
-      translations[tag_id] = target
-    end
-    translations
-  end
-
-  def fail! message
-    raise Error, "Certified tag names #{@path.inspect}: #{ message }"
-  end
-end

backend/app/services/counts_axes.rb

@@ -1,100 +0,0 @@
-class CountsAxes
-  GRAINS = ['year', 'quarter', 'month', 'week', 'day', 'hour'].freeze
-
-  def self.time_expression(expression, grain, timezone)
-    raise ArgumentError unless GRAINS.include?(grain)
-
-    connection = ApplicationRecord.connection
-    # Use the installed TZInfo database, including DST, rather than MySQL timezone tables.
-    first = Time.utc(1900)
-    last = Time.utc(2101)
-    offset = timezone.period_for_utc(first).utc_total_offset
-    clauses = timezone.transitions_up_to(last, first).filter_map do |transition|
-      next if transition.offset.utc_total_offset == offset
-
-      previous = offset
-      offset = transition.offset.utc_total_offset
-      "WHEN #{ expression } < #{ connection.quote(transition.at.to_time) } THEN #{ previous }"
-    end
-    seconds = clauses.empty? ? offset.to_s : "CASE #{ clauses.join(' ') } ELSE #{ offset } END"
-    local = "DATE_ADD(#{ expression }, INTERVAL (#{ seconds }) SECOND)"
-    case grain
-    when 'year' then "DATE_FORMAT(#{ local }, '%Y')"
-    when 'quarter' then "CONCAT(YEAR(#{ local }), '-Q', QUARTER(#{ local }))"
-    when 'month' then "DATE_FORMAT(#{ local }, '%Y-%m')"
-    when 'week' then "DATE_FORMAT(#{ local }, '%x-W%v')"
-    when 'day' then "DATE_FORMAT(#{ local }, '%Y-%m-%d')"
-    else "DATE_FORMAT(#{ local }, '%Y-%m-%d %H:00')"
-    end
-  end
-
-  attr_reader :rows, :columns, :all
-
-  def initialize(population)
-    @population = population
-    @definition = population.definition
-    @rows = parse(population.params.fetch('rows', ''))
-    @columns = parse(population.params.fetch('cols', ''))
-    @all = rows + columns
-    raise ArgumentError if all.length > 8 || all.map { _1['field'] }.uniq.length != all.length
-  end
-
-  def expressions
-    @expressions ||= all.map do |axis|
-      field = axis.fetch('field')
-      case field
-      when 'tags'
-        "CASE WHEN tag_entry.tag_id IS NOT NULL " +
-            "THEN CONCAT('internal:', tag_entry.tag_id) " +
-            "ELSE CONCAT('external:', tag_entry.external_tag_id) END"
-      when 'changed_field' then 'changed.field_key'
-      else
-        expression = @definition.expression(field)
-        @definition.fields[field] == :datetime ? self.class.time_expression(
-            expression, axis.fetch('grain', 'month'), @population.timezone) : expression
-      end
-    end
-  end
-
-  def joins(fields = all.map { _1['field'] })
-    result = ''
-    if fields.include?('tags')
-      result += <<~SQL.squish
-         JOIN JSON_TABLE(CAST(v.tags_json AS JSON), '$[*]' COLUMNS (
-          tag_id BIGINT PATH '$.tag_id' NULL ON EMPTY,
-          external_tag_id BIGINT PATH '$.external_tag_id' NULL ON EMPTY)) tag_entry
-
-      SQL
-    end
-    if fields.include?('changed_field')
-      fields = @definition.changes.map do |field|
-        "SELECT #{ ApplicationRecord.connection.quote(field) } AS field_key"
-      end.join(' UNION ALL ')
-      condition = @definition.changes.map do |field|
-        "(changed.field_key = '#{ field }' AND #{ @population.changed_sql(field) })"
-      end.join(' OR ')
-      result += " JOIN (#{ fields }) changed ON (#{ condition })"
-    end
-    result
-  end
-
-  private
-
-  def parse(value)
-    axes = QueryState.axes(value)
-    raise ArgumentError unless axes.is_a?(Array)
-
-    axes.each do |axis|
-      raise ArgumentError unless axis.is_a?(Hash) && (axis.keys - ['field', 'grain']).empty?
-      raise ArgumentError unless @definition.axes(@population.history).include?(axis['field'])
-      if @definition.fields[axis['field']] == :datetime
-        raise ArgumentError unless GRAINS.include?(axis.fetch('grain', 'month'))
-
-        axis['grain'] ||= 'month'
-      elsif axis.key?('grain')
-        raise ArgumentError
-      end
-    end
-    axes
-  end
-end

backend/app/services/counts_condition.rb

@@ -1,330 +0,0 @@
-# Typed predicates over the same recorded fields used by Counts output.
-# Multi-valued comparisons use EXISTS; inequality means no matching element.
-class CountsCondition
-  CHILD_TYPES = { 'id' => :integer, 'versionNo' => :integer, 'tagId' => :integer,
-                  'tagGroupId' => :integer, 'autoGenerated' => :boolean,
-                  'category' => :enum, 'kind' => :enum, 'sections' => :interval }.freeze
-
-  def self.fields(definition)
-    result = definition.metadata.map { _1.merge(labels: [definition.field_label(_1[:key])]) }
-    visit = lambda do |fields, prefix, labels|
-      fields.each do |field|
-        key = [prefix, field[:key]].compact.join('.')
-        path_labels = labels + [field[:label]]
-        type = if prefix
-                 CHILD_TYPES.fetch(field[:key], :string)
-               elsif key == 'is_latest'
-                 :boolean
-               elsif key == 'parents'
-                 :integer
-               elsif key == 'deprecated'
-                 :datetime
-               elsif key == 'thumbnail'
-                 :string
-               else
-                 definition.fields.fetch(key, :string)
-               end
-        type = :string if field[:children] || type == :json
-        type = :integer if ['parents', 'members'].include?(key)
-        physical = result.find { _1[:key] == key }
-        nullable = prefix.present? || (physical ? physical[:nullable] : key != 'is_latest')
-        nullable = false if field[:children]
-        entry = { key:, type:, labels: path_labels, nullable:,
-                  multiple: prefix.present? || field[:children].present?,
-                  collection: field[:children].present? || type == :interval,
-                  operators: CountsResourceDefinition::OPERATORS.fetch(type) }
-        result.reject! { _1[:key] == key }
-        result << entry
-        visit.call(field[:children] || [], key, path_labels)
-      end
-    end
-    visit.call(definition.output_fields, nil, [])
-    # The raw JSON alias has the same membership semantics as the public tag field.
-    if definition.resource == 'posts'
-      result.find { _1[:key] == 'tags_json' }.merge!(type: :string, multiple: true, collection: true,
-          operators: CountsResourceDefinition::OPERATORS.fetch(:string))
-    end
-    result + [{ key: 'user.id', type: :integer, nullable: true,
-                labels: ['history:actor', 'counts:columns.resourceId'],
-                operators: CountsResourceDefinition::OPERATORS.fetch(:integer) },
-              { key: 'user.name', type: :string, nullable: true,
-                labels: ['history:actor', 'history:name'],
-                operators: CountsResourceDefinition::OPERATORS.fetch(:string) }]
-  end
-
-  def initialize(population)
-    @population = population
-    @definition = population.definition
-    @connection = @definition.model.connection
-    @language = Locale.find_by!(code: population.params.fetch('locale', 'ja')).language_code
-  end
-
-  def apply(scope, entry)
-    raise ArgumentError unless entry.is_a?(Hash)
-    raise ArgumentError if (entry.keys - ['field', 'operator', 'value', 'end', 'blank']).any?
-
-    field = @definition.filter_metadata.find { _1[:key] == entry.fetch('field') }
-    raise ArgumentError unless field && field[:operators].include?(entry.fetch('operator'))
-    raise ArgumentError unless [true, false, nil].include?(entry['blank'])
-    values = [entry['value'], entry['end']]
-    raise ArgumentError unless values.all? { _1.nil? || (_1.is_a?(String) && _1.bytesize <= 8192) }
-    if entry['operator'] != 'range' && !values[1].nil? && values[1] != ''
-      raise ArgumentError
-    end
-
-    source, expression = source_for(field[:key])
-    type = field[:type]
-    blank = blank_sql(expression, type)
-    if entry['blank']
-      sql = if source && field[:collection]
-              "NOT EXISTS (SELECT 1 FROM #{ source })"
-            elsif source
-              "NOT EXISTS (SELECT 1 FROM #{ source } WHERE NOT (#{ blank }))"
-            else blank
-            end
-      return scope.where(Arel.sql(sql))
-    end
-    return scope if values.all? { _1.nil? || _1 == '' }
-
-    parsed = values.map { parse(_1, type) }
-    validate_enum!(field[:key], parsed.first) if type == :enum
-    operator = entry['operator']
-    if type == :interval
-      return interval(scope, source, expression, parsed)
-    end
-    clauses = []
-    binds = []
-    if operator == 'range'
-      ['>=', '<='].each_with_index do |comparison, index|
-        next if parsed[index].nil?
-
-        clauses << "#{ expression } #{ comparison } ?"
-        binds << parsed[index]
-      end
-      raise ArgumentError if parsed.all? && parsed.first > parsed.last
-    elsif !parsed.first.nil?
-      comparison = { 'eq' => '=', 'ne' => '=', 'gt' => '>', 'lt' => '<',
-                     'ge' => '>=', 'le' => '<=', 'like' => 'LIKE' }.fetch(operator)
-      clauses << "#{ expression } #{ comparison } ?"
-      binds << parsed.first
-    end
-    return scope if clauses.empty?
-
-    predicate = clauses.join(' AND ')
-    if source
-      predicate = "EXISTS (SELECT 1 FROM #{ source } WHERE #{ predicate })"
-      predicate = "NOT (#{ predicate })" if operator == 'ne'
-    elsif operator == 'ne'
-      predicate = "(NOT (#{ predicate }) OR #{ blank })"
-    end
-    scope.where(predicate, *binds)
-  end
-
-  private
-
-  def parse(value, type)
-    return nil if value.nil? || value == ''
-
-    case type
-    when :integer, :interval
-      raise ArgumentError unless value.match?(/\A-?[0-9]{1,19}\z/)
-
-      number = Integer(value, 10)
-      raise ArgumentError unless number.between?(-9_223_372_036_854_775_808, 9_223_372_036_854_775_807)
-
-      number
-    when :boolean
-      raise ArgumentError unless ['true', 'false'].include?(value)
-
-      value == 'true' ? 1 : 0
-    when :datetime then @population.timestamp(value)
-    else value
-    end
-  end
-
-  def validate_enum!(key, value)
-    allowed = case key.split('.').last
-              when 'event_type' then VersionRecord::DEFAULT_EVENT_TYPE_MAP.values
-              when 'category', 'tag_category' then TagVersion.categories.keys
-              when 'kind' then ['internal', 'external']
-              when 'source_kind' then Material::SOURCE_KINDS
-              end
-    raise ArgumentError if value && allowed && !allowed.include?(value)
-  end
-
-  def blank_sql(expression, type)
-    if [:string, :enum].include?(type)
-      "(#{ expression } IS NULL OR #{ expression } = '')"
-    else
-      "#{ expression } IS NULL"
-    end
-  end
-
-  def scalar(document, key, type = :string)
-    path = @connection.quote("$.#{ key }")
-    value = "JSON_EXTRACT(#{ document }, #{ path })"
-    text = "JSON_UNQUOTE(NULLIF(#{ value }, CAST('null' AS JSON)))"
-    return "CAST(#{ text } AS SIGNED)" if type == :integer
-    return "CASE #{ text } WHEN 'true' THEN 1 WHEN 'false' THEN 0 END" if type == :boolean
-
-    text
-  end
-
-  def json_table(document, path, name)
-    "JSON_TABLE(COALESCE(#{ document }, JSON_ARRAY()), #{ @connection.quote(path) } " +
-        "COLUMNS (item JSON PATH '$')) #{ name }"
-  end
-
-  def names_source(document, alias_name)
-    source = "JSON_TABLE(COALESCE(JSON_KEYS(#{ document }), JSON_ARRAY()), '$[*]' " +
-        "COLUMNS (language_code TEXT PATH '$')) #{ alias_name }"
-    path = "CONCAT('$.', JSON_QUOTE(#{ alias_name }.language_code))"
-    [source, "JSON_EXTRACT(#{ document }, #{ path })", "#{ alias_name }.language_code"]
-  end
-
-  def point
-    @point ||= @population.history ? @population.at :
-        @population.timestamp(@population.params['at']) || @population.at
-  end
-
-  def name_expression(row)
-    language_path = @connection.quote("$.#{ @language.to_json }.name")
-    name = "JSON_UNQUOTE(JSON_EXTRACT(#{ row }.names, #{ language_path }))"
-    "COALESCE(#{ name }, #{ row }.name)"
-  end
-
-  def parent_source
-    column = @definition.resource == 'posts' ? 'parent_post_ids' : 'parent_tag_ids'
-    ids = "TRIM(REGEXP_REPLACE(COALESCE(v.#{ column }, ''), '[[:space:]]+', ' '))"
-    document = "CONCAT('[', REPLACE(#{ ids }, ' ', ','), ']')"
-    source = json_table(document, '$[*]', 'parent_entry')
-    id = 'CAST(parent_entry.item AS UNSIGNED)'
-    return [source, id] if @definition.resource == 'posts'
-
-    bound = @connection.quote(point)
-    source += ' LEFT JOIN tag_versions parent_version ON parent_version.tag_id = ' + id +
-        " AND parent_version.created_at <= #{ bound } AND " +
-        HistoryAccess.sql(TagVersion, 'parent_version') +
-        ' AND NOT EXISTS (SELECT 1 FROM tag_versions newer_parent ' +
-        'WHERE newer_parent.tag_id = parent_version.tag_id ' +
-        "AND newer_parent.created_at <= #{ bound } " +
-        'AND newer_parent.version_no > parent_version.version_no)'
-    [source, id]
-  end
-
-  def post_tag_source
-    source = json_table('v.tags_json', '$[*]', 'tag_snapshot')
-    source += ' LEFT JOIN tag_versions referenced_tag ON referenced_tag.tag_id = ' +
-        scalar('tag_snapshot.item', 'tag_id', :integer) +
-        ' AND referenced_tag.version_no = ' + scalar('tag_snapshot.item', 'version_no', :integer) +
-        " AND referenced_tag.created_at <= #{ @connection.quote(point) } AND " +
-        HistoryAccess.sql(TagVersion, 'referenced_tag')
-    source
-  end
-
-  def language_field(source, document, parts, inherited_language = nil)
-    return [source, scalar(document, 'name')] if parts.empty?
-    return [source, inherited_language] if parts == ['languageCode'] && inherited_language
-
-    if parts.first == 'aliases'
-      aliases = "JSON_EXTRACT(#{ document }, '$.aliases')"
-      source += ' JOIN ' + json_table(aliases, '$[*]', 'alias_entry')
-      child = parts[1] || 'name'
-      return [source, inherited_language] if child == 'languageCode' && inherited_language
-      columns = { 'scriptCode' => 'script_code', 'name' => 'name' }
-      key = columns.fetch(child) { raise ArgumentError }
-      return [source, scalar('alias_entry.item', key)]
-    end
-    key = { 'name' => 'name', 'scriptCode' => 'script_code',
-            'autoGenerated' => 'auto_generated' }.fetch(parts.first) { raise ArgumentError }
-    [source, scalar(document, key, CHILD_TYPES.fetch(parts.first, :string))]
-  end
-
-  def source_for(key)
-    root, *parts = key.split('.')
-    return [nil, "(#{ @population.latest_sql })"] if root == 'is_latest'
-    if root == 'user' && parts.any?
-      expression = parts.first == 'id' ? 'v.created_by_user_id' :
-          '(SELECT actor.name FROM users actor WHERE actor.id = v.created_by_user_id)'
-      return [nil, expression]
-    end
-    return [nil, 'v.thumbnail_base'] if root == 'thumbnail'
-    return [nil, 'v.deprecated_at'] if root == 'deprecated'
-    if root == 'parents'
-      source, id = parent_source
-      return [source, parts.first == 'name' ? name_expression('parent_version') : id]
-    end
-    if root == 'tags' || root == 'tags_json'
-      return post_field(parts)
-    end
-    if root == 'names' || root == 'aliases'
-      source, document, language = names_source('v.names', 'name_language')
-      parts = ['aliases', *parts] if root == 'aliases'
-      return language_field(source, document, parts, language)
-    end
-    if root == 'members' || root == 'placements'
-      path = '$[*]' + parts.take_while { _1 == 'children' }.map { '.children[*]' }.join
-      source = json_table("v.#{ root }", path, 'structure_entry')
-      if root == 'members' && parts.empty?
-        value = "COALESCE(#{ scalar('structure_entry.item', 'tag_id', :integer) }, " +
-            "#{ scalar('structure_entry.item', 'tag_group_id', :integer) })"
-        return [source, value]
-      end
-      leaf = parts.last || 'sectionName'
-      leaf = 'sectionName' if leaf == 'children'
-      column = leaf.gsub(/[A-Z]/) { "_#{ _1.downcase }" }
-      return [source, scalar('structure_entry.item', column, CHILD_TYPES.fetch(leaf, :string))]
-    end
-    if root == 'export_paths_json'
-      source, value, profile = names_source('v.export_paths_json', 'export_profile')
-      return [source, parts.first == 'profile' ? profile : "JSON_UNQUOTE(#{ value })"]
-    end
-    return [nil, name_expression('v')] if root == 'name' && @definition.resource == 'tags'
-
-    [nil, @definition.expression(root)]
-  end
-
-  def post_field(parts)
-    source = post_tag_source
-    document = 'tag_snapshot.item'
-    case parts.first
-    when 'names'
-      names, entry, language = names_source('referenced_tag.names', 'tag_language')
-      language_field("#{ source } JOIN #{ names }", entry, parts.drop(1), language)
-    when 'sections'
-      sections = "JSON_EXTRACT(#{ document }, '$.sections')"
-      source += ' JOIN ' + json_table(sections, '$[*]', 'section_entry')
-      [source, 'section_entry.item']
-    when 'id'
-      [source, "COALESCE(#{ scalar(document, 'tag_id', :integer) }, " +
-          "#{ scalar(document, 'external_tag_id', :integer) })"]
-    when 'versionNo' then [source, scalar(document, 'version_no', :integer)]
-    when 'kind'
-      [source, "CASE WHEN #{ scalar(document, 'tag_id') } IS NULL " +
-          "THEN 'external' ELSE 'internal' END"]
-    when 'category'
-      [source, "COALESCE(#{ scalar(document, 'category') }, referenced_tag.category)"]
-    else
-      [source, "COALESCE(#{ scalar(document, 'name') }, referenced_tag.name)"]
-    end
-  end
-
-  def interval(scope, source, document, bounds)
-    raise ArgumentError if bounds.compact.any?(&:negative?)
-    raise ArgumentError if bounds.all? && bounds.first > bounds.last
-
-    first = scalar(document, 'begin_ms', :integer)
-    last = scalar(document, 'end_ms', :integer)
-    clauses = []
-    values = []
-    if bounds.first
-      clauses << "(#{ last } IS NULL OR #{ last } > ?)"
-      values << bounds.first
-    end
-    if bounds.last
-      clauses << "#{ first } < ?"
-      values << bounds.last
-    end
-    scope.where("EXISTS (SELECT 1 FROM #{ source } WHERE #{ clauses.join(' AND ') })", *values)
-  end
-end

backend/app/services/counts_defaults.rb

@@ -1,63 +0,0 @@
-# Only confirmed combinations belong here; all others retain the existing behaviour.
-class CountsDefaults
-  POST_COLUMNS = ['resource_id', 'version_no', 'is_latest', 'title', 'url',
-                  'thumbnail_base', 'tags', 'parents', 'original_created_from',
-                  'original_created_before', 'event_type', 'user', 'created_at'].freeze
-
-  def self.entries(resource)
-    result = [{ definition: 'history', mode: 'summary', params: {
-        'rows' => 'created_at:month',
-        'cols' => 'event_type',
-        'sort' => 'a0:desc', 'conditions' => '' } }]
-    result << { definition: 'latest', mode: 'summary', params: {
-        'rows' => '', 'cols' => '', 'conditions' => '', 'tags' => '' } }
-    details = detail_params(resource)
-    if details
-      ['latest', 'history'].each do |definition|
-        result << { definition:, mode: 'details', params: details }
-      end
-    end
-    result
-  end
-
-  def self.detail_params(resource)
-    if resource == 'posts'
-      return { 'columns' => POST_COLUMNS.join(' '), 'order' => POST_COLUMNS.join(' '),
-               'children' => 'tags.name parents.id', 'nested_json' => '0',
-               'conditions' => '', 'tags' => '' }
-    end
-    unless resource == 'tags'
-      return { 'columns' => CountsExportProjection.default_columns(resource).join(' '),
-               'children' => CountsExportProjection.default_children(resource).join(' ') }
-    end
-
-    columns = ['resource_id', 'version_no', 'is_latest', 'name', 'category',
-               'aliases', 'parents', 'deprecated', 'event_type', 'user', 'created_at']
-    { 'columns' => columns.join(' '), 'order' => columns.join(' '),
-      'children' => 'aliases.name parents.name', 'nested_json' => '0' }
-  end
-
-  def self.for(params)
-    settings = entries(params.fetch('resource', 'posts')).find do |entry|
-      entry[:definition] == params.fetch('definition', 'latest') &&
-          entry[:mode] == params.fetch('mode', 'summary')
-    end&.fetch(:params) || {}
-    if settings.key?('sort') && params.key?('rows')
-      actual = QueryState.axes(params['rows'])
-      expected = QueryState.axes(settings.fetch('rows'))
-      compatible = actual.is_a?(Array) && actual.first.is_a?(Hash) &&
-          actual.first['field'] == expected.first['field'] &&
-          actual.first.fetch('grain', 'month') == expected.first['grain']
-      unless compatible
-        settings = settings.except('sort')
-      end
-    end
-    settings
-  end
-
-  def self.metadata(resource)
-    entries(resource).map do |entry|
-      entry.merge(params: entry[:params].map { |key, value| { key:, value: } })
-    end
-  end
-end

backend/app/services/counts_export_download.rb

@@ -1,106 +0,0 @@
-require 'tempfile'
-require 'fileutils'
-
-# Finish generation before reporting readiness; downloads never hold the full file in RAM.
-class CountsExportDownload
-  EXPIRY = 10.minutes
-  MAX_CONCURRENT_EXPORTS = 2
-  MAX_FILE_BYTES = 128.megabytes
-  MAX_STORED_BYTES = 512.megabytes
-  MAX_EXPORT_SECONDS = 120
-  class Busy < StandardError; end
-  class TooLarge < StandardError; end
-  class TimedOut < StandardError; end
-  class StorageFull < StandardError; end
-
-  def self.directory
-    Rails.root.join(ENV.fetch('COUNTS_EXPORT_DIRECTORY', 'tmp/counts_exports'))
-  end
-  def self.verifier = Rails.application.message_verifier('counts_export_download')
-
-  def self.prepare(exporter, owner, progress: nil, check: nil, identifier: nil)
-    FileUtils.mkdir_p(directory, mode: 0o700)
-    authority = HistoryAccess.authority
-    slot = acquire_slot
-
-    begin
-      generate(exporter, owner, progress:, check:, identifier:, authority:)
-    ensure
-      slot.close
-    end
-  end
-
-  def self.acquire_slot
-    MAX_CONCURRENT_EXPORTS.times do |index|
-      path = directory.join(".slot-#{ index }.lock")
-      handle = File.open(path, File::RDWR | File::CREAT, 0o600)
-      return handle if handle.flock(File::LOCK_EX | File::LOCK_NB)
-
-      handle.close
-    end
-    raise Busy
-  end
-
-  def self.generate(exporter, owner, progress: nil, check: nil, identifier: nil, authority: HistoryAccess.authority)
-    FileUtils.mkdir_p(directory, mode: 0o700)
-    Dir.glob(directory.join('counts-*.export')).each do |path|
-      File.unlink(path) if File.mtime(path) < Time.current - EXPIRY
-    rescue Errno::ENOENT
-      # Another request may have removed an expired download.
-    end
-    stored = Dir.glob(directory.join('counts-*')).sum do |path|
-      File.size(path)
-    rescue Errno::ENOENT
-      0
-    end
-    reserved = MAX_CONCURRENT_EXPORTS * MAX_FILE_BYTES
-    raise StorageFull if stored + reserved > MAX_STORED_BYTES
-
-    prefix = identifier ? "counts-#{ identifier }-" : 'counts-'
-    file = Tempfile.create([prefix, '.part'], directory)
-    complete = false
-    bytes = 0
-    deadline = Process.clock_gettime(Process::CLOCK_MONOTONIC) + MAX_EXPORT_SECONDS
-    path = file.path
-    begin
-      file.binmode
-      exporter.body(progress:).each do |chunk|
-        check&.call
-        raise TimedOut if Process.clock_gettime(Process::CLOCK_MONOTONIC) > deadline
-
-        bytes += chunk.bytesize
-        raise TooLarge if bytes > MAX_FILE_BYTES
-
-        file.write(chunk)
-      end
-      file.flush
-      file.fsync
-      file.close
-      finished_path = path.sub(/\.part\z/, '.export')
-      File.rename(path, finished_path)
-      path = finished_path
-      token = verifier.generate({ 'file' => File.basename(path),
-                          'resource' => exporter.resource, 'format' => exporter.format,
-                          'owner' => owner, **authority }, expires_in: EXPIRY, purpose: 'counts_export')
-      complete = true
-      token
-    ensure
-      file.close unless file.closed?
-      File.unlink(path) if !complete && File.exist?(path)
-    end
-  end
-
-  def self.find(token, owner)
-    metadata = verifier.verified(token, purpose: 'counts_export')
-    return unless metadata.is_a?(Hash) && metadata['owner'] == owner
-    return unless HistoryAccess.authorised_artifact?(metadata)
-    return unless metadata['file']&.match?(/\Acounts-[a-zA-Z0-9_-]+\.export\z/)
-    return unless CountsResourceDefinition::DEFINITIONS.key?(metadata['resource'])
-    return unless ['json', 'csv'].include?(metadata['format'])
-
-    path = directory.join(metadata['file'])
-    return unless path.file?
-
-    metadata.merge('path' => path.to_s)
-  end
-end

backend/app/services/counts_export_jobs.rb

@@ -1,255 +0,0 @@
-require 'json'
-require 'fileutils'
-
-# A durable, bounded export spool. Web and supervised workers share the same POSIX volume.
-class CountsExportJobs
-  class Cancelled < StandardError; end
-  ACTIVE = ['queued', 'running', 'cancelling'].freeze
-
-  def self.directory = CountsExportDownload.directory.join('jobs')
-  def self.verifier = Rails.application.message_verifier('counts_export_job')
-
-  def self.synchronise
-    FileUtils.mkdir_p(directory, mode: 0o700)
-    File.open(directory.join('.registry.lock'), File::RDWR | File::CREAT, 0o600) do |lock|
-      lock.flock(File::LOCK_EX)
-      yield
-    end
-  end
-
-  def self.write(state)
-    path = directory.join("#{ state.fetch('id') }.json")
-    temporary = "#{ path }.#{ SecureRandom.hex(8) }.tmp"
-    File.open(temporary, 'w', 0o600) do |file|
-      file.write(JSON.generate(state))
-      file.flush
-      file.fsync
-    end
-    File.rename(temporary, path)
-  ensure
-    File.unlink(temporary) if temporary && File.exist?(temporary)
-  end
-
-  def self.read(id)
-    JSON.parse(File.read(directory.join("#{ id }.json")))
-  rescue Errno::ENOENT
-    nil
-  end
-
-  def self.remove_download(state)
-    name = state['download_file']
-    return unless name&.match?(/\Acounts-[a-zA-Z0-9_-]+\.export\z/)
-
-    File.unlink(CountsExportDownload.directory.join(name))
-  rescue Errno::ENOENT
-    nil
-  end
-
-  def self.remove_partial(id)
-    Dir.glob(CountsExportDownload.directory.join("counts-#{ id }-*")).each do |path|
-      File.unlink(path)
-    rescue Errno::ENOENT
-      nil
-    end
-  end
-
-  def self.clean
-    Dir.glob(directory.join('*.tmp')).each do |path|
-      File.unlink(path) if File.mtime(path) < Time.current - CountsExportDownload::EXPIRY
-    end
-    Dir.glob(directory.join('*.run')).each do |path|
-      id = File.basename(path, '.run')
-      next if directory.join("#{ id }.json").exist?
-
-      File.open(path, File::RDWR) do |lock|
-        File.unlink(path) if lock.flock(File::LOCK_EX | File::LOCK_NB)
-      end
-    end
-    Dir.glob(directory.join('*.json')).each do |path|
-      state = JSON.parse(File.read(path))
-      next if state.fetch('expires_at') > Time.current.to_f
-
-      run = directory.join("#{ state.fetch('id') }.run")
-      File.open(run, File::RDWR | File::CREAT, 0o600) do |lock|
-        unless lock.flock(File::LOCK_EX | File::LOCK_NB)
-          state['status'] = 'cancelling'
-          write(state)
-          next
-        end
-        remove_download(state)
-        remove_partial(state.fetch('id'))
-        File.unlink(path)
-        File.unlink(run)
-      end
-    end
-  end
-
-  def self.start(payload, owner, language)
-    payload = payload.dup
-    time_key = payload.key?('definition') ? 'execution_at' : 'at'
-    if payload[time_key].nil? || payload[time_key] == ''
-      payload[time_key] = Time.current.utc.iso8601(6)
-    end
-    exporter = CountsExporter.new(payload, language_code: language)
-    synchronise do
-      clean
-      active = Dir.glob(directory.join('*.json')).count do |path|
-        ACTIVE.include?(JSON.parse(File.read(path))['status'])
-      end
-      raise CountsExportDownload::Busy if active >= CountsExportDownload::MAX_CONCURRENT_EXPORTS
-
-      id = SecureRandom.hex(24)
-      state = { 'id' => id, 'owner' => owner, 'payload' => payload,
-          'language' => language, 'resource' => exporter.resource,
-          'format' => exporter.format, 'status' => 'queued',
-          'processed' => 0, 'total' => nil,
-          **HistoryAccess.authority,
-          'expires_at' => (Time.current + CountsExportDownload::EXPIRY).to_f }
-      write(state)
-      verifier.generate({ 'id' => id, 'owner' => owner },
-          expires_in: CountsExportDownload::EXPIRY, purpose: 'counts_export_job')
-    end
-  end
-
-  def self.authorised_id(token, owner)
-    value = verifier.verified(token, purpose: 'counts_export_job')
-    return unless value.is_a?(Hash) && value['owner'] == owner
-    return unless value['id']&.match?(/\A[a-f0-9]{48}\z/)
-
-    value['id']
-  end
-
-  def self.status(token, owner, cancel: false)
-    id = authorised_id(token, owner)
-    return unless id
-
-    synchronise do
-      state = read(id)
-      return unless state && state['owner'] == owner && state['expires_at'] > Time.current.to_f
-
-      return unless HistoryAccess.authorised_artifact?(state)
-
-      if cancel && ['queued', 'running', 'ready'].include?(state['status'])
-        state['status'] = state['status'] == 'running' ? 'cancelling' : 'cancelled'
-        remove_download(state)
-        state.delete('token')
-        write(state)
-      end
-      state.slice('status', 'processed', 'total', 'format', 'token', 'error')
-    end
-  end
-
-  def self.checkpoint(id, processed = nil, total = nil)
-    synchronise do
-      state = read(id)
-      unless state && state['status'] == 'running' && state['expires_at'] > Time.current.to_f
-        raise Cancelled
-      end
-      if processed
-        state.merge!('processed' => processed, 'total' => total)
-        write(state)
-      end
-    end
-  end
-
-  def self.work_one
-    FileUtils.mkdir_p(directory, mode: 0o700)
-    Dir.glob(directory.join('*.json')).sort.each do |path|
-      id = File.basename(path, '.json')
-      File.open(directory.join("#{ id }.run"), File::RDWR | File::CREAT, 0o600) do |lock|
-        next unless lock.flock(File::LOCK_EX | File::LOCK_NB)
-
-        state = synchronise do
-          candidate = read(id)
-          next unless candidate && ACTIVE.include?(candidate['status'])
-
-          if candidate['status'] != 'queued'
-            candidate['status'] = candidate['status'] == 'cancelling' ? 'cancelled' : 'failed'
-            remove_partial(id)
-            candidate['error'] = 'interrupted'
-            write(candidate)
-            next
-          end
-          candidate['status'] = 'running'
-          write(candidate)
-          candidate
-        end
-        next unless state
-
-        perform(state)
-        return true
-      end
-    end
-    synchronise { clean }
-    false
-  end
-
-  def self.perform(state)
-    actor = state['actor_id'] && User.find_by(id: state['actor_id'])
-    actor = nil unless state['admin'] && actor&.admin? && !actor.banned?
-    HistoryAccess.set(user: actor) do
-      unless HistoryAccess.authorised_artifact?(state)
-        synchronise do
-          current = read(state.fetch('id'))
-          current['status'] = 'failed'
-          write(current)
-        end
-        return
-      end
-      perform_authorised(state)
-    end
-  end
-
-  def self.perform_authorised(state)
-    id = state.fetch('id')
-    token = nil
-    begin
-      checkpoint(id)
-      exporter = CountsExporter.new(state.fetch('payload'), language_code: state.fetch('language'))
-      last = Process.clock_gettime(Process::CLOCK_MONOTONIC)
-      progress = lambda do |processed, total|
-        now = Process.clock_gettime(Process::CLOCK_MONOTONIC)
-        if processed.zero? || now - last >= 0.25 || processed == total
-          checkpoint(id, processed, total)
-          last = now
-        end
-      end
-      check = lambda do
-        now = Process.clock_gettime(Process::CLOCK_MONOTONIC)
-        if now - last >= 0.25
-          checkpoint(id)
-          last = now
-        end
-      end
-      token = CountsExportDownload.prepare(exporter, state.fetch('owner'),
-          progress:, check:, identifier: id)
-      synchronise do
-        current = read(id)
-        if (!current || current['status'] != 'running' ||
-            current['expires_at'] <= Time.current.to_f)
-          file = CountsExportDownload.find(token, state.fetch('owner'))
-          File.unlink(file.fetch('path')) if file
-          raise Cancelled
-        end
-        file = CountsExportDownload.find(token, state.fetch('owner'))
-        current.merge!('status' => 'ready', 'token' => token,
-            'download_file' => File.basename(file.fetch('path')))
-        current['total'] = current['processed']
-        write(current)
-      end
-    rescue StandardError => error
-      remove_partial(id)
-      synchronise do
-        current = read(id)
-        if current
-          current['status'] = error.is_a?(Cancelled) ? 'cancelled' : 'failed'
-          current['error'] = error.class.name
-          write(current)
-        end
-      end
-      unless error.is_a?(Cancelled)
-        Rails.logger.error("counts_export_job #{ id }: #{ error.class }")
-      end
-    end
-  end
-end

backend/app/services/counts_export_projection.rb

@@ -1,265 +0,0 @@
-require 'set'
-
-# Export identifiers are stable; UI translations are supplied separately as column labels.
-class CountsExportProjection
-  ROOT_KEYS = { 'event_type' => 'event_type', 'resource_id' => 'resource_id', 'version_no' => 'version_no',
-                'is_latest' => 'is_latest', 'thumbnail' => 'thumbnail_url',
-                'thumbnail_base' => 'thumbnail_base', 'title' => 'title', 'url' => 'url',
-                'tags' => 'tags', 'parents' => 'parents', 'video_ms' => 'video_ms',
-                'original_created_from' => 'original_created_from',
-                'original_created_before' => 'original_created_before',
-                'user' => 'created_by_user', 'created_at' => 'created_at',
-                'name' => 'name', 'names' => 'names', 'category' => 'category',
-                'aliases' => 'aliases', 'deprecated' => 'deprecated' }.freeze
-  CHILD_KEYS = { 'id' => 'id', 'versionNo' => 'version_no', 'kind' => 'kind',
-                 'name' => 'name', 'category' => 'category', 'sections' => 'sections',
-                 'names' => 'names', 'aliases' => 'aliases',
-                 'languageCode' => 'language', 'scriptCode' => 'script',
-                 'autoGenerated' => 'auto_generated' }.freeze
-  NAME_FIELDS = ['languageCode', 'scriptCode', 'name', 'aliases', 'autoGenerated'].freeze
-  CHILDREN = {
-      'tags' => ['id', 'versionNo', 'kind', 'name', 'category', 'sections', 'names'],
-      'names' => NAME_FIELDS,
-      'aliases' => ['languageCode', 'name', 'scriptCode'],
-      'parents' => ['id', 'name'] }.freeze
-
-  def self.default_columns(resource)
-    fields = { 'posts' => ['title', 'url', 'tags', 'parents', 'video_ms'],
-        'tags' => ['name', 'category', 'aliases', 'parents'],
-        'nico_tags' => ['name', 'linked_tags'],
-        'tag_descriptions' => ['language_code', 'body', 'source_language_code'],
-        'tag_groups' => ['name', 'description', 'members'],
-        'materials' => ['url', 'tag_id', 'source_kind'] }.fetch(resource)
-    ['resource_id', 'version_no'] + fields + ['event_type', 'user', 'created_at']
-  end
-
-  def self.default_children(resource)
-    { 'posts' => ['tags.name', 'parents.id'],
-        'tags' => ['aliases.name', 'parents.id'],
-        'tag_groups' => ['members.tagId'] }.fetch(resource, [])
-  end
-
-  attr_reader :columns, :headers
-
-  def initialize(params, resource:, summary:, tags:)
-    if params.key?('definition')
-      params = CountsDefaults.for(params.merge('resource' => resource)).merge(params)
-    end
-    @params = params
-    @labels = JSON.parse(params.fetch('labels', '{}'))
-    raise ArgumentError unless @labels.is_a?(Hash) && @labels.length <= 100
-    raise ArgumentError unless @labels.all? { |key, value| valid_label?(key, value) }
-    @summary = summary
-    allowed = if summary
-                tags ? ['kind', 'tag_id', 'name', 'deprecated', 'count'] : ['group', 'count']
-              else
-                CountsQuery::COLUMNS.fetch(resource)
-              end
-    defaults = summary ? allowed : self.class.default_columns(resource)
-    @columns = QueryState.list(params.fetch('columns', defaults.join(' ')))
-    raise ArgumentError if (@columns - allowed).any? || @columns.uniq != @columns
-
-    @headers = JSON.parse(params.fetch('headers', JSON.generate(@columns)))
-    unless (@headers.is_a?(Array) && @headers.length == @columns.length &&
-        @headers.all? { _1.is_a?(String) && _1.length <= 200 })
-      raise ArgumentError
-    end
-    @order = QueryState.list(params.fetch('order', ''))
-    @sorts = QueryState.list(params.fetch('child_sort', '')).map do |entry|
-      parts = entry.split(':', -1)
-      raise ArgumentError unless parts.length == 2
-
-      path, direction = parts
-      raise ArgumentError unless ['asc', 'desc'].include?(direction)
-
-      [path, direction]
-    end
-    @selected = params.key?('children') ? QueryState.list(params['children']).to_set :
-        self.class.default_children(resource).to_set
-    valid = CHILDREN.flat_map do |key, fields|
-      paths(fields, key)
-    end
-    raise ArgumentError if @selected && (@selected.to_a - valid).any?
-    prune = lambda do |fields, prefix|
-      fields.each do |key|
-        path = "#{ prefix }.#{ key }"
-        children = fields_for(key)
-        next if children.empty?
-
-        prune.call(children, path)
-        @selected.delete(path) unless children.any? { @selected.include?("#{ path }.#{ _1 }") }
-      end
-    end
-    CHILDREN.each { |key, fields| prune.call(fields, key) }
-    pairs = @columns.zip(@headers).select do |key, _header|
-      !CHILDREN.key?(key) || CHILDREN[key].any? { @selected.include?("#{ key }.#{ _1 }") }
-    end
-    @columns = pairs.map(&:first)
-    @headers = pairs.map(&:last)
-    allowed_sorts = valid.map do |path|
-      parts = path.split('.')
-      field = parts.pop
-      "#{ parts.join('.') }/#{ field }"
-    end
-    if ((@sorts.map(&:first) - allowed_sorts).any? ||
-        @sorts.map(&:first).uniq.length != @sorts.length)
-      raise ArgumentError
-    end
-  end
-
-  def metadata
-    @columns.each_with_index.map do |key, index|
-      { key: @summary ? key : ROOT_KEYS.fetch(key), label: @headers[index] }
-    end
-  end
-
-  def project(row)
-    @columns.to_h do |key|
-      if @summary
-        value = key == 'group' ? { key: row[:key], name: row[:name] } : row[key.to_sym]
-        [key, value]
-      else
-        value = case key
-                when 'thumbnail' then row[:thumbnail_base]
-                else row[ROOT_KEYS.fetch(key).to_sym]
-                end
-        value = nested(value || [], CHILDREN.fetch(key), key) if CHILDREN.key?(key)
-        [ROOT_KEYS.fetch(key), value]
-      end
-    end
-  end
-
-  def csv_record(record, row)
-    record.map do |key, value|
-      key == 'group' ? row[:name] || row[:key] : csv_display(value, key)
-    end
-  end
-
-  private
-
-  def valid_label?(key, value)
-    key.is_a?(String) && value.is_a?(String) && key.length <= 100 && value.length <= 300
-  end
-
-  def csv_display(value, key)
-    return @labels.fetch('bot', '') if key == 'created_by_user' && value.nil?
-    return @labels.fetch("event_type:#{ value }", value) if key == 'event_type'
-    return value ? @labels.fetch('latest', 'true') : '' if key == 'is_latest'
-    return @labels.fetch("kind:#{ value }", value) if key == 'kind'
-    return @labels.fetch("category:#{ value }", value) if key == 'category'
-    if ['language', 'script'].include?(key) && value.is_a?(Hash)
-      return @labels.fetch("#{ key }:#{ value[:code] }",
-                           value[:name] || @labels.fetch('notRegistered', ''))
-    end
-    return @labels.fetch("boolean:#{ value }", value) if value == true || value == false
-    return value.map { csv_display(_1, key) } if value.is_a?(Array)
-    if value.is_a?(Hash)
-      return value.to_h do |field, item|
-        [@labels.fetch(field.to_s, field), csv_display(item, field.to_s)]
-      end
-    end
-
-    value
-  end
-
-  def fields_for(key)
-    return NAME_FIELDS if key == 'names'
-    return ['name', 'scriptCode'] if key == 'aliases'
-
-    []
-  end
-
-  def paths(fields, prefix)
-    fields.flat_map do |key|
-      path = "#{ prefix }.#{ key }"
-      [path, *paths(fields_for(key), path)]
-    end
-  end
-
-  def fetch(row, key) = row.key?(key) ? row[key] : row[key.to_sym]
-
-  def named_code(row, kind)
-    code = fetch(row, "#{ kind }_code")
-    name = fetch(row, "#{ kind }_name")
-    { code:, name: name.present? && name.to_s.downcase != code.to_s.downcase ? name : nil }
-  end
-
-  def stable(value)
-    case value
-    when Hash
-      value.stringify_keys.sort.to_h.transform_values { stable(_1) }
-    when Array
-      value.map { stable(_1) }
-    else
-      value
-    end
-  end
-
-  def sort_value(row, key)
-    if key == 'languageCode'
-      language = named_code(row, 'language')
-      return @labels.fetch("language:#{ language[:code] }", language[:name])
-    end
-    return named_code(row, 'script')[:name] if key == 'scriptCode'
-
-    fetch(row, CHILD_KEYS.fetch(key))
-  end
-
-  def comparison(left, right)
-    if left.is_a?(Array) && right.is_a?(Array) && left.length != right.length
-      return left.length <=> right.length
-    end
-    return left <=> right if left.is_a?(Numeric) && right.is_a?(Numeric)
-    if [true, false].include?(left) && [true, false].include?(right)
-      return (left ? 1 : 0) <=> (right ? 1 : 0)
-    end
-
-    a = left.is_a?(Hash) || left.is_a?(Array) ? JSON.generate(stable(left)) : left.to_s
-    b = right.is_a?(Hash) || right.is_a?(Array) ? JSON.generate(stable(right)) : right.to_s
-    a <=> b
-  end
-
-  def sorted(rows, prefix, fields)
-    sorts = @sorts.filter_map do |path, direction|
-      parent, key = path.split('/', 2)
-      next unless parent == prefix
-      raise ArgumentError unless fields.include?(key)
-
-      [key, direction]
-    end
-    rows.each_with_index.sort do |(left, a), (right, b)|
-      result = 0
-      sorts.each do |key, direction|
-        first = sort_value(left, key)
-        last = sort_value(right, key)
-        result = if first.nil? || last.nil?
-                   first.nil? == last.nil? ? 0 : (first.nil? ? 1 : -1)
-                 else
-                   comparison(first, last) * (direction == 'asc' ? 1 : -1)
-                 end
-        break unless result.zero?
-      end
-      result.zero? ? a <=> b : result
-    end.map(&:first)
-  end
-
-  def nested(rows, fields, prefix)
-    keys = fields.select { !@selected || @selected.include?("#{ prefix }.#{ _1 }") }
-    ordered = (@order + keys.map { "#{ prefix }.#{ _1 }" }).uniq
-        .filter_map { |path| keys.find { "#{ prefix }.#{ _1 }" == path } }
-    sorted(rows, prefix, fields).map do |row|
-      ordered.to_h do |key|
-        stable = CHILD_KEYS.fetch(key)
-        value = case key
-                when 'languageCode' then named_code(row, 'language')
-                when 'scriptCode' then named_code(row, 'script')
-                else fetch(row, stable)
-                end
-        if fields_for(key).any?
-          value = nested(value || [], fields_for(key), "#{ prefix }.#{ key }")
-        end
-        [stable, value]
-      end
-    end
-  end
-end

backend/app/services/counts_exporter.rb

@@ -1,152 +0,0 @@
-require 'csv'
-
-# One read view and a fixed point for the entire download; only one page is held in memory.
-class CountsExporter
-  EXTRA_KEYS = ['basis', 'format', 'resource', 'headers', 'children', 'order',
-                'summary_columns', 'nested_json', 'child_sort', 'labels'].freeze
-  attr_reader :format, :resource
-
-  def initialize(params, language_code:)
-    if params.key?('definition')
-      @table_export = CountsTableExport.new(params)
-      @format = @table_export.format
-      @resource = @table_export.resource
-      return
-    end
-    allowed = (CountsQuery::KEYS + CountsStateQuery::KEYS + EXTRA_KEYS).uniq
-    QueryState.validate!(params, allowed)
-    raise ArgumentError unless params.values.all? { _1.is_a?(String) }
-
-    @params = params
-    @language = language_code
-    @format = params.fetch('format', 'json')
-    @resource = params.fetch('resource', 'posts')
-    @mode = params.fetch('mode', 'summary')
-    @history = params.fetch('basis', 'state') == 'history'
-    raise ArgumentError unless ['json', 'csv'].include?(format)
-    raise ArgumentError unless ['posts', 'tags'].include?(resource)
-    raise ArgumentError unless ['summary', 'details'].include?(@mode)
-    raise ArgumentError unless ['state', 'history'].include?(params.fetch('basis', 'state'))
-
-    @query_params = params.except(*EXTRA_KEYS, 'columns', 'page', 'limit')
-        .merge('mode' => @mode, 'page' => '1', 'limit' => '100')
-    point = CountsStateQuery.new(@query_params.slice('at').merge('resource' => resource),
-                                 language_code: @language).at
-    @query_params['at'] = point.iso8601(6)
-    @query_params['axis'] ||= !@history && resource == 'posts' ? 'tags' : 'month'
-    @projection = CountsExportProjection.new(params, resource:,
-                                             summary: @mode == 'summary',
-                                             tags: !@history && @query_params['axis'] == 'tags')
-    if @history
-      @query_params.delete('kind')
-      history_query
-    else
-      @query_params.delete('latest')
-      @state_query = CountsStateQuery.new(@query_params.merge('resource' => resource),
-                                          language_code: @language).validate_query!
-    end
-  end
-
-  def body(progress: nil)
-    return @table_export.body(progress:) if @table_export
-
-    Enumerator.new do |output|
-      ApplicationRecord.transaction(isolation: :repeatable_read) do
-        processed = 0
-        first_row = true
-        started = false
-        pages do |page|
-          total = @mode == 'details' ? page[:count] : nil
-          progress&.call(processed, total)
-          unless started
-            meta = { resource:, mode: @mode, basis: @history ? 'history' : 'state',
-                     at: @query_params['at'], boundary: 'inclusive',
-                     filters: @query_params.except('sort', 'page', 'limit', 'mode'),
-                     sort: export_sort,
-                     child_sort: sort_metadata(@params.fetch('child_sort', '')),
-                     columns: @projection.metadata,
-                     children: @params['children']&.split(' '),
-                     field_order: @QueryState.list(params.fetch('order', '')), timezone: 'UTC',
-                     count: @mode == 'summary' ? page[:group_count] : page[:count],
-                     totals: page[:totals], coverage: page[:coverage] }
-            output << (format == 'json' ? "{\"metadata\":#{ meta.to_json },\"data\":[" : "\uFEFF")
-            if format == 'csv'
-              output << CSV.generate_line(
-                  @projection.headers.map { csv_value(_1) }, row_sep: "\r\n")
-            end
-            started = true
-          end
-          (page[:versions] || page[:state_rows] || page[:groups] || []).each do |row|
-            record = @projection.project(row)
-            if format == 'json'
-              output << ',' unless first_row
-              output << record.to_json
-            else
-              values = @projection.csv_record(record, row).map { csv_value(_1) }
-              output << CSV.generate_line(values, row_sep: "\r\n")
-            end
-            first_row = false
-            processed += 1
-            progress&.call(processed, total)
-          end
-        end
-        progress&.call(processed, processed)
-        output << ']}' if format == 'json'
-      end
-    end
-  end
-
-  private
-
-  def export_sort
-    sort_metadata(@query_params.fetch('sort', default_sort))
-  end
-
-  def sort_metadata(value)
-    value.split(' ').map do |entry|
-      key, direction = entry.split(':', 2)
-      { key:, direction: }
-    end
-  end
-
-  def default_sort
-    return 'created_at:desc' if @mode == 'details'
-
-    !@history && @query_params['axis'] == 'tags' ? 'count:desc' : 'group:asc'
-  end
-
-  def history_query
-    scope = resource == 'posts' ? PostVersion.with_previous_version : TagVersion.all
-    CountsQuery.new(scope, resource, @query_params)
-  end
-
-  def pages
-    unless @history
-      @state_query.each_page { yield _1 }
-      return
-    end
-    query = history_query
-    total = @mode == 'summary' ? nil : query.count
-    loop do
-      result = if @mode == 'summary'
-                 query.summary(include_count: @query_params['page'] == '1')
-               else
-                 { count: total,
-                   versions: CountsVersionRepr.many(query.details, resource:,
-                                                   language_code: @language,
-                                                   at: Time.iso8601(@query_params['at'])) }
-               end
-      total ||= result[:group_count]
-      yield result
-      break if @query_params['page'].to_i * 100 >= total
-
-      @query_params['page'] = (@query_params['page'].to_i + 1).to_s
-      query = history_query
-    end
-  end
-
-  def csv_value(value)
-    text = value.is_a?(Hash) || value.is_a?(Array) ? JSON.generate(value) : value.to_s
-    text.match?(/\A\s*[=+@-]/) ? "'#{ text }" : text
-  end
-end

backend/app/services/counts_pivot_layout.rb

@@ -1,28 +0,0 @@
-# Coordinates carry their aggregation depth; SQL NULL remains an ordinary axis value.
-class CountsPivotLayout
-  def self.depths(keys, sorts, measures: true)
-    return [] if keys.length < 2 || sorts.first&.first == 'count' ||
-        sorts.first&.first&.match?(/\Am[0-9]+\z/)
-
-    relevant = sorts.map(&:first).select { keys.include?(_1) || (measures && (_1 == 'count' ||
-        _1.match?(/\Am[0-9]+\z/))) }
-    order = (relevant + keys).uniq
-    (1...keys.length).select { order.first(_1).sort == keys.first(_1).sort }
-  end
-
-  def self.headers(values, depths, next_values: nil, grand: false)
-    result = []
-    values.each_with_index do |tuple, index|
-      result << { values: tuple, kind: 'data' }
-      following = values[index + 1] || next_values
-      depths.reverse_each do |depth|
-        prefix = tuple.first(depth)
-        next if following && following.first(depth) == prefix
-
-        result << { values: prefix, kind: 'subtotal' }
-      end
-    end
-    result << { values: [], kind: 'grand_total' } if grand
-    result
-  end
-end

backend/app/services/counts_pivot_totals.rb

@@ -1,59 +0,0 @@
-# Re-aggregate the population, never add up already grouped cells (multi-value axes overlap).
-class CountsPivotTotals
-  def initialize(population, axes, row_headers, column_headers)
-    @population = population
-    @axes = axes
-    @rows = row_headers.group_by { _1[:values].length }
-    @columns = column_headers.group_by { _1[:values].length }
-  end
-
-  def call
-    queries = @rows.flat_map do |row_depth, headers|
-      @columns.filter_map do |column_depth, columns|
-        next if row_depth == @axes.rows.length && column_depth == @axes.columns.length
-
-        indexes = (0...row_depth).to_a +
-            (0...column_depth).map { @axes.rows.length + _1 }
-        expressions = indexes.map { @axes.expressions[_1] }
-        scope = @population.scope.except(:select, :order)
-        joins = @axes.joins(indexes.map { @axes.all[_1].fetch('field') })
-        scope = scope.joins(joins) unless joins.empty?
-        if row_depth.positive?
-          tuples = headers.map { _1[:values] }.uniq
-          predicate = tuples.map { Array.new(row_depth) { |i|
-            "(#{ expressions[i] }) <=> ?"
-          }.join(' AND ') }.map { "(#{ _1 })" }.join(' OR ')
-          scope = scope.where(predicate, *tuples.flatten)
-        end
-        if column_depth.positive?
-          tuples = columns.map { _1[:values] }.uniq
-          predicate = tuples.map { Array.new(column_depth) { |i|
-            "(#{ expressions[row_depth + i] }) <=> ?"
-          }.join(' AND ') }.map { "(#{ _1 })" }.join(' OR ')
-          scope = scope.where(predicate, *tuples.flatten)
-        end
-        grouped = expressions.each_with_index.map { |expression, index|
-          "#{ expression } AS v#{ index }"
-        }
-        # Fixed-width result coordinates, independent of NULL and of translated labels.
-        padding = (expressions.length...@axes.all.length).map { "NULL AS v#{ _1 }" }
-        fields = ["#{ row_depth } AS row_depth", "#{ column_depth } AS column_depth"] +
-            grouped + padding + ['COUNT(DISTINCT v.id) AS count']
-        scope = scope.group(*expressions.map { Arel.sql(_1) }) if expressions.any?
-        scope.optimizer_hints('MAX_EXECUTION_TIME(10000)')
-            .select(Arel.sql(fields.join(', '))).to_sql
-      end
-    end
-    return [] if queries.empty?
-
-    sql = queries.map { "(#{ _1 })" }.join(' UNION ALL ')
-    @population.definition.model.connection.select_all(
-        "SELECT /*+ MAX_EXECUTION_TIME(10000) */ * FROM (#{ sql }) totals").map do |record|
-      row_depth = record['row_depth'].to_i
-      column_depth = record['column_depth'].to_i
-      { row: (0...row_depth).map { record["v#{ _1 }"] },
-        column: (0...column_depth).map { record["v#{ row_depth + _1 }"] },
-        count: record['count'].to_i }
-    end
-  end
-end

backend/app/services/counts_population.rb

@@ -1,168 +0,0 @@
-# Population first, then filters. Previous versions are joined before period filtering.
-class CountsPopulation
-  attr_reader :definition, :params, :at, :history, :scope, :timezone
-
-  def initialize(params)
-    @params = params
-    @definition = CountsResourceDefinition.new(params.fetch('resource', 'posts'))
-    @history = params.fetch('definition', 'latest') == 'history'
-    raise ArgumentError unless ['latest', 'history'].include?(params.fetch('definition', 'latest'))
-
-    @timezone = TZInfo::Timezone.get(params.fetch('timezone', 'UTC'))
-    @at = timestamp(params['execution_at']) || Time.current.utc
-    raise ArgumentError if at > Time.current + 1.second
-
-    @scope = definition.model.all
-    make_population
-    @scope = HistoryAccess.visible(@scope, 'v')
-    apply_filters
-  rescue TZInfo::InvalidTimezoneIdentifier
-    raise ArgumentError
-  end
-
-  def timestamp(value, end_date: false)
-    return nil if value.nil? || value == ''
-    raise ArgumentError unless value.is_a?(String)
-
-    if value.match?(/\A\d{4}-\d{2}-\d{2}\z/)
-      date = Date.iso8601(value)
-      date += 1 if end_date
-      timezone.local_to_utc(Time.utc(date.year, date.month, date.day))
-    else
-      raise ArgumentError unless value.match?(/(?:Z|[+-]\d{2}:\d{2})\z/)
-
-      time = Time.iso8601(value).utc
-      raise ArgumentError unless time.year.between?(1900, 2100)
-
-      time
-    end
-  rescue TZInfo::AmbiguousTime, TZInfo::PeriodNotFound, Date::Error
-    raise ArgumentError
-  end
-
-  def latest_sql(row = 'v', cutoff = nil)
-    cutoff ||= history ? at : timestamp(params['at']) || at
-    keys = definition.keys.map { "newer.#{ _1 } = #{ row }.#{ _1 }" }.join(' AND ')
-    bound = definition.model.connection.quote(cutoff)
-    <<~SQL.squish
-      NOT EXISTS (SELECT 1 FROM #{ definition.table } newer
-        WHERE #{ keys } AND newer.version_no > #{ row }.version_no
-          AND newer.#{ definition.time_column } <= #{ bound })
-    SQL
-  end
-
-  def changed_sql(key)
-    column = definition.column(key)
-    "(previous.id IS NOT NULL AND NOT (v.#{ column } <=> previous.#{ column }))"
-  end
-
-  private
-
-  def make_population
-    table = definition.table
-    @scope = @scope.from("#{ table } v").select('v.*')
-    keys = definition.keys.map { "previous.#{ _1 } = v.#{ _1 }" }.join(' AND ')
-    @scope = @scope.joins("LEFT JOIN #{ table } previous ON #{ keys } " +
-        'AND previous.version_no = v.version_no - 1 AND ' +
-        HistoryAccess.sql(definition.model, 'previous'))
-    @scope = @scope.where("v.#{ definition.time_column } <= ?", at)
-    unless history
-      point = timestamp(params['at']) || at
-      raise ArgumentError if point > at
-
-      @scope = @scope.where("v.#{ definition.time_column } <= ?", point)
-          .where(Arel.sql(latest_sql('v', point))).where.not('v.event_type' => 'discard')
-      return
-    end
-    first = timestamp(params['from'])
-    last = timestamp(params['through'], end_date: true)
-    raise ArgumentError if first && last && first > last
-
-    @scope = @scope.where('v.created_at >= ?', first) if first
-    if last
-      operator = params['through'].length == 10 ? '<' : '<='
-      @scope = @scope.where("v.created_at #{ operator } ?", last)
-    end
-  end
-
-  def apply_filters
-    ['resource_id', 'user'].each do |key|
-      value = params[key == 'user' ? 'user_id' : key]
-      next if value.nil? || value == ''
-      raise ArgumentError unless value.match?(/\A[0-9]+\z/)
-
-      @scope = @scope.where("#{ definition.expression(key) } = ?", value.to_i)
-    end
-    apply_tags if params['tags'].present? || params['tag'].present?
-    conditions = QueryState.records(params, 'conditions')
-    raise ArgumentError unless conditions.is_a?(Array) && conditions.length <= 100
-
-    conditions.each { apply_condition(_1) }
-    drills = QueryState.records(params, 'drill')
-    raise ArgumentError unless drills.is_a?(Array) && drills.length <= 100
-
-    drills.each do |entry|
-      key = entry.fetch('field')
-      if key == 'tags'
-        @scope = @scope.where(snapshot_membership(entry.fetch('value')))
-      elsif key == 'changed_field'
-        raise ArgumentError unless history && definition.changes.include?(entry['value'])
-
-        @scope = @scope.where(Arel.sql(changed_sql(entry['value'])))
-      elsif definition.fields[key] == :datetime && entry['grain']
-        @scope = @scope.where(definition.expression(key) => nil) if entry['value'].nil?
-        if entry['value']
-          expression = CountsAxes.time_expression(definition.expression(key),
-                                                  entry['grain'], timezone)
-          @scope = @scope.where("#{ expression } = ?", entry['value'])
-        end
-      else
-        raise ArgumentError unless definition.axes(history).include?(key)
-
-        @scope = @scope.where(definition.expression(key) => entry['value'])
-      end
-    end
-  end
-
-  def apply_condition(entry)
-    @condition ||= CountsCondition.new(self)
-    @scope = @condition.apply(@scope, entry)
-  end
-
-  def snapshot_membership(value)
-    kind, identifier = value.to_s.split(':', 2)
-    raise ArgumentError unless ['internal', 'external'].include?(kind)
-    raise ArgumentError unless identifier&.match?(/\A[1-9]\d*\z/)
-
-    key = kind == 'internal' ? 'tag_id' : 'external_tag_id'
-    definition.model.sanitize_sql_array([
-        "JSON_CONTAINS(v.tags_json, JSON_OBJECT('#{ key }', CAST(? AS UNSIGNED)))",
-        identifier.to_i])
-  end
-
-  def apply_tags
-    raise ArgumentError unless definition.resource == 'posts'
-
-    if params['tag'].present?
-      @scope = @scope.where(snapshot_membership(
-          "#{ params.fetch('kind', 'internal') }:#{ params['tag'] }"))
-    end
-    return unless params['tags'].present?
-
-    locale = Locale.find_by!(code: params.fetch('locale', 'ja'))
-    refs = PostTagSearch.parse_references(params, params['tags'].split)
-    raw = params['tags']
-    literals = PostTagSearch.literals(locale, raw, references: refs)
-    predicates = literals.map do |literal|
-      internal, external = PostTagSearch.references(locale, literal)
-      parts = Array(internal).map { snapshot_membership("internal:#{ _1 }") } +
-          Array(external).map { snapshot_membership("external:#{ _1 }") }
-      expression = parts.empty? ? 'FALSE' : "(#{ parts.join(' OR ') })"
-      literal[:negative] ? "NOT #{ expression }" : expression
-    end
-    return if predicates.empty?
-
-    any = params['match'] == 'any' || raw.split.any? { _1.downcase == 'type:or' }
-    @scope = @scope.where(Arel.sql("(#{ predicates.join(any ? ' OR ' : ' AND ') })"))
-  end
-end

backend/app/services/counts_query.rb

@@ -1,207 +0,0 @@
-# Read-only history queries. SQL identifiers and expressions are server-owned.
-class CountsQuery
-  AXES = ['month', 'user'].freeze
-  COMMON_COLUMNS = ['resource_id', 'version_no', 'is_latest', 'event_type', 'user', 'created_at'].freeze
-  COLUMNS = {
-      'posts' => COMMON_COLUMNS + [
-          'thumbnail', 'thumbnail_base', 'title', 'url', 'tags', 'parents', 'video_ms',
-          'original_created_from', 'original_created_before'],
-      'tags' => COMMON_COLUMNS + [
-          'name', 'names', 'category', 'aliases', 'parents', 'deprecated'] }.freeze
-  KEYS = [
-      'locale', 'mode', 'axis', 'page', 'limit', 'from', 'before', 'latest',
-      'user_id', 'resource_id', 'tag', 'columns', 'sort', 'at'].freeze
-
-  attr_reader :page, :limit, :mode, :axis
-
-  def initialize(scope, resource, params, latest_expression: nil)
-    @table = scope.klass.table_name
-    @resource = resource
-    @id_column = resource == 'posts' ? 'post_id' : 'tag_id'
-    @params = params
-    @latest_expression = latest_expression
-    scope = HistoryAccess.visible(scope)
-    @scope = scope.except(:select, :order).joins(
-        "LEFT JOIN users counts_actor ON counts_actor.id = #{ @table }.created_by_user_id")
-    raise ArgumentError unless (params.keys - KEYS).empty?
-    raise ArgumentError unless params.values.all? { _1.is_a?(String) }
-
-    @mode = params.fetch('mode', 'summary')
-    @axis = params.fetch('axis', 'month')
-    raise ArgumentError unless ['summary', 'details'].include?(mode) && AXES.include?(axis)
-
-    @page = integer('page', 1)
-    @limit = integer('limit', 20)
-    raise ArgumentError if limit > 100 || page > 1_000_000
-
-    columns = QueryState.list(params.fetch('columns', COLUMNS.fetch(resource).join(' ')))
-    if columns.empty? || columns.uniq != columns || (columns - COLUMNS.fetch(resource)).any?
-      raise ArgumentError
-    end
-    @sort = parse_sort
-    apply_filters
-  end
-
-  def relation = @scope
-
-  def count = @scope.count
-
-  def latest_expression
-    return @latest_expression if @latest_expression
-
-    at = timestamp('at')
-    cutoff = at ? "AND newer.created_at <= #{ @scope.klass.connection.quote(at) }" : ''
-    <<~SQL.squish
-      NOT EXISTS (SELECT 1 FROM #{ @table } newer
-        WHERE newer.#{ @id_column } = #{ @table }.#{ @id_column }
-          AND newer.version_no > #{ @table }.version_no #{ cutoff })
-    SQL
-  end
-
-  def details
-    @scope.select("#{ @table }.*", Arel.sql("(#{ latest_expression }) AS counts_latest"))
-        .order(Arel.sql(order_sql(detail_expressions, ["#{ @table }.id DESC"])))
-        .limit(limit).offset((page - 1) * limit)
-  end
-
-  def summary(include_count: true)
-    expression = axis == 'month' ? "DATE_FORMAT(#{ @table }.created_at, '%Y-%m')" :
-                                  "#{ @table }.created_by_user_id"
-    group_order = axis == 'user' ? 'MAX(counts_actor.name)' : expression
-    sql = @scope.group(Arel.sql(expression)).select(Arel.sql(
-        "#{ expression } AS group_key, COUNT(*) AS record_count, " +
-        "#{ group_order } AS group_name")).to_sql
-    connection = @scope.klass.connection
-    source = "FROM (#{ sql }) counts_groups"
-    groups = include_count ? connection.select_value("SELECT COUNT(*) #{ source }") : nil
-    expressions = { 'group' => ['group_name'], 'count' => ['record_count'] }
-    order = order_sql(expressions, ['group_key IS NULL ASC', 'group_key ASC'])
-    rows = connection.select_all(
-        "SELECT group_key, group_name, record_count #{ source } " +
-        "ORDER BY #{ order } LIMIT #{ limit } OFFSET #{ (page - 1) * limit }")
-    { group_count: groups&.to_i,
-      groups: rows.map do |row|
-        { key: row['group_key'],
-          name: axis == 'user' ? row['group_name'] : nil,
-          count: row['record_count'].to_i }
-      end }
-  end
-
-  private
-
-  def parse_sort
-    value = @params.fetch('sort', mode == 'details' ? 'created_at:desc' : 'group:asc')
-    return [] if value.empty?
-
-    values = QueryState.list(value).map { _1.split(':', -1) }
-    allowed = mode == 'details' ? COLUMNS.fetch(@resource) - ['thumbnail'] : ['group', 'count']
-    invalid = values.any? do |key, direction, *extra|
-      extra.any? || !allowed.include?(key) || !['asc', 'desc'].include?(direction)
-    end
-    if (values.empty? || values.length > allowed.length || invalid ||
-        values.map(&:first).uniq.length != values.length)
-      raise ArgumentError
-    end
-    values
-  end
-
-  # NULL always sorts last, including descending keys. The final key is unique.
-  def order_sql(expressions, final)
-    (@sort.flat_map do |key, direction|
-      expressions.fetch(key).flat_map do |expression|
-        ["(#{ expression }) IS NULL ASC", "#{ expression } #{ direction.upcase }"]
-      end
-    end + final).join(', ')
-  end
-
-  def detail_expressions
-    common = { 'resource_id' => ["#{ @table }.#{ @id_column }"],
-               'version_no' => ["#{ @table }.version_no"],
-               'is_latest' => ["(#{ latest_expression })"],
-               'user' => ['counts_actor.name', "#{ @table }.created_by_user_id"],
-               'created_at' => ["#{ @table }.created_at"] }
-    if @resource == 'posts'
-      fields = [
-          'title', 'url', 'thumbnail_base', 'video_ms',
-          'original_created_from', 'original_created_before']
-      common.merge(fields.to_h { [_1, ["#{ @table }.#{ _1 }"]] }).merge(
-          'tags' => json_order('post_versions.tags_json'),
-          'parents' => list_order('post_versions.parent_post_ids'))
-    else
-      name = "JSON_UNQUOTE(JSON_EXTRACT(tag_versions.names, #{ locale_name_path }))"
-      common.merge(
-          'name' => ["COALESCE(#{ name }, tag_versions.name)"],
-          'names' => json_order('tag_versions.names'),
-          'category' => ['tag_versions.category'],
-          'aliases' => json_order(
-              "JSON_EXTRACT(tag_versions.names, '$.*.aliases[*]')"),
-          'parents' => list_order('tag_versions.parent_tag_ids'),
-          'deprecated' => ['tag_versions.deprecated_at'])
-    end
-  end
-
-  def locale_name_path
-    language = Locale.find_by(code: @params.fetch('locale', 'ja'))&.language_code || 'ja'
-    path = "$.#{ JSON.generate(language) }.name"
-    @scope.klass.connection.quote(path)
-  end
-
-  # Multi-valued parent cells sort by cardinality, then binary snapshot text.
-  def json_order(column)
-    [
-        "COALESCE(JSON_LENGTH(#{ column }), 0)",
-        "COALESCE(CAST(#{ column } AS CHAR), '[]') COLLATE utf8mb4_bin"]
-  end
-
-  def list_order(column)
-    [
-        "CASE WHEN #{ column } = '' THEN 0 ELSE " +
-            "1 + LENGTH(#{ column }) - LENGTH(REPLACE(#{ column }, ' ', '')) END",
-        "#{ column } COLLATE utf8mb4_bin"]
-  end
-
-  def integer(key, default = nil)
-    value = @params[key].presence
-    return default if value.nil? && !default.nil?
-    raise ArgumentError unless value&.match?(/\A[1-9][0-9]{0,9}\z/)
-
-    value.to_i
-  end
-
-  def timestamp(key)
-    value = @params[key].presence
-    return nil unless value
-    pattern = /\A\d{4}-\d{2}-\d{2}T\d{2}:\d{2}(?::\d{2}(?:\.\d{1,6})?)?Z\z/
-    raise ArgumentError unless value.match?(pattern)
-
-    Date.iso8601(value.first(10))
-    Time.iso8601(value.length == 17 ? value.sub(/Z\z/, ':00Z') : value)
-  end
-
-  def apply_filters
-    at = timestamp('at')
-    first = timestamp('from')
-    last = timestamp('before')
-    raise ArgumentError if first && last && first >= last
-    raise ArgumentError unless ['0', '1'].include?(@params.fetch('latest', '1'))
-
-    @scope = @scope.where("#{ @table }.created_at <= ?", at) if at
-    @scope = @scope.where(Arel.sql(latest_expression)) unless @params['latest'] == '0'
-    @scope = @scope.where("#{ @table }.created_at >= ?", first) if first
-    @scope = @scope.where("#{ @table }.created_at < ?", last) if last
-    if @params['resource_id'].present?
-      @scope = @scope.where(@table => { @id_column => integer('resource_id') })
-    end
-    if @params['user_id'].present?
-      user = @params['user_id'] == 'none' ? nil : integer('user_id')
-      @scope = @scope.where(@table => { created_by_user_id: user })
-    end
-    return unless @params['tag'].present?
-    raise ArgumentError unless @resource == 'posts'
-
-    id = integer('tag')
-    @scope = @scope.where(
-        "JSON_CONTAINS(post_versions.tags_json, JSON_OBJECT('tag_id', CAST(? AS UNSIGNED))) OR " +
-        "JSON_CONTAINS(prev.tags_json, JSON_OBJECT('tag_id', CAST(? AS UNSIGNED)))", id, id)
-  end
-end

backend/app/services/counts_resource_definition.rb

@@ -1,146 +0,0 @@
-# Server-owned version identities, attributes and field types. No client SQL is accepted.
-class CountsResourceDefinition
-  COMMON = { 'resource_id' => :integer, 'version_no' => :integer,
-             'event_type' => :enum, 'user' => :integer, 'created_at' => :datetime }.freeze
-  DEFINITIONS = {
-      'posts' => [PostVersion, ['post_id'], {
-          'title' => :string, 'url' => :string, 'thumbnail_base' => :string,
-          'tags_json' => :json, 'parent_post_ids' => :string, 'video_ms' => :integer,
-          'original_created_from' => :datetime, 'original_created_before' => :datetime }],
-      'tags' => [TagVersion, ['tag_id'], {
-          'name' => :string, 'names' => :json, 'category' => :enum, 'aliases' => :string,
-          'parent_tag_ids' => :string, 'deprecated_at' => :datetime }],
-      'nico_tags' => [NicoTagVersion, ['tag_id'], {
-          'name' => :string, 'linked_tags' => :string }],
-      'tag_descriptions' => [TagDescriptionVersion, ['tag_id', 'language_code'], {
-          'language_code' => :enum, 'body' => :string,
-          'source_language_code' => :enum, 'source_version_no' => :integer }],
-      'tag_groups' => [TagGroupVersion, ['tag_group_id'], {
-          'name' => :string, 'description' => :string,
-          'members' => :json, 'placements' => :json }],
-      'materials' => [MaterialVersion, ['material_id'], {
-          'url' => :string, 'parent_id' => :integer, 'tag_id' => :integer,
-          'tag_name' => :string, 'tag_category' => :enum,
-          'discarded_at' => :datetime, 'updated_at' => :datetime,
-          'updated_by_user_id' => :integer, 'export_paths_json' => :json,
-          'file_blob_id' => :integer, 'file_filename' => :string,
-          'file_content_type' => :string, 'file_byte_size' => :integer,
-          'file_checksum' => :string, 'file_sha256' => :string,
-          'source_kind' => :enum, 'source_uri' => :string, 'source_path' => :string,
-          'source_file_id' => :string, 'normalized_source_key' => :string }] }.freeze
-  OPERATORS = {
-      string: ['eq', 'ne', 'like'], enum: ['eq', 'ne'], boolean: ['eq', 'ne'],
-      integer: ['eq', 'ne', 'gt', 'lt', 'ge', 'le', 'range'],
-      datetime: ['eq', 'ne', 'gt', 'lt', 'ge', 'le', 'range'],
-      interval: ['range'] }.freeze
-  LABELS = {
-      'resource_id' => 'counts:columns.resourceId', 'version_no' => 'counts:columns.version',
-      'is_latest' => 'counts:columns.latest', 'event_type' => 'counts:eventType',
-      'user' => 'history:actor', 'created_at' => 'counts:columns.changedAt',
-      'thumbnail' => 'counts:columns.thumbnailDisplay',
-      'thumbnail_base' => 'counts:columns.thumbnailURL',
-      'title' => 'counts:columns.postTitle', 'url' => 'counts:columns.postURL',
-      'tags' => 'counts:columns.tags', 'tags_json' => 'counts:columns.tags',
-      'video_ms' => 'shared:labels.duration', 'name' => 'counts:columns.name',
-      'names' => 'counts:columns.names', 'category' => 'common:categoryLabel',
-      'aliases' => 'tags:aliases', 'deprecated' => 'counts:columns.deprecated',
-      'original_created_from' => 'counts:columns.originalFrom',
-      'original_created_before' => 'counts:columns.originalBefore',
-      'language_code' => 'counts:columns.language' }.freeze
-  CHILD_LABELS = {
-      'id' => 'counts:columns.resourceId', 'versionNo' => 'history:version',
-      'kind' => 'counts:columns.kind', 'name' => 'history:name',
-      'category' => 'common:categoryLabel', 'sections' => 'counts:columns.sections',
-      'names' => 'counts:columns.names', 'aliases' => 'tags:aliases',
-      'languageCode' => 'counts:columns.language',
-      'scriptCode' => 'history:nameChanges.script',
-      'autoGenerated' => 'history:nameChanges.automatic',
-      'tagId' => 'counts:attributes.tag_id', 'tagGroupId' => 'counts:resources.tag_groups',
-      'sectionName' => 'counts:attributes.section_name',
-      'description' => 'counts:attributes.description',
-      'children' => 'counts:attributes.children', 'profile' => 'counts:attributes.profile',
-      'path' => 'counts:attributes.path' }.freeze
-
-  attr_reader :model, :keys, :fields, :resource
-
-  def initialize(resource)
-    @resource = resource
-    @model, @keys, attributes = DEFINITIONS.fetch(resource) { raise ArgumentError }
-    @fields = COMMON.merge(attributes)
-  end
-
-  def table = model.table_name
-  def time_column = 'created_at'
-  def column(key)
-    raise ArgumentError unless fields.key?(key)
-
-    { 'resource_id' => keys.first, 'user' => 'created_by_user_id' }.fetch(key, key)
-  end
-
-  def expression(key, row = 'v') = "#{ row }.#{ column(key) }"
-  def changes = fields.keys - COMMON.keys - ['updated_at', 'updated_by_user_id']
-  def axes(history)
-    fields.reject { |_key, type| type == :json }.keys +
-        (resource == 'posts' ? ['tags'] : []) + (history ? ['changed_field'] : [])
-  end
-
-  def metadata
-    fields.map do |key, type|
-      { key:, type:, nullable: model.columns_hash.fetch(column(key)).null,
-        operators: OPERATORS.fetch(type, []) }
-    end
-  end
-
-  def field_label(key) = LABELS.fetch(key, "counts:attributes.#{ key }")
-
-  def output_fields
-    content = case resource
-              when 'posts'
-                ['thumbnail', 'title', 'url', 'thumbnail_base', 'tags', 'parents', 'video_ms',
-                 'original_created_from', 'original_created_before']
-              when 'tags' then ['name', 'names', 'category', 'aliases', 'parents', 'deprecated']
-              else fields.keys - COMMON.keys
-              end
-    keys = ['resource_id', 'version_no', 'is_latest'] + (content - ['thumbnail']) +
-        ['event_type', 'user', 'created_at']
-    keys.unshift('thumbnail') if content.include?('thumbnail')
-    keys.map do |key|
-      label = if key == 'parents'
-                resource == 'tags' ? 'tags:parents' : 'shared:labels.parentPosts'
-              else field_label(key)
-              end
-      field = { key:, label: }
-      children = child_fields(key)
-      field[:children] = children if children.any?
-      field
-    end
-  end
-
-  def child_fields(key, depth = 4)
-    keys = case key
-           when 'tags' then ['id', 'versionNo', 'kind', 'name', 'category', 'sections', 'names']
-           when 'names' then ['languageCode', 'scriptCode', 'name', 'aliases', 'autoGenerated']
-           when 'aliases'
-             resource == 'tags' ? ['languageCode', 'name', 'scriptCode'] : ['name', 'scriptCode']
-           when 'parents' then resource == 'tags' ? ['id', 'name'] : ['id']
-           when 'members' then ['tagId', 'tagGroupId', 'versionNo']
-           when 'export_paths_json' then ['profile', 'path']
-           when 'placements', 'children'
-             scalar = ['sectionName', 'description', 'tagId', 'tagGroupId']
-             scalar + (depth > 0 ? ['children'] : [])
-           else []
-           end
-    keys.map do |child|
-      label = key == 'members' && child == 'tagId' ?
-          'counts:columns.resourceId' : CHILD_LABELS.fetch(child)
-      field = { key: child, label: }
-      children = child_fields(child, child == 'children' ? depth - 1 : depth)
-      # Language is attached to top-level aliases, not to aliases inside a language entry.
-      children = children.reject { _1[:key] == 'languageCode' } if child == 'aliases'
-      field[:children] = children if children.any?
-      field
-    end
-  end
-
-  def filter_metadata = @filter_metadata ||= CountsCondition.fields(self)
-end

backend/app/services/counts_state_query.rb

@@ -1,370 +0,0 @@
-# Restores one recorded version per resource before applying record filters.
-class CountsStateQuery
-  KEYS = ['locale', 'resource', 'mode', 'at', 'from', 'before', 'grain',
-          'kind', 'tag', 'page', 'limit', 'sort', 'axis', 'user_id', 'resource_id'].freeze
-  MAX_PERIODS = 366
-
-  attr_reader :at
-
-  def initialize(params, language_code:)
-    raise ArgumentError unless (params.keys - KEYS).empty?
-    raise ArgumentError unless params.values.all? { _1.is_a?(String) }
-
-    @params = params
-    @connection = ApplicationRecord.connection
-    @language = language_code
-    @resource = params.fetch('resource', 'posts')
-    @mode = params.fetch('mode', 'summary')
-    @inclusive = ['summary', 'details'].include?(@mode)
-    @axis = params.fetch('axis', @resource == 'posts' ? 'tags' : 'month')
-    @kind = params.fetch('kind', 'internal')
-    @grain = params.fetch('grain', 'month')
-    raise ArgumentError unless ['posts', 'tags'].include?(@resource)
-    raise ArgumentError unless ['summary', 'details', 'state', 'trend', 'snapshot'].include?(@mode)
-    raise ArgumentError unless ['internal', 'external', 'all'].include?(@kind)
-    raise ArgumentError unless ['day', 'month', 'year'].include?(@grain)
-
-    raise ArgumentError unless ['tags', 'month', 'user'].include?(@axis)
-    if @mode == 'summary' && @axis == 'tags' && @resource != 'posts'
-      raise ArgumentError
-    end
-
-    @page = integer('page', 1)
-    @limit = integer('limit', 20)
-    @tag = integer('tag') if params['tag'].present?
-    raise ArgumentError if @limit > 100 || @page > 1_000_000
-    raise ArgumentError if @tag && @kind == 'all'
-    raise ArgumentError if @mode == 'trend' && !@tag
-
-    @now = Time.current.utc
-    @at = timestamp('at') || @now
-    raise ArgumentError if @at > @now
-  end
-
-  def call(metadata: true)
-    @metadata = metadata
-    # Keep totals, rows, and detail pages on one repeatable-read database snapshot.
-    ApplicationRecord.transaction do
-      unless @history_loaded
-        @post_start = HistoryAccess.visible(PostVersion.all).minimum(:created_at)
-        @tag_start = HistoryAccess.visible(TagVersion.all).minimum(:created_at)
-        @history_loaded = true
-      end
-      result = { population_count: HistoryAccess.population_count(
-          @resource == 'posts' ? PostVersion : TagVersion), page: @page, limit: @limit, at: @at.iso8601(6),
-                 coverage: { basis: 'recorded_snapshots', exact: false,
-                             post_history_from: @post_start&.iso8601(6),
-                             tag_history_from: @tag_start&.iso8601(6) } }
-      case @mode
-      when 'trend'
-        result.merge(trend)
-      when 'summary'
-        result.merge(metadata ? { totals: totals } : {}).merge(summary)
-      when 'details', 'snapshot'
-        result.merge(metadata ? { totals: totals } : {}).merge(details)
-      else
-        result.merge(metadata ? { totals: totals } : {}).merge(listing)
-      end
-    end
-  end
-
-  def each_page
-    return enum_for(:each_page) unless block_given?
-
-    @page = 1
-    first = call
-    yield first
-    total = ['summary', 'state', 'trend'].include?(@mode) ? first[:group_count] : first[:count]
-    while @page * @limit < total
-      @page += 1
-      yield call(metadata: false)
-    end
-  end
-
-  def validate_query!
-    filtered_scope(@resource)
-    case @mode
-    when 'summary'
-      if @axis == 'tags'
-        order_sql({ 'name' => 'name', 'count' => 'post_count', 'tag_id' => 'tag_id',
-                    'kind' => 'kind', 'deprecated' => 'deprecated_at' }, 'count:desc', [])
-      else
-        record_query('summary')
-      end
-    when 'details', 'snapshot'
-      record_query('details')
-    end
-    self
-  end
-
-  private
-
-  def quote(value) = @connection.quote(value)
-
-  def integer(key, default = nil)
-    value = @params[key]
-    return default if value.blank? && default
-    raise ArgumentError unless value&.match?(/\A[1-9][0-9]{0,9}\z/)
-
-    value.to_i
-  end
-
-  def timestamp(key)
-    value = @params[key].presence
-    return nil unless value
-    pattern = /\A\d{4}-\d{2}-\d{2}T\d{2}:\d{2}(?::\d{2}(?:\.\d{1,6})?)?Z\z/
-    raise ArgumentError unless value.match?(pattern)
-
-    Date.iso8601(value.first(10))
-    Time.iso8601(value.length == 17 ? value.sub(/Z\z/, ':00Z') : value).utc
-  end
-
-  # Version numbers order equal-timestamp writes. Do not filter tags before this selection.
-  def latest_condition(table, row, boundary)
-    key = table == 'post_versions' ? 'post_id' : 'tag_id'
-    <<~SQL.squish
-      #{ row }.created_at #{ @inclusive ? '<=' : '<' } #{ boundary } AND NOT EXISTS (
-        SELECT 1 FROM #{ table } newer
-        WHERE newer.#{ key } = #{ row }.#{ key }
-          AND newer.created_at #{ @inclusive ? '<=' : '<' } #{ boundary }
-          AND newer.version_no > #{ row }.version_no)
-    SQL
-  end
-
-  def state_sql(table, boundary, live: false)
-    model = { 'post_versions' => PostVersion, 'tag_versions' => TagVersion,
-              'nico_tag_versions' => NicoTagVersion }.fetch(table)
-    sql = "SELECT v.* FROM #{ table } v WHERE #{ latest_condition(table, 'v', boundary) } " +
-        "AND (#{ HistoryAccess.sql(model, 'v') })"
-    live ? "#{ sql } AND v.event_type <> 'discard'" : sql
-  end
-
-  def membership(row)
-    key = @kind == 'external' ? 'external_tag_id' : 'tag_id'
-    "JSON_CONTAINS(#{ row }.tags_json, JSON_OBJECT('#{ key }', #{ @tag }))"
-  end
-
-  # Deprecation keeps a tag in existence. Only an explicit discard removes it.
-  def tag_visible(boundary)
-    return '1 = 1' if @kind == 'external'
-
-    <<~SQL.squish
-      NOT EXISTS (SELECT 1 FROM tag_versions t
-        WHERE t.tag_id = #{ @tag } AND t.event_type = 'discard'
-          AND #{ latest_condition('tag_versions', 't', boundary) })
-    SQL
-  end
-
-  def ctes
-    boundary = quote(@at)
-    <<~SQL
-      WITH post_at AS (#{ filtered_scope('posts').to_sql }),
-      tag_at AS (#{ state_sql('tag_versions', boundary) }),
-      nico_at AS (#{ state_sql('nico_tag_versions', boundary) }),
-      pairs AS (
-        SELECT DISTINCT p.post_id,
-          CASE WHEN entry.tag_id IS NULL THEN 'external' ELSE 'internal' END AS kind,
-          COALESCE(entry.tag_id, entry.external_tag_id) AS tag_id
-        FROM post_at p
-        JOIN JSON_TABLE(CAST(p.tags_json AS JSON), '$[*]' COLUMNS (
-          tag_id BIGINT PATH '$.tag_id' NULL ON EMPTY ERROR ON ERROR,
-          external_tag_id BIGINT PATH '$.external_tag_id' NULL ON EMPTY ERROR ON ERROR
-        )) entry
-        LEFT JOIN tag_at t ON t.tag_id = entry.tag_id
-        WHERE COALESCE(entry.tag_id, entry.external_tag_id) IS NOT NULL
-          AND (t.event_type IS NULL OR t.event_type <> 'discard')
-      )
-    SQL
-  end
-
-  def totals
-    row = @connection.select_one(<<~SQL)
-      #{ ctes }
-      SELECT (SELECT COUNT(*) FROM post_at) AS posts,
-        (SELECT COUNT(*) FROM (#{ filtered_scope('tags').to_sql }) counted_tags) AS tags,
-        (SELECT COUNT(*) FROM tag_at WHERE event_type <> 'discard'
-          AND deprecated_at IS NOT NULL) AS deprecated_tags,
-        (SELECT COUNT(*) FROM pairs) AS assignments,
-        (SELECT COUNT(*) FROM pairs WHERE kind = 'external') AS external_assignments
-    SQL
-    row.transform_values!(&:to_i)
-    row.merge('posts' => available?(@post_start, @at) ? row['posts'] : nil,
-              'assignments' => available?(@post_start, @at) ? row['assignments'] : nil,
-              'external_assignments' => available?(@post_start, @at) ?
-                  row['external_assignments'] : nil,
-              'tags' => available?(@tag_start, @at) ? row['tags'] : nil,
-              'deprecated_tags' => available?(@tag_start, @at) ? row['deprecated_tags'] : nil)
-  end
-
-  def available?(first, boundary)
-    first && (@inclusive ? first <= boundary : first < boundary)
-  end
-
-  def listing
-    path = quote("$.#{ JSON.generate(@language) }.name")
-    catalog = <<~SQL
-      , tag_counts AS (
-        SELECT membership.kind, membership.tag_id, COUNT(*) AS post_count
-        FROM pairs membership GROUP BY membership.kind, membership.tag_id
-      ), catalog AS (
-        SELECT 'internal' AS kind, tag_id,
-          COALESCE(JSON_UNQUOTE(JSON_EXTRACT(names, #{ path })), name) AS name,
-          deprecated_at FROM tag_at WHERE event_type <> 'discard'
-        UNION ALL
-        SELECT tc.kind, tc.tag_id, external_version.name, NULL AS deprecated_at
-        FROM tag_counts tc
-        LEFT JOIN tag_at internal_version
-          ON tc.kind = 'internal' AND internal_version.tag_id = tc.tag_id
-        LEFT JOIN nico_at external_version
-          ON tc.kind = 'external' AND external_version.tag_id = tc.tag_id
-        WHERE tc.kind = 'external' OR internal_version.tag_id IS NULL
-      )
-    SQL
-    filters = []
-    filters << "c.kind = #{ quote(@kind) }" unless @kind == 'all'
-    filters << "c.tag_id = #{ @tag }" if @tag
-    where = filters.any? ? "WHERE #{ filters.join(' AND ') }" : ''
-    source = 'FROM catalog c LEFT JOIN tag_counts n ON n.kind = c.kind AND n.tag_id = c.tag_id'
-    prefix = "#{ ctes } #{ catalog }"
-    total = if @metadata
-              @connection.select_value("#{ prefix } SELECT COUNT(*) #{ source } #{ where }").to_i
-            end
-    order = order_sql({ 'name' => 'c.name', 'count' => 'COALESCE(n.post_count, 0)',
-                        'tag_id' => 'c.tag_id', 'kind' => 'c.kind',
-                        'deprecated' => 'c.deprecated_at' }, 'count:desc',
-                      ['c.kind ASC', 'c.tag_id ASC'])
-    rows = @connection.select_all(<<~SQL)
-      #{ prefix }
-      SELECT c.kind, c.tag_id, c.name, c.deprecated_at,
-        COALESCE(n.post_count, 0) AS post_count #{ source } #{ where }
-      ORDER BY #{ order } LIMIT #{ @limit } OFFSET #{ (@page - 1) * @limit }
-    SQL
-    { count: total, group_count: total, state_rows: rows.map do |row|
-        { kind: row['kind'], tag_id: row['tag_id'], name: row['name'],
-          deprecated: row['deprecated_at']&.iso8601,
-          count: available?(@post_start, @at) ? row['post_count'].to_i : nil,
-          at: @at.iso8601(6) }
-      end }
-  end
-
-  # All record filters apply after selecting the latest recorded version at the point.
-  def restored_scope(resource)
-    model = resource == 'posts' ? PostVersion : TagVersion
-    table = model.table_name
-    scope = model.where(Arel.sql(latest_condition(table, table, quote(@at))))
-        .where.not(event_type: 'discard')
-    if @tag
-      scope = if resource == 'posts'
-                scope.where("#{ membership(table) } AND #{ tag_visible(quote(@at)) }")
-              elsif @kind == 'internal'
-                scope.where(tag_id: @tag)
-              else
-                scope.none
-              end
-    end
-    scope
-  end
-
-  def filtered_scope(resource)
-    params = @params.slice('from', 'before', 'user_id', 'resource_id')
-        .merge('mode' => 'details', 'latest' => '0', 'sort' => '')
-    CountsQuery.new(restored_scope(resource), resource, params,
-                    latest_expression: '1').relation
-  end
-
-  def record_query(mode)
-    params = @params.slice('locale', 'sort', 'page', 'limit')
-        .merge('mode' => mode, 'latest' => '0', 'axis' => @axis == 'tags' ? 'month' : @axis,
-               'page' => @page.to_s, 'limit' => @limit.to_s)
-    CountsQuery.new(filtered_scope(@resource), @resource, params, latest_expression: '1')
-  end
-
-  def summary
-    return listing if @axis == 'tags'
-
-    query = record_query('summary')
-    { count: @metadata ? query.count : nil }.merge(query.summary(include_count: @metadata))
-  end
-
-  def details
-    query = record_query('details')
-    { count: @metadata ? query.count : nil,
-      versions: CountsVersionRepr.many(query.details, resource: @resource,
-                                      language_code: @language, at: @at,
-                                      inclusive: @inclusive) }
-  end
-
-  def next_boundary(time)
-    case @grain
-    when 'day' then time.beginning_of_day + 1.day
-    when 'month' then time.beginning_of_month.next_month
-    else time.beginning_of_year.next_year
-    end
-  end
-
-  def periods
-    @before = timestamp('before') || @now.beginning_of_month
-    @from = timestamp('from') || @before.prev_year
-    raise ArgumentError if @from >= @before || @before > @now
-
-    rows = []
-    boundary = next_boundary(@from)
-    while boundary <= @before
-      raise ArgumentError if rows.length >= MAX_PERIODS
-
-      format = { 'day' => '%Y-%m-%d', 'month' => '%Y-%m', 'year' => '%Y' }.fetch(@grain)
-      rows << [boundary, (boundary - 1.second).strftime(format)]
-      boundary = next_boundary(boundary)
-    end
-    rows
-  end
-
-  def trend
-    points = periods
-    metadata = { count: points.length, group_count: points.length, state_rows: [],
-                 from: @from.iso8601, before: @before.iso8601, grain: @grain }
-    return metadata if points.empty?
-
-    values = points.map do |boundary, period|
-      "SELECT CAST(#{ quote(boundary) } AS DATETIME(6)) AS boundary, " +
-          "#{ quote(period) } AS period"
-    end.join(' UNION ALL ')
-    order = order_sql({ 'period' => 'boundary', 'count' => 'post_count' },
-                      'period:asc', ['boundary ASC'])
-    known = @post_start ? "periods.boundary > #{ quote(@post_start) }" : 'FALSE'
-    rows = @connection.select_all(<<~SQL)
-      WITH periods AS (#{ values }), series AS (
-        SELECT periods.boundary, periods.period,
-          CASE WHEN #{ known } THEN COUNT(DISTINCT p.post_id) ELSE NULL END AS post_count
-        FROM periods
-        LEFT JOIN post_versions p ON p.event_type <> 'discard'
-          AND #{ membership('p') }
-          AND #{ latest_condition('post_versions', 'p', 'periods.boundary') }
-          AND #{ tag_visible('periods.boundary') }
-        GROUP BY periods.boundary, periods.period
-      )
-      SELECT * FROM series ORDER BY #{ order }
-      LIMIT #{ @limit } OFFSET #{ (@page - 1) * @limit }
-    SQL
-    metadata.merge(state_rows: rows.map do |row|
-      { kind: @kind, tag_id: @tag, name: nil, period: row['period'],
-        at: row['boundary'].utc.iso8601(6), count: row['post_count']&.to_i }
-    end)
-  end
-
-  def order_sql(expressions, default, final)
-    raw = @params.fetch('sort', default)
-    sorts = raw.empty? ? [] : QueryState.list(raw).map { _1.split(':', -1) }
-    raise ArgumentError if sorts.map(&:first).uniq.length != sorts.length
-    raise ArgumentError if sorts.length > expressions.length
-
-    order = sorts.flat_map do |key, direction, *extra|
-      if extra.any? || !expressions.key?(key) || !['asc', 'desc'].include?(direction)
-        raise ArgumentError
-      end
-      expression = expressions.fetch(key)
-      ["(#{ expression }) IS NULL ASC", "#{ expression } #{ direction.upcase }"]
-    end
-    (order + final).join(', ')
-  end
-end

backend/app/services/counts_table_export.rb

@@ -1,329 +0,0 @@
-require 'csv'
-
-# Export uses the exact same population and grouped cells as the interactive table.
-class CountsTableExport
-  EXTRA = ['format', 'columns', 'headers', 'labels', 'children', 'child_sort',
-           'order', 'nested_json', 'summary_columns', 'basis', 'axis', 'latest'].freeze
-  attr_reader :format, :resource
-
-  def initialize(params)
-    QueryState.validate!(params, CountsTableQuery::KEYS + EXTRA + ['measures'])
-    raise ArgumentError unless params.values.all? { _1.is_a?(String) }
-
-    params = CountsDefaults.for(params).merge(params)
-    unless params.key?('children')
-      params = params.merge('children' => CountsExportProjection.default_children(
-          params.fetch('resource', 'posts')).join(' '))
-    end
-    @params = params
-    @labels = JSON.parse(params.fetch('labels', '{}'))
-    raise ArgumentError unless @labels.is_a?(Hash)
-    @selected = params.key?('children') ? QueryState.list(params['children']) : nil
-    @order = QueryState.list(params.fetch('order', ''))
-    @child_sort = QueryState.list(params.fetch('child_sort', '')).map { _1.split(':', -1) }
-    @child_sort.each do |path, direction, *extra|
-      raise ArgumentError if extra.any? || !['asc', 'desc'].include?(direction)
-      raise ArgumentError unless path&.match?(/\A[a-zA-Z_]+(?:\.[a-zA-Z]+)*\/[a-zA-Z]+\z/)
-    end
-    @format = params.fetch('format', 'json')
-    @resource = params.fetch('resource', 'posts')
-    raise ArgumentError unless ['json', 'csv'].include?(format)
-
-    @query = CountsTableQuery.new(params.select { |key, _value| CountsTableQuery::KEYS.include?(key) ||
-        key == 'measures' || QueryState.valid_key?(key) }
-        .merge('page' => '1', 'limit' => '100'))
-    @summary = @query.mode == 'summary'
-    default_columns = @summary ? @query.axes.all.map { _1.fetch('field') } + ['count'] :
-        CountsExportProjection.default_columns(resource)
-    @columns = QueryState.list(params.fetch('columns', default_columns.join(' ')))
-    default_headers = @columns.map { @labels.fetch(_1, _1) }
-    @headers = JSON.parse(params.fetch('headers', default_headers.to_json))
-    raise ArgumentError unless @headers.is_a?(Array) && @headers.all? { _1.is_a?(String) }
-    if @summary && @headers.any? && @headers.length != @query.axes.all.length + 1
-      raise ArgumentError
-    end
-
-    unless @summary
-      if ['posts', 'tags'].include?(resource)
-        @projection = CountsExportProjection.new(params, resource:, summary: false, tags: false)
-      else
-        allowed = @query.population.definition.fields.keys + ['is_latest']
-        raise ArgumentError if (@columns - allowed).any?
-        raise ArgumentError unless @columns.length == @headers.length && @columns.uniq == @columns
-      end
-    end
-  end
-
-  def body(progress: nil)
-    Enumerator.new do |output|
-      ApplicationRecord.transaction(isolation: :repeatable_read) do
-        @language_names = Language.pluck(:code, :name).to_h
-        processed = 0
-        started = false
-        first = true
-        @query.each_page do |page|
-          total = @summary ? nil : page[:count]
-          progress&.call(processed, total)
-          unless started
-            metadata = page.except(:versions, :cells, :axis_labels, :column_values,
-                :row_headers, :column_headers, :pivot_totals).merge(
-                resource:, record_unit: @summary && format == 'json' ? 'pivot_cell' : 'row',
-                body_row_totals: 'once_per_logical_group', page_grand_total_repeated: false,
-                filters: @params.select { |key, _value| CountsTableQuery::KEYS.include?(key) ||
-        key == 'measures' || QueryState.valid_key?(key) },
-                fields: @query.population.definition.metadata,
-                columns: @columns, column_labels: @headers,
-                order: @params['order'], children: @params['children'],
-                sort: @params.fetch('sort', ''), child_sort: @params['child_sort'])
-            output << (format == 'json' ? "{\"metadata\":#{ metadata.to_json },\"data\":[" : "\uFEFF")
-            if format == 'csv'
-              headings = @summary ? pivot_headers(page) : [@headers]
-              headings.each do |headers|
-                output << CSV.generate_line(headers.map { safe(_1) }, row_sep: "\r\n")
-              end
-            end
-            started = true
-          end
-          if @summary
-            pivot_rows(page).each do |header|
-              records = page.fetch(:column_headers).map { pivot_cell(header, _1, page) }
-              if format == 'csv'
-                labels = pivot_row_labels(header, page)
-                values = labels + records.map { _1[:count] }
-                output << CSV.generate_line(values.map { safe(_1) }, row_sep: "\r\n")
-                processed += records.length
-                progress&.call(processed, nil)
-              else
-                records.each do |record|
-                  output << ',' unless first
-                  output << record.to_json
-                  first = false
-                  processed += 1
-                  progress&.call(processed, nil)
-                end
-              end
-            end
-            next
-          end
-          page[:versions].each do |row|
-            record = project(row)
-            if format == 'json'
-              output << ',' unless first
-              output << record.to_json
-            else
-              values = if @summary
-                         row[:values].each_with_index.map do |value, index|
-                           summary_value(value, index, page)
-                         end + [row[:count]]
-                       elsif @projection
-                         @projection.csv_record(record, row)
-                       else
-                         record.map { |key, value| csv_logical(key, value) }
-                       end
-              output << CSV.generate_line(values.map { safe(_1) }, row_sep: "\r\n")
-            end
-            first = false
-            processed += 1
-            progress&.call(processed, total)
-          end
-        end
-        progress&.call(processed, processed)
-        output << ']}' if format == 'json'
-      end
-    end
-  end
-
-  private
-
-  def pivot_rows(page)
-    page.fetch(:row_headers).reject do |header|
-      header[:kind] == 'grand_total' && @query.page * @query.limit < page[:group_count]
-    end
-  end
-
-  def total_label(header)
-    key = header[:kind] == 'grand_total' ?
-        (@query.axes.all.length == 1 ? 'singleTotal' : 'grandTotal') : 'subtotal'
-    @labels.fetch(key, '')
-  end
-
-  def pivot_row_labels(header, page)
-    values = header[:values].each_with_index.map { |value, index|
-      summary_value(value, index, page)
-    }
-    if header[:kind] == 'data' && @previous_pivot_row&.fetch(:kind) == 'data'
-      values.each_index do |index|
-        if header[:values].first(index + 1) == @previous_pivot_row[:values].first(index + 1)
-          values[index] = ''
-        end
-      end
-    end
-    @previous_pivot_row = header
-    values << total_label(header) unless header[:kind] == 'data'
-    leading = [@query.axes.rows.length, @query.axes.columns.any? ? 1 : 0].max
-    Array.new(leading) { values[_1] || '' }
-  end
-
-  def pivot_headers(page)
-    row_count = @query.axes.rows.length
-    cols = @query.axes.columns
-    leading = [row_count, cols.any? ? 1 : 0].max
-    columns = page.fetch(:column_headers)
-    headings = Array.new([cols.length, 1].max) do |level|
-      prefix = Array.new(leading) { level.zero? ? @headers[_1].to_s : '' }
-      prefix = [''] if row_count.zero? && cols.any?
-      prefix + columns.each_with_index.map do |header, index|
-        values = header[:values]
-        if cols.empty?
-          @labels.fetch('count', '')
-        elsif level == values.length && header[:kind] != 'data'
-          total_label(header)
-        elsif level < values.length
-          previous = index.positive? ? columns[index - 1][:values] : nil
-          if level < cols.length - 1 && previous && previous.first(level + 1) == values.first(level + 1)
-            ''
-          else
-            summary_value(values[level], row_count + level, page)
-          end
-        else
-          ''
-        end
-      end
-    end
-    return headings if cols.empty?
-
-    cols.each_index.flat_map do |level|
-      prefix = Array.new(leading, '')
-      prefix = headings.first.first(leading) if level.zero?
-      names = columns.each_with_index.map do |header, index|
-        next '' if header[:values].length < level
-
-        previous = index.positive? ? columns[index - 1][:values] : nil
-        if (previous && previous.length >= level &&
-            previous.first(level) == header[:values].first(level))
-          ''
-        else
-          @headers[row_count + level].to_s
-        end
-      end
-      headings[level] = Array.new(leading, '') + headings[level].drop(leading) if level.zero?
-      [prefix + names, headings[level]]
-    end
-  end
-
-  def pivot_cell(row, column, page)
-    @pivot_lookup ||= {}
-    # Discard the previous page's lookup; memory remains bounded by one page and its rollups.
-    unless @pivot_page.equal?(page)
-      @pivot_page = page
-      @pivot_lookup = page.fetch(:cells).to_h do |cell|
-        values = cell[:values]
-        [[values.first(@query.axes.rows.length), values.drop(@query.axes.rows.length)], cell[:count]]
-      end
-      page.fetch(:pivot_totals).each { @pivot_lookup[[_1[:row], _1[:column]]] = _1[:count] }
-    end
-    kinds = [row[:kind], column[:kind]]
-    kind = if kinds.include?('grand_total')
-             'grand_total'
-           elsif kinds.include?('subtotal')
-             'subtotal'
-           else
-             'data'
-           end
-    { row: row[:values], column: column[:values], kind:,
-      row_kind: row[:kind], column_kind: column[:kind],
-      row_depth: row[:values].length, column_depth: column[:values].length,
-      count: @pivot_lookup.fetch([row[:values], column[:values]], 0) }
-  end
-
-  def summary_value(value, index, page)
-    field = @query.axes.all.fetch(index).fetch('field')
-    return @labels.fetch('bot', '') if field == 'user' && value.nil?
-
-    names = page[:axis_labels].fetch("a#{ index }", {})
-    name = field == 'user' ? names[value.to_s] : names.fetch(value.to_s, value)
-    if field == 'user'
-      name = @labels.fetch('notRegistered', '') if name.nil? || name == ''
-      pattern = @labels.fetch('actorFormat', '{{name}} (#{{id}})')
-      return pattern.gsub(/\{\{(name|id)\}\}/) do
-        Regexp.last_match(1) == 'name' ? name.to_s : value.to_s
-      end
-    end
-    @labels.fetch("#{ field }:#{ value }", name)
-  end
-
-  def project(row)
-    return { row: row[:values].first(@query.axes.rows.length),
-             column: row[:values].drop(@query.axes.rows.length), count: row[:count] } if @summary
-    return @projection.project(row) if @projection
-
-    @columns.to_h do |column|
-      key = column == 'user' ? 'created_by_user' : column
-      value = row[key]
-      value = project_nested(value, key) if ['members', 'placements', 'export_paths_json'].include?(key)
-      [key, value]
-    end
-  end
-
-  def camel(key)
-    key.gsub(/_([a-z])/) { Regexp.last_match(1).upcase }
-  end
-
-  def project_nested(value, path)
-    rows = path == 'export_paths_json' ? (value || {}).map { |profile, target|
-      { 'profile' => profile, 'path' => target }
-    } : Array(value)
-    sorts = @child_sort.filter_map do |key, direction|
-      parent, field = key.split('/', 2)
-      [field, direction] if parent == path
-    end
-    rows = rows.each_with_index.sort do |(first, a), (last, b)|
-      compared = 0
-      sorts.each do |field, direction|
-        left = first.find { |key, _| camel(key) == field }&.last
-        right = last.find { |key, _| camel(key) == field }&.last
-        compared = if left.nil? || right.nil?
-                     left.nil? == right.nil? ? 0 : (left.nil? ? 1 : -1)
-                   else
-                     (left.to_s <=> right.to_s) * (direction == 'asc' ? 1 : -1)
-                   end
-        break unless compared.zero?
-      end
-      compared.zero? ? a <=> b : compared
-    end.map(&:first)
-    rows.map do |row|
-      fields = row.keys.select { !@selected || @selected.include?("#{ path }.#{ camel(_1) }") }
-      ordered = (@order + fields.map { "#{ path }.#{ camel(_1) }" }).uniq
-          .filter_map { |item| fields.find { "#{ path }.#{ camel(_1) }" == item } }
-      ordered.to_h do |key|
-        item = row[key]
-        item = project_nested(item, "#{ path }.#{ camel(key) }") if item.is_a?(Array)
-        [key, item]
-      end
-    end
-  end
-
-  def csv_logical(key, value)
-    return @labels.fetch('bot', '') if key == 'created_by_user' && value.nil?
-    return value ? @labels.fetch('latest', '') : '' if key == 'is_latest'
-    if value.is_a?(Time) || value.is_a?(ActiveSupport::TimeWithZone)
-      return @query.population.timezone.utc_to_local(value.utc).iso8601
-    end
-    if key == 'language_code' || key == 'source_language_code'
-      return @language_names[value]
-    end
-    if value.is_a?(Hash)
-      return value.to_h { |field, item| [@labels.fetch(camel(field.to_s), ''), csv_logical(field, item)] }
-    end
-    return value.map { csv_logical(key, _1) } if value.is_a?(Array)
-
-    @labels.fetch("#{ key }:#{ value }", value)
-  end
-
-  def safe(value)
-    return value if value.is_a?(Numeric)
-
-    text = value.is_a?(Hash) || value.is_a?(Array) ? JSON.generate(value) : value.to_s
-    text.match?(/\A\s*[=+@-]/) ? "'#{ text }" : text
-  end
-end

backend/app/services/counts_table_query.rb

@@ -1,345 +0,0 @@
-# Common entrypoint for six version resources; legacy endpoints keep their contracts.
-class CountsTableQuery
-  MAX_COLUMNS = 100
-  class TooManyColumns < ArgumentError; end
-
-  KEYS = ['resource', 'definition', 'mode', 'at', 'execution_at', 'from', 'through',
-          'timezone', 'rows', 'cols', 'conditions', 'drill', 'tags', 'tag_refs', 'match',
-          'tag', 'kind', 'resource_id', 'user_id', 'page', 'limit', 'sort', 'locale', 'measures', 'column_order'].freeze
-  attr_reader :population, :axes, :page, :limit, :mode
-
-  def initialize(params)
-    QueryState.validate!(params, KEYS + ['measures'])
-    raise ArgumentError unless params.values.all? { _1.is_a?(String) }
-
-    params = CountsDefaults.for(params).slice(*KEYS).merge(params)
-    @params = params
-    @population = CountsPopulation.new(params)
-    @definition = population.definition
-    @connection = @definition.model.connection
-    @axes = CountsAxes.new(population)
-    @page = Integer(params.fetch('page', '1'), 10)
-    @limit = Integer(params.fetch('limit', '20'), 10)
-    @mode = params.fetch('mode', 'summary')
-    raise ArgumentError unless page.between?(1, 1_000_000) && limit.between?(1, 100)
-    raise ArgumentError unless ['summary', 'details'].include?(mode)
-
-    @sort = QueryState.list(params.fetch('sort', '')).map { _1.split(':', -1) }
-    allowed = mode == 'summary' ? axes.all.each_index.map { "a#{ _1 }" } + ['count'] :
-        @definition.fields.keys + ['is_latest'] +
-        (@definition.resource == 'posts' ? ['tags', 'parents'] : []) +
-        (@definition.resource == 'tags' ? ['parents', 'deprecated'] : [])
-    raise ArgumentError if @sort.map(&:first).uniq.length != @sort.length
-    @sort.each do |key, direction, *extra|
-      measure = mode == 'summary' && key&.match?(/\Am[0-9]{1,2}\z/)
-      measure_values(key) if measure
-      if extra.any? || (!allowed.include?(key) && !measure) || !['asc', 'desc'].include?(direction)
-        raise ArgumentError
-      end
-    end
-  end
-
-  def call
-    ApplicationRecord.transaction do
-      result = metadata.merge(mode == 'summary' ? summary : details)
-      visible = mode == 'summary' ? result[:visible_output_count] : result[:count]
-      # A scalar result has one row regardless of private attributes or predicates.
-      public_count = result[:excluded_count].zero? || (mode == 'summary' && axes.rows.empty?)
-      result.merge(visible_output_count: visible,
-          output_count: public_count ? visible : nil,
-          output_count_scope: public_count ?
-              'before_authorisation' : 'unavailable_private_history',
-          output_count_unit: 'table_body_rows',
-          page_grand_total_repeated: mode == 'summary' && axes.rows.any? &&
-              result[:group_count] > limit)
-    end
-  end
-
-  def metadata
-    @population_counts ||= HistoryAccess.population_counts(@definition.model)
-    @population_counts.merge(
-        population_count_scope: 'all_recorded_versions',
-        page:, limit:, execution_at: population.at.iso8601(6), at: @params['at'].presence,
-        definition: population.history ? 'history' : 'latest',
-        timezone: population.timezone.identifier, rows: axes.rows, cols: axes.columns,
-        fields: @definition.metadata,
-        count: population.scope.except(:select).optimizer_hints('MAX_EXECUTION_TIME(10000)').count)
-  end
-
-  def summary
-    expressions = axes.expressions
-    selected = expressions.each_with_index.map do |expression, index|
-      "#{ expression } AS a#{ index }"
-    end
-    source = population.scope
-    source = source.joins(axes.joins) unless axes.joins.empty?
-    sql = if expressions.empty?
-            source.except(:select).select('COUNT(DISTINCT v.id) AS count').to_sql
-          else
-            source.except(:select).group(*expressions.map { Arel.sql(_1) })
-                .select(Arel.sql((selected + ['COUNT(DISTINCT v.id) AS count']).join(', ')))
-                .to_sql
-          end
-    prefix = "WITH cells AS (#{ sql })"
-    column_values = limited_columns(prefix)
-    row_keys = axes.rows.each_index.map { "a#{ _1 }" }
-    # Page whole row tuples so a pivot row is never split across display pages.
-    row_selection = row_keys.empty? ? '1 AS singleton' : row_keys.join(', ')
-    row_group = row_keys.empty? ? 'GROUP BY singleton' : "GROUP BY #{ row_keys.join(', ') }"
-    measure_joins = []
-    extras = []
-    row_sort = @sort.each_with_index.filter_map do |(key, direction), index|
-      if key == 'count' || key.match?(/\Am[0-9]+\z/)
-        values = key == 'count' ? [] : measure_values(key)
-        relation = measure_relation(values)
-        matches = row_keys.map { "grouped.#{ _1 } <=> sort#{ index }.#{ _1 }" }
-        condition = matches.empty? ? 'TRUE' : matches.join(' AND ')
-        measure_joins << "LEFT JOIN (#{ relation }) sort#{ index } ON #{ condition }"
-        extras << "sort#{ index }.count AS sort#{ index }"
-        "sort#{ index } IS NULL ASC, sort#{ index } #{ direction.upcase }"
-      elsif row_keys.include?(key)
-        "#{ key } IS NULL ASC, #{ key } #{ direction.upcase }"
-      end
-    end
-    order_rows = row_sort + row_keys.map { "#{ _1 } IS NULL, #{ _1 }" }
-    order_rows = ['singleton ASC'] if order_rows.empty?
-    grouped = "SELECT #{ row_selection } FROM cells #{ row_group }"
-    selection = (['grouped.*'] + extras).join(', ')
-    group_sql = "SELECT #{ selection } FROM (#{ grouped }) grouped #{ measure_joins.join(' ') }"
-    total = @connection.select_value(<<~SQL).to_i
-      #{ prefix }
-      SELECT /*+ MAX_EXECUTION_TIME(10000) */ COUNT(*) FROM (#{ group_sql }) row_groups
-    SQL
-    join = row_keys.map { "cells.#{ _1 } <=> page_rows.#{ _1 }" }.join(' AND ')
-    join = 'TRUE' if row_keys.empty?
-    order_cols = @sort.filter_map do |key, direction|
-      next unless key.match?(/\Aa[0-9]+\z/) && !row_keys.include?(key)
-
-      "cells.#{ key } #{ direction.upcase }"
-    end
-    order = ['page_rows.position'] + order_cols +
-        axes.all.each_index.map { "cells.a#{ _1 }" }
-    records = @connection.select_all(<<~SQL)
-      #{ prefix }, row_groups AS (#{ group_sql }),
-      numbered AS (SELECT row_groups.*,
-        ROW_NUMBER() OVER (ORDER BY #{ order_rows.join(', ') }) AS position FROM row_groups),
-      page_rows AS (SELECT * FROM numbered
-        WHERE position > #{ (page - 1) * limit } AND position <= #{ page * limit })
-      SELECT /*+ MAX_EXECUTION_TIME(10000) */ cells.* FROM cells JOIN page_rows ON #{ join }
-      ORDER BY #{ order.join(', ') }
-    SQL
-    next_row = @connection.select_one(<<~SQL)
-      #{ prefix }, row_groups AS (#{ group_sql }),
-      numbered AS (SELECT row_groups.*,
-        ROW_NUMBER() OVER (ORDER BY #{ order_rows.join(', ') }) AS position FROM row_groups)
-      SELECT /*+ MAX_EXECUTION_TIME(10000) */ * FROM numbered
-      WHERE position = #{ page * limit + 1 }
-    SQL
-    row_values = records.map { |record| row_keys.map { record[_1] } }.uniq
-    row_depths = CountsPivotLayout.depths(row_keys, @sort)
-    column_keys = axes.columns.each_index.map { "a#{ axes.rows.length + _1 }" }
-    column_sort = @sort
-    if @params.key?('column_order') && column_keys.any? &&
-        !(@sort.first&.first == 'count' || @sort.first&.first&.match?(/\Am[0-9]+\z/))
-      column_sort = [[column_keys.first, 'asc']] + @sort
-    end
-    column_depths = CountsPivotLayout.depths(column_keys, column_sort, measures: false)
-    row_headers = CountsPivotLayout.headers(row_values, row_depths,
-        next_values: next_row && row_keys.map { next_row[_1] }, grand: axes.rows.any?)
-    if axes.rows.empty? && axes.columns.any?
-      row_headers = [{ values: [], kind: 'grand_total' }]
-    end
-    column_headers = CountsPivotLayout.headers(column_values, column_depths,
-        grand: axes.columns.any?)
-    if axes.columns.empty? && axes.rows.any?
-      column_headers = [{ values: [], kind: 'grand_total' }]
-    end
-    totals = CountsPivotTotals.new(population, axes, row_headers, column_headers).call
-    { group_count: total,
-      visible_output_count: output_row_count(prefix, row_keys, row_depths, total),
-      column_values:, row_headers:, column_headers:, pivot_totals: totals,
-      cells: records.map do |row|
-        { values: axes.all.each_index.map { row["a#{ _1 }"] }, count: row['count'].to_i }
-      end, axis_labels: axis_labels(records, column_values) }
-  end
-
-  def details
-    query = population.scope.select(Arel.sql(
-        "(#{ population.latest_sql }) AS counts_latest"))
-    order = sort_sql { detail_expression(_1) }
-    query = query.order(Arel.sql((order + ['v.id DESC']).join(', ')))
-        .limit(limit).offset((page - 1) * limit)
-    rows = query.to_a
-    if ['posts', 'tags'].include?(@definition.resource)
-      point = population.history ? population.at :
-          population.timestamp(@params['at']) || population.at
-      values = CountsVersionRepr.many(rows, resource: @definition.resource,
-          language_code: locale.language_code,
-          at: point)
-      values.each_with_index { |value, index| value[:event_type] = rows[index].event_type }
-    else
-      users = User.where(id: rows.map(&:created_by_user_id)).pluck(:id, :name).to_h
-      values = rows.map do |row|
-        attributes = @definition.fields.keys.to_h do |key|
-          [key, row.public_send(@definition.column(key))]
-        end
-        attributes.merge('id' => row.id, 'resource_id' => row[@definition.keys.first],
-            'is_latest' => row.attributes['counts_latest'].to_i == 1,
-            'created_by_user' => row.created_by_user_id &&
-                { id: row.created_by_user_id, name: users[row.created_by_user_id] })
-      end
-    end
-    { versions: values }
-  end
-
-  def each_page
-    total = nil
-    loop do
-      result = call
-      total ||= mode == 'summary' ? result[:group_count] : result[:count]
-      yield result
-      break if page * limit >= total
-
-      @page += 1
-    end
-  end
-
-  private
-
-  def output_row_count(prefix, row_keys, depths, total)
-    # With no row axes the table has exactly one body row, including the zero-count case.
-    return 1 if row_keys.empty?
-
-    subtotals = depths.map do |depth|
-      keys = row_keys.first(depth).join(', ')
-      "(SELECT COUNT(*) FROM (SELECT #{ keys } FROM cells GROUP BY #{ keys }) subtotal_rows)"
-    end
-    expressions = [total.to_s, '1'] + subtotals
-    @output_row_count ||= @connection.select_value(<<~SQL).to_i
-      #{ prefix }
-      SELECT /*+ MAX_EXECUTION_TIME(10000) */ #{ expressions.join(' + ') } AS output_rows
-    SQL
-  end
-
-  def locale = @locale ||= Locale.find_by!(code: @params.fetch('locale', 'ja'))
-
-  def limited_columns(prefix)
-    keys = axes.columns.each_index.map { "a#{ axes.rows.length + _1 }" }
-    return [[]] if keys.empty?
-
-    configured = @sort.select { |key, _direction| keys.include?(key) }
-    directions = (configured + keys.map { [_1, 'asc'] }).uniq { _1.first }
-    order = directions.map { |key, direction|
-      "#{ key } IS NULL ASC, #{ key } #{ direction.upcase }"
-    }.join(', ')
-    custom = QueryState.records(@params, 'column_order')
-    if custom.any?
-      raise ArgumentError unless custom.map { _1['value'] }.uniq.length == custom.length
-
-      cases = custom.each_with_index.map do |record, index|
-        "WHEN #{ keys.first } <=> #{ @connection.quote(record['value']) } THEN #{ index }"
-      end.join(' ')
-      order = "CASE #{ cases } ELSE #{ custom.length } END, #{ order }"
-    end
-    values = @connection.select_rows(<<~SQL)
-      #{ prefix }
-      SELECT /*+ MAX_EXECUTION_TIME(10000) */ DISTINCT #{ keys.join(', ') } FROM cells
-      ORDER BY #{ order } LIMIT #{ MAX_COLUMNS + 1 }
-    SQL
-    raise TooManyColumns if values.length > MAX_COLUMNS
-
-    values
-  end
-
-  def measure_relation(values)
-    row_expressions = axes.expressions.first(axes.rows.length)
-    source = population.scope.except(:select, :order)
-    fields = axes.rows + axes.columns.first(values.length)
-    joins = axes.joins(fields.map { _1.fetch('field') })
-    source = source.joins(joins) unless joins.empty?
-    values.each_with_index do |value, index|
-      expression = axes.expressions[axes.rows.length + index]
-      source = source.where("(#{ expression }) <=> ?", value)
-    end
-    selected = row_expressions.each_with_index.map { |expression, index|
-      "#{ expression } AS a#{ index }"
-    } + ['COUNT(DISTINCT v.id) AS count']
-    source = source.group(*row_expressions.map { Arel.sql(_1) }) if row_expressions.any?
-    source.select(Arel.sql(selected.join(', '))).to_sql
-  end
-
-  def measure_values(key)
-    records = QueryState.records(@params, 'measures').select { _1['key'] == key }
-    raise ArgumentError if records.empty?
-    raise ArgumentError unless records.map { _1['axis'] }.sort == (0...records.length).to_a
-
-    value = records.sort_by { _1['axis'] }.map { _1['value'] }
-    unless (value.is_a?(Array) && value.length <= axes.columns.length &&
-        value.all? { _1.nil? || _1.is_a?(String) || _1.is_a?(Numeric) })
-      raise ArgumentError
-    end
-    value
-  end
-
-  def detail_expression(key)
-    return "(#{ population.latest_sql })" if key == 'is_latest'
-    aliases = { 'tags' => 'tags_json', 'parents' => @definition.resource == 'posts' ?
-        'parent_post_ids' : 'parent_tag_ids', 'deprecated' => 'deprecated_at' }
-    field = aliases.fetch(key, key)
-    expression = @definition.expression(field)
-    return expression unless @definition.fields[field] == :json
-
-    "CAST(#{ expression } AS CHAR) COLLATE utf8mb4_bin"
-  end
-
-  def sort_sql
-    @sort.flat_map do |key, direction|
-      expression = yield key
-      ["(#{ expression }) IS NULL ASC", "#{ expression } #{ direction.upcase }"]
-    end
-  end
-
-  def axis_labels(records, column_values)
-    result = {}
-    axes.all.each_with_index do |axis, index|
-      values = records.map { _1["a#{ index }"] }
-      if index >= axes.rows.length
-        values += column_values.map { _1[index - axes.rows.length] }
-      end
-      values = values.compact.uniq
-      labels = case axis['field']
-               when 'user'
-                 User.where(id: values).pluck(:id, :name).to_h.transform_keys(&:to_s)
-               when 'language_code', 'source_language_code'
-                 Language.where(code: values).pluck(:code, :name).to_h
-               when 'tags'
-                 tag_labels(values)
-               else {}
-               end
-      result["a#{ index }"] = labels
-    end
-    result
-  end
-
-  def tag_labels(values)
-    internal = values.grep(/\Ainternal:/).map { _1.split(':').last.to_i }
-    external = values.grep(/\Aexternal:/).map { _1.split(':').last.to_i }
-    point = population.timestamp(@params['at']) || population.at
-    labels = {}
-    sources = [[TagVersion, internal, 'internal'], [NicoTagVersion, external, 'external']]
-    sources.each do |model, ids, kind|
-      next if ids.empty?
-
-      table = model.table_name
-      model.where(tag_id: ids).where('created_at <= ?', point)
-          .where("NOT EXISTS (SELECT 1 FROM #{ table } newer " +
-              "WHERE newer.tag_id = #{ table }.tag_id AND newer.created_at <= ? " +
-              "AND newer.version_no > #{ table }.version_no)", point).each do |row|
-        names = row.respond_to?(:names) ? row.names : nil
-        name = names&.dig(locale.language_code, 'name') || row.name
-        labels["#{ kind }:#{ row.tag_id }"] = name
-      end
-    end
-    labels
-  end
-end

backend/app/services/material_version_recorder.rb

@@ -31,7 +31,7 @@ class MaterialVersionRecorder < VersionRecorder
     { url: @record.url,
       parent: @record.parent,
       tag: @record.tag,
-      tag_name: @record.tag&.name('ja'),
+      tag_name: @record.tag&.name,
       tag_category: @record.tag&.category,
       source_kind: @record.source_kind,
       source_uri: @record.source_uri,

backend/app/services/nico_tag_version_recorder.rb

@@ -1,22 +1,19 @@
 class NicoTagVersionRecorder < VersionRecorder
-  def self.record! external_tag:, event_type:, created_by_user:
+  def self.record! tag:, event_type:, created_by_user:
-    new(external_tag:, event_type:, created_by_user:).record!
+    new(tag:, event_type:, created_by_user:).record!
   end
 
-  def initialize external_tag:, event_type:, created_by_user:
+  def initialize tag:, event_type:, created_by_user:
-    super(record: external_tag, event_type:, created_by_user:)
+    super(record: tag, event_type:, created_by_user:)
   end
 
   private
 
   def version_class = NicoTagVersion
   def version_association = :nico_tag_versions
-  def record_key = :external_tag
+  def record_key = :tag
 
   def snapshot_attributes
-    { name: "#{ @record.platform }:#{ @record.name }",
+    { name: @record.name, linked_tags: @record.snapshot_linked_tag_names.join(' ') }
-      linked_tags: @record.snapshot_linked_tag_names.join(' ') }
   end
-
-  def tracks_version_no_on_record? = false
 end

backend/app/services/post_bulk_creator.rb

@@ -6,7 +6,7 @@ class PostBulkCreator
     @host = host
   end
 
-  def run locale
+  def run
     results = Array.new(@posts.length)
     mutex = Mutex.new
     next_index = 0
@@ -27,7 +27,7 @@ class PostBulkCreator
                 break if index >= @posts.length
 
                 attributes = @posts[index]
-                results[index] = create_row(locale, actor, attributes, index)
+                results[index] = create_row(actor, attributes, index)
               rescue StandardError => e
                 Rails.logger.error(
                     "post_bulk_creator_worker_failure #{ { error: e.class.name,
@@ -61,12 +61,12 @@ class PostBulkCreator
 
   private
 
-  def create_row locale, actor, attributes, index
+  def create_row actor, attributes, index
     preflight =
         PostCreatePreflight.new(
             attributes: attributes,
             thumbnail: thumbnail_for(index, attributes),
-            host: @host).run(locale)
+            host: @host).run
     if preflight[:existing_post_id].present?
       return {
           status: 'skipped',
@@ -76,7 +76,7 @@ class PostBulkCreator
 
     post = PostCreator.new(
         actor: actor,
-        attributes: normalised_attributes(attributes, preflight, index)).create!(locale)
+        attributes: normalised_attributes(attributes, preflight, index)).create!
     result = {
         status: 'created',
         post: { id: post.id } }

backend/app/services/post_create_plan.rb

@@ -8,50 +8,50 @@ class PostCreatePlan
     @existing_tags_by_name = nil
   end
 
-  def build! locale
+  def build!
-    direct_tag_specs, tag_sections = parse_direct_tag_specs(locale)
+    direct_tag_specs, tag_sections = parse_direct_tag_specs
-    default_tag_specs = build_default_tag_specs(locale, direct_tag_specs)
+    default_tag_specs = build_default_tag_specs(direct_tag_specs)
     snapshot_tag_specs = merge_tag_specs(direct_tag_specs + default_tag_specs)
-    preload_existing_tags_by_name!(locale, snapshot_tag_specs.map { _1[:name] })
+    preload_existing_tags_by_name!(snapshot_tag_specs.map { _1[:name] })
-    validate_new_tag_specs!(locale, snapshot_tag_specs)
+    validate_new_tag_specs!(snapshot_tag_specs)
-    post_tag_specs = expand_parent_tag_specs(locale, snapshot_tag_specs)
+    post_tag_specs = expand_parent_tag_specs(snapshot_tag_specs)
     video_ms = normalise_video_ms(snapshot_tag_specs)
     validate_video_sections!(video_ms, tag_sections)
     parent_post_ids = normalise_parent_post_ids
     validate_parent_post_ids!(parent_post_ids)
 
-    { url: @attributes[:url],
+    {
-      title: @attributes[:title].to_s,
+        url: @attributes[:url],
-      thumbnail_base: @attributes[:thumbnail_base].presence,
+        title: @attributes[:title].to_s,
-      original_created_from: @attributes[:original_created_from].presence,
+        thumbnail_base: @attributes[:thumbnail_base].presence,
-      original_created_before: @attributes[:original_created_before].presence,
+        original_created_from: @attributes[:original_created_from].presence,
-      tags: serialised_tags(direct_tag_specs, tag_sections),
+        original_created_before: @attributes[:original_created_before].presence,
-      display_tags: display_tags(direct_tag_specs, tag_sections),
+        tags: serialised_tags(direct_tag_specs, tag_sections),
-      duration: @attributes[:duration].to_s,
+        display_tags: display_tags(direct_tag_specs, tag_sections),
-      video_ms: video_ms,
+        duration: @attributes[:duration].to_s,
-      parent_post_ids: parent_post_ids.join(' '),
+        video_ms: video_ms,
-      direct_tag_specs: direct_tag_specs,
+        parent_post_ids: parent_post_ids.join(' '),
-      default_tag_specs: default_tag_specs,
+        direct_tag_specs: direct_tag_specs,
-      snapshot_tag_specs: snapshot_tag_specs,
+        default_tag_specs: default_tag_specs,
-      post_tag_specs: post_tag_specs,
+        snapshot_tag_specs: snapshot_tag_specs,
-      tag_sections: tag_sections,
+        post_tag_specs: post_tag_specs,
-      normalised_parent_post_ids: parent_post_ids }
+        tag_sections: tag_sections,
+        normalised_parent_post_ids: parent_post_ids }
   end
 
   private
 
   def tag_names = @attributes[:tags].to_s.split
 
-  def parse_direct_tag_specs locale
+  def parse_direct_tag_specs
     tag_sections = { }
     direct_tag_specs = []
 
     tag_names.each do |raw_name|
-      tag_name, category, sections = parse_raw_tag_name(locale, raw_name)
+      tag_name, category, sections = parse_raw_tag_name(raw_name)
-      existing_tag = existing_tags_by_name(locale)[tag_name]
+      existing_tag = existing_tags_by_name[tag_name]
-      if existing_tag&.deprecated?
+      raise Tag::NicoTagNormalisationError if existing_tag&.nico?
-        raise Tag::DeprecatedTagNormalisationError, [existing_tag.name(locale.language_code)]
+      raise Tag::DeprecatedTagNormalisationError, [existing_tag.name] if existing_tag&.deprecated?
-      end
 
       direct_tag_specs << {
           name: tag_name,
@@ -67,7 +67,7 @@ class PostCreatePlan
     [merge_tag_specs(direct_tag_specs), tag_sections]
   end
 
-  def parse_raw_tag_name locale, raw_name
+  def parse_raw_tag_name raw_name
     name = raw_name.to_s
     prefix, category =
         Tag::CATEGORY_PREFIXES.find {
@@ -76,7 +76,7 @@ class PostCreatePlan
     name = name.sub(/\A#{ prefix }/i, '')
 
     sections = []
-    while match = name.match(/\A(\S*?)\[([^\[\]\s]*)-([^\[\]\s]*)\](\S*)\z/)
+    while (match = name.match(/\A(\S*?)\[([^\[\]\s]*)-([^\[\]\s]*)\](\S*)\z/))
       name = "#{ match[1] }#{ match[4] }"
       next if match[2].empty? && match[3].empty?
 
@@ -89,33 +89,36 @@ class PostCreatePlan
       raise Tag::SectionLiteralParseError.new(raw_name, raw_name)
     end
 
-    [resolved_tag_name(locale, name), category&.to_sym, sections]
+    [resolved_tag_name(name), category&.to_sym, sections]
   end
 
-  def build_default_tag_specs locale, direct_tag_specs
+  def build_default_tag_specs direct_tag_specs
     default_tag_specs = []
     if direct_tag_specs.length < 10 && direct_tag_specs.none? { _1[:name] == TAGME_TAG_NAME }
-      default_tag_specs << { name: TAGME_TAG_NAME, category: :meta }
+      default_tag_specs << {
+          name: TAGME_TAG_NAME,
+          category: :meta }
     end
-    if direct_tag_specs.none? { deerjikist_tag_spec?(locale, _1) }
+    if direct_tag_specs.none? { deerjikist_tag_spec?(_1) }
-      default_tag_specs << { name: NO_DEERJIKIST_TAG_NAME, category: :meta }
+      default_tag_specs << {
+          name: NO_DEERJIKIST_TAG_NAME,
+          category: :meta }
     end
 
     default_tag_specs
   end
 
-  def validate_new_tag_specs! locale, specs
+  def validate_new_tag_specs! specs
     Array(specs).each do |spec|
-      next if existing_tags_by_name(locale).key?(spec[:name])
+      next if existing_tags_by_name.key?(spec[:name])
 
-      validate_new_tag_spec!(locale, spec)
+      validate_new_tag_spec!(spec)
     end
   end
 
-  def validate_new_tag_spec! locale, spec
+  def validate_new_tag_spec! spec
-    tag = Tag.new(category: spec[:category])
+    tag_name = TagName.new(name: spec[:name])
-    tag_name = TagName.new(tag:, name: spec[:name], language_code: locale.language_code,
+    tag = Tag.new(category: spec[:category], tag_name:)
-                           script_code: locale.script_code, primary_flg: true)
     return if tag_name.valid? && tag.valid?
 
     post = Post.new
@@ -130,22 +133,20 @@ class PostCreatePlan
     raise ActiveRecord::RecordInvalid, post
   end
 
-  def expand_parent_tag_specs locale, snapshot_tag_specs
+  def expand_parent_tag_specs snapshot_tag_specs
-    existing_snapshot_tags = snapshot_tag_specs.filter_map do
+    existing_snapshot_tags = snapshot_tag_specs.filter_map { existing_tags_by_name[_1[:name]] }
-      existing_tags_by_name(locale)[_1[:name]]
-    end
     expanded_parent_specs =
         Tag.expand_parent_tags(existing_snapshot_tags)
-        .reject(&:deprecated?)
+           .reject(&:deprecated?)
-        .map { |tag|
+           .map { |tag|
-          { name: tag.name(locale.language_code),
+             {
-            category: tag.category.to_sym }
+                 name: tag.name,
-        }
+                 category: tag.category.to_sym } }
     merge_tag_specs(snapshot_tag_specs + expanded_parent_specs)
   end
 
   def merge_tag_specs specs
-    specs.each_with_object({ }) { |spec, merged|
+    specs.each_with_object({ }) do |spec, merged|
       merged[spec[:name]] =
           if merged.key?(spec[:name]) && merged[spec[:name]][:category] != :general
             merged[spec[:name]]
@@ -154,41 +155,36 @@ class PostCreatePlan
                 name: spec[:name],
                 category: spec[:category] }
           end
-    }.values.sort_by { _1[:name] }
+    end.values.sort_by { _1[:name] }
   end
 
-  def existing_tags_by_name locale
+  def existing_tags_by_name
     @existing_tags_by_name ||= begin
-      names = tag_names.map { canonical_tag_name_without_sections(locale, _1) }.uniq
+      names = tag_names.map { canonical_tag_name_without_sections(_1) }.uniq
-      TagName
+      Tag.joins(:tag_name).where(tag_names: { name: names }).index_by(&:name)
-          .where(language_code: locale.language_code, primary_flg: true, name: names)
-          .includes(:tag)
-          .each_with_object({ }) { |tag_name, tags| tags[tag_name.name] = tag_name.tag }
     end
   end
 
-  def preload_existing_tags_by_name! locale, names
+  def preload_existing_tags_by_name! names
     wanted_names = Array(names).map { _1.to_s }.reject(&:blank?).uniq
-    missing_names = wanted_names - existing_tags_by_name(locale).keys
+    missing_names = wanted_names - existing_tags_by_name.keys
     return if missing_names.empty?
 
-    existing_tags_by_name(locale).merge!(
+    existing_tags_by_name.merge!(
-        TagName.where(language_code: locale.language_code,
+        Tag.joins(:tag_name)
-                      primary_flg: true,
+           .where(tag_names: { name: missing_names })
-                      name: missing_names)
+           .index_by(&:name))
-               .includes(:tag)
-               .each_with_object({ }) { |tag_name, tags| tags[tag_name.name] = tag_name.tag })
   end
 
-  def canonical_tag_name_without_sections locale, raw_name
+  def canonical_tag_name_without_sections raw_name
-    name, = parse_raw_tag_name(locale, raw_name)
+    name, = parse_raw_tag_name(raw_name)
     name
   end
 
-  def deerjikist_tag_spec? locale, spec
+  def deerjikist_tag_spec? spec
     return true if spec[:category] == :deerjikist
 
-    existing_tags_by_name(locale)[spec[:name]]&.deerjikist?
+    existing_tags_by_name[spec[:name]]&.deerjikist?
   end
 
   def normalise_parent_post_ids
@@ -266,12 +262,10 @@ class PostCreatePlan
     end
   end
 
-  def resolved_tag_name locale, name
+  def resolved_tag_name name
-    tag_name = TagName.find_by(language_code: locale.language_code, name:)
+    tag_name = TagName.includes(:canonical).find_by(name:)
-    return name if !(tag_name) || tag_name.primary?
+    return name if tag_name.nil?
 
-    TagName.find_by(language_code: locale.language_code,
+    (tag_name.canonical || tag_name).name
-                    tag_id: tag_name.tag_id,
-                    primary_flg: true)&.name || name
   end
 end

backend/app/services/post_create_preflight.rb

@@ -15,10 +15,9 @@ class PostCreatePreflight
     @host = host
   end
 
-  def run locale
+  def run
     preview = PostImportPreviewer.new.preview_rows(
         rows: [preview_row],
-        locale:,
         fetch_metadata: false).first
     if preview[:existing_post_id].present?
       return {
@@ -54,7 +53,7 @@ class PostCreatePreflight
             original_created_from: preview[:attributes]['original_created_from'],
             original_created_before: preview[:attributes]['original_created_before'],
             duration: preview[:attributes]['duration'],
-            video_ms: preview[:attributes]['video_ms'] }).build!(locale)
+            video_ms: preview[:attributes]['video_ms'] }).build!
 
     {
         url: plan[:url],

backend/app/services/post_creator.rb

@@ -9,7 +9,7 @@ class PostCreator
     @field_warnings = { }
   end
 
-  def create! locale
+  def create!
     thumbnail_attachment = prepare_thumbnail_attachment
     post = Post.new(title: @attributes[:title].presence,
                     url: @attributes[:url],
@@ -21,17 +21,14 @@ class PostCreator
     ApplicationRecord.transaction do
       post.save!
       post.thumbnail.attach(thumbnail_attachment) if thumbnail_attachment.present?
-      snapshot_tags = planned_snapshot_tags(locale)
+      snapshot_tags = planned_snapshot_tags
-      post_tags = planned_post_tags(locale)
+      post_tags = planned_post_tags
-      sections = planned_sections(locale)
+      sections = planned_sections
-      TagVersioning.record_tag_snapshots!(
+      TagVersioning.record_tag_snapshots!(snapshot_tags, created_by_user: @actor)
-          snapshot_tags,
+      post.video_ms = planned_video_ms
-          created_by_user: @actor,
-          language_code: locale.language_code)
-      post.video_ms = planned_video_ms(locale)
       post.save!
       sync_post_tags!(post, post_tags, sections)
-      sync_parent_posts!(post, planned_parent_post_ids(locale))
+      sync_parent_posts!(post, planned_parent_post_ids)
       PostVersionRecorder.record!(post:, event_type: :create, created_by_user: @actor)
     end
     post
@@ -48,73 +45,69 @@ class PostCreator
         thumbnail_base: @attributes[:thumbnail_base].presence)
   end
 
-  def planned_snapshot_tags(locale) = planned_create_attributes(locale)[:snapshot_tags]
+  def planned_snapshot_tags = planned_create_attributes[:snapshot_tags]
 
-  def planned_post_tags(locale) = planned_create_attributes(locale)[:post_tags]
+  def planned_post_tags = planned_create_attributes[:post_tags]
 
-  def planned_sections(locale) = planned_create_attributes(locale)[:tag_sections]
+  def planned_sections = planned_create_attributes[:tag_sections]
 
-  def planned_parent_post_ids locale
+  def planned_parent_post_ids = planned_create_attributes[:normalised_parent_post_ids]
-    planned_create_attributes(locale)[:normalised_parent_post_ids]
+
+  def planned_video_ms
+    planned_create_attributes[:video_ms]
   end
 
-  def planned_video_ms locale
+  def planned_create_attributes
-    planned_create_attributes(locale)[:video_ms]
-  end
-
-  def planned_create_attributes locale
     @planned_create_attributes ||= begin
       if @attributes.key?(:snapshot_tag_specs)
-        snapshot_tags = materialise_tags(locale, @attributes[:snapshot_tag_specs] || [])
+        snapshot_tags = materialise_tags(@attributes[:snapshot_tag_specs] || [])
-        post_tags = materialise_tags(locale, @attributes[:post_tag_specs] || [])
+        post_tags = materialise_tags(@attributes[:post_tag_specs] || [])
         {
             snapshot_tags: snapshot_tags,
             post_tags: post_tags,
             tag_sections: materialise_sections(
                 @attributes[:tag_sections] || { },
                 snapshot_tags,
-                post_tags,
+                post_tags),
-                locale),
             normalised_parent_post_ids: @attributes[:normalised_parent_post_ids] || [],
             video_ms: @attributes[:video_ms] }
       else
-        build_materialised_plan(locale)
+        build_materialised_plan
       end
     end
   end
 
-  def build_materialised_plan locale
+  def build_materialised_plan
-    plan = PostCreatePlan.new(attributes: @attributes).build!(locale)
+    plan = PostCreatePlan.new(attributes: @attributes).build!
-    snapshot_tags = materialise_tags(locale, plan[:snapshot_tag_specs] || [])
+    snapshot_tags = materialise_tags(plan[:snapshot_tag_specs] || [])
-    post_tags = materialise_tags(locale, plan[:post_tag_specs] || [])
+    post_tags = materialise_tags(plan[:post_tag_specs] || [])
     {
         snapshot_tags: snapshot_tags,
         post_tags: post_tags,
         tag_sections: materialise_sections(
             plan[:tag_sections] || { },
             snapshot_tags,
-            post_tags,
+            post_tags),
-            locale),
         normalised_parent_post_ids: plan[:normalised_parent_post_ids] || [],
         video_ms: plan[:video_ms] }
   end
 
-  def materialise_tags locale, specs
+  def materialise_tags specs
     Array(specs).each_with_object({ }) do |spec, tags|
       name = spec[:name] || spec['name']
       category = spec[:category] || spec['category']
       next if name.blank? || category.blank?
 
-      tag = Tag.find_or_create_by_tag_name!(locale, name, category:, created_by_user: @actor)
+      tag = Tag.find_or_create_by_tag_name!(name, category:)
       tag.update!(category:) if tag.category.to_sym != category.to_sym
       tags[name] ||= tag
     end.values
   end
 
-  def materialise_sections sections_by_name, snapshot_tags, post_tags, locale = Locale.nipponese
+  def materialise_sections sections_by_name, snapshot_tags, post_tags
-    tags_by_name = post_tags.index_by { _1.name(locale.language_code) }
+    tags_by_name = post_tags.index_by(&:name)
     snapshot_tags.each do |tag|
-      tags_by_name[tag.name(locale.language_code)] ||= tag
+      tags_by_name[tag.name] ||= tag
     end
 
     sections_by_name.each_with_object({ }) do |(tag_name, ranges), sections|
@@ -128,11 +121,9 @@ class PostCreator
   def sync_post_tags! post, desired_tags, sections
     desired_ids = desired_tags.map(&:id).to_set
     current_ids = post.tags.pluck(:id).to_set
-
     Tag.where(id: desired_ids - current_ids).find_each do |tag|
       PostTag.create_or_find_by!(post:, tag:, created_user: @actor)
     end
-
     PostTagSection.where(post_id: post.id).destroy_all
     sections.each do |tag_id, ranges|
       ranges.each do |begin_ms, end_ms|
@@ -142,9 +133,10 @@ class PostCreator
                                end_ms:)
       end
     end
-
     PostTag.where(post_id: post.id,
-                  tag_id: (current_ids - desired_ids).to_a).destroy_all
+                  tag_id: (current_ids - desired_ids).to_a).kept.find_each do |post_tag|
+      post_tag.discard_by!(@actor)
+    end
   end
 
   def sync_parent_posts! post, ids

backend/app/services/post_import_previewer.rb

@@ -16,7 +16,7 @@ class PostImportPreviewer
   THUMBNAIL_FETCH_WARNING = 'サムネールを取得できませんでした．'.freeze
   METADATA_FETCH_WARNING = '自動取得に失敗しました．'.freeze
 
-  def preview_rows rows:, locale: Locale.nipponese, fetch_metadata: true, metadata_cache: { }
+  def preview_rows rows:, fetch_metadata: true, metadata_cache: { }
     prepared_rows = rows.map { prepare_row(_1) }
     url_counts = prepared_rows.filter_map { _1[:normal_url] }.tally
     existing_posts =
@@ -30,14 +30,13 @@ class PostImportPreviewer
         url_counts)
     known_tags =
         preload_known_tags(
-            locale,
             prepared_rows,
             fetch_metadata,
             metadata_cache,
             existing_posts,
             url_counts)
     prepared_rows.map { |row|
-      preview_row(locale, row,
+      preview_row(row,
                   fetch_metadata:,
                   metadata_cache:,
                   existing_posts:,
@@ -60,7 +59,7 @@ class PostImportPreviewer
     source.merge(url_text: url, normal_url:, url_error: validate_url_safety(normal_url))
   end
 
-  def preview_row locale, row,
+  def preview_row row,
                   fetch_metadata:,
                   metadata_cache:,
                   existing_posts:,
@@ -125,8 +124,7 @@ class PostImportPreviewer
 
     attributes['url'] = url
     validate_basic_data(attributes, validation_errors)
-    validate_preview_tags(locale,
+    validate_preview_tags(merged_tags(tag_sources, provenance['tags']),
-                          merged_tags(tag_sources, provenance['tags']),
                           validation_errors,
                           known_tags)
     validate_parents(attributes['parent_post_ids'], validation_errors, existing_parent_ids)
@@ -348,7 +346,7 @@ class PostImportPreviewer
     { data: { }, warnings: { 'url' => [METADATA_FETCH_WARNING] }, validation_errors: { } }
   end
 
-  def preload_known_tags locale, prepared_rows, fetch_metadata, metadata_cache, existing_posts, url_counts
+  def preload_known_tags prepared_rows, fetch_metadata, metadata_cache, existing_posts, url_counts
     names = prepared_rows.flat_map { |row|
       attributes = initial_attributes(row)
       provenance = initial_provenance(row)
@@ -360,13 +358,15 @@ class PostImportPreviewer
         metadata = metadata_for(row[:normal_url], metadata_cache)
         apply_metadata!(attributes, provenance, tag_sources, metadata[:data])
       end
-      preview_tag_names(locale, merged_tags(tag_sources, provenance['tags']))
+      preview_tag_names(merged_tags(tag_sources, provenance['tags']))
     }.compact.uniq
     return { } if names.empty?
 
-    TagName.where(language_code: locale.language_code, primary_flg: true, name: names)
+    Tag.joins(:tag_name)
-           .includes(:tag)
+      .where(tag_names: { name: names })
-           .each_with_object({ }) { |tag_name, tags| tags[tag_name.name] = tag_name.tag }
+      .includes(:tag_name)
+      .to_a
+      .index_by(&:name)
   end
 
   def preload_parent_ids prepared_rows
@@ -437,21 +437,19 @@ class PostImportPreviewer
     sources['automatic'].to_s
   end
 
-  def preview_tag_names locale, raw
+  def preview_tag_names raw
     names = raw.to_s.split
     return [] if names.empty?
     if names.any? { _1.downcase.start_with?('nico:') }
       return []
     end
 
-    names.map { |name|
+    names.map { |name| TagName.canonicalise(name.sub(/\[.*\]\z/, '')).first }
-      TagName.canonicalise(locale, name.sub(/\[.*\]\z/, '')).first
-    }
   rescue Tag::SectionLiteralParseError
     []
   end
 
-  def validate_preview_tags locale, raw, errors, known_tags
+  def validate_preview_tags raw, errors, known_tags
     names = raw.to_s.split
     return if names.empty?
     if names.any? { _1.downcase.start_with?('nico:') }
@@ -459,11 +457,9 @@ class PostImportPreviewer
       return
     end
 
-    parsed = names.map { |name|
+    parsed = names.map { |name| TagName.canonicalise(name.sub(/\[.*\]\z/, '')).first }
-      TagName.canonicalise(locale, name.sub(/\[.*\]\z/, '')).first
-    }
     existing = parsed.filter_map { known_tags[_1] }
-    deprecated = existing.select(&:deprecated?).map { _1.name(locale.language_code) }
+    deprecated = existing.select(&:deprecated?).map(&:name)
     errors[:tags] = ["廃止済みタグがあります: #{ deprecated.join(' ') }"] if deprecated.present?
   rescue Tag::SectionLiteralParseError
     errors[:tags] = ['タグ区間の記法が不正です．']

backend/app/services/post_metadata_fetcher.rb

@@ -47,9 +47,9 @@ class PostMetadataFetcher
 
     existing_tags =
         Tag
-        .joins(:tag_names)
+        .joins(:tag_name)
-        .where(tag_names: { language_code: 'ja', primary_flg: true, name: names })
+        .where(tag_names: { name: names })
-        .index_by { _1.name('ja') }
+        .index_by(&:name)
 
     names.map { |name|
       tag = existing_tags[name]

backend/app/services/post_tag_search.rb

@@ -1,42 +0,0 @@
-# Shared literal parsing and reference resolution for current and snapshot post search.
-class PostTagSearch
-  def self.parse_references(raw, literals)
-    return nil unless raw.key?('tag_refs')
-
-    references = QueryState.records(raw, 'tag_refs')
-    valid = references.is_a?(Array) && references.all? do |reference|
-      next false unless reference.is_a?(Hash) && reference['name'].is_a?(String)
-
-      ids = [reference['tag_id'], reference['external_tag_id']].compact
-      ((ids.any? || reference['missing'] == true) &&
-          ids.all? { _1.is_a?(Integer) && _1.positive? } &&
-          (!reference.key?('missing') || (reference['missing'] == true && ids.empty?)) &&
-          (!reference.key?('negative') || [true, false].include?(reference['negative'])))
-    end
-    raise ArgumentError unless valid
-
-    references.map { _1['name'] } == literals ? references : nil
-  end
-
-  def self.literals(locale, raw, references: nil)
-    raw.split.each_with_index.filter_map do |token, index|
-      next if token.downcase == 'type:or'
-
-      reference = references&.[](index)
-      name = token.sub(/\Anot:/i, '')
-      name = TagName.canonicalise(locale, name).first unless reference
-      negative = reference&.key?('negative') ? reference['negative'] :
-          token.downcase.start_with?('not:')
-      { name:, negative:, reference: }
-    end
-  end
-
-  def self.references(locale, literal)
-    reference = literal[:reference]
-    return [reference['tag_id'], reference['external_tag_id']] if reference
-
-    [TagName.where(language_code: locale.language_code, name: literal[:name])
-         .distinct.pluck(:tag_id),
-     ExternalTag.where("CONCAT(platform, ':', name) = ?", literal[:name]).pluck(:id)]
-  end
-end

backend/app/services/post_version_recorder.rb

@@ -25,7 +25,6 @@ class PostVersionRecorder < VersionRecorder
       thumbnail_base: @record.thumbnail_base,
       video_ms: @record.video_ms,
       tags: @record.snapshot_tag_names.join(' '),
-      tags_json: @record.snapshot_tags_json,
       parent_post_ids: @record.snapshot_parent_post_ids.join(' '),
       original_created_from: @record.original_created_from,
       original_created_before: @record.original_created_before }

backend/app/services/query_state.rb

@@ -1,88 +0,0 @@
-# Public query grammar shared with frontend/src/lib/queryState.ts.
-class QueryState
-  SCHEMAS = {
-      'conditions' => ['field', 'operator', 'value', 'end', 'blank'],
-      'drill' => ['field', 'grain', 'value', 'value_type'],
-      'tag_refs' => ['name', 'tag_id', 'external_tag_id', 'negative', 'missing'],
-      'column_order' => ['value', 'value_type'],
-      'measures' => ['key', 'axis', 'value', 'value_type'] }.freeze
-
-  def self.list(value)
-    return [] if value.nil? || value == ''
-    raise ArgumentError unless value.is_a?(String) && value.bytesize <= 32_768
-
-    values = value.split(/ /, -1)
-    raise ArgumentError if values.length > 200 || values.any?(&:empty?) || values.uniq != values
-
-    values
-  end
-
-  def self.valid_key?(key)
-    root, index, field, extra = key.split('.', -1)
-    !extra && SCHEMAS.key?(root) && index&.match?(/\A(?:0|[1-9][0-9]?)\z/) &&
-        SCHEMAS[root].include?(field)
-  end
-
-  def self.validate!(params, allowed)
-    raise ArgumentError if params.length > 1000
-    params.each do |key, value|
-      raise ArgumentError unless allowed.include?(key) || valid_key?(key)
-      raise ArgumentError unless value.is_a?(String) && value.bytesize <= 131_072
-    end
-    SCHEMAS.each_key { records(params, _1) }
-  end
-
-  def self.records(params, key)
-    records = {}
-    params.each do |name, value|
-      next unless name.start_with?("#{ key }.")
-      raise ArgumentError unless params.key?(key) && valid_key?(name)
-
-      _root, index, field = name.split('.')
-      record = records[index.to_i] ||= {}
-      if ['blank', 'negative', 'missing'].include?(field)
-        raise ArgumentError unless ['0', '1'].include?(value)
-
-        value = value == '1'
-      elsif ['tag_id', 'external_tag_id', 'axis'].include?(field)
-        raise ArgumentError unless value.match?(/\A[0-9]{1,18}\z/)
-
-        value = value.to_i
-      end
-      record[field] = value
-    end
-    raise ArgumentError if params.key?(key) && params[key] != ''
-    raise ArgumentError unless records.keys.sort == (0...records.length).to_a
-
-    records.sort.map do |_index, record|
-      type = record.delete('value_type')
-      if ['drill', 'measures', 'column_order'].include?(key)
-        raise ArgumentError unless ['null', 'string', 'number', 'boolean'].include?(type)
-
-        raise ArgumentError unless type == 'null' || record.key?('value')
-
-        value = record['value']
-        record['value'] = case type
-                          when 'null' then nil
-                          when 'number'
-                            raise ArgumentError unless value&.match?(/\A-?[0-9]{1,18}\z/)
-                            Integer(value, 10)
-                          when 'boolean'
-                            raise ArgumentError unless ['true', 'false'].include?(value)
-                            value == 'true'
-                          else value
-                          end
-      end
-      record
-    end
-  end
-
-  def self.axes(value)
-    list(value).map do |token|
-      field, grain, extra = token.split(':', -1)
-      raise ArgumentError if extra || !field.match?(/\A[a-z_]+\z/) || grain == ''
-
-      grain ? { 'field' => field, 'grain' => grain } : { 'field' => field }
-    end
-  end
-end

backend/app/services/shared_resource_id_allocator.rb

@@ -1,137 +0,0 @@
-require 'digest'
-
-
-class SharedResourceIdAllocator
-  SPACES = {
-      'tags' => ['tags', 'external_tags'],
-      'wiki' => ['tag_descriptions', 'tag_groups'],
-      'wiki_history' => ['tag_description_versions', 'tag_group_versions'] }.freeze
-  HISTORY_IDS = {
-      'tags' => [
-          ['tag_versions', 'tag_id'],
-          ['nico_tag_versions', 'tag_id'],
-          ['tag_description_versions', 'tag_id']],
-      'wiki' => [['tag_group_versions', 'tag_group_id']] }.freeze
-  LOCK_TIMEOUT = 10
-  CONNECTION_LOCKS = ObjectSpace::WeakMap.new
-
-  class LockCleanup
-    def initialize &finish
-      @finish = finish
-    end
-
-    def trigger_transactional_callbacks? = false
-    def before_committed! = nil
-    def committed!(**) = @finish.call(self)
-    def rolledback!(**) = @finish.call(self)
-  end
-  private_constant :LockCleanup
-
-  def self.assign! record
-    space, tables = SPACES.find { |_name, names| names.include?(record.class.table_name) }
-    raise ArgumentError, 'Unknown shared ID resource' unless tables
-
-    connection = record.class.connection
-    lock_space!(record.class, space)
-
-    # 明示-tic な Id. 指定は許可する．
-    return if record.id.present?
-
-    next_id = record.class.uncached do
-      historical_max = HISTORY_IDS.fetch(space, []).map { |table, column|
-        quoted_table = connection.quote_table_name(table)
-        quoted_column = connection.quote_column_name(column)
-        connection.select_value("SELECT MAX(#{ quoted_column }) FROM #{ quoted_table }").to_i
-      }.max.to_i
-
-      [next_ids(connection, tables).max, historical_max + 1].max
-    end
-
-    record.id = next_id
-  end
-
-  def self.lock_space! record_class, space
-    raise ArgumentError, 'Unknown shared ID resource' unless SPACES.key?(space)
-
-    connection = record_class.connection
-    transaction = connection.current_transaction
-    unless transaction.open?
-      raise ActiveRecord::ActiveRecordError, 'Shared ID requires a transaction'
-    end
-
-    database = connection.pool.db_config.database
-    name = "btrc:ids:#{ Digest::SHA256.hexdigest(database)[0, 16] }:#{ space }"
-    held_locks = CONNECTION_LOCKS[connection] ||= {}
-    return if held_locks.key?(name)
-
-    quoted_name = connection.quote(name)
-    acquired = record_class.uncached do
-      connection.select_value("SELECT GET_LOCK(#{ quoted_name }, #{ LOCK_TIMEOUT })")
-    end
-    unless acquired.to_i == 1
-      raise ActiveRecord::LockWaitTimeout, "Shared ID lock unavailable: #{ space }"
-    end
-    held_locks[name] = true
-
-    finish = lambda do |cleanup|
-      next unless held_locks.key?(name)
-
-      # Savepoint callbacks may run before the outer transaction ends.
-      # Keep one session lock and transfer its cleanup to the remaining parent.
-      parent = connection.current_transaction
-      if parent.open?
-        connection.add_transaction_record(cleanup)
-        next
-      end
-      result = record_class.uncached do
-        connection.select_value("SELECT RELEASE_LOCK(#{ quoted_name })")
-      end
-      unless result.to_i == 1
-        raise ActiveRecord::ActiveRecordError, "Shared ID lock release failed: #{ space }"
-      end
-      held_locks.delete(name)
-    rescue StandardError
-      # Never return a session with an untracked advisory lock to the pool.
-      begin
-        connection.disconnect!
-      ensure
-        held_locks.clear
-      end
-      raise
-    end
-    begin
-      # Rails finalises remaining records in ensure if another callback raises.
-      connection.add_transaction_record(LockCleanup.new(&finish))
-    rescue StandardError
-      begin
-        connection.disconnect!
-      ensure
-        held_locks.clear
-      end
-      raise
-    end
-  end
-
-  def self.next_ids connection, tables
-    expiry = connection.select_value('SELECT @@SESSION.information_schema_stats_expiry').to_i
-    begin
-      connection.execute('SET SESSION information_schema_stats_expiry = 0')
-      quoted_tables = tables.map { connection.quote(_1) }.join(', ')
-      rows = connection.select_all(<<~SQL).to_a
-        SELECT TABLE_NAME, ENGINE, AUTO_INCREMENT
-        FROM information_schema.tables
-        WHERE TABLE_SCHEMA = DATABASE() AND TABLE_NAME IN (#{ quoted_tables })
-      SQL
-      persistent = rows.all? do |row|
-        row['ENGINE'] == 'InnoDB' && row['AUTO_INCREMENT'].to_i.positive?
-      end
-      unless rows.size == tables.size && persistent
-        raise ActiveRecord::ActiveRecordError, 'Persistent InnoDB counters are required'
-      end
-      rows.map { _1.fetch('AUTO_INCREMENT').to_i }
-    ensure
-      connection.execute("SET SESSION information_schema_stats_expiry = #{ expiry }")
-    end
-  end
-  private_class_method :next_ids
-end

backend/app/services/tag_description_operation.rb

@@ -1,69 +0,0 @@
-class TagDescriptionOperation
-  class Conflict < StandardError
-    ;
-  end
-
-  def self.create! locale, tag:, body:, created_by_user:, message: nil
-    ApplicationRecord.transaction do
-      SharedResourceIdAllocator.lock_space!(Tag, 'tags')
-      tag.lock!
-      raise ActiveRecord::RecordNotUnique if TagDescription.exists?(
-          tag_id: tag.id, language_code: locale.language_code)
-
-      description = TagDescription.new(tag:, language_code: locale.language_code)
-      body = normalise_body!(description, body)
-      record_version!(description:, body:, user: created_by_user)
-
-      description
-    end
-  end
-
-  def self.update! locale, tag:, body:, updated_by_user:, message: nil, base_revision_id: nil
-    ApplicationRecord.transaction do
-      SharedResourceIdAllocator.lock_space!(Tag, 'tags')
-      tag.lock!
-      description = TagDescription.lock.find_by!(
-          tag_id: tag.id, language_code: locale.language_code)
-      body = normalise_body!(description, body)
-      latest = TagDescriptionVersion.find_by(
-          tag_id: tag.id, language_code: locale.language_code,
-          version_no: description.version_no)
-      if base_revision_id.present? && latest&.id != base_revision_id.to_i
-        raise Conflict, '競合が発生してゐます．'
-      end
-      next if latest && normalise_body(description.body) == body
-
-      record_version!(description:, body:, user: updated_by_user)
-    end
-  end
-
-  def self.record_version! description:, body:, user:
-    creating = description.new_record?
-    version_no = creating ? 1 : description.version_no + 1
-    description.created_by_user_id = user.id if creating
-    description.update!(body:, version_no:, updated_by_user_id: user.id)
-    TagDescriptionVersion.create!(
-        tag_id: description.tag_id, language_code: description.language_code,
-        version_no:, event_type: creating ? 'create' : 'update', body:,
-        source_language_code: description.source_language_code,
-        source_version_no: description.source_version_no,
-        created_by_user_id: user.id, created_at: Time.current)
-  end
-  private_class_method :record_version!
-
-  def self.normalise_body body
-    body.to_s.gsub(/\r\n?/, "\n")
-        .encode('UTF-8', invalid: :replace, undef: :replace, replace: '🖕')
-        .sub(/\n+\z/, '')
-  end
-  private_class_method :normalise_body
-
-  def self.normalise_body! description, body
-    normalised = normalise_body(body)
-    return normalised if normalised.present?
-
-    description.errors.add(:body, :blank)
-    raise ActiveRecord::RecordInvalid, description
-  end
-  private_class_method :normalise_body!
-end

backend/app/services/tag_group_markdown_parser.rb

@@ -1,178 +0,0 @@
-require 'kramdown'
-
-class TagGroupMarkdownParser
-  PREFIX = TagGroupRepr::PREFIX
-
-  class InvalidDocument < StandardError
-    ;
-  end
-
-  def self.parse(locale, body:, group: nil)
-    new(body:, group:).parse(locale)
-  end
-
-  def initialize(body:, group:)
-    @body = body.to_s.gsub(/\r\n?/, "\n")
-    @group = group
-  end
-
-  def parse locale
-    parse_body = @body.gsub(/([^\n])\n([\x23]{1,6}[ \t]+\S)/, "\\1\n\n\\2")
-    root = Kramdown::Document.new(parse_body, input: 'GFM', auto_ids: false).root
-    placements = []
-    description_parts = []
-    headings = []
-
-    root.children.each do |node|
-      case node.type
-      when :blank
-        next
-      when :header
-        level = node.options.fetch(:level)
-        name = inline_text(node).strip
-        invalid!('Section 名は必須です．') if name.blank?
-
-        headings.pop while headings.any? && headings.last.fetch(:level) >= level
-        parent = headings.last&.fetch(:node)
-        section = { 'section_name' => name, 'description' => nil,
-                    'children' => [] }
-        (parent ? parent.fetch('children') : placements) << section
-        headings << { level:, node: section }
-      when :p
-        text = inline_markdown(node).strip
-        next if text.empty?
-
-        if headings.any?
-          section = headings.last.fetch(:node)
-          section['description'] = [section['description'], text].compact.join("\n\n")
-        else
-          description_parts << text
-        end
-      when :ul
-        invalid!('未対応のリスト項目があります．') if node.attr.any?
-        target = headings.any? ? headings.last.fetch(:node).fetch('children') : placements
-        target.concat(node.children.map { parse_list_item(locale, _1) })
-      else
-        invalid!("未対応の Markdown 構文 #{ node.type.inspect } です．")
-      end
-    end
-
-    description = description_parts.presence&.join("\n\n")
-    { description:, placements:, members: build_members(placements),
-      compatibility_body: @body.sub(/\n+\z/, '') }
-  rescue Kramdown::Error => error
-    invalid!("Markdown を解析できません：#{ error.message }")
-  end
-
-  private
-
-  def parse_list_item locale, item
-    invalid!('リスト項目ではありません．') unless item.type == :li
-    invalid!('未対応のリスト項目があります．') if item.attr.any?
-
-    nested, own = item.children.partition { |child| child.type == :ul }
-    invalid!('順序付きリストは使用できません．') if own.any? { _1.type == :ol }
-    own.reject! { _1.type == :blank }
-    invalid!('未対応のリスト項目があります．') unless own.all? do |child|
-      [:p, :text].include?(child.type)
-    end
-    invalid!('リスト項目は一つの段落で指定してください．') if own.size != 1
-
-    label = own.map { inline_text(_1) }.join(' ').strip
-    invalid!('空のリスト項目は使用できません．') if label.blank?
-
-    node =
-        if (group_name = TagGroupRepr.name_from_title(label))
-          group_name = group_name.strip
-          child_group = TagGroup.find_by(name: group_name)
-          invalid!("Tag Group #{ group_name.inspect } を解決できません．") unless child_group
-          invalid!('自分自身を含めることはできません．') if child_group.id == @group&.id
-          { 'tag_group_id' => child_group.id }
-        elsif label.end_with?(':', '：')
-          section_name = label.sub(/[:：]\z/, '').strip
-          invalid!('Section 名は必須です．') if section_name.blank?
-          { 'section_name' => section_name, 'description' => nil }
-        else
-          { 'tag_id' => resolve_tag(locale, label) }
-        end
-
-    node['children'] = nested.flat_map do |list|
-      invalid!('未対応の入れ子リストです．') unless list.type == :ul
-      list.children.map { parse_list_item(locale, _1) }
-    end
-    node
-  end
-
-  def resolve_tag locale, name
-    names = TagName.where(language_code: locale.language_code, name:).to_a
-    tag_ids = names.filter_map(&:tag_id).uniq
-    invalid!("Tag #{ name.inspect } を解決できません．") if tag_ids.empty?
-    invalid!("Tag 名 #{ name.inspect } が曖昧です．") if tag_ids.many?
-
-    tag_ids.first
-  end
-
-  def build_members(nodes)
-    references = []
-    visit = lambda do |items|
-      items.each do |node|
-        references << node.slice('tag_id', 'tag_group_id') if
-            node.key?('tag_id') || node.key?('tag_group_id')
-        visit.call(node.fetch('children'))
-      end
-    end
-    visit.call(nodes)
-    references.uniq
-  end
-
-  def inline_text(node)
-    if node.attr.any?
-      invalid!("未対応のインライン構文 #{ node.type.inspect } です．")
-    end
-
-    case node.type
-    when :text
-      node.value.to_s
-    when :p, :header
-      node.children.map { inline_text(_1) }.join
-    else
-      invalid!("未対応のインライン構文 #{ node.type.inspect } です．")
-    end
-  end
-
-  def inline_markdown(node)
-    case node.type
-    when :text
-      node.value.to_s
-    when :codespan
-      text = node.value.to_s
-      longest = text.scan(/`+/).map(&:length).max || 0
-      delimiter = '`' * (longest + 1)
-      padded = text.start_with?('`', ' ') || text.end_with?('`', ' ')
-      content = padded ? " #{ text } " : text
-      "#{ delimiter }#{ content }#{ delimiter }"
-    when :softbreak
-      "\n"
-    when :br
-      "  \n"
-    when :p
-      node.children.map { inline_markdown(_1) }.join
-    when :a
-      href = node.attr.fetch('href')
-      label = node.children.map { inline_markdown(_1) }.join
-      title = node.attr['title']&.gsub('"', '\\"')
-      suffix = title ? " \"#{ title }\"" : ''
-      "[#{ label }](#{ href }#{ suffix })"
-    when :em
-      "*#{ node.children.map { inline_markdown(_1) }.join }*"
-    when :strong
-      "**#{ node.children.map { inline_markdown(_1) }.join }**"
-    else
-      invalid!("未対応の説明文構文 #{ node.type.inspect } です．")
-    end
-  end
-
-  def invalid! message
-    raise InvalidDocument, message
-  end
-end

backend/app/services/tag_group_markdown_renderer.rb

@@ -1,123 +0,0 @@
-class TagGroupMarkdownRenderer
-  class UnsupportedStructure < StandardError
-    ;
-  end
-
-  def self.render group
-    new(group).render
-  end
-
-  def initialize group
-    @group = group
-  end
-
-  def render
-    placements = @group.fetch('placements')
-    validate_membership!(placements)
-    blocks = []
-    if @group['description'].present?
-      blocks << clean_text(@group['description'])
-    end
-
-    first_section = placements.index { _1.key?('section_name') }
-
-    if first_section
-      if placements.drop(first_section).any? { !_1.key?('section_name') }
-        raise UnsupportedStructure,
-            '見出し Section の後に最上位 Placement は配置できません．'
-      end
-
-      if first_section.positive?
-        blocks << render_list(placements.take(first_section), 0)
-      end
-
-      blocks.concat(placements.drop(first_section).map { render_heading(_1, 2) })
-    elsif placements.any?
-      blocks << render_list(placements, 0)
-    end
-
-    blocks.compact.reject(&:empty?).join("\n").sub(/\n+\z/, '')
-  end
-
-  private
-
-  def validate_membership! placements
-    placed = placement_members(placements).uniq.sort
-    members = @group.fetch('members').map { |member|
-      if member['tag']
-        ['tag', member.fetch('tag').fetch('id')]
-      else
-        ['tag_group', member.fetch('tag_group').fetch('id')]
-      end
-    }.uniq.sort
-    missing = members - placed
-    if missing.any?
-      raise UnsupportedStructure, '掲載位置のない membership は Markdown にできません．'
-    end
-  end
-
-  def placement_members nodes
-    nodes.flat_map do |node|
-      own = if node['tag']
-        [['tag', node.fetch('tag').fetch('id')]]
-      elsif node['tag_group']
-        [['tag_group', node.fetch('tag_group').fetch('id')]]
-      else
-        []
-      end
-      own + placement_members(node.fetch('children'))
-    end
-  end
-
-  def render_heading section, level
-    header = "#{ '#' * level } #{ clean_text(section.fetch('section_name')) }"
-    blocks = [header]
-    blocks << clean_text(section['description']) if section['description'].present?
-    children = section.fetch('children')
-    section_indexes = children.each_index.select { |index| children[index].key?('section_name') }
-    if section_indexes.any?
-      first_section = section_indexes.first
-      if children[first_section..].any? { !_1.key?('section_name') }
-        raise UnsupportedStructure,
-              '見出し Section と兄弟 Placement の交互配置は Markdown にできません．'
-      end
-      list_nodes = children[0...first_section]
-      blocks << render_list(list_nodes, 0) if list_nodes.any?
-      blocks.concat(children[first_section..].map { render_heading(_1, level + 1) })
-    elsif children.any?
-      blocks << render_list(children, 0)
-    end
-    blocks.join("\n")
-  end
-
-  def render_list nodes, depth
-    nodes.map do |node|
-      indent = '  ' * depth
-      if node.key?('section_name')
-        raise UnsupportedStructure,
-            'リスト内 Section の説明文は Markdown にできません．' if node['description'].present?
-
-        children = node.fetch('children')
-        head = "#{ indent }- #{ clean_text(node.fetch('section_name')) }:"
-        tail = render_list(children, depth + 1) if children.any?
-        [head, tail].compact.join("\n")
-      else
-        label = if node['tag']
-          node.fetch('tag').fetch('name')
-        else
-          "#{ TagGroupRepr::PREFIX }#{ node.fetch('tag_group').fetch('name') }"
-        end
-        raise UnsupportedStructure, '名称のない Placement は Markdown にできません．' if label.blank?
-
-        head = "#{ indent }- #{ clean_text(label) }"
-        children = node.fetch('children')
-        tail = render_list(children, depth + 1) if children.any?
-        [head, tail].compact.join("\n")
-      end
-    end.join("\n")
-  end
-
-  def clean_text value
-    value.to_s.gsub(/\r\n?/, "\n").lines.map(&:rstrip).join("\n").strip
-  end
-end

backend/app/services/tag_group_operation.rb

@@ -1,362 +0,0 @@
-class TagGroupOperation
-  MAX_PLACEMENT_DEPTH = 5
-
-  def self.create!(locale, name:, description:, members:, placements:, actor:,
-                   require_compatibility: false)
-    ApplicationRecord.transaction do
-      group = TagGroup.new(version_no: 1, created_by_user: actor,
-                           updated_by_user: actor)
-      attributes = prepare!(locale:, group:, name:, description:, members:, placements:)
-      group.assign_attributes(name: attributes.fetch(:name),
-                              description: attributes.fetch(:description))
-      group.save!
-      persist_structure!(group:, attributes:, actor:)
-      validate_compatibility!(locale, group) if require_compatibility
-      persist_version!(group:, attributes:, actor:, event_type: 'create')
-      group
-    end
-  end
-
-  def self.update!(locale, group:, name:, description:, members:, placements:, actor:,
-                   require_compatibility: false)
-    ApplicationRecord.transaction do
-      group.lock!
-      attributes = prepare!(locale:, group:, name:, description:, members:, placements:)
-      if unchanged?(group, attributes)
-        validate_compatibility!(locale, group) if require_compatibility
-        next group
-      end
-
-      group.update!(name: attributes.fetch(:name),
-                    description: attributes.fetch(:description),
-                    version_no: group.version_no + 1,
-                    updated_by_user: actor)
-      persist_structure!(group:, attributes:, actor:)
-      validate_compatibility!(locale, group) if require_compatibility
-      persist_version!(group:, attributes:, actor:, event_type: 'update')
-      group
-    end
-  end
-
-  def self.validate_compatibility! locale, group
-    TagGroupMarkdownRenderer.render(TagGroupRepr.one(locale, group))
-  rescue TagGroupMarkdownRenderer::UnsupportedStructure => error
-    invalid!(group, :base, error.message)
-  end
-  private_class_method :validate_compatibility!
-
-  def self.prepare!(locale:, group:, name:, description:, members:, placements:)
-    record = group
-    normalized_name = name.to_s.strip
-    invalid!(record, :name, 'グループ名は必須です．') if normalized_name.blank?
-    if normalized_name.length > 255
-      invalid!(record, :name, 'グループ名は 255 文字以内で入力してください．')
-    end
-    invalid!(record, :description, '説明文は文字列で入力してください．') unless
-        description.nil? || description.is_a?(String)
-
-    record.name = normalized_name
-    record.description = description
-    unless record.valid?
-      raise ActiveRecord::RecordInvalid, record
-    end
-
-    member_rows = normalize_members(record, members)
-    placement_tree = normalize_placements(record, placements)
-    validate_references!(locale, record, member_rows, placement_tree)
-    validate_containment!(record, member_rows)
-    { name: normalized_name, description:, members: member_rows,
-      placements: placement_tree }
-  end
-  private_class_method :prepare!
-
-  def self.unchanged?(group, attributes)
-    unless (group.name_in_database == attributes.fetch(:name) &&
-            group.description_in_database == attributes.fetch(:description))
-      return false
-    end
-
-    members = TagGroupTag.where(tag_group_id: group.id).pluck(:tag_id)
-        .map { { 'tag_id' => _1 } }
-    members.concat(TagGroupGroup.where(tag_group_id: group.id)
-        .pluck(:child_tag_group_id).map { { 'tag_group_id' => _1 } })
-    requested_members = attributes.fetch(:members).map do |member|
-      member.except('version_no')
-    end
-    return false unless members.to_set == requested_members.to_set
-
-    rows = TagGroupPlacement.where(tag_group_id: group.id).order(:position)
-        .group_by(&:parent_position)
-    build = lambda do |parent_position|
-      rows.fetch(parent_position, []).map do |row|
-        node = if row.tag_id
-          { 'tag_id' => row.tag_id }
-        elsif row.child_tag_group_id
-          { 'tag_group_id' => row.child_tag_group_id }
-        else
-          { 'section_name' => row.section_name, 'description' => row.description }
-        end
-        node.merge('children' => build.call(row.position))
-      end
-    end
-    build.call(nil) == comparable_placements(attributes.fetch(:placements))
-  end
-  private_class_method :unchanged?
-
-  def self.comparable_placements(nodes)
-    nodes.map do |node|
-      result = node.dup
-      result['description'] = node['description'] if node.key?('section_name')
-      result['children'] = comparable_placements(node.fetch('children'))
-      result
-    end
-  end
-  private_class_method :comparable_placements
-
-  def self.normalize_members(record, members)
-    invalid!(record, :members, 'members は配列で指定してください．') unless members.is_a?(Array)
-
-    result = members.map do |member|
-      member = stringify_keys(member)
-      unless member.is_a?(Hash) && member.keys.one? &&
-             (member.key?('tag_id') || member.key?('tag_group_id'))
-        invalid!(record, :members, '各 member は tag_id または tag_group_id を一つ指定してください．')
-      end
-      key = member.keys.first
-      id = integer_id(record, member.fetch(key), :members)
-      { key => id }
-    end
-    invalid!(record, :members, 'membership に重複があります．') unless result.uniq == result
-    result
-  end
-  private_class_method :normalize_members
-
-  def self.normalize_placements(record, placements)
-    invalid!(record, :placements, 'placements は配列で指定してください．') unless
-        placements.is_a?(Array)
-
-    normalize_placement_nodes(record, placements, 1)
-  end
-  private_class_method :normalize_placements
-
-  def self.normalize_placement_nodes(record, nodes, depth)
-    if depth > MAX_PLACEMENT_DEPTH && nodes.any?
-      invalid!(record, :placements, '配置の深さは 5 階層までです．')
-    end
-
-    nodes.map do |raw_node|
-      node = stringify_keys(raw_node)
-      unless node.is_a?(Hash) && node.key?('children')
-        invalid!(record, :placements, '各 Placement に children 配列が必要です．')
-      end
-      children = node.delete('children')
-      unless children.is_a?(Array)
-        invalid!(record, :placements, 'children は配列で指定してください．')
-      end
-
-      kinds = ['tag_id', 'tag_group_id', 'section_name'].select { node.key?(_1) }
-      invalid!(record, :placements, 'Placement の種類を一つ指定してください．') unless
-          kinds.one?
-      kind = kinds.first
-      allowed = [kind, 'description']
-      invalid!(record, :placements, '不明な Placement 項目があります．') unless
-          (node.keys - allowed).empty?
-
-      case kind
-      when 'tag_id', 'tag_group_id'
-        node[kind] = integer_id(record, node.fetch(kind), :placements)
-        if node['description'].present?
-          invalid!(record, :placements, 'Tag / Tag Group Placement に説明文は指定できません．')
-        end
-        node.delete('description')
-      when 'section_name'
-        section_name = node.fetch(kind)
-        unless section_name.is_a?(String) && section_name.strip.present? &&
-               section_name.length <= 255
-          invalid!(record, :placements, 'Section 名は 1〜255 文字で入力してください．')
-        end
-        node[kind] = section_name.strip
-        unless node['description'].nil? || node['description'].is_a?(String)
-          invalid!(record, :placements, 'Section の説明文は文字列で入力してください．')
-        end
-      end
-
-      node['children'] = normalize_placement_nodes(record, children, depth + 1)
-      node
-    end
-  end
-  private_class_method :normalize_placement_nodes
-
-  def self.validate_references!(locale, record, members, placements)
-    membership = members.to_set
-    placed = placement_references(placements).uniq
-    missing = placed - membership.to_a
-    invalid!(record, :placements, 'Placement の対象は membership に含めてください．') if
-        missing.any?
-
-    tag_ids = members.filter_map { _1['tag_id'] }
-    group_ids = members.filter_map { _1['tag_group_id'] }
-    tags = Tag.where(id: tag_ids).index_by(&:id)
-    groups = TagGroup.where(id: group_ids).index_by(&:id)
-    invalid!(record, :members, '存在しない Tag が含まれています．') if
-        tags.size != tag_ids.uniq.size
-    invalid!(record, :members, '存在しない Tag Group が含まれています．') if
-        groups.size != group_ids.uniq.size
-
-    tag_versions = TagVersion.where(tag_id: tag_ids).pluck(:tag_id, :version_no).to_set
-    group_versions = TagGroupVersion.where(tag_group_id: group_ids)
-        .pluck(:tag_group_id, :version_no).to_set
-    tag_names = TagName.where(
-        tag_id: tag_ids, language_code: locale.language_code, primary_flg: true)
-        .pluck(:tag_id).uniq
-
-    members.each do |member|
-      if member.key?('tag_id')
-        id = member.fetch('tag_id')
-        invalid!(record, :members, :invalid) unless
-            tag_names.include?(id)
-        version_no = tags.fetch(id).version_no
-        unless tag_versions.include?([id, version_no])
-          invalid!(record, :members, "Tag ##{ id } の Version がありません．")
-        end
-        member['version_no'] = version_no
-      else
-        id = member.fetch('tag_group_id')
-        version_no = groups.fetch(id).version_no
-        unless group_versions.include?([id, version_no])
-          invalid!(record, :members, "Tag Group ##{ id } の Version がありません．")
-        end
-        member['version_no'] = version_no
-      end
-    end
-  end
-  private_class_method :validate_references!
-
-  def self.validate_containment!(record, members)
-    target_id = record.id
-    return unless target_id
-
-    children = members.filter_map { _1['tag_group_id'] }
-    edges = TagGroupGroup.pluck(:tag_group_id, :child_tag_group_id)
-        .group_by(&:first).transform_values { _1.map(&:last) }
-    edges[target_id] = children
-    visiting = Set.new
-    visited = Set.new
-    visit = lambda do |group_id|
-      invalid!(record, :members, 'Tag Group の循環包含はできません．') if
-          group_id == target_id && visiting.include?(group_id)
-      return if visited.include?(group_id)
-
-      visiting.add(group_id)
-      edges.fetch(group_id, []).each do |child_id|
-        invalid!(record, :members, 'Tag Group の循環包含はできません．') if
-            child_id == target_id
-        visit.call(child_id)
-      end
-      visiting.delete(group_id)
-      visited.add(group_id)
-    end
-    children.each { visit.call(_1) }
-  end
-  private_class_method :validate_containment!
-
-  def self.persist_structure!(group:, attributes:, actor:)
-    tag_members = TagGroupTag.where(tag_group_id: group.id).index_by(&:tag_id)
-    group_members = TagGroupGroup.where(tag_group_id: group.id).index_by(&:child_tag_group_id)
-    attributes.fetch(:members).each do |member|
-      if member.key?('tag_id')
-        next if tag_members.delete(member.fetch('tag_id'))
-
-        TagGroupTag.create!(tag_group_id: group.id, tag_id: member.fetch('tag_id'),
-                            created_by_user: actor)
-      else
-        next if group_members.delete(member.fetch('tag_group_id'))
-
-        TagGroupGroup.create!(tag_group_id: group.id,
-                              child_tag_group_id: member.fetch('tag_group_id'),
-                              created_by_user: actor)
-      end
-    end
-    existing = TagGroupPlacement.where(tag_group_id: group.id).index_by(&:position)
-    persist_placements!(group:, nodes: attributes.fetch(:placements), actor:, existing:)
-    existing.values.sort_by(&:position).reverse_each(&:delete)
-    tag_members.each_value(&:delete)
-    group_members.each_value(&:delete)
-  end
-  private_class_method :persist_structure!
-
-  def self.persist_placements!(group:, nodes:, actor:, existing:, parent_position: nil,
-                               next_position: [0])
-    nodes.each do |node|
-      next_position[0] += 1
-      position = next_position[0]
-      row = existing.delete(position) || TagGroupPlacement.new(
-          tag_group_id: group.id, position:, created_by_user: actor)
-      row.assign_attributes(
-          parent_position:,
-          section_name: node['section_name'], description: node['description'],
-          tag_id: node['tag_id'], child_tag_group_id: node['tag_group_id'])
-      if row.new_record? || row.changed?
-        row.updated_by_user = actor
-        row.save!
-      end
-      persist_placements!(group:, nodes: node.fetch('children'), actor:, existing:,
-                          parent_position: position, next_position:)
-    end
-  end
-  private_class_method :persist_placements!
-
-  def self.persist_version!(group:, attributes:, actor:, event_type:)
-    TagGroupVersion.create!(tag_group: group, version_no: group.version_no,
-                            event_type:, name: group.name,
-                            description: group.description,
-                            members: attributes.fetch(:members),
-                            placements: attributes.fetch(:placements),
-                            created_by_user: actor)
-  end
-  private_class_method :persist_version!
-
-  def self.placement_references(nodes)
-    nodes.flat_map do |node|
-      own = if node.key?('tag_id')
-        [{ 'tag_id' => node.fetch('tag_id') }]
-      elsif node.key?('tag_group_id')
-        [{ 'tag_group_id' => node.fetch('tag_group_id') }]
-      else
-        []
-      end
-      own + placement_references(node.fetch('children'))
-    end
-  end
-  private_class_method :placement_references
-
-  def self.stringify_keys(value)
-    case value
-    when ActionController::Parameters
-      value.to_unsafe_h.deep_stringify_keys
-    when Hash
-      value.deep_stringify_keys
-    else
-      value
-    end
-  end
-  private_class_method :stringify_keys
-
-  def self.integer_id(record, value, attribute)
-    integer = if value.is_a?(Integer)
-      value
-    elsif value.is_a?(String) && value.match?(/\A[0-9]+\z/)
-      value.to_i
-    end
-    invalid!(record, attribute, 'ID は正の整数で指定してください．') unless integer
-    invalid!(record, attribute, 'ID は正の整数で指定してください．') unless integer.positive?
-    integer
-  end
-  private_class_method :integer_id
-
-  def self.invalid!(record, attribute, message)
-    record.errors.add(attribute, message)
-    raise ActiveRecord::RecordInvalid, record
-  end
-  private_class_method :invalid!
-end