#41

backend/AGENTS.md

@@ -1,20 +1,5 @@
 # backend/AGENTS.md
 
-## Mandatory security and URL contracts
-
-- URL クエリへの JSON 格納を禁止する。Base64・16進数等による迂回も禁止。
-  単純な配列は半角スペース区切り、構造化条件は番号付きの個別キーで表す。
-  未指定と明示的な空選択を区別し、自由入力を区切り文字で分割しない。
-  Frontend / Backend で同じ文法・上限・検証を使い、新しい URL 状態の追加前に確認する。
-- 一度除票されたリソースの除票以前（除票版を含む）の履歴は admin 以外へ公開しない。
-  恢復しても解除しない。すべての後続除票を確定版番号と正しい複合キーで判定する。
-  本文・属性・ID・版番号・関連参照・旧値を API / SSR / キャッシュ / 出力から漏らさない。
-  サーバーが確定した current_user の role のみを認可に使用する。
-  秘匿版も安全な全体件数には含めるが、公開明細・内訳・検索件数とは分離する。
-  秘匿属性を条件・軸・ソート・差分件数から推測できる実装を禁止する。
-- 履歴読み取りは HistoryAccess の共通認可を必須とする。書込用 relation を公開しない。
-  export は生成時と取得時に認可し、権限変更・除票後に古いファイルを再利用しない。
-
 ## Scope
 
 These rules apply to work under `backend/`.
@@ -82,214 +67,22 @@ pass or the remaining failure is clearly blocked.
 Before changing behavior, inspect the matching route, controller, model,
 service, representation, and spec.
 
-## Shared backend systems
-
-Before adding backend behaviour, search the existing backend first. At minimum,
-check these locations:
-
-- `app/controllers`
-- `app/controllers/concerns`
-- `app/models`
-- `app/models/concerns`
-- `app/representations`
-- `app/services`
-- `app/services/*`
-- `app/jobs`
-- `lib`
-- `lib/tasks`
-- `config/initializers`
-
-Do not infer commonality from directory names alone. Read the actual
-responsibility and representative usage sites.
-
-### Controller reuse
-
-Before adding logic to a controller, inspect:
-
-- `ApplicationController` authentication, authorization, BAN, and IP BAN
-- existing render and validation-error helpers
-- existing param parsing
-- controller concerns
-- the controller for the same resource
-- existing services
-- existing representations
-
-Keep controllers focused on:
-
-- authentication and authorization
-- parameter intake
-- service and model invocation
-- HTTP status selection
-- representation selection
-
-Do not reimplement these per controller when an existing path already owns
-them:
-
-- authentication and role checks
-- validation error JSON
-- URL normalisation
-- tag normalisation
-- thumbnail handling
-- version recording
-- complex transactions
-- external HTTP fetching
-- response representation assembly
-
-### Authentication, authorization, and BAN
-
-Treat these as the canonical backend entrypoints:
-
-- `ApplicationController#authenticate_user`
-- `current_user`
-- `X-Transfer-Code`
-- `reject_banned_ip_address!`
-- `reject_banned_user!`
-- `gte_member?`
-- `admin?`
-
-Do not create feature-local permission services, role comparisons, or header
-parsing when the existing authentication boundary already owns the behaviour.
-If the current boundary is insufficient, extend it minimally instead of adding
-another permission path.
-
-### Representations
-
-If an endpoint for the same resource already uses `app/representations`, do not
-assemble a separate JSON shape directly inside the controller without first
-checking the existing representation contract.
-
-Inspect at least:
-
-- `PostRepr`
-- `TagRepr`
-- `MaterialRepr`
-- `TheatreRepr`
-- `UserRepr`
-- `WikiPageRepr`
-- `DeerjikistRepr`
-
-When a lightweight response is genuinely different in purpose, keep it
-deliberate and compatible with the surrounding contracts. Do not force every
-identifier list into a large representation, but do not fork the same resource
-shape casually either.
-
-### Domain services
-
-When work touches multiple models, transactions, external APIs, file handling,
-history creation, or multi-step workflow, search `app/services` first.
-
-At minimum, search for existing services in these responsibility areas:
-
-- version recorder and versioning
-- wiki commit
-- YouTube or Google Drive API client
-- material sync or ZIP export
-- similarity calculation
-- theatre selection or skip finalisation
-- metadata, thumbnail, or file processing
-- URL normaliser or sanitisation
-- import or export
-- preview safety or HTTP fetch
-
-Do not create a same-responsibility service under another namespace or another
-name. If an existing service is close, extend that API minimally instead of
-wrapping it in a feature-local service.
-
-### Versioning
-
-When a feature writes history, snapshots, or restore roots, search the existing
-versioning path first. At minimum, inspect:
-
-- `VersionRecorder`
-- `PostVersionRecorder`
-- `TagVersionRecorder`
-- `TagVersioning`
-- `MaterialVersionRecorder`
-- `NicoTagVersionRecorder`
-- `WikiVersionRecorder`
-
-Do not implement history writes in controllers, callbacks, or ad hoc feature
-services when the recorder layer already owns the transaction boundary and
-meaning.
-
-### Normalisation, sanitisation, and parsing
-
-For URLs, tag names, times, video durations, identifiers, and paths, search the
-existing normaliser, sanitisation rule, parser, and model-callback path first.
-
-Do not let frontend, controller, service, and model each invent different rules
-for the same value. Use one canonical normalisation path and keep input
-validation distinct from pre-persistence normalisation.
-
-### External HTTP and URL safety
-
-When fetching external URLs, reuse the existing preview-safety stack. Search at
-least for:
-
-- URL safety
-- redirect validation
-- response size limits
-- timeouts
-- network failure mapping
-- HTML metadata extraction
-- known-site extraction
-- thumbnail fetching
-
-Do not add direct `Net::HTTP`, `Faraday`, or equivalent feature-local HTTP code
-that reimplements SSRF checks, redirect restrictions, size limits, or timeouts.
-If the current fetcher is insufficient, extend its existing safety contract.
-
-### Storage, files, and Active Storage
-
-When handling files, thumbnails, ZIP output, object storage, or Active Storage
-blobs, inspect existing storage helpers, exporters, thumbnail generators, and
-checksum helpers first. Do not reimplement the same attach, export path,
-download, resize, or checksum flow in a controller or one-off service.
-
-### Concerns
-
-Do not create controller or model concerns merely because some code is shared.
-Use a concern only when multiple classes share the same lifecycle, macro,
-callback, or tightly cohesive behaviour. Utility collections belong in explicit
-objects or services, not in `CommonConcern`, `SharedMethods`, or `Utils`.
-
-### Model boundaries
-
-Model-specific invariants, associations, validations, and normalisation may
-live in the model. Multi-model workflow, external access, complex transaction
-flow, and feature orchestration belong in services. Do not hide feature
-workflow in model callbacks.
-
-### Transactions, locking, and race handling
-
-If transactions, locking, idempotency, or race recovery already exist in a
-service or model method, do not add a second implementation in a controller or
-new service. Inspect the existing transaction boundary first, avoid wrapping
-the same operation in needless nested transactions, and handle unique-constraint
-races according to the target constraint's business meaning.
-
 ## Ruby style
 
 - Prefer precise, minimal changes.
 - Use single quotes unless interpolation or escaping makes double quotes better.
 - Do not put a space before Ruby method-call parentheses.
-- For `render`-family method calls, omit parentheses even when passing
-  keyword arguments.
 - Never put a line break immediately before `)` in Ruby.
 - Do not use `%w` or `%i` in new Ruby code.
 - Never write a Ruby line longer than 99 characters.
 - Aim to keep Ruby lines within 79 characters where practical.
 - For small Ruby method definitions that take keyword arguments, match the
   local no-parentheses style when nearby code uses it.
-- When an `if` condition is split across multiple lines and combines clauses
-  with `&&` or `||`, wrap the whole condition in parentheses.
 - Treat Ruby hash `{ ... }` style and Ruby block `{ ... }` style as separate
   rules.
 - Do not format Ruby hashes like Ruby blocks.
 - For Ruby hashes, keep the closing `}` on the same line as the final pair.
 - Keep the first pair on the same line as `{` by default.
-- Short Ruby hashes may stay visually compact across two lines with the first
-  pair kept on the opening line and aligned continuation pairs below it.
 - If the hash would exceed the line limit, break after `{` and indent pairs
   by 4 spaces.
 - Put one logical pair per line when the expression would otherwise become

backend/app/controllers/application_controller.rb

@@ -1,63 +1,20 @@
 class ApplicationController < ActionController::API
-  include RequestAccess
-  include ActionController::Cookies
-  include ActionController::RequestForgeryProtection
-
-  class UnsupportedLocale < StandardError
-    ;
-  end
-
   rescue_from ActiveRecord::RecordInvalid, with: :render_record_invalid
   rescue_from ActiveRecord::RecordNotUnique, with: :render_record_not_unique
-  rescue_from(ActiveRecord::RecordNotFound) { head :not_found }
-  rescue_from(UnsupportedLocale) { head :not_found }
 
   before_action :reject_banned_ip_address!
   before_action :authenticate_user
   before_action :reject_banned_user!
-  before_action :validate_explicit_locale!
+
-  before_action :enforce_query_contract!
+  def current_user = @current_user
-  around_action :with_history_authority
-  rescue_from(HistoryAccess::Forbidden) { head :not_found }
-  protect_from_forgery with: :exception
-  self.forgery_protection_origin_check = true
-  rescue_from ActionController::InvalidAuthenticityToken, with: :reject_invalid_csrf
-  after_action :prevent_authenticated_caching
 
   private
 
-  def enforce_query_contract!
+  def authenticate_user
-    pairs = URI.decode_www_form(request.query_string)
+    code = request.headers['X-Transfer-Code'] || request.headers['HTTP_X_TRANSFER_CODE']
-    return head :bad_request if pairs.length > 1000 || pairs.map(&:first).uniq.length != pairs.length
+    return if code.blank?
-    return head :bad_request if request.query_parameters.values.any? { !_1.is_a?(String) }
-    if (controller_name.start_with?('counts_') &&
-        (request.query_parameters.keys & ['headers', 'labels']).any?)
-      return head :bad_request
-    end
-  rescue ArgumentError
-    head :bad_request
-  end
 
-  def with_history_authority(&action)
+    @current_user = User.find_by(inheritance_code: code)
-    response.headers['Cache-Control'] = 'private, no-store'
-    HistoryAccess.set(user: current_user, &action)
-  end
-
-  def reject_invalid_csrf
-    response.headers['X-CSRF-Invalid'] = '1'
-    response.headers['Cache-Control'] = 'private, no-store'
-    head :forbidden
-  end
-
-  def prevent_authenticated_caching
-    response.headers['Cache-Control'] = 'private, no-store' if current_user
-  end
-
-  def establish_user_session user
-    reset_session
-    session[:inheritance_code] = user.inheritance_code
-    @current_user = user
-    response.headers['Cache-Control'] = 'private, no-store'
   end
 
   def bool? key, default: false
@@ -92,6 +49,19 @@ class ApplicationController < ActionController::API
     render_validation_error base: ['すでに存在してゐます．']
   end
 
+  def reject_banned_ip_address!
+    ip_address = IpAddress.find_by(ip_address: IPAddr.new(request.remote_ip).hton)
+    return unless ip_address&.banned?
+
+    head :forbidden
+  end
+
+  def reject_banned_user!
+    return unless current_user&.banned?
+
+    head :forbidden
+  end
+
   def render_validation_error record = nil, fields: { }, base: [], status: :unprocessable_entity
     errors = { }
 
@@ -115,29 +85,4 @@ class ApplicationController < ActionController::API
                    base_errors: },
            status:
   end
-
-  def normalise_json value
-    return nil if value.nil?
-    return JSON.parse(value) if value.is_a?(String)
-
-    value
-  end
-
-  def resolve_locale!
-    return @resolved_locale if defined?(@resolved_locale)
-
-    code = params.fetch(:locale, 'ja')
-    unless code.is_a?(String) && Locale.codes.value?(code)
-      raise UnsupportedLocale
-    end
-
-    locale = Locale.find_by(code:)
-    raise UnsupportedLocale unless locale && locale.deprecated_at.nil?
-
-    @resolved_locale = locale
-  end
-
-  def validate_explicit_locale!
-    resolve_locale! if params.key?(:locale)
-  end
 end

backend/app/controllers/concerns/counts_listing.rb

@@ -1,21 +0,0 @@
-module CountsListing
-  extend ActiveSupport::Concern
-
-  def index
-    locale = resolve_locale!
-    resource = is_a?(CountsPostsController) ? 'posts' : 'tags'
-    query = CountsQuery.new(version_scope, resource, request.query_parameters.except('locale')
-        .merge('locale' => locale.code))
-    result = { population_count_scope: 'all_recorded_versions', population_count: HistoryAccess.population_count(query.relation.klass), count: query.count, page: query.page, limit: query.limit,
-               at: request.query_parameters['at'].presence || Time.current.utc.iso8601(6) }
-    if query.mode == 'summary'
-      result.merge!(query.summary)
-    else
-      result[:versions] = CountsVersionRepr.many(
-          query.details, resource:, language_code: locale.language_code)
-    end
-    render json: result
-  rescue ArgumentError, TypeError
-    head :bad_request
-  end
-end

backend/app/controllers/concerns/history_storage_access.rb

@@ -1,52 +0,0 @@
-# Permanent storage URLs must not bypass a resource's later discard boundary.
-module HistoryStorageAccess
-  extend ActiveSupport::Concern
-
-  included do
-    include RequestAccess
-    prepend_before_action :authorise_history_storage!
-  end
-
-  private
-
-  def authorise_history_storage!
-    response.headers['Cache-Control'] = 'private, no-store'
-    return unless request.get? || request.head?
-
-    reject_banned_ip_address!
-    return if performed?
-
-    authenticate_user
-    reject_banned_user!
-    return if performed?
-
-    signed = params[:signed_blob_id] || params[:signed_id]
-    blob = if signed
-             ActiveStorage::Blob.find_signed(signed)
-           elsif params[:encoded_key]
-             payload = ActiveStorage.verifier.verified(params[:encoded_key], purpose: :blob_key)
-             key = payload && (payload[:key] || payload['key'])
-             if key
-               original = key.start_with?('variants/') ? key.split('/')[1] : key
-               ActiveStorage::Blob.find_by(key: original)
-             end
-           end
-    return head :not_found unless blob
-
-    allowed = HistoryAccess.set(user: current_user) { HistoryAccess.blob_visible?(blob) }
-    head :not_found unless allowed
-  rescue ActiveSupport::MessageVerifier::InvalidSignature
-    head :not_found
-  end
-
-  # Override the engine's public-forever caching before a streaming response commits.
-  def http_cache_forever(**_options)
-    response.headers['Cache-Control'] = 'private, no-store'
-    yield
-  end
-
-  def expires_in(*arguments, **options)
-    super
-    response.headers['Cache-Control'] = 'private, no-store'
-  end
-end

backend/app/controllers/concerns/request_access.rb

@@ -1,28 +0,0 @@
-# Canonical read-side session and BAN boundary, shared by API and storage controllers.
-module RequestAccess
-  extend ActiveSupport::Concern
-
-  def current_user = @current_user
-
-  private
-
-  def authenticate_user
-    code = session[:inheritance_code]
-    return if code.blank?
-
-    @current_user = User.find_by(inheritance_code: code)
-  end
-
-  def reject_banned_ip_address!
-    ip_address = IpAddress.find_by(ip_address: IPAddr.new(request.remote_ip).hton)
-    return unless ip_address&.banned?
-
-    head :forbidden
-  end
-
-  def reject_banned_user!
-    return unless current_user&.banned?
-
-    head :forbidden
-  end
-end

backend/app/controllers/counts_exports_controller.rb

@@ -1,83 +0,0 @@
-class CountsExportsController < ApplicationController
-  wrap_parameters false
-
-  def create_job
-    locale = resolve_locale!
-    payload = request.request_parameters.merge(request.query_parameters)
-    session[:counts_export_owner] ||= SecureRandom.hex(32)
-    job = CountsExportJobs.start(payload, session[:counts_export_owner], locale.language_code)
-    response.headers['Cache-Control'] = 'private, no-store'
-    render json: { job: }, status: :accepted
-  rescue CountsExportDownload::Busy
-    head :too_many_requests
-  rescue ArgumentError, TypeError, KeyError, JSON::ParserError
-    head :bad_request
-  end
-
-  def job_status
-    job_state(cancel: false)
-  end
-
-  def cancel_job
-    job_state(cancel: true)
-  end
-
-  def prepare
-    locale = resolve_locale!
-    payload = request.request_parameters.merge(request.query_parameters)
-    exporter = CountsExporter.new(payload, language_code: locale.language_code)
-    session[:counts_export_owner] ||= SecureRandom.hex(32)
-    token = CountsExportDownload.prepare(exporter, session[:counts_export_owner])
-    response.headers['Cache-Control'] = 'private, no-store'
-    render json: { token: }
-  rescue CountsExportDownload::Busy
-    head :too_many_requests
-  rescue CountsExportDownload::TooLarge
-    head 413
-  rescue CountsExportDownload::TimedOut
-    head :gateway_timeout
-  rescue CountsExportDownload::StorageFull, Errno::ENOSPC
-    head :insufficient_storage
-  rescue ArgumentError, TypeError, KeyError, JSON::ParserError
-    head :bad_request
-  end
-
-  def download
-    file = CountsExportDownload.find(params[:token], session[:counts_export_owner])
-    return head :not_found unless file
-
-    response.headers['Cache-Control'] = 'private, no-store'
-    send_file file.fetch('path'),
-              filename: "counts-#{ file.fetch('resource') }.#{ file.fetch('format') }",
-              type: file['format'] == 'json' ? 'application/json' : 'text/csv',
-              disposition: 'attachment'
-  end
-
-  def job_state(cancel:)
-    result = CountsExportJobs.status(params[:token], session[:counts_export_owner], cancel:)
-    return head :not_found unless result
-
-    response.headers['Cache-Control'] = 'private, no-store'
-    render json: result
-  end
-  private :job_state
-
-  def index
-    locale = resolve_locale!
-    payload = request.request_parameters.except('authenticity_token')
-        .merge(request.query_parameters)
-    exporter = CountsExporter.new(payload,
-                                  language_code: locale.language_code)
-    session[:counts_export_owner] ||= SecureRandom.hex(32)
-    token = CountsExportDownload.prepare(exporter, session[:counts_export_owner])
-    file = CountsExportDownload.find(token, session[:counts_export_owner])
-    return head :not_found unless file
-
-    send_file file.fetch('path'),
-        filename: "counts-#{ exporter.resource }.#{ exporter.format }",
-        type: exporter.format == 'json' ? 'application/json' : 'text/csv',
-        disposition: 'attachment'
-  rescue ArgumentError, TypeError, KeyError, JSON::ParserError
-    head :bad_request
-  end
-end

backend/app/controllers/counts_posts_controller.rb

@@ -1,3 +0,0 @@
-class CountsPostsController < PostVersionsController
-  include CountsListing
-end

backend/app/controllers/counts_states_controller.rb

@@ -1,10 +0,0 @@
-class CountsStatesController < ApplicationController
-  def index
-    locale = resolve_locale!
-    result = CountsStateQuery.new(request.query_parameters,
-                                  language_code: locale.language_code).call
-    render json: result
-  rescue ArgumentError, TypeError
-    head :bad_request
-  end
-end

backend/app/controllers/counts_tables_controller.rb

@@ -1,23 +0,0 @@
-class CountsTablesController < ApplicationController
-  def definition
-    resolve_locale!
-    definition = CountsResourceDefinition.new(params[:resource])
-    render json: { fields: definition.metadata,
-                   output_fields: definition.output_fields,
-                   filter_fields: definition.filter_metadata,
-                   axes: { latest: definition.axes(false), history: definition.axes(true) },
-                   grains: CountsAxes::GRAINS, defaults: CountsDefaults.metadata(params[:resource]) }
-  rescue ArgumentError
-    head :bad_request
-  end
-
-  def index
-    resolve_locale!
-    render json: CountsTableQuery.new(request.query_parameters).call
-  rescue CountsTableQuery::TooManyColumns
-    render json: { error: 'too_many_columns', max_columns: CountsTableQuery::MAX_COLUMNS },
-           status: :unprocessable_entity
-  rescue ArgumentError, TypeError, KeyError, JSON::ParserError
-    head :bad_request
-  end
-end

backend/app/controllers/counts_tags_controller.rb

@@ -1,3 +0,0 @@
-class CountsTagsController < TagVersionsController
-  include CountsListing
-end

backend/app/controllers/deerjikists_controller.rb

@@ -1,7 +1,5 @@
 class DeerjikistsController < ApplicationController
   def show
-    locale = resolve_locale!
-
     platform = params[:platform].to_s.strip
     code = params[:code].to_s.strip
     return render_bad_request('platform は必須です．') if platform.blank?
@@ -9,10 +7,10 @@ class DeerjikistsController < ApplicationController
 
     deerjikist = Deerjikist
                  .joins(:tag)
-                 .includes(tag: :tag_names)
+                 .includes(tag: :tag_name)
                  .find_by(platform:, code:)
     if deerjikist
-      render json: DeerjikistRepr.base(deerjikist, language_code: locale.language_code)
+      render json: DeerjikistRepr.base(deerjikist)
     else
       head :not_found
     end
@@ -22,7 +20,6 @@ class DeerjikistsController < ApplicationController
     return head :unauthorized unless current_user
     return head :forbidden unless current_user.gte_member?
 
-    locale = resolve_locale!
     platform = params[:platform].to_s.strip
     code = params[:code].to_s.strip
     tag_id = params[:tag_id].to_i
@@ -35,7 +32,7 @@ class DeerjikistsController < ApplicationController
       d.save!
     end
 
-    render json: DeerjikistRepr.base(deerjikist, language_code: locale.language_code)
+    render json: DeerjikistRepr.base(deerjikist)
   end
 
   def destroy

backend/app/controllers/gekanator_games_controller.rb

@@ -1,6 +1,6 @@
 class GekanatorGamesController < ApplicationController
   def create
-    return head :unauthorized unless current_user
+    return head :not_found unless current_user&.admin?
 
     guessed_post_id = params.require(:guessed_post_id)
     correct_post_id = params[:correct_post_id].presence
@@ -14,52 +14,47 @@ class GekanatorGamesController < ApplicationController
         question_count: answers.length,
         answers:)
 
-    if game.invalid?
+    if game.save
+      render json: { id: game.id }, status: :created
+    else
       render json: { errors: game.errors.full_messages }, status: :unprocessable_entity
-      return
     end
-
-    learned_example_count = 0
-
-    ActiveRecord::Base.transaction do
-      game.save!
-      learned_example_count = learn_answers_from_game!(game)
-    end
-
-    render json: {
-      id: game.id,
-      learned_example_count:
-    }, status: :created
-  rescue ActiveRecord::RecordInvalid => e
-    render json: { errors: e.record.errors.full_messages }, status: :unprocessable_entity
   end
 
   def extra_questions
-    game = find_owned_game
+    return head :not_found unless current_user&.admin?
-    return if performed?
 
+    game = GekanatorGame.find_by(id: params[:id])
+    return head :not_found unless game
+
+    asked_ids = Array(game.answers).filter_map { |answer| answer_question_id(answer) }
+    existing_example_ids =
+        GekanatorQuestionExample.where(post_id: game.correct_post_id)
+                                .select(:gekanator_question_id)
+    # Direct examples only for now; post_similarity-based expansion is deferred.
     questions =
         GekanatorQuestion
         .accepted
-        .includes(:gekanator_question_examples)
         .where(kind: 'post_similarity', source: 'user_suggested')
-        .to_a
+        .where.not(id: existing_example_ids)
-
+        .order(priority_weight: :desc, id: :asc)
-    selected =
-        prioritized_extra_questions(
-            questions,
-            post_id: game.correct_post_id,
-            user: current_user,
-            limit: 6)
 
     render json: {
-      questions: selected.map { |question| extra_question_json(question) }
+      questions: questions.filter_map { |question|
+        json = extra_question_json(question)
+        next if asked_ids.include?(json[:id].to_s)
+        next if asked_ids.include?("post-similarity:#{ json[:id] }")
+
+        json
+      }.first(2)
     }
   end
 
   def extra_question_answers
-    game = find_owned_game
+    return head :not_found unless current_user&.admin?
-    return if performed?
+
+    game = GekanatorGame.find_by(id: params[:id])
+    return head :not_found unless game
 
     answer_params = params.require(:answers)
     if !answer_params.is_a?(Array)
@@ -89,10 +84,11 @@ class GekanatorGamesController < ApplicationController
                 gekanator_question: question,
                 post: game.correct_post,
                 user: current_user)
-        example.record_answer!(
+        example.assign_attributes(
+            gekanator_game: game,
             answer: item[:answer],
             source: 'post_game_extra',
-            gekanator_game: game)
+            weight: 1.0)
         example.save!
       end
     end
@@ -111,162 +107,12 @@ class GekanatorGamesController < ApplicationController
     }
   end
 
-  def prioritized_extra_questions questions, post_id:, user:, limit:
+  def answer_question_id answer
-    answered_question_ids =
+    value = if answer.is_a?(Hash)
-        GekanatorQuestionExample
+      answer['question_id'].presence || answer[:question_id].presence ||
-        .where(user:, gekanator_question_id: questions.map(&:id))
+        answer['questionId'].presence || answer[:questionId].presence
-        .distinct
-        .pluck(:gekanator_question_id)
-    unanswered, answered =
-        questions.partition { |question| !answered_question_ids.include?(question.id) }
-    selected = weighted_sample_questions(unanswered, post_id:, limit:)
-    return selected if selected.length >= limit
-
-    selected + weighted_sample_questions(
-        answered.reject { |question| selected.any? { _1.id == question.id } },
-        post_id:,
-        limit: limit - selected.length)
     end
 
-  def weighted_sample_questions questions, post_id:, limit:
+    value&.to_s
-    remaining = questions.uniq(&:id)
-    selected = []
-
-    while selected.length < limit && remaining.any?
-      weighted =
-          remaining.map { |question|
-            [question, selection_weight_for(question, post_id: post_id)]
-          }
-      total_weight = weighted.sum { |_question, weight| weight }
-      break if total_weight <= 0
-
-      target = rand * total_weight
-      cumulative = 0.0
-      chosen =
-          weighted.find do |_question, weight|
-            cumulative += weight
-            cumulative >= target
-          end&.first || weighted.first.first
-
-      selected << chosen
-      remaining.reject! { |question| question.id == chosen.id }
-    end
-
-    selected
-  end
-
-  def selection_weight_for question, post_id:
-    sample_count =
-        question.gekanator_question_examples.sum { |example|
-          next 0 unless example.post_id == post_id
-
-          example.sample_count.presence || 1
-        }
-
-    question.priority_weight.to_f / (1.0 + sample_count * 0.15)
-  end
-
-  def find_owned_game
-    return head :unauthorized unless current_user
-
-    game = GekanatorGame.find_by(id: params[:id])
-    return head :not_found unless game
-    if !current_user.admin? && game.user_id != current_user.id
-      return head :not_found
-    end
-
-    game
-  end
-
-  def learn_answers_from_game! game
-    correct_post = game.correct_post
-    return 0 if correct_post.blank?
-
-    accepted_questions =
-        GekanatorQuestion
-        .accepted
-        .index_by { |question| public_question_id_for(question) }
-    learned_count = 0
-
-    Array(game.answers).each do |answer|
-      answer_value = answer['answer'].to_s
-      next if answer_value.blank? || answer_value == 'unknown'
-
-      question_id = game_answer_question_id(answer)
-      next if question_id.blank?
-
-      question = accepted_questions[question_id.to_s]
-      next unless learnable_game_answer_question?(question)
-
-      example =
-          GekanatorQuestionExample.find_or_initialize_by(
-              gekanator_question: question,
-              post: correct_post,
-              user: current_user)
-      example.record_answer!(
-          answer: answer_value,
-          source: 'post_game_answer',
-          gekanator_game: game)
-      example.save!
-      learned_count += 1
-    end
-
-    learned_count
-  end
-
-  def public_question_id_for question
-    condition = normalize_condition(question.condition)
-
-    case condition[:type]
-    when 'tag'
-      "tag:#{condition[:key]}"
-    when 'source'
-      "source:#{condition[:host]}"
-    when 'original-year'
-      "original-year:#{condition[:year]}"
-    when 'original-month'
-      "original-month:#{condition[:month]}"
-    when 'original-month-day'
-      "original-month-day:#{condition[:monthDay] || condition[:month_day]}"
-    when 'title-length-at-least'
-      "title:length-at-least:#{condition[:length]}"
-    when 'title-length-greater-than'
-      "title:length-at-least:#{condition[:length].to_i + 1}"
-    when 'title-has-ascii'
-      'title:ascii'
-    when 'title-contains'
-      "title:contains:#{condition[:text]}"
-    when 'post-similarity'
-      "post-similarity:#{question.id}"
-    else
-      "catalog:#{question.id}"
-    end
-  end
-
-  def normalize_condition condition
-    json = condition.deep_dup.as_json
-
-    if json['type'] == 'original-month-day' && json['monthDay'].blank?
-      json['monthDay'] = json.delete('month_day')
-    end
-
-    json.deep_symbolize_keys
-  end
-
-  def learnable_game_answer_question? question
-    return false if question.nil?
-    return true if question.kind == 'post_similarity'
-    return false unless question.kind == 'tag'
-
-    condition = normalize_condition(question.condition)
-    key = condition[:key].to_s
-    !key.start_with?('nico:')
-  end
-
-  def game_answer_question_id answer
-    answer['question_id'] ||
-      answer[:question_id] ||
-      answer['questionId'] ||
-      answer[:questionId]
   end
 end

backend/app/controllers/gekanator_posts_controller.rb

@@ -1,30 +1,21 @@
 class GekanatorPostsController < ApplicationController
   def index
+    return head :not_found unless current_user&.admin?
+
     posts =
         Post
-        .preload(:post_similarities, tags: :tag_names)
+        .preload(tags: :tag_name)
         .with_attached_thumbnail
         .order(Arel.sql(
             'COALESCE(posts.original_created_before - INTERVAL 1 MINUTE, ' \
             'posts.original_created_from, posts.created_at) DESC, posts.id DESC'))
-        .to_a
 
-    active_tags_by_post_id =
+    render json: { posts: posts.map { |post| post_json(post) } }
-        posts.each_with_object({ }) do |post, h|
-          h[post.id] = post.tags.reject(&:deprecated?)
-        end
-
-    render json: {
-      posts: posts.map { |post|
-        post_json(post,
-                  active_tags_by_post_id:)
-      }
-    }
   end
 
   private
 
-  def post_json post, active_tags_by_post_id:
+  def post_json post
     {
       id: post.id,
       url: post.url,
@@ -33,30 +24,14 @@ class GekanatorPostsController < ApplicationController
       thumbnail_base: post.thumbnail_base,
       original_created_from: post.original_created_from,
       original_created_before: post.original_created_before,
-      post_similarity_edges: post_similarity_edges_json(
+      tags: post.tags.map { |tag| tag_json(tag) }
-          post,
-          active_tags_by_post_id:),
-      tags: active_tags_by_post_id.fetch(post.id, []).map { |tag| tag_json(tag) }
     }
   end
 
-  def post_similarity_edges_json post, active_tags_by_post_id:
-    post
-        .post_similarities
-        .filter_map do |similarity|
-          next unless active_tags_by_post_id.key?(similarity.target_post_id)
-
-          {
-            target_post_id: similarity.target_post_id,
-            cos: similarity.cos.to_f
-          }
-        end
-  end
-
   def tag_json tag
     {
       id: tag.id,
-      name: tag.name('ja'),
+      name: tag.name,
       category: tag.category
     }
   end

backend/app/controllers/gekanator_question_suggestions_controller.rb

@@ -1,41 +1,9 @@
 class GekanatorQuestionSuggestionsController < ApplicationController
   def create
-    return head :unauthorized unless current_user
+    return head :not_found unless current_user&.admin?
 
     game = GekanatorGame.find_by(id: params.require(:gekanator_game_id))
     return head :not_found unless game
-    if !current_user.admin? && game.user_id != current_user.id
-      return head :not_found
-    end
-
-    existing_question_id = params[:existing_question_id].presence
-    if existing_question_id
-      question = GekanatorQuestion.accepted.find_by(id: existing_question_id)
-      return head :not_found unless question
-      unless learnable_existing_question?(question)
-        return render_validation_error fields: { existing_question_id: ['質問が不正です．'] }
-      end
-
-      example =
-          GekanatorQuestionExample.find_or_initialize_by(
-              gekanator_question: question,
-              post: game.correct_post,
-              user: current_user)
-      example.record_answer!(
-          answer: params.require(:answer),
-          source: 'post_game_extra',
-          gekanator_game: game)
-
-      if example.save
-        render json: {
-          id: question.id,
-          count: game.question_suggestions.count
-        }, status: :created
-      else
-        render_validation_error example
-      end
-      return
-    end
 
     suggestion = GekanatorQuestionSuggestion.new(
         gekanator_game: game,
@@ -82,14 +50,4 @@ class GekanatorQuestionSuggestionsController < ApplicationController
   rescue NotImplementedError
     head :not_implemented
   end
-
-  private
-
-  def learnable_existing_question? question
-    return true if question.kind == 'post_similarity'
-    return false unless question.kind == 'tag'
-
-    key = question.condition.as_json['key'].to_s
-    !key.start_with?('nico:')
-  end
 end

backend/app/controllers/gekanator_questions_controller.rb

@@ -1,43 +1,38 @@
 class GekanatorQuestionsController < ApplicationController
   def index
+    return head :not_found unless current_user&.admin?
+
     questions =
         GekanatorQuestion
         .accepted
         .includes(:gekanator_question_examples)
         .order(priority_weight: :desc, id: :asc)
-        .to_a
-    deprecated_tag_keys = deprecated_tag_keys_for(questions)
 
     render json: {
-      questions: questions.filter_map { |question|
+      questions: questions.map { |question| question_json(question) }
-        json = question_json(question)
-        next if hidden_question?(json[:condition], deprecated_tag_keys)
-
-        json
-      }
     }
   end
 
   private
 
   def question_json question
-    condition = condition_json(question.condition).deep_symbolize_keys
     json = {
-      record_id: question.id,
+      id: question_id_for(question),
-      id: question_id_for(question, condition),
+      text: question.text,
-      text: question_text_for(question, condition),
       kind: question.kind,
-      condition: condition,
+      condition: condition_json(question.condition),
       source: question.source,
       priority_weight: question.priority_weight
     }
-    if question.kind == 'post_similarity' || question.kind == 'tag'
+    if question.kind == 'post_similarity'
       json[:example_answers] = example_answers_json(question)
     end
     json
   end
 
-  def question_id_for question, condition
+  def question_id_for question
+    condition = condition_json(question.condition).deep_symbolize_keys
+
     case condition[:type]
     when 'tag'
       "tag:#{ condition[:key] }"
@@ -49,14 +44,10 @@ class GekanatorQuestionsController < ApplicationController
       "original-month:#{ condition[:month] }"
     when 'original-month-day'
       "original-month-day:#{ condition[:monthDay] || condition[:month_day] }"
-    when 'title-length-at-least'
-      "title:length-at-least:#{ condition[:length] }"
     when 'title-length-greater-than'
-      "title:length-at-least:#{ condition[:length].to_i + 1 }"
+      "title:length-greater-than:#{ condition[:length] }"
     when 'title-has-ascii'
       'title:ascii'
-    when 'title-contains'
-      "title:contains:#{ condition[:text] }"
     when 'post-similarity'
       "post-similarity:#{ question.id }"
     else
@@ -71,27 +62,9 @@ class GekanatorQuestionsController < ApplicationController
       json['monthDay'] = json.delete('month_day')
     end
 
-    if json['type'] == 'title-length-greater-than'
-      json['type'] = 'title-length-at-least'
-      json['length'] = json['length'].to_i + 1
-    end
-
     json
   end
 
-  def question_text_for question, condition
-    return question.text unless question.kind == 'title'
-
-    case condition[:type]
-    when 'title-length-at-least'
-      "タイトルは #{ condition[:length] } 文字以上?"
-    when 'title-contains'
-      "題名に「#{ condition[:text] }」が含まれる?"
-    else
-      question.text
-    end
-  end
-
   def example_answers_json question
     question
         .gekanator_question_examples
@@ -107,41 +80,4 @@ class GekanatorQuestionsController < ApplicationController
         .first
         &.first
   end
-
-  def deprecated_tag_keys_for questions
-    tag_keys = questions.filter_map { |question|
-      condition = condition_json(question.condition)
-      next unless condition['type'] == 'tag'
-
-      condition['key'].to_s.presence
-    }.uniq
-    return {} if tag_keys.empty?
-
-    categories = []
-    names = []
-    tag_keys.each do |key|
-      category, name = parse_tag_key(key)
-      categories << category
-      names << name
-    end
-
-    Tag
-        .joins(:tag_names)
-        .where(category: categories.uniq)
-        .where(tag_names: { language_code: 'ja', primary_flg: true, name: names.uniq })
-        .where.not(deprecated_at: nil)
-        .pluck('tags.category', 'tag_names.name')
-        .each_with_object({ }) do |(category, name), h|
-          h["#{ category }:#{ name }"] = true
-        end
-  end
-
-  def hidden_question? condition, deprecated_tag_keys
-    condition[:type] == 'tag' && deprecated_tag_keys[condition[:key].to_s]
-  end
-
-  def parse_tag_key key
-    parts = key.to_s.split(':')
-    [parts.first.to_s, parts.drop(1).join(':')]
-  end
 end

backend/app/controllers/locales_controller.rb

@@ -1,10 +0,0 @@
-class LocalesController < ApplicationController
-  def index
-    locales = Locale.where(deprecated_at: nil).order(:created_at).map do |locale|
-      { code: locale.code_before_type_cast, name: locale.name }
-    end
-
-    response.headers['Cache-Control'] = 'no-store'
-    render json: { locales: }
-  end
-end

backend/app/controllers/material_sync_suppressions_controller.rb

@@ -1,49 +0,0 @@
-class MaterialSyncSuppressionsController < ApplicationController
-  def index
-    return head :unauthorized unless current_user
-    return head :forbidden unless current_user.gte_member?
-
-    suppressions =
-        MaterialSyncSuppression
-        .includes(:created_by_user)
-        .order(created_at: :desc, id: :desc)
-
-    render json: { suppressions: suppressions.map { |s| repr(s) } }
-  end
-
-  def create
-    return head :unauthorized unless current_user
-    return head :forbidden unless current_user.gte_member?
-
-    suppression =
-        MaterialSyncSuppressionRegistrar.create!(suppression_params,
-                                                 created_by_user: current_user)
-
-    render json: repr(suppression), status: :created
-  rescue ActiveRecord::RecordInvalid => e
-    render_validation_error e.record
-  end
-
-  private
-
-  def suppression_params
-    params.permit(:source_kind,
-                  :source_uri,
-                  :drive_path,
-                  :drive_file_id,
-                  :reason)
-  end
-
-  def repr suppression
-    { id: suppression.id,
-      source_kind: suppression.source_kind,
-      source_uri: suppression.source_uri,
-      drive_path: suppression.drive_path,
-      drive_file_id: suppression.drive_file_id,
-      normalized_source_key: suppression.normalized_source_key,
-      reason: suppression.reason,
-      created_by_user: suppression.created_by_user&.as_json(UserRepr::BASE),
-      created_at: suppression.created_at,
-      updated_at: suppression.updated_at }
-  end
-end

backend/app/controllers/material_versions_controller.rb

@@ -1,54 +0,0 @@
-class MaterialVersionsController < ApplicationController
-  def index
-    page = (params[:page].presence || 1).to_i
-    limit = (params[:limit].presence || 20).to_i
-
-    page = 1 if page < 1
-    limit = 1 if limit < 1
-
-    offset = (page - 1) * limit
-
-    q = HistoryAccess.visible(MaterialVersion.includes(:created_by_user))
-    q = q.where(material_id: params[:material_id]) if params[:material_id].present?
-    q = q.where(tag_name: params[:tag].to_s.strip) if params[:tag].present?
-    q = q.where(event_type: params[:event_type]) if params[:event_type].present?
-
-    count = q.except(:order, :limit, :offset).count
-    versions =
-        q
-        .order(Arel.sql('material_versions.created_at DESC, material_versions.id DESC'))
-        .limit(limit)
-        .offset(offset)
-
-    render json: { versions: serialise_versions(versions), count:, population_count: HistoryAccess.population_count(MaterialVersion) }
-  end
-
-  private
-
-  def serialise_versions rows
-    rows.map do |row|
-      { id: row.id,
-        material_id: row.material_id,
-        version_no: row.version_no,
-        event_type: row.event_type,
-        tag_id: row.tag_id,
-        tag_name: row.tag_name,
-        tag_category: row.tag_category,
-        url: row.url,
-        source_kind: row.source_kind,
-        source_uri: row.source_uri,
-        source_path: row.source_path,
-        source_file_id: row.source_file_id,
-        normalized_source_key: row.normalized_source_key,
-        file_blob_id: row.file_blob_id,
-        file_filename: row.file_filename,
-        file_content_type: row.file_content_type,
-        file_byte_size: row.file_byte_size,
-        file_sha256: row.file_sha256,
-        export_paths_json: row.export_paths_hash,
-        discarded_at: row.discarded_at,
-        created_by_user: row.created_by_user&.as_json(UserRepr::BASE),
-        created_at: row.created_at }
-    end
-  end
-end

backend/app/controllers/materials_controller.rb

@@ -1,14 +1,5 @@
-require 'set'
-
-
 class MaterialsController < ApplicationController
-  rescue_from MaterialZipExporter::EmptyExportError, with: :render_zip_empty
-  rescue_from MaterialZipExporter::DuplicatePathError, with: :render_zip_duplicate_path
-  rescue_from MaterialZipExporter::MissingFileError, with: :render_zip_missing_file
-
   def index
-    locale = resolve_locale!
-
     page = (params[:page].presence || 1).to_i
     limit = (params[:limit].presence || 20).to_i
 
@@ -17,111 +8,55 @@ class MaterialsController < ApplicationController
 
     offset = (page - 1) * limit
 
-    filters = material_index_filters
+    tag_id = params[:tag_id].presence
-    tag_graph = material_index_tag_graph(filters)
+    parent_id = params[:parent_id].presence
-    q = Material.includes(:material_export_items,
-                          thumbnail_attachment: :blob,
-                          file_attachment: :blob,
-                          tag: :tag_names)
-    q = q.where(tag_id: nil) if filters[:tag_state] == 'untagged'
-    q = q.where.not(tag_id: nil) if filters[:tag_state] == 'tagged'
-    q = apply_material_tag_filter(q, filters, tag_graph)
-    q = q.where('materials.created_at >= ?', filters[:created_from]) if filters[:created_from]
-    q = q.where('materials.created_at <= ?', filters[:created_to]) if filters[:created_to]
-    q = q.where('materials.updated_at >= ?', filters[:updated_from]) if filters[:updated_from]
-    q = q.where('materials.updated_at <= ?', filters[:updated_to]) if filters[:updated_to]
 
-    q = material_index_join_tag_name(q) if material_index_needs_tag_name?(filters)
+    q = Material.includes(:tag, :created_by_user).with_attached_file
-    q = material_index_join_file_blob(q) if material_index_needs_file_blob?(filters)
+    q = q.where(tag_id:) if tag_id
-    q = apply_material_query(q, filters[:q]) if filters[:q].present?
+    q = q.where(parent_id:) if parent_id
-    q = apply_material_media_kind(q, filters[:media_kind])
 
-    count = q.distinct.count(:id)
+    count = q.count
-    materials =
+    materials = q.order(created_at: :desc, id: :desc).limit(limit).offset(offset)
-        q
-        .order(Arel.sql(material_index_order_sql(filters)))
-        .limit(limit)
-        .offset(offset)
-        .to_a
 
-    response = { materials: MaterialRepr.list_many(
+    render json: { materials: MaterialRepr.many(materials, host: request.base_url), count: count }
-                   materials,
-                   host: request.base_url,
-                   language_code: locale.language_code),
-                 count: }
-    if filters[:tag_id]
-      response[:tag_scope] = material_index_tag_scope(filters, tag_graph)
-    end
-    if filters[:group_by] == 'parent_tag'
-      response[:groups] = material_index_groups(materials, tag_graph)
-    end
-
-    render json: response
   end
 
   def show
-    locale = resolve_locale!
-
     material =
         Material
-        .includes(:tag, :material_export_items)
+        .includes(:tag)
-        .with_attached_thumbnail
         .with_attached_file
         .find_by(id: params[:id])
     return head :not_found unless material
 
-    wiki_page_body =
+    wiki_page_body = material.tag.tag_name.wiki_page&.current_revision&.body
-        TagDescription
-        .find_by(tag_id: material.tag_id, language_code: locale.language_code)
-        &.body
 
-    render json: MaterialRepr.base(material, host: request.base_url,
+    render json: MaterialRepr.base(material, host: request.base_url).merge(wiki_page_body:)
-                                   language_code: locale.language_code).merge(wiki_page_body:)
   end
 
   def create
     return head :unauthorized unless current_user
-    return head :forbidden unless current_user.gte_member?
 
-    locale = resolve_locale!
     tag_name_raw = params[:tag].to_s.strip
     file = params[:file]
-    file_sha256 = MaterialFileSha256.from_upload(file)
     url = params[:url].to_s.strip.presence
     return render_unprocessable_entity('タグは必須です．', field: :tag) if tag_name_raw.blank?
     if file.blank? && url.blank?
       return render_validation_error fields: { file: ['ファイルまたは URL は必須です．'],
                                                url: ['ファイルまたは URL は必須です．'] }
     end
-    block = MaterialImportBlockMatcher.match_for_sha256(file_sha256)
-    return render_material_import_block(block) if block
 
-    uploaded_blob = build_uploaded_material_blob!(file, file_sha256)
+    tag_name = TagName.find_undiscard_or_create_by!(name: tag_name_raw)
-    material = nil
+    tag = tag_name.tag
+    tag = Tag.create!(tag_name:, category: :material) unless tag
 
-    begin
-      Material.transaction do
-        tag = resolve_material_tag!(locale, tag_name_raw)
     material = Material.new(tag:, url:,
                             created_by_user: current_user,
                             updated_by_user: current_user)
-        material.file.attach(uploaded_blob) if uploaded_blob
+    material.file.attach(file)
-        material.save!
-        TagVersioning.record_tag_snapshot!(tag, created_by_user: current_user,
-                                           language_code: locale.language_code)
-        upsert_export_paths!(material)
-        MaterialVersionRecorder.record!(material:, event_type: :create,
-                                        created_by_user: current_user)
-      end
-    rescue StandardError
-      uploaded_blob&.purge_later
-      raise
-    end
 
-    if material
+    if material.save
-      log_thumbnail_generation(material, MaterialThumbnailGenerator.generate!(material))
+      render json: MaterialRepr.base(material, host: request.base_url), status: :created
-      render json: MaterialRepr.base(material, host: request.base_url,
-                                     language_code: locale.language_code), status: :created
     else
       render_validation_error material
     end
@@ -136,46 +71,29 @@ class MaterialsController < ApplicationController
 
     tag_name_raw = params[:tag].to_s.strip
     file = params[:file]
-    file_sha256 = MaterialFileSha256.from_upload(file)
+    url = params[:url].to_s.strip.presence
-    url = params.key?(:url) ? params[:url].to_s.strip.presence : material.url
     return render_unprocessable_entity('タグは必須です．', field: :tag) if tag_name_raw.blank?
-    if file.blank? && url.blank? && !material.file.attached?
+    if file.blank? && url.blank?
       return render_validation_error fields: { file: ['ファイルまたは URL は必須です．'],
                                                url: ['ファイルまたは URL は必須です．'] }
     end
-    block = MaterialImportBlockMatcher.match_for_sha256(file_sha256)
-    return render_material_import_block(block) if block
 
-    locale = resolve_locale!
+    tag_name = TagName.find_undiscard_or_create_by!(name: tag_name_raw)
+    tag = tag_name.tag
+    tag = Tag.create!(tag_name:, category: :material) unless tag
 
-    uploaded_blob = build_uploaded_material_blob!(file, file_sha256)
+    material.update!(tag:, url:, updated_by_user: current_user)
-
+    if file
-    begin
+      material.file.attach(file)
-      Material.transaction do
+    else
-        MaterialVersionRecorder.ensure_snapshot!(material, created_by_user: current_user)
+      material.file.purge
-        tag = resolve_material_tag!(locale, tag_name_raw)
-        material.assign_attributes(tag:, url:, updated_by_user: current_user)
-        if uploaded_blob
-          material.file.attach(uploaded_blob)
-        elsif params.key?(:url) && url.present? && file.blank?
-          material.file.detach
-        end
-        material.save!
-        TagVersioning.record_tag_snapshot!(tag, created_by_user: current_user,
-                                           language_code: locale.language_code)
-        upsert_export_paths!(material)
-        MaterialVersionRecorder.record!(material:, event_type: :update,
-                                        created_by_user: current_user)
-      end
-    rescue StandardError
-      uploaded_blob&.purge_later
-      raise
     end
 
-    log_thumbnail_generation(material, MaterialThumbnailGenerator.generate!(material))
+    if material.save
-
+      render json: MaterialRepr.base(material, host: request.base_url)
-    render json: MaterialRepr.base(material, host: request.base_url,
+    else
-                                   language_code: locale.language_code)
+      render_validation_error material
+    end
   end
 
   def destroy
@@ -185,388 +103,7 @@ class MaterialsController < ApplicationController
     material = Material.find_by(id: params[:id])
     return head :not_found unless material
 
-    Material.transaction do
+    material.discard
-      MaterialVersionRecorder.ensure_snapshot!(material, created_by_user: current_user)
-      material.discard!
-      MaterialVersionRecorder.record!(material:, event_type: :discard,
-                                      created_by_user: current_user)
-    end
     head :no_content
   end
-
-  def download
-    zip = MaterialZipExporter.new(profile: params[:profile],
-                                  tag_id: params[:tag_id]).export
-    profile = params[:profile].presence || 'legacy_drive'
-    send_data zip,
-              type: 'application/zip',
-              disposition: 'attachment',
-              filename: "btrc-materials-#{ profile }.zip"
-  end
-
-  private
-
-  def material_index_filters
-    tag_state = params[:tag_state].to_s.presence
-    tag_state = 'untagged' if bool?(:unclassified)
-    tag_state = 'all' unless ['all', 'tagged', 'untagged'].include?(tag_state)
-
-    media_kind = params[:media_kind].to_s.presence
-    unless ['all', 'image', 'video', 'audio', 'file_other', 'url_only'].include?(media_kind)
-      media_kind = 'all'
-    end
-
-    sort = params[:sort].to_s.presence
-    unless ['created_at', 'updated_at', 'tag_name', 'media_kind', 'file_byte_size',
-            'version_no', 'id'].include?(sort)
-      sort = 'created_at'
-    end
-
-    direction = params[:direction].to_s.downcase
-    direction = 'desc' unless ['asc', 'desc'].include?(direction)
-
-    group_by = params[:group_by].to_s.presence
-    group_by = 'none' unless ['none', 'parent_tag'].include?(group_by)
-
-    tag_id = params[:tag_id].to_i
-    tag_id = nil if tag_id <= 0
-
-    { q: params[:q].to_s.strip.presence,
-      tag_state:,
-      media_kind:,
-      tag_id:,
-      include_descendants: bool?(:include_descendants),
-      group_by:,
-      created_from: parse_time_param(:created_from),
-      created_to: parse_time_param(:created_to),
-      updated_from: parse_time_param(:updated_from),
-      updated_to: parse_time_param(:updated_to),
-      sort:,
-      direction: }
-  end
-
-  def parse_time_param name
-    value = params[name].to_s.strip
-    return nil if value.blank?
-
-    Time.zone.parse(value)
-  rescue ArgumentError
-    nil
-  end
-
-  def resolve_material_tag! locale, tag_name_raw
-    Tag.find_or_create_by_tag_name!(locale, tag_name_raw, category: :material,
-                                   created_by_user: current_user)
-  end
-
-  def material_index_needs_tag_name? filters
-    filters[:q].present? || filters[:sort] == 'tag_name'
-  end
-
-  def material_index_needs_file_blob? filters
-    filters[:q].present? ||
-        filters[:media_kind] != 'all' ||
-        ['media_kind', 'file_byte_size'].include?(filters[:sort])
-  end
-
-  def material_index_join_tag_name q
-    q.left_joins(:tag).joins(<<~SQL.squish)
-      LEFT JOIN tag_names
-        ON tag_names.tag_id = tags.id
-       AND tag_names.language_code = 'ja'
-       AND tag_names.primary_flg = TRUE
-    SQL
-  end
-
-  def material_index_join_file_blob q
-    q.joins(<<~SQL.squish)
-      LEFT JOIN active_storage_attachments material_file_attachments
-        ON material_file_attachments.record_type = 'Material'
-       AND material_file_attachments.record_id = materials.id
-       AND material_file_attachments.name = 'file'
-      LEFT JOIN active_storage_blobs material_file_blobs
-        ON material_file_blobs.id = material_file_attachments.blob_id
-    SQL
-  end
-
-  def apply_material_query q, term
-    like = "%#{ ActiveRecord::Base.sanitize_sql_like(term) }%"
-    q.where(<<~SQL.squish, q: like)
-      tag_names.name LIKE :q
-      OR materials.url LIKE :q
-      OR material_file_blobs.filename LIKE :q
-    SQL
-  end
-
-  def apply_material_media_kind q, media_kind
-    case media_kind
-    when 'image'
-      q.where('material_file_blobs.content_type LIKE ?', 'image/%')
-    when 'video'
-      q.where('material_file_blobs.content_type LIKE ?', 'video/%')
-    when 'audio'
-      q.where('material_file_blobs.content_type LIKE ?', 'audio/%')
-    when 'file_other'
-      q.where('material_file_attachments.id IS NOT NULL')
-       .where.not('material_file_blobs.content_type LIKE ?', 'image/%')
-       .where.not('material_file_blobs.content_type LIKE ?', 'video/%')
-       .where.not('material_file_blobs.content_type LIKE ?', 'audio/%')
-    when 'url_only'
-      q.where('material_file_attachments.id IS NULL').where.not(url: [nil, ''])
-    else
-      q
-    end
-  end
-
-  def apply_material_tag_filter q, filters, tag_graph
-    return q unless filters[:tag_id]
-
-    tag_ids =
-        if tag_graph
-          tag_graph[:scope_tag_ids]
-        else
-          [filters[:tag_id]]
-        end
-
-    q.where(tag_id: tag_ids)
-  end
-
-  def material_index_tag_graph filters
-    return nil unless filters[:tag_id]
-
-    children_by_parent_id = Hash.new { |h, k| h[k] = [] }
-    TagImplication.pluck(:parent_tag_id, :tag_id).each do |parent_id, child_id|
-      children_by_parent_id[parent_id] << child_id
-    end
-
-    scope_tag_ids =
-        if filters[:include_descendants]
-          collect_material_descendant_ids(filters[:tag_id], children_by_parent_id)
-        else
-          Set.new([filters[:tag_id]])
-        end
-    tags_by_id =
-        Tag
-        .joins(:tag_names)
-        .where(id: scope_tag_ids,
-               tag_names: { language_code: 'ja', primary_flg: true })
-        .pluck('tags.id', 'tag_names.name', 'tags.category', 'tags.deprecated_at')
-        .each_with_object({ }) do |(id, name, category, deprecated_at), hash|
-          hash[id] = { id:, name:, category:, deprecated: deprecated_at.present? }
-        end
-
-    selected_tag = tags_by_id[filters[:tag_id]]
-    return nil unless selected_tag
-
-    group_tag_id_by_tag_id = { }
-    children_by_parent_id[filters[:tag_id]].each do |child_tag_id|
-      assign_material_group_branch(child_tag_id,
-                                   children_by_parent_id:,
-                                   tags_by_id:,
-                                   group_tag_id_by_tag_id:,
-                                   current_group_tag_id: nil,
-                                   seen: Set.new([filters[:tag_id]]))
-    end
-
-    { selected_tag_id: filters[:tag_id],
-      selected_tag:,
-      scope_tag_ids: scope_tag_ids.to_a,
-      tags_by_id:,
-      group_tag_id_by_tag_id: }
-  end
-
-  def collect_material_descendant_ids selected_tag_id, children_by_parent_id
-    ids = Set.new([selected_tag_id])
-    stack = [selected_tag_id]
-
-    until stack.empty?
-      tag_id = stack.pop
-      children_by_parent_id[tag_id].each do |child_tag_id|
-        next if ids.include?(child_tag_id)
-
-        ids << child_tag_id
-        stack << child_tag_id
-      end
-    end
-
-    ids
-  end
-
-  def assign_material_group_branch tag_id, children_by_parent_id:, tags_by_id:,
-                                   group_tag_id_by_tag_id:, current_group_tag_id:, seen:
-    return if seen.include?(tag_id)
-
-    tag = tags_by_id[tag_id]
-    return unless tag
-
-    seen = seen.dup << tag_id
-    next_group_tag_id = current_group_tag_id
-    next_group_tag_id = tag_id if next_group_tag_id.nil? && !tag[:deprecated]
-    group_tag_id_by_tag_id[tag_id] = next_group_tag_id if next_group_tag_id
-
-    children_by_parent_id[tag_id].each do |child_tag_id|
-      assign_material_group_branch(child_tag_id,
-                                   children_by_parent_id:,
-                                   tags_by_id:,
-                                   group_tag_id_by_tag_id:,
-                                   current_group_tag_id: next_group_tag_id,
-                                   seen:)
-    end
-  end
-
-  def material_index_groups materials, tag_graph
-    return [] unless tag_graph
-
-    groups_by_key = { }
-
-    materials.each do |material|
-      next unless material.tag_id
-
-      group_tag_id =
-          if material.tag_id == tag_graph[:selected_tag_id]
-            tag_graph[:selected_tag_id]
-          else
-            tag_graph[:group_tag_id_by_tag_id][material.tag_id] || tag_graph[:selected_tag_id]
-          end
-
-      group_tag = tag_graph[:tags_by_id][group_tag_id]
-      next unless group_tag
-
-      key = "tag:#{ group_tag_id }"
-      group =
-          groups_by_key[key] ||= { key:,
-                                   tag: group_tag_repr(group_tag),
-                                   material_ids: [],
-                                   count: 0 }
-      group[:material_ids] << material.id
-      group[:count] += 1
-    end
-
-    groups_by_key.values.sort_by { |group| group[:tag][:name] }
-  end
-
-  def material_index_tag_scope filters, tag_graph
-    return nil unless tag_graph
-
-    {
-      tag: group_tag_repr(tag_graph[:selected_tag]),
-      include_descendants: filters[:include_descendants]
-    }
-  end
-
-  def group_tag_repr group_tag
-    { id: group_tag[:id],
-      name: group_tag[:name],
-      category: group_tag[:category] }
-  end
-
-  def material_index_order_sql filters
-    direction = filters[:direction] == 'asc' ? 'ASC' : 'DESC'
-    sort_sql =
-        case filters[:sort]
-        when 'tag_name'
-          'tag_names.name'
-        when 'media_kind'
-          material_media_kind_sql
-        when 'file_byte_size'
-          'material_file_blobs.byte_size'
-        when 'updated_at'
-          'materials.updated_at'
-        when 'version_no'
-          'materials.version_no'
-        when 'id'
-          'materials.id'
-        else
-          'materials.created_at'
-        end
-
-    "#{ sort_sql } #{ direction }, materials.id #{ direction }"
-  end
-
-  def material_media_kind_sql
-    "CASE " \
-      "WHEN material_file_attachments.id IS NULL AND materials.url IS NOT NULL THEN 5 " \
-      "WHEN material_file_blobs.content_type LIKE 'image/%' THEN 1 " \
-      "WHEN material_file_blobs.content_type LIKE 'video/%' THEN 2 " \
-      "WHEN material_file_blobs.content_type LIKE 'audio/%' THEN 3 " \
-      "WHEN material_file_attachments.id IS NOT NULL THEN 4 " \
-      "ELSE 6 END"
-  end
-
-  def upsert_export_paths! material
-    raw = params[:export_paths]
-    return if raw.blank?
-
-    export_paths = raw.respond_to?(:to_unsafe_h) ? raw.to_unsafe_h : raw.to_h
-    export_paths.each do |profile, export_path|
-      profile = profile.to_s
-      export_path = export_path.to_s.strip
-      item = material.material_export_items.find_or_initialize_by(profile:)
-
-      if export_path.blank?
-        item.destroy! if item.persisted?
-        next
-      end
-
-      item.export_path = export_path
-      item.enabled = true
-      item.created_by_user ||= current_user
-      item.save!
-    end
-  end
-
-  def render_zip_empty
-    render_unprocessable_entity('ZIP export 対象の素材がありません．')
-  end
-
-  def render_zip_duplicate_path error
-    render_unprocessable_entity("ZIP export path が重複してゐます: #{ error.message }")
-  end
-
-  def render_zip_missing_file error
-    missing_files = error.missing_files.map do |missing_file|
-      { material_id: missing_file.material_id,
-        export_path: missing_file.export_path,
-        blob_id: missing_file.blob_id,
-        filename: missing_file.filename }
-    end
-
-    render json: { type: 'validation_error',
-                   message: 'ZIP export に必要な素材ファイルが欠損しています．',
-                   errors: { },
-                   base_errors: ['ZIP export に必要な素材ファイルが欠損しています．'],
-                   missing_files: },
-           status: :unprocessable_entity
-  end
-
-  def build_uploaded_material_blob! file, file_sha256
-    return nil unless file
-
-    file.tempfile.rewind
-
-    blob = ActiveStorage::Blob.create_and_upload!(
-        io: file.tempfile,
-        filename: file.original_filename,
-        content_type: file.content_type,
-    )
-
-    MaterialFileSha256.assign_metadata_sha256!(blob, file_sha256)
-
-    blob
-  ensure
-    file.tempfile.rewind if file&.tempfile
-  end
-
-  def render_material_import_block block
-    render_validation_error fields: { file: ["抑止された素材です: #{ block.reason }"] }
-  end
-
-  def log_thumbnail_generation material, result
-    material.reload
-    Rails.logger.info(
-        "Material thumbnail generation: material_id=#{ material.id } " \
-        "result=#{ result } " \
-        "thumbnail_attached=#{ material.thumbnail.attached? } " \
-        "content_type=#{ material.content_type }")
-  end
 end

backend/app/controllers/nico_tags_controller.rb

@@ -1,7 +1,5 @@
 class NicoTagsController < ApplicationController
   def index
-    locale = resolve_locale!
-
     name = params[:name].presence
     linked_tag = params[:linked_tag].presence
     link_status = params[:link_status].presence
@@ -17,27 +15,21 @@ class NicoTagsController < ApplicationController
     limit = 1 if limit < 1
 
     post_tag_max_sql =
-        PostExternalTag
+        PostTag
-        .select('external_tag_id, MAX(created_at) AS max_created_at')
+        .select('tag_id, MAX(created_at) AS max_created_at')
-        .group('external_tag_id')
+        .group('tag_id')
         .to_sql
 
-    q =
+    q = Tag.nico_tags
-        ExternalTag
+           .joins(:tag_name)
-        .joins("LEFT JOIN (#{ post_tag_max_sql }) post_tag_max " +
+           .joins("LEFT JOIN (#{ post_tag_max_sql }) post_tag_max " \
-               'ON post_tag_max.external_tag_id = external_tags.id')
+                  'ON post_tag_max.tag_id = tags.id')
-        .includes(linked_tags: [:tag_names, :tag_descriptions])
+           .includes(:tag_name, tag_name: :wiki_page, linked_tags: { tag_name: :wiki_page })
-    if name
+    q = q.where('tag_names.name LIKE ?', "%#{ name }%") if name
-      q = q.where(('external_tags.name LIKE ? ' +
-                   "OR CONCAT(external_tags.platform, ':', external_tags.name) LIKE ?"),
-                  "%#{ name }%", "%#{ name }")
-    end
-
     if linked_tag
       linked_tag_ids =
         Tag
-          .joins(:tag_names)
+        .joins(:tag_name)
-          .where(tag_names: { language_code: locale.language_code, primary_flg: true })
         .where('tag_names.name LIKE ?', "%#{ linked_tag }%")
         .pluck(:id)
       linked_nico_tag_ids = NicoTagRelation.where(tag_id: linked_tag_ids).pluck(:nico_tag_id)
@@ -45,8 +37,8 @@ class NicoTagsController < ApplicationController
     end
     if link_status.in?(['linked', 'unlinked'])
       exists_sql =
-          'EXISTS (SELECT 1 FROM nico_tag_relations ' +
+          'EXISTS (SELECT 1 FROM nico_tag_relations ' \
-          'WHERE nico_tag_relations.nico_tag_id = external_tags.id)'
+          'WHERE nico_tag_relations.nico_tag_id = tags.id)'
       q = link_status == 'linked' ? q.where(exists_sql) : q.where("NOT #{ exists_sql }")
     end
 
@@ -54,69 +46,54 @@ class NicoTagsController < ApplicationController
     sort_sql =
         case order[0]
         when 'name'
-          'external_tags.name'
+          'tag_names.name'
         when 'updated_at'
           'post_tag_max.max_created_at'
         else
-          "external_tags.#{ order[0] }"
+          "tags.#{ order[0] }"
         end
-    tags = q.reselect('external_tags.*',
+    tags = q.reselect('tags.*',
                       Arel.sql('post_tag_max.max_created_at AS recent_post_tag_created_at'))
-            .order(Arel.sql("#{ sort_sql } #{ order[1] }, external_tags.id #{ order[1] }"))
+            .order(Arel.sql("#{ sort_sql } #{ order[1] }, tags.id #{ order[1] }"))
             .limit(limit)
             .offset((page - 1) * limit)
             .to_a
 
     render json: { tags: tags.map { |tag|
-      external_tag_json(tag).merge(
+      TagRepr.base(tag).merge(
           recent_post_tag_created_at: tag.recent_post_tag_created_at,
-          linked_tags: tag.linked_tags.map { |lt|
+          linked_tags: tag.linked_tags.map { |lt| TagRepr.base(lt) })
-            TagRepr.base(lt, language_code: locale.language_code)
-          })
     }, count: }
   end
 
-  def show
-    tag = ExternalTag.find(params[:id])
-
-    render json: external_tag_json(tag)
-  end
-
   def update
     return head :unauthorized unless current_user
     return head :forbidden unless current_user.gte_member?
 
-    locale = resolve_locale!
-
     id = params[:id].to_i
 
-    tag = ExternalTag.find(id)
+    tag = Tag.find(id)
+    return render_bad_request('ニコニコ・タグを指定してください．') unless tag.nico?
 
     linked_tag_names = params[:tags].to_s.split
     linked_tags = nil
 
     ApplicationRecord.transaction do
-      linked_tags = Tag.normalise_tags!(locale, linked_tag_names,
+      linked_tags = Tag.normalise_tags!(linked_tag_names, with_tagme: false,
-                                        with_tagme: false,
+                                                          with_no_deerjikist: false)
-                                        with_no_deerjikist: false,
+      if linked_tags.any? { |t| t.nico? }
-                                        created_by_user: current_user)
+        raise Tag::NicoTagNormalisationError
+      end
 
-      TagVersioning.record_tag_snapshots!(
+      TagVersioning.record_tag_snapshots!(linked_tags, created_by_user: current_user)
-          linked_tags,
-          created_by_user: current_user,
-          language_code: locale.language_code)
 
       tag.linked_tags = linked_tags
       tag.save!
 
-      NicoTagVersionRecorder.record!(external_tag: tag,
+      NicoTagVersionRecorder.record!(tag:, event_type: :update, created_by_user: current_user)
-                                     event_type: :update,
-                                     created_by_user: current_user)
     end
 
-    render json: tag.linked_tags.map { |t|
+    render json: tag.linked_tags.map { |t| TagRepr.base(t) }, status: :ok
-      TagRepr.base(t, language_code: locale.language_code)
-    }, status: :ok
   rescue Tag::NicoTagNormalisationError
     render_validation_error fields: { tags: ['ニコニコ・タグ同士は連携できません．'] }
   rescue ActiveRecord::RecordInvalid => e
@@ -125,39 +102,11 @@ class NicoTagsController < ApplicationController
 
   private
 
-  def external_tag_json tag
-    { id: tag.id,
-      name: "#{ tag.platform }:#{ tag.name }",
-      category: 'nico',
-      post_count: tag.post_count,
-      created_at: tag.created_at,
-      updated_at: tag.created_at,
-      deprecated_at: nil,
-      aliases: [],
-      parents: [],
-      has_wiki: false,
-      material_id: nil,
-      has_deerjikists: false }
-  end
-
   def render_nico_tag_form_record_invalid record
-    names =
+    if record.is_a?(TagName) || record.is_a?(Tag)
-        case record
+      render_validation_error fields: { tags: record.errors.full_messages.map { |message|
-        when TagName
+        "タグ名 “#{ record.name }”: #{ message }"
-          [record]
+      } }
-        when Tag
-          record.tag_names.to_a
-        else
-          []
-        end
-    messages = names.flat_map do |tag_name|
-      tag_name.errors.full_messages.map do |message|
-        "タグ名 “#{ tag_name.name }”: #{ message }"
-      end
-    end
-
-    if messages.any?
-      render_validation_error fields: { tags: messages }
     else
       render_validation_error record
     end

backend/app/controllers/post_versions_controller.rb

@@ -1,11 +1,7 @@
 class PostVersionsController < ApplicationController
   def index
-    locale = resolve_locale!
     post_id = params[:post].presence
-    tag_id = params[:tag].presence&.to_i
+    tag_id = params[:tag].presence
-    external_tag_id = params[:external_tag].presence&.to_i
-    return head :bad_request if tag_id && external_tag_id
-
     page = (params[:page].presence || 1).to_i
     limit = (params[:limit].presence || 20).to_i
 
@@ -14,16 +10,29 @@ class PostVersionsController < ApplicationController
 
     offset = (page - 1) * limit
 
-    q = HistoryAccess.visible(version_scope)
+    tag_name =
+        if tag_id
+          TagName.joins(:tag).find_by(tag: { id: tag_id })
+        end
+    return render json: { versions: [], count: 0 } if tag_id && tag_name.blank?
+
+    q = PostVersion.joins(<<~SQL.squish)
+          LEFT JOIN
+            post_versions prev
+            ON
+              prev.post_id = post_versions.post_id
+              AND prev.version_no = post_versions.version_no - 1
+        SQL
+        .select('post_versions.*', 'prev.title AS prev_title', 'prev.url AS prev_url',
+                'prev.thumbnail_base AS prev_thumbnail_base', 'prev.tags AS prev_tags',
+                'prev.original_created_from AS prev_original_created_from',
+                'prev.original_created_before AS prev_original_created_before')
     q = q.where('post_versions.post_id = ?', post_id) if post_id
-    if external_tag_id || (tag_id && !(Tag.exists?(id: tag_id)))
+    if tag_name
-      q = q.where('JSON_CONTAINS(post_versions.tags_json,' +
+      escaped = ActiveRecord::Base.sanitize_sql_like(tag_name.name)
-                  "JSON_OBJECT('external_tag_id', #{ external_tag_id || tag_id })) " +
+      q = q.where(("CONCAT(' ', post_versions.tags, ' ') LIKE :kw " +
-                  'OR JSON_CONTAINS(prev.tags_json,' +
+                   "OR CONCAT(' ', prev.tags, ' ') LIKE :kw"),
-                  "JSON_OBJECT('external_tag_id', #{ external_tag_id || tag_id }))")
+                  kw: "% #{ escaped } %")
-    elsif tag_id
-      q = q.where("JSON_CONTAINS(post_versions.tags_json, JSON_OBJECT('tag_id', #{ tag_id })) " +
-                  "OR JSON_CONTAINS(prev.tags_json, JSON_OBJECT('tag_id', #{ tag_id }))")
     end
 
     count = q.except(:select, :order, :limit, :offset).count
@@ -32,129 +41,79 @@ class PostVersionsController < ApplicationController
                 .limit(limit)
                 .offset(offset)
 
-    render json: { versions: serialise_versions(versions, locale.language_code), count:, population_count: HistoryAccess.population_count(PostVersion) }
+    render json: { versions: serialise_versions(versions), count: }
   end
 
   private
 
-  def version_scope
+  def serialise_versions rows
-    PostVersion.with_previous_version
-        .select('post_versions.*',
-                'prev.title AS prev_title',
-                'prev.url AS prev_url',
-                'prev.thumbnail_base AS prev_thumbnail_base',
-                'prev.tags_json AS prev_tags_json',
-                'prev.video_ms AS prev_video_ms',
-                'prev.original_created_from AS prev_original_created_from',
-                'prev.original_created_before AS prev_original_created_before')
-  end
-
-  def serialise_versions rows, language_code
-    rows = rows.to_a
     user_ids = rows.map(&:created_by_user_id).compact.uniq
     users_by_id = User.where(id: user_ids).pluck(:id, :name).to_h
-    snapshots = rows.flat_map { |row|
-      [normalise_json(row.tags_json),
-       normalise_json(row.attributes['prev_tags_json']) || []]
-    }
-    tag_ids = snapshots.flatten.filter_map { _1['tag_id'] }.uniq
-    tag_names = TagName.where(tag_id: tag_ids, language_code:, primary_flg: true)
-                       .pluck(:tag_id, :name).to_h
-    external_tag_names = external_tag_names_for(snapshots)
 
     rows.map do |row|
-      cur_tags = snapshot_tag_literals(
+      cur_tags = split_tags(row.tags)
-          normalise_json(row.tags_json), tag_names, external_tag_names)
+      prev_tags = split_tags(row.attributes['prev_tags'])
-      prev_tags = snapshot_tag_literals(
-          normalise_json(row.attributes['prev_tags_json']) || [], tag_names, external_tag_names)
 
-      { post_id: row.post_id,
+      {
+        post_id: row.post_id,
         version_no: row.version_no,
         event_type: row.event_type,
-        title: { current: row.title, prev: row.attributes['prev_title'] },
+        title: {
-        url: { current: row.url, prev: row.attributes['prev_url'] },
+          current: row.title,
-        thumbnail: { current: nil, prev: nil },
+          prev: row.attributes['prev_title']
-        thumbnail_base: { current: row.thumbnail_base,
+        },
-                          prev: row.attributes['prev_thumbnail_base'] },
+        url: {
-        video_ms: { current: row.video_ms, prev: row.attributes['prev_video_ms'] },
+          current: row.url,
+          prev: row.attributes['prev_url']
+        },
+        thumbnail: {
+          current: nil,
+          prev: nil
+        },
+        thumbnail_base: {
+          current: row.thumbnail_base,
+          prev: row.attributes['prev_thumbnail_base']
+        },
         tags: build_version_tags(cur_tags, prev_tags),
-        parent_post_ids: row.parent_post_ids,
         original_created_from: {
           current: row.original_created_from&.iso8601,
-            prev: row.attributes['prev_original_created_from']&.iso8601 },
+          prev: row.attributes['prev_original_created_from']&.iso8601
+        },
         original_created_before: {
           current: row.original_created_before&.iso8601,
-            prev: row.attributes['prev_original_created_before']&.iso8601 },
+          prev: row.attributes['prev_original_created_before']&.iso8601
+        },
         created_at: row.created_at.iso8601,
         created_by_user:
           if row.created_by_user_id
-              { id: row.created_by_user_id,
+            {
-                name: users_by_id[row.created_by_user_id] }
+              id: row.created_by_user_id,
-            end }
+              name: users_by_id[row.created_by_user_id]
-    end
-  end
-
-  def external_tag_names_for snapshots
-    ids = snapshots.flatten.filter_map { _1['external_tag_id'] }.uniq
-    names = ExternalTag.where(id: ids).pluck(:id, :platform, :name).to_h { |id, platform, name|
-      [id, "#{ platform }:#{ name }"]
             }
-    missing_ids = ids - names.keys
-
-    HistoryAccess.visible(NicoTagVersion.all)
-        .where(tag_id: missing_ids)
-        .order(:tag_id, version_no: :desc)
-        .pluck(:tag_id, :name)
-        .each { |id, name| names[id] ||= name }
-
-    names
-  end
-
-  def snapshot_tag_literals snapshots, tag_names, external_tag_names
-    snapshots
-        .filter_map { |snapshot|
-          if snapshot.key?('tag_id')
-            sections = snapshot.fetch('sections', []).map { |section|
-              [section.fetch('begin_ms'), section['end_ms']]
-            }.sort_by { |begin_ms, end_ms| [begin_ms, end_ms || Float::INFINITY] }
-            name = tag_names[snapshot['tag_id']] || snapshot.fetch('name')
-            [tag_category_order(snapshot['category']),
-             { identity: [:tag, snapshot['tag_id'], sections],
-               name: Post.tag_snapshot_literal(snapshot.merge('name' => name)),
-               restore_name: tag_names[snapshot['tag_id']] &&
-                   Post.tag_snapshot_literal(snapshot.merge('name' => name)),
-               external: false }]
-          elsif external_tag_names[snapshot['external_tag_id']]
-            [6, { identity: [:external_tag, snapshot['external_tag_id']],
-                  name: external_tag_names[snapshot['external_tag_id']],
-                  restore_name: nil, external: true }]
           end
       }
-        .sort_by { |order, tag| [order, tag[:name].downcase] }
+    end
-        .map(&:second)
   end
 
-  def tag_category_order category
+  def build_version_tags(cur_tags, prev_tags)
-    ['deerjikist', 'meme', 'character', 'general', 'material', 'meta'].index(category) || 6
+    (cur_tags | prev_tags).map do |name|
-  end
-
-  def build_version_tags cur_tags, prev_tags
-    current = cur_tags.index_by { _1[:identity] }
-    previous = prev_tags.index_by { _1[:identity] }
-
-    (current.keys | previous.keys).map do |identity|
       type =
-          if current.key?(identity) && previous.key?(identity)
+        if cur_tags.include?(name) && prev_tags.include?(name)
           'context'
-          elsif current.key?(identity)
+        elsif cur_tags.include?(name)
           'added'
         else
           'removed'
         end
 
-      entry = current[identity] || previous[identity]
+      {
-      { name: entry[:name], restore_name: entry[:restore_name],
+        name:,
-        external: entry[:external], type: }
+        type:
+      }
     end
   end
+
+  def split_tags(tags)
+    tags.to_s.split(/\s+/).reject(&:blank?)
+  end
 end

backend/app/controllers/posts_controller.rb

@@ -1,14 +1,7 @@
 class PostsController < ApplicationController
   Event = Struct.new(:post, :tag, :user, :change_type, :timestamp, keyword_init: true)
-  MAX_BULK_REQUEST_BYTES = 40 * 1024 * 1024
-
-  class VideoMsParseError < ArgumentError
-    ;
-  end
 
   def index
-    locale = resolve_locale!
-
     url = params[:url].presence
     title = params[:title].presence
     original_created_from = params[:original_created_from].presence
@@ -38,8 +31,8 @@ class PostsController < ApplicationController
     offset = (page - 1) * limit
 
     pt_max_sql =
-        HistoryAccess.visible(PostVersion.all)
+        PostTag
-        .select('post_id, MAX(created_at) AS max_updated_at')
+        .select('post_id, MAX(updated_at) AS max_updated_at')
         .group('post_id')
         .to_sql
 
@@ -48,12 +41,11 @@ class PostsController < ApplicationController
         'COALESCE(pt_max.max_updated_at, posts.updated_at))'
 
     q =
-        filtered_posts(locale)
+        filtered_posts
         .joins("LEFT JOIN (#{ pt_max_sql }) pt_max ON pt_max.post_id = posts.id")
         .reselect('posts.*', Arel.sql("#{ updated_at_all_sql } AS updated_at_all"))
-        .preload(:external_tags, :uploaded_user, :parents, :children,
+        .preload(:uploaded_user, :parents, :children,
-                 post_tags: [:sections, { tag: [:deerjikists, :materials,
+                 tags: [:deerjikists, :materials, { tag_name: :wiki_page }])
-                                                :tag_names, :tag_descriptions] }])
         .with_attached_thumbnail
 
     q = q.where('posts.url LIKE ?', "%#{ url }%") if url
@@ -92,8 +84,7 @@ class PostsController < ApplicationController
     q = q.except(:select, :order)
 
     render json: { posts: posts.map { |post|
-      PostRepr.base(post, language_code: locale.language_code)
+      PostRepr.base(post).merge(updated_at: post.updated_at_all).tap do |json|
-              .merge(updated_at: post.updated_at_all).tap do |json|
         json['thumbnail'] =
             if post.thumbnail.attached?
               rails_storage_proxy_url(post.thumbnail, only_path: false)
@@ -105,100 +96,21 @@ class PostsController < ApplicationController
   end
 
   def random
-    locale = resolve_locale!
+    post = filtered_posts.preload(:uploaded_user, :parents, :children,
-
+                                  tags: [:deerjikists, :materials, { tag_name: :wiki_page }])
-    post =
-        filtered_posts(locale)
-        .preload(:uploaded_user, :parents, :children,
-                 post_tags: [:sections, { tag: [:deerjikists, :materials,
-                                                :tag_names, :tag_descriptions] }])
                          .with_attached_thumbnail
                          .order('RAND()')
                          .first
     return head :not_found unless post
 
-    render json: PostRepr.base(post, current_user, language_code: locale.language_code)
+    render json: PostRepr.base(post, current_user)
-  end
-
-  def metadata
-    return head :unauthorized unless current_user
-    return head :forbidden unless current_user.gte_member?
-    return render_bad_request('URL は必須です．') if params[:url].blank?
-
-    normal_url = PostUrlNormaliser.normalise(params[:url].to_s)
-    return render_validation_error(fields: { url: ['URL が不正です．'] }) if normal_url.blank?
-
-    Preview::UrlSafety.validate(normal_url)
-    existing_post = Post.with_attached_thumbnail.find_by(url: normal_url)
-    if existing_post.present?
-      return render json: {
-          url: normal_url,
-          title: nil,
-          thumbnail_base: nil,
-          tags: nil,
-          display_tags: [],
-          original_created_from: nil,
-          original_created_before: nil,
-          duration: nil,
-          video_ms: nil,
-          field_warnings: { },
-          base_warnings: [],
-          existing_post_id: existing_post.id,
-          existing_post: compact_post(existing_post.id) }
-    end
-
-    metadata = PostMetadataFetcher.fetch(normal_url)
-    field_warnings = { }
-    field_warnings[:title] = ['タイトルを取得できませんでした．'] if metadata[:title].blank?
-    if metadata[:thumbnail_base].blank?
-      field_warnings[:thumbnail_base] = ['サムネールを取得できませんでした．']
-    end
-
-    render json: {
-        url: normal_url,
-        title: metadata[:title],
-        thumbnail_base: metadata[:thumbnail_base],
-        tags: metadata[:tags],
-        display_tags: metadata[:display_tags],
-        original_created_from: metadata[:original_created_from],
-        original_created_before: metadata[:original_created_before],
-        duration: metadata[:duration],
-        video_ms: metadata[:video_ms],
-        field_warnings: field_warnings,
-        base_warnings: [],
-        existing_post_id: nil,
-        existing_post: nil }
-  rescue ArgumentError => e
-    render_bad_request e.message
-  rescue Preview::UrlSafety::UnsafeUrl => e
-    render_validation_error fields: { url: [e.message] }
-  rescue Preview::HttpFetcher::FetchFailed,
-         Preview::HttpFetcher::FetchTimeout,
-         Preview::HttpFetcher::ResponseTooLarge
-    render json: {
-        url: normal_url,
-        title: nil,
-        thumbnail_base: nil,
-        tags: nil,
-        display_tags: [],
-        original_created_from: nil,
-        original_created_before: nil,
-        duration: nil,
-        video_ms: nil,
-        field_warnings: { url: ['自動取得に失敗しました．'] },
-        base_warnings: [],
-        existing_post_id: nil,
-        existing_post: nil }
   end
 
   def show
-    locale = resolve_locale!
-
     post =
         Post
         .includes(:uploaded_user, :parents, :children,
-                  post_tags: [:sections, { tag: [:deerjikists, :materials,
+                  tags: [:deerjikists, :materials, { tag_name: :wiki_page }])
-                                                 :tag_names, :tag_descriptions] }])
         .with_attached_thumbnail
         .find_by(id: params[:id])
     return head :not_found unless post
@@ -212,94 +124,53 @@ class PostsController < ApplicationController
                                  parent_posts:,
                                  child_posts:,
                                  sibling_posts:,
-                                 related:,
+                                 related:)
-                                 language_code: locale.language_code)
+                         .merge(tags: build_tag_tree_for(post.tags))
-                         .merge(tags: build_tag_tree_for(post, locale.language_code))
   end
 
   def create
     return head :unauthorized unless current_user
     return head :forbidden unless current_user.gte_member?
 
-    locale = resolve_locale!
+    # TODO: サイトに応じて thumbnail_base 設定
+    title = params[:title].presence
+    url = params[:url]
+    thumbnail = params[:thumbnail]
+    tag_names = params[:tags].to_s.split
+    original_created_from = params[:original_created_from]
+    original_created_before = params[:original_created_before]
+    parent_post_ids = parse_parent_post_ids
 
-    preflight = PostCreatePreflight.new(
+    post = Post.new(title:, url:, thumbnail_base: nil, uploaded_user: current_user,
-        attributes: post_create_attributes,
+                    original_created_from:, original_created_before:)
-        thumbnail: params[:thumbnail],
+    post.thumbnail.attach(thumbnail) if thumbnail.present?
-        host: request.base_url).run(locale)
+
-    return render json: dry_run_json(preflight) if bool?(:dry)
+    ApplicationRecord.transaction do
-    if preflight[:existing_post_id].present?
+      post.save!
-      post = Post.new(url: preflight[:url])
+
-      post.errors.add :url, :taken
+      tags = Tag.normalise_tags!(tag_names)
-      return render_post_form_record_invalid post
+      TagVersioning.record_tag_snapshots!(tags, created_by_user: current_user)
+
+      tags = Tag.expand_parent_tags(tags)
+      sync_post_tags!(post, tags)
+
+      sync_parent_posts!(post, parent_post_ids)
+
+      post.resized_thumbnail!
+
+      PostVersionRecorder.record!(post:, event_type: :create, created_by_user: current_user)
     end
 
-    post = PostCreator.new(actor: current_user,
-                           attributes: post_create_attributes.merge(
-                               preflight.slice(
-                                   :url,
-                                   :title,
-                                   :thumbnail_base,
-                                   :tags,
-                                   :parent_post_ids,
-                                   :original_created_from,
-                                   :original_created_before,
-                                   :duration,
-                                   :video_ms,
-                                   :direct_tag_specs,
-                                   :default_tag_specs,
-                                   :snapshot_tag_specs,
-                                   :post_tag_specs,
-                                   :tag_sections,
-                                   :normalised_parent_post_ids).symbolize_keys).merge(
-                               thumbnail: params[:thumbnail])).create!(locale)
-
     post.reload
-    render json: PostRepr.base(post, language_code: locale.language_code), status: :created
+    render json: PostRepr.base(post), status: :created
-  rescue PostCreatePreflight::ValidationFailed => e
-    render_validation_error fields: e.fields, base: e.base_errors
   rescue Tag::NicoTagNormalisationError
     render_validation_error fields: { tags: 'ニコニコ・タグは直接指定できません．' }
-  rescue Tag::DeprecatedTagNormalisationError
-    render_unprocessable_entity '廃止済みタグは付与できません．', field: :tags
-  rescue Tag::SectionLiteralParseError
-    render_validation_error fields: { tags: ['タグ区間の記法が不正です．'] }
-  rescue PostCreator::VideoMsParseError
-    render_validation_error fields: { video_ms: ['動画時間の記法が不正です．'] }
-  rescue Post::RemoteThumbnailFetchFailed
-    render_validation_error fields: { thumbnail_base: ['サムネイル画像の取得に失敗しました．'] }
-  rescue MiniMagick::Error
-    render_validation_error fields: { thumbnail: ['サムネイル画像の変換に失敗しました．'] }
   rescue ArgumentError => e
     render_validation_error fields: { parent_post_ids: [e.message] }
   rescue ActiveRecord::RecordInvalid => e
     render_post_form_record_invalid e.record
   end
 
-  def bulk
-    return head :unauthorized unless current_user
-    return head :forbidden unless current_user.gte_member?
-    return head :unsupported_media_type unless request.content_mime_type == Mime[:multipart_form]
-    return head :payload_too_large if request.content_length.to_i > MAX_BULK_REQUEST_BYTES
-
-    locale = resolve_locale!
-
-    posts = parse_bulk_posts_manifest
-    thumbnails = parse_bulk_thumbnails(posts.length)
-
-    result = PostBulkCreator.new(
-        actor: current_user,
-        posts:,
-        thumbnails:,
-        host: request.base_url).run(locale)
-
-    render json: result
-  rescue JSON::ParserError
-    render_bad_request 'posts manifest の JSON が不正です．'
-  rescue ArgumentError => e
-    render_validation_error base: [e.message]
-  end
-
   def viewed
     return head :unauthorized unless current_user
 
@@ -325,7 +196,6 @@ class PostsController < ApplicationController
     base_version_no = parse_base_version_no
     return render_bad_request('base_version_no は必須です．') if !(force) && !(base_version_no)
 
-    locale = Locale.find_by(code: params[:locale].presence) || Locale.nipponese
     title = params[:title].presence
     tag_names = params[:tags].to_s.split
     original_created_from = params[:original_created_from]
@@ -344,17 +214,13 @@ class PostsController < ApplicationController
       unless force
         base_version = post.post_versions.find_by!(version_no: base_version_no)
 
-        HistoryAccess.require!(base_version)
         base_snapshot = post_snapshot_from_version(base_version)
-        current_snapshot = post_snapshot_from_record(post, locale)
+        current_snapshot = post_snapshot_from_record(post)
       end
-      incoming_snapshot = post_incoming_snapshot(locale:,
+      incoming_snapshot = post_incoming_snapshot(title:,
-                                                 title:,
                                                  original_created_from:,
                                                  original_created_before:,
                                                  tag_names:,
-                                                 video_ms_param: params[:video_ms],
-                                                 duration_param: params[:duration],
                                                  parent_post_ids:)
 
       snapshot_to_apply =
@@ -378,107 +244,94 @@ class PostsController < ApplicationController
             end
           end
 
-      apply_post_snapshot!(locale, post, snapshot_to_apply)
+      apply_post_snapshot!(post, snapshot_to_apply)
     end
 
     return render json: conflict_json, status: :conflict if conflict_json
 
     post.reload
-    json = PostRepr.base(post, current_user, language_code: locale.language_code)
+    json = PostRepr.base(post, current_user)
-    json['tags'] = build_tag_tree_for(post, locale.language_code)
+    json['tags'] = build_tag_tree_for(post.tags)
     render json:, status: :ok
   rescue Tag::NicoTagNormalisationError
     render_validation_error fields: { tags: ['ニコニコ・タグは直接指定できません．'] }
-  rescue Tag::DeprecatedTagNormalisationError
-    render_unprocessable_entity '廃止済みタグは付与できません．', field: :tags
-  rescue Tag::SectionLiteralParseError
-    render_validation_error fields: { tags: ['タグ区間の記法が不正です．'] }
-  rescue VideoMsParseError
-    render_validation_error fields: { video_ms: ['動画時間の記法が不正です．'] }
   rescue ArgumentError => e
     render_validation_error fields: { parent_post_ids: [e.message] }
   rescue ActiveRecord::RecordInvalid => e
     render_post_form_record_invalid e.record
   end
 
-  def tag_filters
+  def changes
-    source = resolve_locale!
+    id = params[:id].presence
-    code = params[:target_locale]
+    tag_id = params[:tag].presence
-    return head :not_found unless code.is_a?(String) && Locale.codes.value?(code)
+    page = (params[:page].presence || 1).to_i
+    limit = (params[:limit].presence || 20).to_i
 
-    target = Locale.find_by(code:, deprecated_at: nil)
+    page = 1 if page < 1
-    return head :not_found unless target
+    limit = 1 if limit < 1
 
-    literals = params[:tags].to_s.split
+    offset = (page - 1) * limit
-    references = tag_filter_references(literals)
-    names = literals.each_with_index.map do |literal, index|
-      reference = references&.[](index)
-      (reference && reference['negative'] == false) ? literal : literal.sub(/\Anot:/i, '')
-    end
-    internal = references ? {} : TagName.where(
-        language_code: source.language_code, name: names).pluck(:name, :tag_id).to_h
-    canonical = references ? {} : TagName.where(
-        language_code: source.language_code, primary_flg: true,
-        tag_id: internal.values).pluck(:tag_id, :name).to_h
-    external_names = names.map { canonical[internal[_1]] || _1 }
-    external = references ? {} : ExternalTag.where(
-        "CONCAT(platform, ':', name) IN (?)", external_names).pluck(
-        Arel.sql("CONCAT(platform, ':', name)"), :id).to_h
-    ids = references ? references.filter_map { _1['tag_id'] } : internal.values
-    translated = TagName.where(language_code: target.language_code,
-                               primary_flg: true, tag_id: ids).pluck(:tag_id, :name).to_h
 
-    filters = literals.each_with_index.map do |literal, index|
+    pts = PostTag.with_discarded
-      name = names[index]
+    pts = pts.where(post_id: id) if id.present?
-      reference = if references
+    pts = pts.where(tag_id:) if tag_id.present?
-                    references[index].except('name')
+    pts = pts.includes(:post, :created_user, :deleted_user,
-                  else
+                       tag: [:deerjikists, :materials, { tag_name: :wiki_page }])
-                    matches = {}
+
-                    matches['tag_id'] = internal[name] if internal[name]
+    events = []
-                    external_id = external[external_names[index]]
+    pts.each do |pt|
-                    matches['external_tag_id'] = external_id if external_id
+      tag = TagRepr.base(pt.tag)
-                    matches.empty? ? { 'missing' => true } : matches
+      post = pt.post
+
+      events << Event.new(
+          post:,
+          tag:,
+          user: pt.created_user && { id: pt.created_user.id, name: pt.created_user.name },
+          change_type: 'add',
+          timestamp: pt.created_at)
+
+      if pt.discarded_at
+        events << Event.new(
+            post:,
+            tag:,
+            user: pt.deleted_user && { id: pt.deleted_user.id, name: pt.deleted_user.name },
+            change_type: 'remove',
+            timestamp: pt.discarded_at)
       end
-      name = translated[reference['tag_id']] || name
-      negative = reference.fetch('negative', literal.downcase.start_with?('not:'))
-      prefix = negative ? literal[/\Anot:/i] || 'not:' : ''
-      reference.merge('name' => "#{ prefix }#{ name }", 'negative' => negative)
     end
-    render json: { references: filters }
+    events.sort_by!(&:timestamp)
+    events.reverse!
+
+    render json: { changes: (events.slice(offset, limit) || []).as_json, count: events.size }
   end
 
   private
 
-  def tag_filter_references literals
+  def filtered_posts
-    PostTagSearch.parse_references(params.to_unsafe_h, literals)
-  rescue ArgumentError, JSON::ParserError, TypeError
-    raise ActionController::BadRequest
-  end
-
-  def filtered_posts locale
     tag_names = params[:tags].to_s.split
     match_type = params[:match]
     if tag_names.present?
-      filter_posts_by_tags(locale, tag_names, match_type)
+      filter_posts_by_tags(tag_names, match_type)
     else
       Post.all
     end
   end
 
-  def filter_posts_by_tags locale, tag_names, match_type
+  def filter_posts_by_tags tag_names, match_type
-    references = tag_filter_references(tag_names)
+    literals = tag_names.map do |raw_name|
-    literals = PostTagSearch.literals(locale, tag_names.join(' '), references:)
+      { name: TagName.canonicalise(raw_name.sub(/\Anot:/i, '')).first,
+        negative: raw_name.downcase.start_with?('not:') }
+    end
 
     return Post.all if literals.empty?
 
     if match_type == 'any'
       literals.reduce(Post.none) do |posts, literal|
-        posts.or(tag_literal_relation(locale, literal[:name], negative: literal[:negative],
+        posts.or(tag_literal_relation(literal[:name], negative: literal[:negative]))
-                                      reference: literal[:reference]))
       end
     else
       literals.reduce(Post.all) do |posts, literal|
-        ids = tagged_post_ids_for(locale, literal[:name], reference: literal[:reference])
+        ids = tagged_post_ids_for(literal[:name])
         if literal[:negative]
           posts.where.not(id: ids)
         else
@@ -488,8 +341,8 @@ class PostsController < ApplicationController
     end
   end
 
-  def tag_literal_relation locale, name, negative:, reference: nil
+  def tag_literal_relation name, negative:
-    ids = tagged_post_ids_for(locale, name, reference:)
+    ids = tagged_post_ids_for(name)
     if negative
       Post.where.not(id: ids)
     else
@@ -497,14 +350,10 @@ class PostsController < ApplicationController
     end
   end
 
-  def tagged_post_ids_for locale, name, reference: nil
+  def tagged_post_ids_for(name) =
-    internal, external = PostTagSearch.references(locale, { name:, reference: })
+      Post.joins(tags: :tag_name).where(tag_names: { name: }).select(:id)
-    Post.where(id: PostTag.where(tag_id: Array(internal)).select(:post_id))
-        .or(Post.where(id: PostExternalTag.where(external_tag_id: Array(external))
-            .select(:post_id))).select(:id)
-  end
 
-  def sync_post_tags! post, desired_tags, sections
+  def sync_post_tags! post, desired_tags
     desired_tags.each do |t|
       t.save! if t.new_record?
     end
@@ -523,21 +372,13 @@ class PostsController < ApplicationController
       end
     end
 
-    PostTagSection.where(post_id: post.id).destroy_all
+    PostTag.where(post_id: post.id, tag_id: to_remove.to_a).kept.find_each do |pt|
-    sections.each do |tag_id, ranges|
+      pt.discard_by!(current_user)
-      ranges.each do |begin_ms, end_ms|
-        PostTagSection.create!(post_id: post.id, tag_id:, begin_ms:, end_ms:)
     end
   end
 
-    PostTag.where(post_id: post.id, tag_id: to_remove.to_a).find_each do |pt|
+  def build_tag_tree_for tags
-      pt.destroy!
+    tags = tags.to_a
-    end
-  end
-
-  def build_tag_tree_for post, language_code = 'ja'
-    post_tags = post.post_tags.reject { |post_tag| post_tag.tag.deprecated? }
-    tags = post_tags.map(&:tag)
     tag_ids = tags.map(&:id)
 
     implications = TagImplication.where(parent_tag_id: tag_ids, tag_id: tag_ids)
@@ -552,9 +393,6 @@ class PostsController < ApplicationController
     root_ids = tag_ids - child_ids
 
     tags_by_id = tags.index_by(&:id)
-    sections_by_tag_id = post_tags.to_h { |post_tag|
-      [post_tag.tag_id, post_tag.sections.as_json(only: [:begin_ms, :end_ms])]
-    }
 
     memo = { }
 
@@ -562,10 +400,8 @@ class PostsController < ApplicationController
       tag = tags_by_id[tag_id]
       return nil unless tag
 
-      sections = sections_by_tag_id.fetch(tag_id, [])
-
       if path.include?(tag_id)
-        return TagRepr.inline(tag, language_code:).merge(children: [], sections:)
+        return TagRepr.inline(tag).merge(children: [])
       end
 
       if memo.key?(tag_id)
@@ -577,13 +413,10 @@ class PostsController < ApplicationController
 
       children = child_ids.filter_map { |cid| build_node.(cid, new_path) }
 
-      memo[tag_id] = TagRepr.inline(tag, language_code:).merge(children:, sections:)
+      memo[tag_id] = TagRepr.inline(tag).merge(children:)
     end
 
-    internal_tags = root_ids.filter_map { |id| build_node.call(id, []) }
+    root_ids.filter_map { |id| build_node.call(id, []) }
-    external_tags =
-        post.external_tags.map { ExternalTagRepr.inline(_1).merge(children: [], sections: []) }
-    internal_tags + external_tags
   end
 
   def sibling_posts_by_parent parent_post_ids
@@ -611,79 +444,6 @@ class PostsController < ApplicationController
     }.uniq
   end
 
-  def post_create_attributes
-    { title: params[:title],
-      url: params[:url],
-      thumbnail_base: params[:thumbnail_base],
-      tags: params[:tags],
-      original_created_from: params[:original_created_from],
-      original_created_before: params[:original_created_before],
-      parent_post_ids: parse_parent_post_ids,
-      video_ms: params[:video_ms],
-      duration: params[:duration] }
-  end
-
-  def parse_bulk_posts_manifest
-    manifest = params[:posts]
-    raise ArgumentError, 'posts は必須です．' if manifest.blank?
-    raise ArgumentError, 'posts は JSON 文字列で指定してください．' unless manifest.is_a?(String)
-
-    posts = JSON.parse(manifest)
-    raise ArgumentError, 'posts は配列で指定してください．' unless posts.is_a?(Array)
-    raise ArgumentError, '投稿件数は 1 件以上必要です．' if posts.empty?
-    raise ArgumentError, '投稿件数が多すぎます．' if posts.length > 100
-    raise ArgumentError, 'posts 要素の形式が不正です．' unless posts.all? { _1.is_a?(Hash) }
-
-    posts
-  end
-
-  def parse_bulk_thumbnails post_count
-    thumbnails = { }
-    raw = params[:thumbnails]
-    return thumbnails if raw.blank?
-    raise ArgumentError, 'thumbnail key が不正です．' unless raw.respond_to?(:to_unsafe_h)
-
-    raw.to_unsafe_h.each do |key, value|
-      raise ArgumentError, 'thumbnail key が不正です．' unless key.to_s.match?(/\A\d+\z/)
-
-      index = Integer(key, 10)
-      raise ArgumentError, 'thumbnail index が範囲外です．' if index.negative? || index >= post_count
-      raise ArgumentError, 'thumbnail index が重複しています．' if thumbnails.key?(index)
-      unless value.is_a?(ActionDispatch::Http::UploadedFile)
-        raise ArgumentError, 'thumbnail upload が不正です．'
-      end
-
-      thumbnails[index] = value
-    end
-
-    thumbnails
-  end
-
-  def compact_post post_id
-    return nil if post_id.blank?
-
-    post = Post.with_attached_thumbnail.find_by(id: post_id)
-    PostCompactRepr.base(post, host: request.base_url)
-  end
-
-  def dry_run_json preflight
-    preflight.slice(
-        :url,
-        :title,
-        :thumbnail_base,
-        :tags,
-        :display_tags,
-        :parent_post_ids,
-        :original_created_from,
-        :original_created_before,
-        :duration,
-        :video_ms,
-        :field_warnings,
-        :base_warnings,
-        :existing_post_id,
-        :existing_post)
-  end
-
   def sync_parent_posts! post, parent_post_ids
     if parent_post_ids.include?(post.id)
       post.errors.add :parent_post_ids, '自分自身を親投稿にはできません．'
@@ -722,7 +482,6 @@ class PostsController < ApplicationController
 
   def post_snapshot_from_version version
     { title: version.title,
-      video_ms: version.respond_to?(:video_ms) ? version.video_ms : nil,
       original_created_from: snapshot_time(version.original_created_from),
       original_created_before: snapshot_time(version.original_created_before),
       tag_names: editable_tag_names_from_version(version),
@@ -730,62 +489,27 @@ class PostsController < ApplicationController
   end
 
   def editable_tag_names_from_version version
-    version.tags_json
+    version.tags.to_s.split.reject { |name| name.downcase.start_with?('nico:') }.sort
-        .select { _1.key?('tag_id') }
-        .map { Post.tag_snapshot_literal(_1) }
-        .compact
-        .sort
   end
 
-  def post_snapshot_from_record post, locale
+  def post_snapshot_from_record post
     { title: post.title,
-      video_ms: post.video_ms,
       original_created_from: snapshot_time(post.original_created_from),
       original_created_before: snapshot_time(post.original_created_before),
-      tag_names: editable_tag_names_from_post(post, locale.language_code),
+      tag_names: editable_tag_names_from_post(post),
       parent_post_ids: post.parent_posts.order(:id).pluck(:id) }
   end
 
-  def editable_tag_names_from_post post, language_code
+  def editable_tag_names_from_post post
-    post
+    post.tags.not_nico.joins(:tag_name).order('tag_names.name').pluck('tag_names.name')
-        .post_tags
-        .joins(tag: :tag_names)
-        .merge(Tag.where(deprecated_at: nil))
-        .where(tag_names: { language_code:, primary_flg: true })
-        .includes(:sections, tag: :tag_names)
-        .order('tag_names.name')
-        .map do |post_tag|
-      name = post_tag.tag.name(language_code)
-      sections = post_tag.sections.sort_by(&:begin_ms)
-
-      next name if sections.empty?
-
-      "#{ name }#{ sections.map { Post.section_literal(_1) }.join }"
-    end.compact.sort
   end
 
-  def post_incoming_snapshot locale:, title:, original_created_from:, original_created_before:,
+  def post_incoming_snapshot title:, original_created_from:, original_created_before:,
-                             tag_names:, video_ms_param:, duration_param:, parent_post_ids:
+                             tag_names:, parent_post_ids:
-    validate_original_created_values!(original_created_from, original_created_before)
-
-    Tag.normalise_tags!(locale, tag_names,
-                        with_tagme: false,
-                        with_no_deerjikist: false,
-                        deny_deprecated: true,
-                        with_sections: true,
-                        created_by_user: current_user) => { tags:, sections: }
-
-    tags = Tag.expand_parent_tags(tags).reject(&:deprecated?)
-    video_ms = normalise_video_ms(tags, video_ms_param:, duration_param:)
-    validate_video_sections!(video_ms, sections)
-
     { title:,
-      video_ms:,
       original_created_from: snapshot_time(original_created_from),
       original_created_before: snapshot_time(original_created_before),
-      tag_names: tags.uniq(&:id).map { |tag|
+      tag_names: incoming_tag_names_for_snapshot(tag_names),
-        "#{ tag.name(locale.language_code) }#{ sections[tag.id].to_a.map { section_literal(_1) }.join }"
-      }.sort,
       parent_post_ids: parent_post_ids.sort }
   end
 
@@ -808,25 +532,10 @@ class PostsController < ApplicationController
     value.to_s
   end
 
-  def validate_original_created_values! original_created_from, original_created_before
+  def incoming_tag_names_for_snapshot raw_tag_names
-    candidate = Post.new(
+    tags = Tag.normalise_tags!(raw_tag_names, with_tagme: false)
-        url: 'https://example.invalid/original-created-validation',
-        original_created_from:,
-        original_created_before:)
-    candidate.valid?
-    fields = [:original_created_from, :original_created_before, :original_created_at]
-    relevant_errors = candidate.errors.select { fields.include?(_1.attribute) }
-    return if relevant_errors.empty?
 
-    invalid_post = Post.new
+    Tag.expand_parent_tags(tags).map(&:name).uniq.sort
-    relevant_errors.each { |error|
-      invalid_post.errors.add(error.attribute, error.message)
-    }
-    raise ActiveRecord::RecordInvalid, invalid_post
-  end
-
-  def section_literal section
-    "[#{ Post.ms_to_time(section[0]) }-#{ section[1] ? Post.ms_to_time(section[1]) : '' }]"
   end
 
   def post_conflict_json post:, base_version_no:, base_snapshot:,
@@ -847,8 +556,6 @@ class PostsController < ApplicationController
   def post_snapshot_changes base_snapshot, current_snapshot, incoming_snapshot
     [scalar_snapshot_change(:title, 'タイトル',
                             base_snapshot, current_snapshot, incoming_snapshot),
-     scalar_snapshot_change(:video_ms, '動画時間',
-                            base_snapshot, current_snapshot, incoming_snapshot),
      scalar_snapshot_change(:original_created_from, 'オリジナルの作成日時（以降）',
                             base_snapshot, current_snapshot, incoming_snapshot),
      scalar_snapshot_change(:original_created_before, 'オリジナルの作成日時（より前）',
@@ -908,35 +615,29 @@ class PostsController < ApplicationController
     (added_by_current & removed_by_me).present? || (removed_by_current & added_by_me).present?
   end
 
-  def apply_post_snapshot! locale, post, snapshot
+  def apply_post_snapshot! post, snapshot
     PostVersionRecorder.ensure_snapshot!(post, created_by_user: current_user)
 
     post.update!(title: snapshot[:title],
-                 video_ms: snapshot[:video_ms],
                  original_created_from: snapshot[:original_created_from],
                  original_created_before: snapshot[:original_created_before])
 
-    Tag.normalise_tags!(locale, snapshot[:tag_names],
+    editable_tags = Tag.normalise_tags!(snapshot[:tag_names], with_tagme: false)
-                        with_tagme: false,
+    TagVersioning.record_tag_snapshots!(editable_tags, created_by_user: current_user)
-                        deny_deprecated: true,
-                        with_sections: true,
-                        created_by_user: current_user) => { tags:, sections: }
-    TagVersioning.record_tag_snapshots!(tags, created_by_user: current_user,
-                                        language_code: locale.language_code)
 
-    tags = Tag.expand_parent_tags(tags).reject(&:deprecated?)
+    readonly_tags = post.tags.nico.to_a
 
-    post.video_ms = tags.any? { _1.id == Tag.video.id } ? snapshot[:video_ms] : nil
+    tags = readonly_tags + editable_tags
-    validate_video_sections!(post.video_ms, sections)
+    tags = Tag.expand_parent_tags(tags)
-    post.save!
+
-    sync_post_tags!(post, tags, sections)
+    sync_post_tags!(post, tags)
     sync_parent_posts!(post, snapshot[:parent_post_ids])
 
     PostVersionRecorder.record!(post:, event_type: :update, created_by_user: current_user)
   end
 
   def merge_post_snapshots base_snapshot, current_snapshot, incoming_snapshot
-    [:title, :video_ms, :original_created_from, :original_created_before].map {
+    [:title, :original_created_from, :original_created_before].map {
       [_1, merge_scalar_snapshot_value(base_snapshot[_1],
                                        current_snapshot[_1],
                                        incoming_snapshot[_1])]
@@ -980,44 +681,4 @@ class PostsController < ApplicationController
       render_validation_error record
     end
   end
-
-  def normalise_video_ms tags, video_ms_param: params[:video_ms], duration_param: params[:duration]
-    return nil unless tags.any? { _1.id == Tag.video.id }
-
-    if video_ms_param.present?
-      video_ms = Integer(video_ms_param, exception: false)
-      raise VideoMsParseError unless video_ms&.positive?
-
-      return video_ms
-    end
-
-    return nil if duration_param.blank?
-
-    video_ms = Tag.time_to_ms!(duration_param.to_s, tag_name: '動画時間')
-    raise VideoMsParseError unless video_ms.positive?
-
-    video_ms
-  rescue Tag::SectionLiteralParseError
-    raise VideoMsParseError
-  end
-
-  def validate_video_sections! video_ms, sections
-    return unless video_ms
-
-    sections.each_value do |ranges|
-      ranges.each do |begin_ms, end_ms|
-        if begin_ms >= video_ms
-          post = Post.new
-          post.errors.add :video_ms, 'タグ区間の開始が動画時間以上です．'
-          raise ActiveRecord::RecordInvalid, post
-        end
-
-        if end_ms && end_ms > video_ms
-          post = Post.new
-          post.errors.add :video_ms, 'タグ区間の終端が動画時間を超えてゐます．'
-          raise ActiveRecord::RecordInvalid, post
-        end
-      end
-    end
-  end
 end

backend/app/controllers/preview_controller.rb

@@ -1,55 +1,50 @@
 class PreviewController < ApplicationController
-  before_action :require_member!
-
   def title
-    return render_bad_request('URL は必須です．') if params[:url].blank?
+    # TODO: # 既知サイトなら決まったフォーマットで title 取得するやぅに．
+    return head :unauthorized unless current_user
 
-    render json: { title: Preview::ThumbnailFetcher.title(params[:url]) }
+    url = params[:url]
-  rescue Preview::UrlSafety::UnsafeUrl => e
+    return render_bad_request('URL は必須です．') unless url.present?
+
+    unless url.start_with?(/http(s)?:\/\//)
+      url = 'http://' + url
+    end
+
+    html = URI.open(url, open_timeout: 5, read_timeout: 5).read
+    doc = Nokogiri::HTML.parse(html)
+    title = doc.at('title')&.text&.strip
+
+    render json: { title: title }
+  rescue => e
     render_bad_request(e.message)
-  rescue Preview::HttpFetcher::FetchTimeout => e
-    render_preview_error(e.message, :gateway_timeout)
-  rescue Preview::HttpFetcher::ResponseTooLarge => e
-    render_preview_error(e.message, :payload_too_large)
-  rescue Preview::HttpFetcher::FetchFailed => e
-    render_preview_error(e.message, :bad_gateway)
   end
 
   def thumbnail
-    return render_bad_request('URL は必須です．') if params[:url].blank?
+    # TODO: 既知ドメインであれば指定のアドレスからサムネールを取得するやぅにする．
 
-    attachment =
-        Post.resized_thumbnail_attachment(
-            StringIO.new(Preview::ThumbnailFetcher.fetch(params[:url])))
-    send_data attachment[:io].read,
-              type: attachment[:content_type],
-              disposition: 'inline'
-  rescue Preview::UrlSafety::UnsafeUrl => e
-    render_bad_request(e.message)
-  rescue Preview::HttpFetcher::FetchTimeout => e
-    render_preview_error(e.message, :gateway_timeout)
-  rescue Preview::HttpFetcher::ResponseTooLarge => e
-    render_preview_error(e.message, :payload_too_large)
-  rescue Preview::HttpFetcher::FetchFailed => e
-    render_preview_error(e.message, :bad_gateway)
-  rescue Preview::ThumbnailFetcher::GenerationFailed, MiniMagick::Error => e
-    render_unprocessable_entity(e.message)
-  end
-
-  private
-
-  def require_member!
     return head :unauthorized unless current_user
-    return if current_user.gte_member?
 
-    head :forbidden
+    url = params[:url]
+    return render_bad_request('URL は必須です．') if url.blank?
+
+    unless url.start_with?(/http(s)?:\/\//)
+      url = 'http://' + url
     end
 
-  def render_preview_error(message, status)
+    path = Rails.root.join('tmp', "thumb_#{ SecureRandom.hex }.png")
-    render json: { type: status.to_s,
+    system("node #{ Rails.root }/lib/screenshot.js #{ Shellwords.escape(url) } #{ path }")
-                   message:,
+
+    if File.exist?(path)
+      image = MiniMagick::Image.open(path)
+      image.resize '180x180'
+      File.delete(path) rescue nil
+      send_file image.path, type: 'image/png', disposition: 'inline'
+    else
+      render json: { type: 'internal_server_error',
+                     message: 'サムネールを生成できませんでした．',
                      errors: { },
-                   base_errors: [message] },
+                     base_errors: ['サムネールを生成できませんでした．'] },
-           status:
+             status: :internal_server_error
+    end
   end
 end

backend/app/controllers/sitemap_entries_controller.rb

@@ -1,58 +0,0 @@
-class SitemapEntriesController < ApplicationController
-  PAGE_SIZE = 1_000
-  RESOURCES = [
-      'posts', 'tags', 'deerjikists', 'materials',
-      'descriptions', 'tag_groups', 'theatres'].freeze
-
-  def index
-    locale = resolve_locale!
-    resources = RESOURCES.map { |resource| { resource:, count: entries(resource, locale).count } }
-    render json: { resources:, page_size: PAGE_SIZE }
-  end
-
-  def show
-    locale = resolve_locale!
-    resource = params[:resource]
-    return head :not_found unless RESOURCES.include?(resource)
-
-    page = Integer(params.fetch(:page, '1'), exception: false)
-    return head :bad_request unless page && page.positive?
-
-    scope = entries(resource, locale)
-    return head :not_found if page > [(scope.count.to_f / PAGE_SIZE).ceil, 1].max
-
-    ids = scope.order(:id).offset((page - 1) * PAGE_SIZE).limit(PAGE_SIZE).pluck(:id)
-    render json: { ids: }
-  end
-
-  private
-
-  def entries resource, locale
-    named_tags = TagName.where(language_code: locale.language_code, primary_flg: true)
-        .select(:tag_id)
-    case resource
-    when 'posts'
-      Post.all
-    when 'tags'
-      Tag.where(id: named_tags)
-    when 'deerjikists'
-      Tag.where(id: named_tags).where(id: Deerjikist.select(:tag_id))
-    when 'materials'
-      Material.all
-    when 'descriptions'
-      TagDescription.where(language_code: locale.language_code, tag_id: named_tags)
-          .where.not(body: [nil, ''])
-          .where(HistoryAccess.visible(TagDescriptionVersion.all)
-              .where('tag_description_versions.tag_id = tag_descriptions.tag_id')
-              .where('tag_description_versions.language_code = tag_descriptions.language_code')
-              .where('tag_description_versions.version_no = tag_descriptions.version_no')
-              .arel.exists)
-    when 'tag_groups'
-      TagGroup.joins(:versions)
-          .where('tag_group_versions.version_no = tag_groups.version_no')
-          .where(Arel.sql(HistoryAccess.sql(TagGroupVersion)))
-    when 'theatres'
-      Theatre.kept
-    end
-  end
-end

backend/app/controllers/tag_children_controller.rb

@@ -10,6 +10,7 @@ class TagChildrenController < ApplicationController
 
     parent = Tag.find(parent_id)
     child = Tag.find(child_id)
+    return render_bad_request('ニコニコ・タグの階層は変更できません．') if parent.nico? || child.nico?
 
     ApplicationRecord.transaction do
       TagVersioning.ensure_snapshot!(child, created_by_user: current_user)
@@ -32,6 +33,7 @@ class TagChildrenController < ApplicationController
 
     parent = Tag.find(parent_id)
     child = Tag.find(child_id)
+    return render_bad_request('ニコニコ・タグの階層は変更できません．') if parent.nico? || child.nico?
 
     ApplicationRecord.transaction do
       TagVersioning.ensure_snapshot!(child, created_by_user: current_user)

backend/app/controllers/tag_descriptions_controller.rb

@@ -1,63 +0,0 @@
-class TagDescriptionsController < ApplicationController
-  def show
-    locale = resolve_locale!
-    description = TagDescription.find_by(tag_id: params[:id], language_code: locale.language_code)
-    return head :not_found unless description
-
-    render json: representation(description)
-  end
-
-  def show_by_title
-    locale = resolve_locale!
-    tag_name = TagName.find_by(
-        language_code: locale.language_code, name: params[:name].to_s.strip)
-    description = tag_name&.tag && TagDescription.find_by(
-        tag: tag_name.tag, language_code: locale.language_code)
-    return head :not_found unless description
-
-    render json: representation(description)
-  end
-
-  def update
-    return head :unauthorized unless current_user
-    return head :forbidden unless current_user.gte_member?
-
-    body = params[:body].to_s
-    return render_unprocessable_entity('本文は必須です．', field: :body) if body.blank?
-
-    locale = resolve_locale!
-    tag = Tag.find_by(id: params[:id])
-    return head :not_found unless tag
-
-    description = TagDescription.find_by(tag:, language_code: locale.language_code)
-
-    if description
-      TagDescriptionOperation.update!(locale, tag:, body:, updated_by_user: current_user)
-      description.reload
-      render json: representation(description)
-    else
-      TagDescriptionOperation.create!(locale, tag:, body:, created_by_user: current_user)
-      description = TagDescription.find_by!(tag:, language_code: locale.language_code)
-      render json: representation(description), status: :created
-    end
-  rescue ActiveRecord::RecordInvalid => e
-    render_validation_error e.record
-  rescue ActiveRecord::RecordNotUnique
-    render_record_not_unique
-  end
-
-  private
-
-  def representation description
-    HistoryAccess.require_current!(description, TagDescriptionVersion)
-    description.as_json(only: [
-        :tag_id,
-        :language_code,
-        :body,
-        :version_no,
-        :created_by_user_id,
-        :updated_by_user_id,
-        :created_at,
-        :updated_at])
-  end
-end

backend/app/controllers/tag_groups_controller.rb

@@ -1,43 +0,0 @@
-class TagGroupsController < ApplicationController
-  def index
-    locale = resolve_locale!
-    groups = HistoryAccess.current(TagGroup.all, TagGroupVersion).includes(:created_by_user, :updated_by_user).order(:id)
-    render json: TagGroupRepr.many(locale, groups.to_a)
-  end
-
-  def show
-    locale = resolve_locale!
-    group = TagGroup.includes(:created_by_user, :updated_by_user).find_by(id: params[:id])
-    return head :not_found unless group
-
-    render json: TagGroupRepr.one(locale, group)
-  end
-
-  def create
-    return head :unauthorized unless current_user
-    return head :forbidden unless current_user.gte_member?
-
-    locale = resolve_locale!
-    group = TagGroupOperation.create!(locale, **write_attributes, actor: current_user)
-    render json: TagGroupRepr.one(locale, group), status: :created
-  end
-
-  def update
-    return head :unauthorized unless current_user
-    return head :forbidden unless current_user.gte_member?
-
-    locale = resolve_locale!
-    group = TagGroup.find_by(id: params[:id])
-    return head :not_found unless group
-
-    TagGroupOperation.update!(locale, group:, **write_attributes, actor: current_user)
-    render json: TagGroupRepr.one(locale, group.reload)
-  end
-
-  private
-
-  def write_attributes
-    { name: params[:name], description: params[:description],
-      members: params[:members], placements: params[:placements] }
-  end
-end

backend/app/controllers/tag_versions_controller.rb

@@ -1,6 +1,5 @@
 class TagVersionsController < ApplicationController
   def index
-    locale = resolve_locale!
     tag_id = params[:id].presence
     page = (params[:page].presence || 1).to_i
     limit = (params[:limit].presence || 20).to_i
@@ -10,7 +9,15 @@ class TagVersionsController < ApplicationController
 
     offset = (page - 1) * limit
 
-    q = HistoryAccess.visible(version_scope)
+    q = TagVersion.joins(<<~SQL.squish)
+          LEFT JOIN
+            tag_versions prev
+            ON
+              prev.tag_id = tag_versions.tag_id
+              AND prev.version_no = tag_versions.version_no - 1
+        SQL
+      .select('tag_versions.*', 'prev.name AS prev_name', 'prev.category AS prev_category',
+              'prev.aliases AS prev_aliases', 'prev.parent_tag_ids AS prev_parent_tag_ids')
     q = q.where('tag_versions.tag_id = ?', tag_id) if tag_id
 
     count = q.except(:select, :order, :limit, :offset).count
@@ -19,71 +26,42 @@ class TagVersionsController < ApplicationController
                 .limit(limit)
                 .offset(offset)
 
-    render json: { versions: serialise_versions(versions, locale.language_code), count:, population_count: HistoryAccess.population_count(TagVersion) }
+    render json: { versions: serialise_versions(versions), count: }
   end
 
   private
 
-  def version_scope
+  def serialise_versions rows
-    TagVersion.joins(<<~SQL.squish)
-          LEFT JOIN
-            tag_versions prev
-            ON
-              prev.tag_id = tag_versions.tag_id
-              AND prev.version_no = tag_versions.version_no - 1
-              AND #{ HistoryAccess.sql(TagVersion, 'prev') }
-        SQL
-      .select('tag_versions.*', 'prev.name AS prev_name', 'prev.category AS prev_category',
-              'prev.deprecated_at AS prev_deprecated_at',
-              'prev.aliases AS prev_aliases', 'prev.names AS prev_names',
-              'prev.parent_tag_ids AS prev_parent_tag_ids')
-  end
-
-  def serialise_versions rows, language_code
-    rows = rows.to_a
-    tag_ids = rows.map(&:tag_id).uniq
-    current_names_by_tag_id = TagName.where(tag_id: tag_ids, language_code:, primary_flg: true)
-        .pluck(:tag_id, :name, :auto_generated).to_h do |id, name, automatic|
-      [id, { name:, auto_generated: automatic }]
-    end
     user_ids = rows.map(&:created_by_user_id).compact.uniq
     users_by_id = User.where(id: user_ids).pluck(:id, :name).to_h
-    parent_ids = rows.flat_map do |row|
-      split_parent_tag_ids(row.parent_tag_ids) +
-          split_parent_tag_ids(row.attributes['prev_parent_tag_ids'])
-    end.uniq
-    parents_by_id = Tag.includes(
-        :tag_names, :materials, :deerjikists, :tag_descriptions,
-        parents: [:tag_names, :materials, :deerjikists, :tag_descriptions])
-        .where(id: parent_ids).index_by(&:id)
-    parent_json = parents_by_id.transform_values { TagRepr.base(_1, language_code:) }
 
     rows.map do |row|
-      current_names = names_for_locale(row.names, language_code)
+      cur_aliases = split_values(row.aliases)
-      previous_names = names_for_locale(row.attributes['prev_names'], language_code)
+      prev_aliases = split_values(row.attributes['prev_aliases'])
-      cur_aliases = current_names.fetch('aliases', []).map { _1.fetch('name') }
-      prev_aliases = previous_names.fetch('aliases', []).map { _1.fetch('name') }
 
       cur_parent_tag_ids = split_parent_tag_ids(row.parent_tag_ids)
       prev_parent_tag_ids = split_parent_tag_ids(row.attributes['prev_parent_tag_ids'])
 
+      all_parent_tag_ids = (cur_parent_tag_ids | prev_parent_tag_ids)
+
+      tags_by_id =
+          Tag
+          .includes(:tag_name, :materials, { tag_name: :wiki_page })
+          .where(id: all_parent_tag_ids)
+          .index_by(&:id)
+
       parent_tags =
           build_version_values(cur_parent_tag_ids, prev_parent_tag_ids, key: :tag_id)
           .map do |h|
-            { tag: parent_json.fetch(h[:tag_id]),
+            { tag: TagRepr.base(tags_by_id[h[:tag_id]]),
               type: h[:type] }
           end
 
       { tag_id: row.tag_id,
         version_no: row.version_no,
         event_type: row.event_type,
-        names: row.names,
+        name: { current: row.name, prev: row.attributes['prev_name'] },
-        name_changes: name_changes(row.names, row.attributes['prev_names']),
-        current_name_reference: current_names_by_tag_id[row.tag_id],
-        name: { current: current_names['name'], prev: previous_names['name'] },
         category: { current: row.category, prev: row.attributes['prev_category'] },
-        deprecated_at: { current: row.deprecated_at&.iso8601,
-                         prev: row.attributes['prev_deprecated_at']&.iso8601 },
         aliases: build_version_values(cur_aliases, prev_aliases, key: :name),
         parent_tags:,
         created_at: row.created_at.iso8601,
@@ -110,45 +88,5 @@ class TagVersionsController < ApplicationController
 
   def split_values(values) = values.to_s.split(/\s+/).reject(&:blank?)
 
-  def names_for_locale names, language_code
-    values = snapshot_names(names)[language_code]
-    values.is_a?(Hash) ? values : {}
-  end
-
-  def snapshot_names names
-    values = names.is_a?(String) ? JSON.parse(names) : names
-    values.is_a?(Hash) ? values : {}
-  rescue JSON::ParserError
-    {}
-  end
-
-  def name_changes current_names, previous_names
-    current = snapshot_names(current_names)
-    previous = snapshot_names(previous_names)
-    (current.keys | previous.keys).sort.filter_map do |language|
-      current_name = language_snapshot(current[language])
-      previous_name = language_snapshot(previous[language])
-      next if current_name == previous_name
-
-      { language_code: language, current: current_name, prev: previous_name }
-    end
-  end
-
-  def language_snapshot entry
-    return nil unless entry.is_a?(Hash)
-
-    aliases = Array(entry['aliases']).filter_map do |value|
-      value = { 'name' => value } if value.is_a?(String)
-      next unless value.is_a?(Hash) && value['name'].is_a?(String)
-
-      { name: value['name'], script_code: value['script_code'] }
-    end
-    aliases = aliases.uniq.sort_by { [_1[:name], _1[:script_code].to_s] }
-    { name: entry['name'],
-      script_code: entry['script_code'],
-      auto_generated: entry['auto_generated'],
-      aliases: }
-  end
-
   def split_parent_tag_ids(values) = split_values(values).map(&:to_i)
 end

backend/app/controllers/theatre_skip_events_controller.rb

@@ -6,7 +6,7 @@ class TheatreSkipEventsController < ApplicationController
     events =
         TheatreSkipEvent
         .where(theatre_id: params[:theatre_id])
-        .includes(:post, tags: :tag_names)
+        .includes(:post, tags: :tag_name)
         .order(created_at: :desc)
         .limit(limit)
 
@@ -14,7 +14,7 @@ class TheatreSkipEventsController < ApplicationController
       { id: event.id,
         theatre_id: event.theatre_id,
         post: { id: event.post.id, title: event.post.title, url: event.post.url },
-        tags: event.tags.map { |tag| { id: tag.id, name: tag.name('ja') } },
+        tags: event.tags.map { |tag| { id: tag.id, name: tag.name } },
         programme_position: event.programme_position,
         created_at: event.created_at }
     }

backend/app/controllers/user_settings_controller.rb

@@ -1,61 +0,0 @@
-class UserSettingsController < ApplicationController
-  wrap_parameters false
-
-  def show
-    return head :unauthorized unless current_user
-
-    render json: current_setting.serializable_hash
-  end
-
-  def update
-    return head :unauthorized unless current_user
-
-    raw_attributes = editable_raw_attributes
-    field_errors = validate_raw_attributes(raw_attributes)
-    return render_validation_error fields: field_errors if field_errors.present?
-
-    setting = current_setting
-    setting.assign_attributes(raw_attributes)
-
-    if setting.save
-      render json: setting.serializable_hash, status: :ok
-    else
-      render_validation_error setting
-    end
-  end
-
-  private
-
-  def current_setting
-    Setting.find_or_create_by!(user: current_user) do |setting|
-      setting.assign_attributes(Setting.defaults)
-    end
-  rescue ActiveRecord::RecordNotUnique
-    Setting.find_by!(user: current_user)
-  end
-
-  def editable_raw_attributes
-    request.request_parameters.slice(*Setting::EDITABLE_ATTRIBUTES)
-  end
-
-  def validate_raw_attributes raw_attributes
-    raw_attributes.each_with_object({ }) do |(key, value), errors|
-      next if value_matches_type?(key, value)
-
-      errors[key.to_sym] = ['値の型が不正です．']
-    end
-  end
-
-  def value_matches_type? key, value
-    case Setting::TYPE_BY_ATTRIBUTE.fetch(key)
-    when :string
-      value.is_a?(String)
-    when :integer
-      value.is_a?(Integer)
-    when :boolean
-      value == true || value == false
-    else
-      false
-    end
-  end
-end

backend/app/controllers/user_theme_slots_controller.rb

@@ -1,38 +0,0 @@
-class UserThemeSlotsController < ApplicationController
-  wrap_parameters false
-
-  def index
-    return head :unauthorized unless current_user
-
-    render json: current_user
-                 .theme_slots
-                 .order(:base_theme, :slot_no)
-                 .map { |slot| slot.serializable_hash }
-  end
-
-  def update
-    return head :unauthorized unless current_user
-
-    base_theme = params[:base_theme].to_s
-    slot_no = params[:slot_no].to_i
-    tokens = params[:tokens]
-
-    unless UserThemeSlot::BASE_THEMES.include?(base_theme)
-      return render_validation_error fields: { base_theme: ['値が不正です．'] }
-    end
-    unless UserThemeSlot::SLOT_NOS.include?(slot_no)
-      return render_validation_error fields: { slot_no: ['値が不正です．'] }
-    end
-    unless tokens.is_a?(ActionController::Parameters) || tokens.is_a?(Hash)
-      return render_validation_error fields: { tokens: ['JSON object で指定してください．'] }
-    end
-
-    slot = UserThemeSlot.find_or_initialize_by(user: current_user,
-                                               base_theme:,
-                                               slot_no:)
-    slot.tokens = tokens.is_a?(ActionController::Parameters) ? tokens.to_unsafe_h : tokens
-    slot.save!
-
-    render json: slot.serializable_hash, status: :ok
-  end
-end

backend/app/controllers/users_controller.rb

@@ -1,69 +1,39 @@
 class UsersController < ApplicationController
   def create
-    if current_user
-      return render json: { code: current_user.inheritance_code,
-                            user: current_user.slice(:id, :name, :inheritance_code, :role) }
-    end
-
     user = nil
     User.transaction do
       user = User.create!(inheritance_code: SecureRandom.uuid, role: :guest)
       attach_ip_address!(user)
     end
 
-    establish_user_session(user)
     render json: { code: user.inheritance_code,
                    user: user.slice(:id, :name, :inheritance_code, :role) },
            status: :created
   end
 
   def verify
-    user = if bool?(:restore) && current_user
+    user = User.find_by(inheritance_code: params[:code])
-             current_user
-           else
-             User.find_by(inheritance_code: params[:code])
-           end
     return render json: { valid: false } unless user
     return head :forbidden if user.banned?
 
     attach_ip_address!(user)
-    establish_user_session(user)
 
     render json: { valid: true, user: user.slice(:id, :name, :inheritance_code, :role) }
   end
 
-  def show_session
-    return head :unauthorized unless current_user
-
-    render json: current_user.slice(:id, :name, :role)
-  end
-
-  def csrf
-    response.headers['Cache-Control'] = 'private, no-store'
-    render json: { token: form_authenticity_token }
-  end
-
-  def destroy_session
-    reset_session
-    response.headers['Cache-Control'] = 'private, no-store'
-    head :no_content
-  end
-
   def renew
     return head :unauthorized unless current_user
 
     user = current_user
     user.inheritance_code = SecureRandom.uuid
     user.save!
-    establish_user_session(user)
     render json: { code: user.inheritance_code }
   end
 
   def me
-    user = current_user
+    user = User.find_by(inheritance_code: params[:code])
-    return head :unauthorized unless user
+    return head :not_found unless user
 
-    attach_ip_address!(user)
     render json: user.slice(:id, :name, :inheritance_code, :role)
   end
 

backend/app/controllers/wiki_pages_controller.rb

@@ -1,168 +1,84 @@
 class WikiPagesController < ApplicationController
-  # 互換性のための Facade API
+  rescue_from Wiki::Commit::Conflict, with: :render_wiki_conflict
-
-  rescue_from TagDescriptionOperation::Conflict, with: :render_wiki_conflict
 
   def index
-    locale = resolve_locale!
-
     title = params[:title].to_s.strip
-    descriptions =
+    if title.blank?
-        HistoryAccess.current(TagDescription.all, TagDescriptionVersion)
+      return render json: WikiPageRepr.base(WikiPage.joins(:tag_name).includes(:tag_name))
-        .where(language_code: locale.language_code)
-        .includes(tag: :tag_names)
-
-    descriptions = descriptions.order(updated_at: :desc, id: :asc)
-    tag_groups = HistoryAccess.current(TagGroup.all, TagGroupVersion).order(updated_at: :desc, id: :asc)
-    if title.present?
-      pattern = "%#{ ApplicationRecord.sanitize_sql_like(title) }%"
-      descriptions =
-          descriptions.joins(tag: :tag_names)
-          .where(tag_names: { language_code: locale.language_code, primary_flg: true })
-          .where('tag_names.name LIKE ?', pattern).limit(20)
-      tag_groups = tag_groups.where(
-          'CONCAT(?, name) LIKE ? OR CONCAT(?, name) LIKE ?',
-          TagGroupRepr::PREFIX, pattern, 'tag_group:', pattern).limit(20)
     end
 
-    json = descriptions.map { |description| WikiPageRepr.tag_description(locale, description) }
+    q = WikiPage.joins(:tag_name).includes(:tag_name)
-    json.concat(tag_groups.map { WikiPageRepr.tag_group(_1) })
+        .where('tag_names.name LIKE ?', "%#{ WikiPage.sanitize_sql_like(title) }%")
-    json.sort_by! do |row|
+    render json: WikiPageRepr.base(q.limit(20))
-      [-row['updated_at'].to_f, row.fetch('resource_type'), row.fetch('id')]
-    end
-    json = json.first(20) if title.present?
-    render json:
-  end
-
-  def autolink_candidates
-    locale = resolve_locale!
-    language = params.fetch(:language, locale.language_code)
-    language = 'ja' if language == 'und'
-    return head :not_found unless language.is_a?(String)
-    return head :not_found unless Language.exists?(code: language)
-
-    names = HistoryAccess.current(TagDescription.all, TagDescriptionVersion).joins(tag: :tag_names)
-        .where(language_code: locale.language_code, tag_names: { language_code: language })
-        .pluck('tag_descriptions.tag_id', 'tag_descriptions.id', 'tag_names.name')
-    candidates = names.map do |tag_id, description_id, name|
-      WikiPageRepr.autolink_tag(locale, tag_id:, description_id:, name:, language:)
-    end
-    candidates.concat(HistoryAccess.current(TagGroup.all, TagGroupVersion).order(:id).pluck(:id, :name).map do |id, name|
-      WikiPageRepr.autolink_group(locale, id:, name:, language:)
-    end)
-    render json: candidates
   end
 
   def show
-    locale = resolve_locale!
+    page = WikiPage.joins(:tag_name)
-    reference = TagDescription.find_by(id: params[:id])
+                   .includes(:tag_name)
-    tag_description = reference && TagDescription.find_by(
+                   .find_by(id: params[:id])
-        tag_id: reference.tag_id, language_code: locale.language_code)
+    render_wiki_page_or_404 page
-
-    return render_wiki_page_by_tag_description locale, tag_description if tag_description
-    return head :not_found if reference
-
-    tag_group = TagGroup.find_by(id: params[:id])
-    return render_tag_group locale, tag_group if tag_group
-
-    head :not_found
   end
 
   def show_by_title
-    locale = resolve_locale!
     title = params[:title].to_s.strip
-
+    page = WikiPage.joins(:tag_name)
-    if (name = TagGroupRepr.name_from_title(title))
+                   .includes(:tag_name)
-      tag_group = TagGroup.find_by(name:)
+                   .find_by(tag_name: { name: title })
-      return render_tag_group locale, tag_group if tag_group
+    render_wiki_page_or_404 page
-      return head :not_found
-    end
-
-    tag_description = tag_description_by_title(locale, title)
-
-    return render_wiki_page_by_tag_description locale, tag_description if tag_description
-
-    head :not_found
-  end
-
-  def tag_reference
-    locale = resolve_locale!
-    description = TagDescription.find_by(id: params[:id])
-    return head :not_found unless description
-
-    tag = Tag.preload(:tag_names, :materials, :deerjikists, :tag_descriptions,
-                      parents: [:tag_names, :materials, :deerjikists, :tag_descriptions])
-        .find(description.tag_id)
-    render json: TagRepr.base(tag, language_code: locale.language_code)
   end
 
   def exists
-    locale = resolve_locale!
+    if WikiPage.exists?(id: params[:id])
-    if TagDescription.exists?(id: params[:id], language_code: locale.language_code)
+      head :no_content
-      return head :no_content
+    else
-    end
-    return head :no_content if TagGroup.exists?(id: params[:id])
-
       head :not_found
     end
+  end
 
   def exists_by_title
-    locale = resolve_locale!
     title = params[:title].to_s.strip
-    if (name = TagGroupRepr.name_from_title(title))
+    if WikiPage.joins(:tag_name).exists?(tag_names: { name: title })
-      exists = TagGroup.exists?(name:)
+      head :no_content
-      return head exists ? :no_content : :not_found
+    else
-    end
-
-    return head :no_content if tag_description_by_title(locale, title)
-
       head :not_found
     end
+  end
 
   def diff
     id = params[:id]
     return render_bad_request('id は必須です．') if id.blank?
 
-    locale = resolve_locale!
-
     from = params[:from].presence
     to = params[:to].presence
 
-    description = TagDescription.find_by(id:, language_code: locale.language_code)
+    page = WikiPage.joins(:tag_name).includes(:tag_name).find(id)
-    if description
-      versions = description_versions(description)
-      from_rev = from && versions.find(from)
-      to_rev = to ? versions.find(to) : versions.find_by!(version_no: description.version_no)
-      resource = WikiPageRepr.tag_description(locale, description)
-    else
-      tag_group = TagGroup.find_by(id:)
-      if tag_group
-        from_rev = from && group_version(tag_group, from)
-        to_rev = to ? group_version(tag_group, to) : current_group_version(tag_group)
-        return head :not_found if (from && !from_rev) || !to_rev
 
-        from_body = from_rev && render_group_version(locale, from_rev)
+    from_rev = from && page.wiki_revisions.find(from)
-        to_body = render_group_version(locale, to_rev)
+    to_rev = to ? page.wiki_revisions.find(to) : page.current_revision
-        resource = { 'id' => tag_group.id,
+    if ((from_rev && !(from_rev.content?)) || !(to_rev&.content?))
-                     'title' => "#{ TagGroupRepr::PREFIX }#{ tag_group.name }",
+      return render_unprocessable_entity('差分を表示できない版です．')
-                     'deprecated_at' => nil }
-      else
-        return head :not_found
-      end
     end
 
-    from_body ||= from_rev&.body
+    diffs = Diff::LCS.sdiff(from_rev&.body&.lines || [], to_rev.body.lines)
-    to_body ||= to_rev.body
+    diff_json = diffs.map { |change|
-    diff_json = wiki_diff(from_body, to_body)
+      case change.action
+      when ?=
+        { type: 'context', content: change.old_element }
+      when ?!
+        [{ type: 'removed', content: change.old_element },
+         { type: 'added', content: change.new_element }]
+      when ?+
+        { type: 'added', content: change.new_element }
+      when ?-
+        { type: 'removed', content: change.old_element }
+      end
+    }.flatten.compact
 
-    render json: { wiki_page_id:      resource.fetch('id'),
+    render json: { wiki_page_id:      page.id,
-                   title:             resource.fetch('title'),
+                   title:             page.title,
-                   deprecated_at:     resource.fetch('deprecated_at'),
                    older_revision_id: from_rev&.id,
                    newer_revision_id: to_rev.id,
                    diff:              diff_json }
-  rescue TagGroupMarkdownRenderer::UnsupportedStructure => error
-    render_unprocessable_entity(error.message)
   end
 
   def create
@@ -176,37 +92,20 @@ class WikiPagesController < ApplicationController
     return render_unprocessable_entity('タイトルは必須です．', field: :title) if title.blank?
     return render_unprocessable_entity('本文は必須です．', field: :body) if body.blank?
 
-    locale = resolve_locale!
+    tag_name = TagName.find_undiscard_or_create_by!(name: title)
 
-    if (name = TagGroupRepr.name_from_title(title))
+    page =
-      parsed = TagGroupMarkdownParser.parse(locale, body:, group: nil)
+        Wiki::Commit.create_content!(
-      group = TagGroupOperation.create!(
+            tag_name:,
-          locale, name:, **parsed.except(:compatibility_body), actor: current_user,
+            body:,
-          require_compatibility: true)
+            created_by_user: current_user,
-      version = current_group_version(group)
+            message:)
-      return render_group_representation(TagGroupRepr.one(locale, group), version,
-                                         status: :created)
-    end
 
-    description =
+    render json: WikiPageRepr.base(page), status: :created
-        if title.include?(':')
-          return render_unprocessable_entity field: :title
-        else
-          tag_name = TagName.find_by(language_code: locale.language_code, name: title)
-          return render_unprocessable_entity field: :title unless tag_name&.tag
-
-          TagDescriptionOperation.create!(
-              locale,
-              tag: tag_name.tag, body:, created_by_user: current_user, message:)
-        end
-
-    render json: WikiPageRepr.tag_description(locale, description), status: :created
   rescue ActiveRecord::RecordInvalid => e
     render_validation_error e.record
   rescue ActiveRecord::RecordNotUnique
     render_record_not_unique
-  rescue TagGroupMarkdownParser::InvalidDocument => error
-    render_unprocessable_entity(error.message)
   end
 
   def update
@@ -219,229 +118,89 @@ class WikiPagesController < ApplicationController
     return render_unprocessable_entity('タイトルは必須です．', field: :title) if title.blank?
     return render_unprocessable_entity('本文は必須です．', field: :body) if body.blank?
 
-    locale = resolve_locale!
+    page = WikiPage.find(params[:id])
-
-    group = TagGroup.find_by(id: params[:id])
-    if (name = TagGroupRepr.name_from_title(title))
-      return head :not_found unless group
-
-      parsed = TagGroupMarkdownParser.parse(locale, body:, group:)
-      TagGroupOperation.update!(
-          locale, group:, name:, **parsed.except(:compatibility_body), actor: current_user,
-          require_compatibility: true)
-      return head :ok
-    end
-
-    return render_unprocessable_entity field: :title if title.include?(':')
-    description = TagDescription.find_by(id: params[:id], language_code: locale.language_code)
-    unless description
-      return render_unprocessable_entity field: :title if group
-      return head :not_found
-    end
-
-    tag = description.tag
-    tag_name = tag.tag_name(locale.language_code)
-    return render_unprocessable_entity field: :title unless tag_name
-    return render_unprocessable_entity field: :title if tag_name.name.include?(':')
-
     base_revision_id = params[:base_revision_id].presence
 
     ApplicationRecord.transaction do
-      SharedResourceIdAllocator.lock_space!(Tag, 'tags')
+      page.lock!
-      tag.lock!
+
-      old_title = tag_name.name
+      old_title = page.title
-      if title != old_title
+
+      tag = Tag.find_by(tag_name_id: page.tag_name_id)
+
+      if tag && title != old_title
         TagVersioning.ensure_snapshot!(tag, created_by_user: current_user)
       end
 
-      tag_name.update!(name: title) if title != old_title
+      page.tag_name.update!(name: title) if title != old_title
 
       message = params[:message].presence
-      TagDescriptionOperation.update!(locale, tag:, body:, updated_by_user: current_user,
+      Wiki::Commit.content!(page:,
-                                      message:, base_revision_id:)
+                            body:,
+                            created_user: current_user,
+                            message:,
+                            base_revision_id:)
 
-      if title != old_title
+      if tag && title != old_title
         tag.reload
         TagVersionRecorder.record!(tag:, event_type: :update, created_by_user: current_user)
       end
     end
 
     head :ok
-  rescue ActiveRecord::RecordInvalid => e
+  end
-    render_validation_error e.record
+
-  rescue ActiveRecord::RecordNotUnique
+  def search
-    render_record_not_unique
+    index
-  rescue TagGroupMarkdownParser::InvalidDocument => error
-    render_unprocessable_entity(error.message)
   end
 
   def changes
-    locale = resolve_locale!
-
     id = params[:id].presence
-    description = id && TagDescription.find_by(id:, language_code: locale.language_code)
+    q = WikiRevision.joins(wiki_page: :tag_name)
-    if description
+                    .includes(:created_user, wiki_page: :tag_name)
-      return render json: description_changes(locale, TagDescription.where(id: description.id))
+                    .order(id: :desc)
-    end
+    q = q.where(wiki_page_id: id) if id
 
-    tag_group = id && TagGroup.find_by(id:)
+    render json: q.limit(200).map { |rev|
-    if tag_group
+      { revision_id:  rev.id,
-      return render json: tag_group_changes(HistoryAccess.visible(TagGroupVersion.all).where(tag_group_id: tag_group.id))
+        pred:         rev.base_revision_id,
-    end
+        succ:         nil,
-
+        wiki_page:    { id: rev.wiki_page_id, title: rev.wiki_page.title },
-    json = []
+        user:         rev.created_user && { id: rev.created_user.id, name: rev.created_user.name },
-    unless id
+        kind:         rev.kind,
-      json.concat(tag_group_changes(HistoryAccess.visible(TagGroupVersion.all)))
+        message:      rev.message,
-      json.concat(description_changes(locale,
+        timestamp:    rev.created_at }
-                                      TagDescription.where(language_code: locale.language_code)))
+    }.compact
-      json = json.each_with_index.sort_by do |row, index|
-        [-row.fetch(:timestamp).to_f, index]
-      end.map(&:first)
-    end
-    render json: json.first(200)
   end
 
   private
 
-  def render_tag_group locale, group
+  def render_wiki_page_or_404 page
-    version = if params[:version].present?
+    return head :not_found unless page
-      group_version(group, params[:version])
-    else
-      current_group_version(group)
-    end
-    return head :not_found unless version
 
-    representation =
+    rev = find_revision(page)
-        if params[:version].present?
-          TagGroupRepr.historical(locale, version)
-        else
-          TagGroupRepr.one(locale, group)
-        end
-    render_group_representation(representation, version)
-  end
-
-  def render_group_version locale, version
-    representation = TagGroupRepr.historical(locale, version)
-    TagGroupMarkdownRenderer.render(representation)
-  end
-
-  def render_group_representation representation, version, body: nil, status: :ok
-    body ||= TagGroupMarkdownRenderer.render(representation)
-    versions = HistoryAccess.visible(TagGroupVersion.all).where(tag_group_id: version.tag_group_id)
-    pred = versions.find_by(version_no: version.version_no - 1)&.id
-    succ = versions.find_by(version_no: version.version_no + 1)&.id
-    updated_at = version.created_at
-    render json: {
-        'id' => version.tag_group_id,
-        'resource_type' => 'tag_group',
-        'title' => "#{ TagGroupRepr::PREFIX }#{ representation.fetch('name') }",
-        'deprecated_at' => nil,
-        'body' => body,
-        'revision_id' => version.id,
-        'pred' => pred,
-        'succ' => succ,
-        'updated_at' => updated_at }, status:
-  rescue TagGroupMarkdownRenderer::UnsupportedStructure => error
-    render_unprocessable_entity(error.message)
-  end
-
-  def current_group_version group
-    HistoryAccess.visible(TagGroupVersion.all).find_by(tag_group_id: group.id, version_no: group.version_no)
-  end
-
-  def group_version group, version_id
-    HistoryAccess.visible(TagGroupVersion.all).find_by(tag_group_id: group.id, id: version_id)
-  end
-
-  def wiki_diff from_body, to_body
-    diffs = Diff::LCS.sdiff(from_body.to_s.lines, to_body.to_s.lines)
-    diffs.map { |change|
-      case change.action
-      when ?=
-        { type: 'context', content: change.old_element }
-      when ?!
-        [{ type: 'removed', content: change.old_element },
-         { type: 'added', content: change.new_element }]
-      when ?+
-        { type: 'added', content: change.new_element }
-      when ?-
-        { type: 'removed', content: change.old_element }
-      end
-    }.flatten.compact
-  end
-
-  def tag_group_changes versions
-    rows = versions.includes(:created_by_user).order(id: :desc).limit(200).to_a
-    groups = TagGroup.where(id: rows.map(&:tag_group_id).uniq).index_by(&:id)
-    predecessors = HistoryAccess.visible(TagGroupVersion.all).where(tag_group_id: groups.keys,
-                                         version_no: rows.map { _1.version_no - 1 })
-        .pluck(:tag_group_id, :version_no, :id)
-        .to_h { |group_id, number, version_id| [[group_id, number], version_id] }
-    rows.map do |version|
-      group = groups.fetch(version.tag_group_id)
-      user = version.created_by_user
-      { revision_id: version.id,
-        pred: predecessors[[group.id, version.version_no - 1]],
-        succ: nil,
-        wiki_page: { id: group.id, title: "#{ TagGroupRepr::PREFIX }#{ version.name }",
-                     deprecated_at: nil },
-        user: user && { id: user.id, name: user.name },
-        kind: 'content', message: nil, timestamp: version.created_at }
-    end
-  end
-
-  def render_wiki_page_by_tag_description locale, description
-    versions = description_versions(description)
-    historical = params[:version].present?
-    rev = historical ? versions.find_by(id: params[:version]) :
-        versions.find_by(version_no: description.version_no)
     return head :not_found unless rev
 
-    pred = versions.find_by(version_no: rev.version_no - 1)&.id
+    if rev.redirect?
-    succ = versions.find_by(version_no: rev.version_no + 1)&.id
+      return (
-    body = historical ? rev.body : description.body
+          redirect_to wiki_page_by_title_path(title: rev.redirect_page.title),
-    updated_at = historical ? rev.created_at : description.updated_at
+                      status: :moved_permanently)
-    render json: WikiPageRepr.tag_description(locale, description).merge(
-        body:, revision_id: rev.id, pred:, succ:, updated_at:)
     end
 
-  def description_versions description
+    body = rev.body
-    HistoryAccess.visible(TagDescriptionVersion.all).where(tag_id: description.tag_id,
+    revision_id = rev.id
-                                language_code: description.language_code)
+    pred = page.pred_revision_id(revision_id)
+    succ = page.succ_revision_id(revision_id)
+    updated_at = rev.created_at
+
+    render json: WikiPageRepr.base(page).merge(body:, revision_id:, pred:, succ:, updated_at:)
   end
 
-  def tag_description_by_title locale, title
+  def find_revision page
-    HistoryAccess.current(TagDescription.all, TagDescriptionVersion).joins(tag: :tag_names)
+    if params[:version].present?
-        .find_by(language_code: locale.language_code,
+      page.wiki_revisions.find_by(id: params[:version])
-                 tag_names: { language_code: locale.language_code, name: title })
+    else
-  end
+      page.current_revision
-
-  def description_changes locale, descriptions
-    versions = HistoryAccess.visible(TagDescriptionVersion.all).where(language_code: locale.language_code,
-                                           tag_id: descriptions.select(:tag_id))
-        .order(id: :desc).limit(200).to_a
-    return [] if versions.empty?
-
-    by_tag = descriptions.where(tag_id: versions.map(&:tag_id))
-        .includes(tag: :tag_names).index_by(&:tag_id)
-    users = User.where(id: versions.map(&:created_by_user_id)).index_by(&:id)
-    predecessors = HistoryAccess.visible(TagDescriptionVersion.all).where(
-        language_code: locale.language_code, tag_id: by_tag.keys,
-        version_no: versions.map { |version| version.version_no - 1 })
-        .pluck(:tag_id, :version_no, :id)
-        .to_h { |tag_id, version_no, id| [[tag_id, version_no], id] }
-    versions.map do |version|
-      description = by_tag.fetch(version.tag_id)
-      user = users[version.created_by_user_id]
-      { revision_id: version.id,
-        pred: predecessors[[version.tag_id, version.version_no - 1]],
-        succ: nil,
-        wiki_page: { id: description.id, title: description.tag.name(locale.language_code),
-                     deprecated_at: description.tag.deprecated_at },
-        user: user && { id: user.id, name: user.name },
-        kind: 'content',
-        message: nil,
-        timestamp: version.created_at }
     end
   end
 

backend/app/models/external_tag.rb

@@ -1,26 +0,0 @@
-class ExternalTag < ApplicationRecord
-  before_create { SharedResourceIdAllocator.assign!(self) }
-
-  enum :platform, nico: 'nico'
-  validates :platform, presence: true, inclusion: { in: ExternalTag.platforms.keys }
-
-  has_many :post_external_tags, dependent: :delete_all
-  has_many :posts, through: :post_external_tags
-
-  has_many :nico_tag_versions, foreign_key: :tag_id, inverse_of: :external_tag
-
-  has_many :nico_tag_relations,
-           foreign_key: :nico_tag_id,
-           inverse_of: :nico_tag,
-           dependent: :destroy
-
-  has_many :linked_tags, through: :nico_tag_relations, source: :tag
-
-  def snapshot_linked_tag_names language_code = 'ja'
-    linked_tags
-        .joins(:tag_names)
-        .where(tag_names: { language_code:, primary_flg: true })
-        .order('tag_names.name')
-        .pluck('tag_names.name')
-  end
-end

backend/app/models/gekanator_question_example.rb

@@ -1,7 +1,6 @@
 class GekanatorQuestionExample < ApplicationRecord
   ANSWERS = GekanatorQuestionSuggestion::ANSWERS
-  NON_UNKNOWN_ANSWERS = ANSWERS - ['unknown']
+  SOURCES = ['initial_suggestion', 'post_game_extra'].freeze
-  SOURCES = ['initial_suggestion', 'post_game_answer', 'post_game_extra'].freeze
 
   belongs_to :gekanator_question
   belongs_to :post
@@ -9,89 +8,10 @@ class GekanatorQuestionExample < ApplicationRecord
   belongs_to :gekanator_game, optional: true
 
   validates :answer, presence: true, inclusion: { in: ANSWERS }
-  validates :answer_counts, presence: true
-  validates :sample_count,
-            presence: true,
-            numericality: {
-              only_integer: true,
-              greater_than: 0
-            }
   validates :source, presence: true, inclusion: { in: SOURCES }
   validates :weight,
             presence: true,
             numericality: {
               greater_than: 0
             }
-
-  before_validation :normalize_learning_state
-
-  def record_answer!(answer:, source:, gekanator_game: nil)
-    answer = answer.to_s
-    raise ArgumentError, 'invalid answer' unless ANSWERS.include?(answer)
-
-    counts = normalized_answer_counts
-    counts[answer] += 1
-
-    self.answer_counts = counts
-    self.sample_count = counts.values.sum
-    self.gekanator_game = gekanator_game if gekanator_game.present?
-    self.source = source
-
-    apply_aggregated_answer!(preferred_answer: answer)
-    self
-  end
-
-  private
-
-  def normalize_learning_state
-    counts = normalized_answer_counts
-
-    if counts.values.sum.zero? && answer.present?
-      counts[answer] = 1
-    end
-
-    self.answer_counts = counts
-    self.sample_count = counts.values.sum
-
-    apply_aggregated_answer!
-  end
-
-  def apply_aggregated_answer!(preferred_answer: nil)
-    counts = normalized_answer_counts
-    known_counts = counts.slice(*NON_UNKNOWN_ANSWERS)
-    known_total = known_counts.values.sum
-
-    if known_total.zero?
-      self.answer = 'unknown'
-      self.weight = 0.1
-      return
-    else
-      max_count = known_counts.values.max
-      candidates = known_counts.select { |_answer, count| count == max_count }.keys
-      self.answer =
-          if preferred_answer.present? && candidates.include?(preferred_answer)
-            preferred_answer
-          elsif answer.present? && candidates.include?(answer)
-            answer
-          else
-            candidates.first
-          end
-    end
-
-    consensus = max_count.to_f / known_total
-    self.weight = Math.sqrt(known_total) * consensus
-  end
-
-  def normalized_answer_counts
-    base = ANSWERS.index_with(0)
-
-    answer_counts.to_h.each do |key, value|
-      answer_key = key.to_s
-      next unless ANSWERS.include?(answer_key)
-
-      base[answer_key] = value.to_i
-    end
-
-    base
-  end
 end

backend/app/models/gekanator_question_suggestion.rb

@@ -1,4 +1,5 @@
 class GekanatorQuestionSuggestion < ApplicationRecord
+  MAX_QUESTIONS_PER_GAME = 3
   ANSWERS = ['yes', 'no', 'partial', 'probably_no', 'unknown'].freeze
 
   belongs_to :gekanator_game
@@ -9,4 +10,16 @@ class GekanatorQuestionSuggestion < ApplicationRecord
   validates :question_text, presence: true, length: { maximum: 1000 }
   validates :answer, presence: true, inclusion: { in: ANSWERS }
   validates :processed, inclusion: { in: [true, false] }
+  validate :question_suggestion_limit_per_game, on: :create
+
+  private
+
+  def question_suggestion_limit_per_game
+    return if gekanator_game_id.blank?
+
+    count = GekanatorQuestionSuggestion.where(gekanator_game_id:).count
+    if count >= MAX_QUESTIONS_PER_GAME
+      errors.add(:base, '質問追加数を超えてゐます．')
+    end
+  end
 end

backend/app/models/history_access.rb

@@ -1,160 +0,0 @@
-require 'digest'
-
-# Request/worker authority comes only from a server-resolved user. Default is public.
-class HistoryAccess < ActiveSupport::CurrentAttributes
-  attribute :user
-  class Forbidden < StandardError; end
-
-  KEYS = { 'post_versions' => ['post_id'], 'tag_versions' => ['tag_id'],
-           'nico_tag_versions' => ['tag_id'],
-           'tag_description_versions' => ['tag_id', 'language_code'],
-           'tag_group_versions' => ['tag_group_id'], 'material_versions' => ['material_id'] }.freeze
-
-  def self.revision
-    values = KEYS.keys.map do |table|
-      table.classify.constantize.unscoped.where(event_type: 'discard').maximum(:id)
-    end
-    Digest::SHA256.hexdigest(values.to_json)
-  end
-
-  def self.authority
-    { 'policy_revision' => revision, 'admin' => admin?, 'actor_id' => user&.id }
-  end
-
-  def self.authorised_artifact?(metadata)
-    return false unless metadata['policy_revision'].is_a?(String)
-    if metadata['admin']
-      admin? && user.id == metadata['actor_id']
-    else
-      metadata['policy_revision'] == revision
-    end
-  end
-
-  def self.admin? = user&.admin? == true
-
-  def self.sql(model, row = model.table_name, references: true)
-    return 'TRUE' if admin?
-
-    table = model.table_name
-    keys = KEYS.fetch(table)
-    matches = keys.map { "boundary.#{ _1 } = #{ row }.#{ _1 }" }.join(' AND ')
-    clauses = ["NOT EXISTS (SELECT 1 FROM #{ table } boundary WHERE #{ matches } " +
-        "AND boundary.event_type = 'discard' AND boundary.version_no >= #{ row }.version_no)"]
-    clauses += reference_sql(table, row) if references
-    clauses.map { "(#{ _1 })" }.join(' AND ')
-  end
-
-  def self.discard_reference(table, key, expression)
-    "NOT EXISTS (SELECT 1 FROM #{ table } blocked WHERE blocked.event_type = 'discard' " +
-        "AND #{ expression.call("blocked.#{ key }") })"
-  end
-
-  def self.reference_sql(table, row)
-    case table
-    when 'post_versions'
-      [['tag_versions', 'tag_id', 'tag_id'],
-       ['nico_tag_versions', 'tag_id', 'external_tag_id']].map do |target, key, field|
-        discard_reference(target, key, ->(id) {
-          "JSON_CONTAINS(#{ row }.tags_json, JSON_OBJECT('#{ field }', #{ id }))"
-        })
-      end + [discard_reference('post_versions', 'post_id', ->(id) {
-        "FIND_IN_SET(#{ id }, REPLACE(#{ row }.parent_post_ids, ' ', ',')) > 0"
-      })]
-    when 'tag_versions'
-      [discard_reference('tag_versions', 'tag_id', ->(id) {
-        "FIND_IN_SET(#{ id }, REPLACE(#{ row }.parent_tag_ids, ' ', ',')) > 0"
-      })]
-    when 'tag_description_versions'
-      [discard_reference('tag_versions', 'tag_id', ->(id) { "#{ row }.tag_id = #{ id }" })]
-    when 'material_versions'
-      [discard_reference('tag_versions', 'tag_id', ->(id) { "#{ row }.tag_id = #{ id }" }),
-       discard_reference('material_versions', 'material_id', ->(id) {
-         "#{ row }.parent_id = #{ id }"
-       })]
-    when 'tag_group_versions'
-      [['tag_versions', 'tag_id'], ['tag_group_versions', 'tag_group_id']].map do |target, field|
-        discard_reference(target, field, ->(id) {
-          ['members', 'placements'].map do |column|
-            "JSON_CONTAINS(COALESCE(JSON_EXTRACT(#{ row }.#{ column }, " +
-                "'$**.#{ field }'), JSON_ARRAY()), CAST(#{ id } AS CHAR))"
-          end.join(' OR ')
-        })
-      end
-    else []
-    end
-  end
-
-  def self.visible(scope, row = scope.klass.table_name)
-    scope.where(Arel.sql(sql(scope.klass, row)))
-  end
-
-  def self.current(scope, version_model)
-    return scope if admin?
-
-    table = scope.klass.table_name
-    keys = KEYS.fetch(version_model.table_name)
-    matches = keys.map do |key|
-      column = scope.klass.column_names.include?(key) ? key : 'id'
-      "current_snapshot.#{ key } = #{ table }.#{ column }"
-    end.join(' AND ')
-    boundary = "#{ matches } AND current_snapshot.version_no >= #{ table }.version_no " +
-        "AND current_snapshot.event_type = 'discard'"
-    scope = scope.where(Arel.sql("NOT EXISTS (SELECT 1 FROM #{ version_model.table_name } " +
-        "current_snapshot WHERE #{ boundary })"))
-    condition = "#{ matches } AND current_snapshot.version_no = #{ table }.version_no"
-    scope.where(Arel.sql("NOT EXISTS (SELECT 1 FROM #{ version_model.table_name } " +
-        "current_snapshot WHERE #{ condition } AND NOT (#{ sql(version_model, 'current_snapshot') }))"))
-  end
-
-  def self.require_current!(record, version_model)
-    return if admin?
-    raise Forbidden unless current(record.class.where(id: record.id), version_model).exists?
-  end
-
-  def self.blob_visible?(blob, seen = [])
-    return true if admin?
-    return false if seen.include?(blob.id)
-    seen = seen + [blob.id]
-    private_versions = MaterialVersion.where(file_blob_id: blob.id)
-        .where(Arel.sql("NOT (#{ sql(MaterialVersion) })"))
-    return false if private_versions.exists?
-
-    blob.attachments.each do |attachment|
-      model, key = { 'Post' => [PostVersion, 'post_id'],
-                     'Material' => [MaterialVersion, 'material_id'] }[attachment.record_type]
-      if model && model.where(key => attachment.record_id, event_type: 'discard').exists?
-        return false
-      end
-      if attachment.record_type == 'ActiveStorage::VariantRecord'
-        original = ActiveStorage::VariantRecord.find_by(id: attachment.record_id)&.blob
-        return false unless original && blob_visible?(original, seen)
-      end
-    end
-    true
-  end
-
-  def self.require_many!(records)
-    return if admin? || records.empty?
-
-    ids = records.map(&:id).uniq
-    raise Forbidden unless visible(records.first.class.where(id: ids)).count == ids.length
-  end
-
-  def self.require!(record)
-    return record if admin? || visible(record.class.where(id: record.id)).exists?
-
-    raise Forbidden
-  end
-
-  # This number deliberately ignores time, identity and content filters to prevent count probes.
-  def self.population_count(model) = model.unscoped.count
-
-  # Only the resource-wide denied count is public, never a condition-dependent denied count.
-  def self.population_counts(model)
-    total, excluded = model.unscoped.optimizer_hints('MAX_EXECUTION_TIME(10000)').pick(
-        Arel.sql('COUNT(*)'),
-        Arel.sql("COALESCE(SUM(CASE WHEN #{ sql(model) } THEN 0 ELSE 1 END), 0)"))
-    { population_count: total.to_i, excluded_count: excluded.to_i,
-      excluded_count_scope: 'all_recorded_versions' }
-  end
-end

backend/app/models/language.rb

@@ -1,5 +0,0 @@
-class Language < ApplicationRecord
-  has_many :locales, foreign_key: :language_code, primary_key: :code
-
-  enum :code, ja: 'ja', ain: 'ain'
-end

backend/app/models/locale.rb

@@ -1,38 +0,0 @@
-class Locale < ApplicationRecord
-  attr_accessor :certified_tag_names_path
-
-  around_save :save_with_tag_names!, if: :will_generate_tag_names?
-
-  belongs_to :language, foreign_key: :language_code, primary_key: :code
-  belongs_to :script, foreign_key: :script_code, primary_key: :code
-
-  # Enum deserialisation uses the first key for each stored value.
-  # Keep the legacy key for existing predicates, scopes and query arguments.
-  enum :code, { ja: 'ja', 'ain-Kana' => 'ain-Kana', ain_kana: 'ain-Kana' }
-
-  def code= value
-    super(self.class.codes.fetch(value.to_s, value))
-  end
-
-  def self.nipponese = Locale.find('ja')
-
-  private
-
-  def will_generate_tag_names?
-    deprecated_at.nil? && (will_save_change_to_code? || will_save_change_to_deprecated_at?)
-  end
-
-  def save_with_tag_names!
-    completed = self.class.transaction(requires_new: true) do
-      SharedResourceIdAllocator.lock_space!(Tag, 'tags')
-      saved = yield
-      raise ActiveRecord::RecordNotSaved, 'Locale save was halted' unless saved
-
-      TagNameCompletion.for_locale!(self)
-      true
-    end
-    unless completed
-      raise ActiveRecord::RecordNotSaved, 'Locale tag completion was rolled back'
-    end
-  end
-end

backend/app/models/material.rb

@@ -1,9 +1,6 @@
 class Material < ApplicationRecord
   include MyDiscard
 
-  SOURCE_KINDS = ['uri', 'google_drive_path', 'google_drive_file',
-                  'legacy_drive_path'].freeze
-
   default_scope -> { kept }
 
   belongs_to :parent, class_name: 'Material', optional: true
@@ -13,18 +10,9 @@ class Material < ApplicationRecord
   belongs_to :created_by_user, class_name: 'User', optional: true
   belongs_to :updated_by_user, class_name: 'User', optional: true
 
-  has_many :material_versions, dependent: :destroy
-  has_many :material_export_items, dependent: :destroy
-
   has_one_attached :file, dependent: :purge
-  has_one_attached :thumbnail, dependent: :purge
 
-  before_validation :assign_normalized_source_key
+  validates :tag_id, presence: true, uniqueness: true
-
-  validates :tag_id, uniqueness: true, allow_nil: true
-  validates :source_kind,
-            inclusion: { in: SOURCE_KINDS },
-            allow_blank: true
 
   validate :file_must_be_attached
   validate :tag_must_be_material_category
@@ -35,37 +23,8 @@ class Material < ApplicationRecord
     file.blob.content_type
   end
 
-  def file_byte_size
-    return nil unless file&.attached?
-
-    file.blob.byte_size
-  end
-
-  def file_filename
-    return nil unless file&.attached?
-
-    file.blob.filename.to_s
-  end
-
-  def snapshot_export_paths
-    material_export_items.order(:profile).pluck(:profile, :export_path).to_h
-  end
-
   private
 
-  def assign_normalized_source_key
-    if source_kind.blank?
-      self.normalized_source_key = nil
-      return
-    end
-
-    self.normalized_source_key =
-        MaterialSyncSuppression.normalize_source_key(source_kind:,
-                                                     source_uri:,
-                                                     drive_path: source_path,
-                                                     drive_file_id: source_file_id)
-  end
-
   def file_must_be_attached
     return if url.present? || file.attached?
 

backend/app/models/material_export_item.rb

@@ -1,48 +0,0 @@
-class MaterialExportItem < ApplicationRecord
-  VALID_PROFILES = ['legacy_drive'].freeze
-
-  belongs_to :material
-  belongs_to :created_by_user, class_name: 'User', optional: true
-
-  validates :profile, presence: true, inclusion: { in: VALID_PROFILES }
-  validates :export_path, presence: true, uniqueness: { scope: :profile }
-  validates :material_id, uniqueness: { scope: :profile }
-  validate :export_path_must_be_relative_safe_path
-
-  scope :enabled, -> { where(enabled: true) }
-
-  private
-
-  def export_path_must_be_relative_safe_path
-    return if export_path.blank?
-
-    if export_path.start_with?('/')
-      errors.add(:export_path, '絶対パスは使へません．')
-    end
-
-    if export_path.match?(/\A[A-Za-z]:\//)
-      errors.add(:export_path, '絶対パスは使へません．')
-    end
-
-    if export_path.include?('\\')
-      errors.add(:export_path, '/ 区切りで指定してください．')
-    end
-
-    if export_path.include?("\0")
-      errors.add(:export_path, 'NUL は使へません．')
-    end
-
-    parts = export_path.split('/')
-    if export_path.include?('//')
-      errors.add(:export_path, '空の path segment は使へません．')
-    end
-
-    if parts.any? { |part| part.in?(['.', '..']) }
-      errors.add(:export_path, '.. は使へません．')
-    end
-
-    if export_path.end_with?('/')
-      errors.add(:export_path, 'directory path は使へません．')
-    end
-  end
-end

backend/app/models/material_import_block.rb

@@ -1,29 +0,0 @@
-class MaterialImportBlock < ApplicationRecord
-  MATCH_KINDS = ['sha256', 'exact_path', 'path_prefix', 'manual'].freeze
-  REASONS = [
-    'copyright_high_risk',
-    'copyright_takedown',
-    'adult_or_sensitive',
-    'personal_information',
-    'malware_or_dangerous_file',
-    'duplicate_or_low_quality',
-    'source_owner_request',
-    'other'
-  ].freeze
-
-  belongs_to :created_by_user, class_name: 'User', optional: true
-
-  validates :match_kind, presence: true, inclusion: { in: MATCH_KINDS }
-  validates :reason, presence: true, inclusion: { in: REASONS }
-  validates :sha256, length: { is: 64 }, allow_blank: true
-  validate :match_value_must_be_present
-
-  private
-
-  def match_value_must_be_present
-    return if match_kind == 'manual'
-    return if sha256.present? || external_path_pattern.present?
-
-    errors.add(:base, 'sha256 または external_path_pattern は必須です．')
-  end
-end

backend/app/models/material_sync_source.rb

@@ -1,45 +0,0 @@
-class MaterialSyncSource < ApplicationRecord
-  SOURCE_KINDS = ['uri', 'google_drive_path', 'google_drive_file', 'legacy_drive_path'].freeze
-
-  belongs_to :created_by_user, class_name: 'User', optional: true
-  belongs_to :updated_by_user, class_name: 'User', optional: true
-
-  validates :name, presence: true
-  validates :source_kind,
-            presence: true,
-            inclusion: { in: SOURCE_KINDS }
-  validates :profile, presence: true, inclusion: { in: MaterialExportItem::VALID_PROFILES }
-  validate :source_value_must_be_present
-
-  scope :enabled, -> { where(enabled: true) }
-
-  def normalized_source_key
-    MaterialSyncSuppression.normalize_source_key(source_kind:,
-                                                 source_uri:,
-                                                 source_path:,
-                                                 source_file_id:)
-  end
-
-  private
-
-  def source_value_must_be_present
-    return if source_value_present?
-
-    errors.add(:base, '同期元を指定してください．')
-  end
-
-  def source_value_present?
-    case source_kind
-    when 'uri'
-      source_uri.present?
-    when 'google_drive_file'
-      source_file_id.present? || source_uri.present?
-    when 'google_drive_path'
-      source_file_id.present? || source_uri.present?
-    when 'legacy_drive_path'
-      source_path.present?
-    else
-      normalized_source_key.present?
-    end
-  end
-end

backend/app/models/material_sync_suppression.rb

@@ -1,105 +0,0 @@
-class MaterialSyncSuppression < ApplicationRecord
-  SOURCE_KINDS = [
-    'uri',
-    'google_drive_path',
-    'google_drive_path_prefix',
-    'google_drive_file',
-    'legacy_drive_path',
-    'legacy_drive_path_prefix'
-  ].freeze
-
-  REASONS = [
-    'copyright_high_risk',
-    'copyright_takedown',
-    'adult_or_sensitive',
-    'personal_information',
-    'malware_or_dangerous_file',
-    'duplicate_or_low_quality',
-    'source_owner_request',
-    'other'
-  ].freeze
-
-  belongs_to :created_by_user, class_name: 'User', optional: true
-
-  before_validation :assign_normalized_source_key
-
-  validates :source_kind, presence: true, inclusion: { in: SOURCE_KINDS }
-  validates :reason, presence: true, inclusion: { in: REASONS }
-  validates :normalized_source_key, presence: true, uniqueness: true
-  validate :source_value_must_be_present
-  validate :drive_path_must_be_safe_relative_path
-
-  def self.normalize_source_key source_kind:,
-                                source_uri: nil,
-                                drive_path: nil,
-                                drive_file_id: nil,
-                                source_path: nil,
-                                source_file_id: nil
-    kind = source_kind.to_s
-    value =
-        case kind
-        when 'uri'
-          source_uri.to_s.strip
-        when 'google_drive_path', 'google_drive_path_prefix',
-             'legacy_drive_path', 'legacy_drive_path_prefix'
-          (source_path || drive_path).to_s.strip.gsub(%r{/+}, '/')
-        when 'google_drive_file'
-          (source_file_id || drive_file_id).to_s.strip
-        else
-          ''
-        end
-
-    return nil if kind.blank? || value.blank?
-
-    "#{ kind }:#{ value }"
-  end
-
-  private
-
-  def assign_normalized_source_key
-    normalize_path_fields!
-    self.normalized_source_key =
-        self.class.normalize_source_key(source_kind:,
-                                        source_uri:,
-                                        drive_path:,
-                                        drive_file_id:)
-  end
-
-  def source_value_must_be_present
-    return if normalized_source_key.present?
-
-    errors.add(:base, '同期元を指定してください．')
-  end
-
-  def drive_path_must_be_safe_relative_path
-    return unless source_kind.in?(
-        ['google_drive_path', 'google_drive_path_prefix',
-         'legacy_drive_path', 'legacy_drive_path_prefix'])
-
-    value = drive_path.to_s
-    return if value.blank?
-
-    if value.start_with?('/') || value.match?(/\A[A-Za-z]:\//)
-      errors.add(:drive_path, '相対 path を指定してください．')
-    end
-    if value.start_with?('My Drive/', 'マイドライブ/')
-      errors.add(:drive_path, '同期元フォルダからの相対 path を指定してください．')
-    end
-    errors.add(:drive_path, 'NUL は使へません．') if value.include?("\0")
-    errors.add(:drive_path, '連続スラッシュは使へません．') if value.include?('//')
-    errors.add(:drive_path, '末尾スラッシュは使へません．') if value.end_with?('/')
-
-    parts = value.split('/')
-    if parts.any? { |part| part.in?(['.', '..']) }
-      errors.add(:drive_path, '. や .. は使へません．')
-    end
-  end
-
-  def normalize_path_fields!
-    return unless source_kind.in?(
-        ['google_drive_path', 'google_drive_path_prefix',
-         'legacy_drive_path', 'legacy_drive_path_prefix'])
-
-    self.drive_path = drive_path.to_s.tr('\\', '/').strip.presence
-  end
-end

backend/app/models/material_version.rb

@@ -1,17 +0,0 @@
-class MaterialVersion < ApplicationRecord
-  EVENT_TYPE_MAP = { create: 'create',
-                     update: 'update',
-                     discard: 'discard',
-                     restore: 'restore' }.freeze
-
-  include VersionRecord
-
-  belongs_to :material
-  belongs_to :tag, optional: true
-  belongs_to :parent, class_name: 'Material', optional: true
-  belongs_to :updated_by_user, class_name: 'User', optional: true
-
-  def export_paths_hash
-    export_paths_json || {}
-  end
-end

backend/app/models/nico_tag_relation.rb

@@ -1,10 +1,24 @@
 class NicoTagRelation < ApplicationRecord
-  belongs_to :nico_tag,
+  belongs_to :nico_tag, class_name: 'Tag'
-             class_name: 'ExternalTag',
+  belongs_to :tag, class_name: 'Tag'
-             foreign_key: :nico_tag_id,
-             inverse_of: :nico_tag_relations
-  belongs_to :tag, class_name: 'Tag', foreign_key: :tag_id
 
   validates :nico_tag_id, presence: true
   validates :tag_id, presence: true
+
+  validate :nico_tag_must_be_nico
+  validate :tag_mustnt_be_nico
+
+  private
+
+  def nico_tag_must_be_nico
+    if nico_tag && nico_tag.category != 'nico'
+      errors.add :nico_tag_id, 'タグのカテゴリがニコニコである必要があります．'
+    end
+  end
+
+  def tag_mustnt_be_nico
+    if tag && tag.category == 'nico'
+      errors.add :tag_id, '連携先タグのカテゴリはニコニコであってはなりません．'
+    end
+  end
 end

backend/app/models/nico_tag_version.rb

@@ -1,7 +1,7 @@
 class NicoTagVersion < ApplicationRecord
   include VersionRecord
 
-  belongs_to :external_tag, foreign_key: :tag_id, inverse_of: :nico_tag_versions
+  belongs_to :tag
 
   validates :name, presence: true
 end

backend/app/models/post.rb

@@ -1,81 +1,26 @@
 class Post < ApplicationRecord
-  require 'date'
   require 'mini_magick'
-  require 'nokogiri'
-  require 'stringio'
-  require 'timeout'
-
-  class RemoteThumbnailFetchFailed < StandardError; end
-
-  ORIGINAL_CREATED_INVALID_MESSAGE = 'オリジナルの作成日時の形式が不正です．'.freeze
-  ORIGINAL_CREATED_MINUTE_PRECISION_MESSAGE =
-      'オリジナルの作成日時は分単位で入力してください．'.freeze
-  ORIGINAL_CREATED_ORDER_MESSAGE = 'オリジナルの作成日時の順番がをかしぃです．'.freeze
-  ORIGINAL_CREATED_MINIMUM_RANGE_MESSAGE =
-      'オリジナルの作成日時の範囲は1分以上必要です．'.freeze
-  REMOTE_SVG_CONTENT_TYPE = 'image/svg+xml'.freeze
-  MAX_SVG_DIMENSION = 4_096
-  MAX_SVG_PIXELS = 16_777_216
-  THUMBNAIL_PROCESS_TIMEOUT = 5.seconds
-  def self.resized_thumbnail_attachment(upload, content_type: nil)
-    upload.rewind
-    bytes = upload.read
-    blob = Timeout.timeout(THUMBNAIL_PROCESS_TIMEOUT) do
-      image = image_for_thumbnail_upload(bytes, content_type:)
-      image.auto_orient
-      image.resize '180x180'
-      image.format 'jpg'
-      image.to_blob
-    end
-
-    { io: StringIO.new(blob),
-      filename: 'resized_thumbnail.jpg',
-      content_type: 'image/jpeg' }
-  rescue Timeout::Error
-    raise MiniMagick::Error, 'サムネイル画像の変換に失敗しました．'
-  ensure
-    upload.rewind
-  end
-
-  def self.remote_thumbnail_attachment(raw_url)
-    response = Preview::ThumbnailFetcher.fetch_image_response(raw_url)
-    resized_thumbnail_attachment(
-        StringIO.new(response.body),
-        content_type: response.content_type)
-  rescue Preview::UrlSafety::UnsafeUrl,
-         Preview::ThumbnailFetcher::GenerationFailed,
-         Preview::HttpFetcher::FetchFailed,
-         Preview::HttpFetcher::FetchTimeout,
-         Preview::HttpFetcher::ResponseTooLarge,
-         Timeout::Error,
-         MiniMagick::Error => e
-    raise RemoteThumbnailFetchFailed, e.message
-  end
 
   belongs_to :uploaded_user, class_name: 'User', optional: true
 
   has_many :post_tags, dependent: :destroy, inverse_of: :post
-  has_many :tags, through: :post_tags
+  has_many :active_post_tags, -> { kept }, class_name: 'PostTag', inverse_of: :post
-  has_many :active_tags, -> { where(tags: { deprecated_at: nil }) },
+  has_many :post_tags_with_discarded, -> { with_discarded }, class_name: 'PostTag'
-           through: :post_tags,
+  has_many :tags, through: :active_post_tags
-           source: :tag
 
   has_many :user_post_views, dependent: :delete_all
   has_many :post_similarities, dependent: :delete_all
   has_many :post_versions
-
   has_many :gekanator_guessed_games,
            class_name: 'GekanatorGame',
            foreign_key: :guessed_post_id,
            dependent: :delete_all,
            inverse_of: :guessed_post
-
   has_many :gekanator_correct_games,
            class_name: 'GekanatorGame',
            foreign_key: :correct_post_id,
            dependent: :delete_all,
            inverse_of: :correct_post
-
   has_many :gekanator_question_examples, dependent: :delete_all
 
   has_many :parent_post_implications,
@@ -92,17 +37,13 @@ class Post < ApplicationRecord
            inverse_of: :parent_post
   has_many :children, through: :child_post_implications, source: :post
 
-  has_many :post_external_tags, dependent: :destroy
-  has_many :external_tags, through: :post_external_tags
-
   has_one_attached :thumbnail
 
   attribute :version_no, :integer, default: 1
 
-  before_validation :normalise_url, if: :will_save_change_to_url?
+  before_validation :normalise_url
 
-  validates :url, presence: true, uniqueness: true, length: { maximum: 768 }
+  validates :url, presence: true, uniqueness: true
-  validates :video_ms, numericality: { only_integer: true, greater_than: 0 }, allow_nil: true
 
   validate :validate_original_created_range
   validate :url_must_be_http_url
@@ -126,85 +67,7 @@ class Post < ApplicationRecord
     super(options).merge(thumbnail: nil)
   end
 
-  def snapshot_tag_names language_code = 'ja'
+  def snapshot_tag_names = tags.joins(:tag_name).order('tag_names.name').pluck('tag_names.name')
-    post_tags
-        .joins(tag: :tag_names)
-        .where(tag_names: { language_code:, primary_flg: true })
-        .includes(:sections, tag: :tag_names)
-        .order('tag_names.name')
-        .map do |post_tag|
-      name = post_tag.tag.name(language_code)
-      sections = post_tag.sections.sort_by(&:begin_ms)
-
-      next name if sections.empty?
-
-      "#{ name }#{ sections.map { Post.section_literal(_1) }.join }"
-    end
-  end
-
-  def self.tag_snapshot_literal tag
-    sections = tag.fetch('sections', []).map do |sec|
-      begin_ms = sec.fetch('begin_ms')
-      end_ms = sec['end_ms']
-
-      "[#{ Post.ms_to_time(begin_ms) }-#{ end_ms ? Post.ms_to_time(end_ms) : '' }]"
-    end
-
-    "#{ tag.fetch('name') }#{ sections.join }"
-  end
-
-  def snapshot_tags_json language_code = 'ja'
-    tag_snapshots =
-        post_tags
-            .joins(tag: :tag_names)
-            .where(tag_names: { language_code:, primary_flg: true })
-            .includes(:sections, tag: :tag_names)
-            .order('tags.id')
-            .map { |pt|
-          { 'tag_id' => pt.tag.id,
-            'version_no' => pt.tag.version_no,
-            'name' => pt.tag.name(language_code),
-            'category' => pt.tag.category,
-            'sections' => pt.sections.sort_by(&:begin_ms).map {
-                            { 'begin_ms' => _1.begin_ms, 'end_ms' => _1.end_ms }
-                          } }
-        }
-
-    external_tag_snapshots =
-        post_external_tags.order(:external_tag_id).map {
-          { 'external_tag_id' => _1.external_tag_id }
-        }
-
-    tag_snapshots + external_tag_snapshots
-  end
-
-  def self.section_literal section
-    end_ms =
-        section.end_ms ? Post.ms_to_time(section.end_ms) : ''
-
-    "[#{ Post.ms_to_time(section.begin_ms) }-#{ end_ms }]"
-  end
-
-  def self.ms_to_time ms
-    total_s = ms / 1_000
-    s = total_s % 60
-    min = (total_s / 60) % 60
-    h = total_s / 3_600
-    remainder_ms = ms % 1_000
-
-    base =
-        if h.positive?
-          '%d:%02d:%02d' % [h, min, s]
-        else
-          '%d:%02d' % [min, s]
-        end
-
-    if remainder_ms.positive?
-      "#{ base }.#{ remainder_ms.to_s.rjust(3, '0') }"
-    else
-      base
-    end
-  end
 
   def snapshot_parent_post_ids = parents.order(:id).pluck(:id)
 
@@ -222,29 +85,27 @@ class Post < ApplicationRecord
   def resized_thumbnail!
     return unless thumbnail.attached?
 
-    thumbnail.attach(self.class.resized_thumbnail_attachment(StringIO.new(thumbnail.download)))
+    image = MiniMagick::Image.read(thumbnail.download)
-  end
+    image.resize '180x180'
-
+    thumbnail.purge
-  def attach_thumbnail_from_url! raw_url
+    thumbnail.attach(io:            File.open(image.path),
-    thumbnail.attach(self.class.remote_thumbnail_attachment(raw_url))
+                     filename:      'resized_thumbnail.jpg',
+                     content_type:  'image/jpeg')
   end
 
   private
 
   def validate_original_created_range
-    return if skip_original_created_validation?
+    f = original_created_from
+    b = original_created_before
+    return if f.blank? || b.blank?
 
-    f = parse_original_created_value(:original_created_from)
+    f = Time.zone.parse(f) if String === f
-    b = parse_original_created_value(:original_created_before)
+    b = Time.zone.parse(b) if String === b
-    return if f.nil? || b.nil?
+    return if !(f) || !(b)
 
-    if b <= f
+    if f >= b
-      errors.add :original_created_at, ORIGINAL_CREATED_ORDER_MESSAGE
+      errors.add :original_created_at, 'オリジナルの作成日時の順番がをかしぃです．'
-      return
-    end
-
-    if b - f < 1.minute
-      errors.add :original_created_at, ORIGINAL_CREATED_MINIMUM_RANGE_MESSAGE
     end
   end
 
@@ -265,307 +126,15 @@ class Post < ApplicationRecord
   def normalise_url
     return if url.blank?
 
-    self.url = PostUrlNormaliser.normalise(url) || url.strip
+    self.url = url.strip
-  end
 
-  def self.image_for_thumbnail_upload(bytes, content_type: nil)
+    u = URI.parse(url)
-    if svg_content_type?(content_type) || svg_document_bytes?(bytes)
+    return unless u in URI::HTTP
-      return decode_svg_thumbnail(bytes)
-    end
 
-    raise MiniMagick::Error, 'サムネイル画像の形式が不正です．' unless raster_thumbnail_bytes?(bytes)
+    u.host = u.host.downcase if u.host
-
+    u.path = u.path.sub(/\/\Z/, '') if u.path.present?
-    decode_raster_thumbnail(bytes)
+    self.url = u.to_s
-  end
+  rescue URI::InvalidURIError
-
+    ;
-  def self.raster_thumbnail_bytes?(bytes)
-    raster_thumbnail_format(bytes).present?
-  end
-
-  def self.remote_thumbnail_image_bytes?(bytes, content_type: nil)
-    return true if svg_content_type?(content_type) || svg_document_bytes?(bytes)
-
-    raster_thumbnail_bytes?(bytes)
-  end
-
-  def self.svg_content_type?(content_type)
-    content_type.to_s.split(';', 2).first.to_s.downcase.strip == REMOTE_SVG_CONTENT_TYPE
-  end
-
-  def self.svg_document_bytes?(bytes)
-    document = Nokogiri::XML(
-        bytes,
-        nil,
-        nil,
-        Nokogiri::XML::ParseOptions::STRICT |
-        Nokogiri::XML::ParseOptions::NONET)
-    document.root&.name == 'svg'
-  rescue Nokogiri::XML::SyntaxError
-    false
-  end
-
-  def self.decode_raster_thumbnail(bytes)
-    MiniMagick::Image.read(bytes)
-  end
-
-  def self.decode_svg_thumbnail(bytes)
-    MiniMagick::Image.read(sanitised_svg_bytes(bytes))
-  end
-
-  def self.raster_thumbnail_format(bytes)
-    binary = bytes.to_s.b
-    return 'jpeg' if binary.start_with?("\xFF\xD8\xFF".b)
-    return 'png' if binary.start_with?("\x89PNG\r\n\x1A\n".b)
-    return 'gif' if binary.start_with?('GIF87a'.b) || binary.start_with?('GIF89a'.b)
-    return 'webp' if binary.bytesize >= 12 &&
-                     binary.start_with?('RIFF'.b) &&
-                     binary.byteslice(8, 4) == 'WEBP'
-
-    nil
-  end
-
-  def self.sanitised_svg_bytes(bytes)
-    parse_options =
-        Nokogiri::XML::ParseOptions::STRICT |
-        Nokogiri::XML::ParseOptions::NONET
-    document = Nokogiri::XML(
-        bytes,
-        nil,
-        nil,
-        parse_options)
-    root = document.root
-    raise MiniMagick::Error, 'SVG が不正です．' if root == nil || root.name != 'svg'
-    raise MiniMagick::Error, 'SVG が不正です．' if document.internal_subset != nil
-    raise MiniMagick::Error, 'SVG が不正です．' if svg_uses_disallowed_features?(document)
-
-    width, height = svg_dimensions(root)
-    raise MiniMagick::Error, 'SVG が大きすぎます．' if width == nil || height == nil
-    if width > MAX_SVG_DIMENSION || height > MAX_SVG_DIMENSION || width * height > MAX_SVG_PIXELS
-      raise MiniMagick::Error, 'SVG が大きすぎます．'
-    end
-
-    document.to_xml
-  rescue Nokogiri::XML::SyntaxError
-    raise MiniMagick::Error, 'SVG が不正です．'
-  end
-
-  def self.svg_uses_disallowed_features?(document)
-    document.traverse.any? do |node|
-      next false unless node.element?
-
-      name = node.name.to_s.downcase
-      next true if name == 'script' || name == 'foreignobject'
-      next style_contains_disallowed_urls?(node.text.to_s) if name == 'style'
-
-      node.attribute_nodes.any? do |attribute|
-        attribute_name = attribute.name.to_s.downcase
-        attribute_value = attribute.value.to_s
-        attribute_name.start_with?('on') ||
-            external_svg_reference?(attribute_name, attribute_value)
-      end
-    end
-  end
-
-  def self.external_svg_reference?(attribute_name, attribute_value)
-    if ['href', 'xlink:href', 'src'].include?(attribute_name)
-      return external_svg_url?(attribute_value)
-    end
-    return style_contains_disallowed_urls?(attribute_value) if attribute_name == 'style'
-    return svg_url_function_disallowed?(attribute_value) if attribute_value.match?(/url\s*\(/i)
-
-    false
-  end
-
-  def self.external_svg_url?(value)
-    stripped = value.to_s.strip
-    return false if stripped.blank? || stripped.start_with?('#')
-
-    true
-  end
-
-  def self.style_contains_disallowed_urls?(value)
-    text = value.to_s
-    text.match?(/@import/i) || svg_url_function_disallowed?(text)
-  end
-
-  def self.svg_url_function_disallowed?(value)
-    value.to_s.scan(/url\s*\(([^)]*)\)/i).flatten.any? do |entry|
-      reference =
-          entry.to_s.strip
-               .delete_prefix("'")
-               .delete_prefix('"')
-               .delete_suffix("'")
-               .delete_suffix('"')
-      reference.present? && !(reference.start_with?('#'))
-    end
-  end
-
-  def self.svg_dimensions(root)
-    width = svg_length_to_pixels(root['width'])
-    height = svg_length_to_pixels(root['height'])
-    return [width, height] if width && height
-
-    view_box = root['viewBox'].to_s.strip.split(/\s+/).map { Float(_1) rescue nil }
-    return [nil, nil] if view_box.length != 4 || view_box.any?(&:nil?)
-    return [nil, nil] unless view_box[2].finite? && view_box[2].positive?
-    return [nil, nil] unless view_box[3].finite? && view_box[3].positive?
-
-    [view_box[2], view_box[3]]
-  end
-
-  def self.svg_length_to_pixels(value)
-    return nil if value.blank?
-
-    matched = /\A([0-9]+(?:\.[0-9]+)?)(px)?\z/i.match(value.to_s.strip)
-    return nil if matched == nil
-
-    pixels = Float(matched[1])
-    return nil unless pixels.finite? && pixels.positive?
-
-    pixels
-  rescue ArgumentError
-    nil
-  end
-
-  private_class_method :image_for_thumbnail_upload,
-                       :svg_content_type?,
-                       :decode_raster_thumbnail,
-                       :decode_svg_thumbnail,
-                       :raster_thumbnail_format,
-                       :sanitised_svg_bytes,
-                       :svg_uses_disallowed_features?,
-                       :external_svg_reference?,
-                       :external_svg_url?,
-                       :style_contains_disallowed_urls?,
-                       :svg_url_function_disallowed?,
-                       :svg_dimensions,
-                       :svg_length_to_pixels
-
-  def parse_original_created_value field
-    raw_value = public_send("#{ field }_before_type_cast")
-    value = public_send(field)
-    return nil if raw_value.blank? && value.blank?
-
-    time =
-        case raw_value
-        when String
-          parse_original_created_string(raw_value)
-        when Time, ActiveSupport::TimeWithZone
-          raw_value.in_time_zone
-        else
-          value&.in_time_zone
-        end
-    if time.nil?
-      errors.add field, ORIGINAL_CREATED_INVALID_MESSAGE
-      return nil
-    end
-    unless minute_precision_time?(time)
-      errors.add field, ORIGINAL_CREATED_MINUTE_PRECISION_MESSAGE
-      return nil
-    end
-    time
-  end
-
-  def parse_original_created_string raw_value
-    value = raw_value.to_s.strip
-    return nil if value.blank?
-
-    match = value.match(/\A(\d{4})-(\d{2})-(\d{2})T(\d{2}):(\d{2})\z/)
-    if match
-      year = match[1].to_i
-      month = match[2].to_i
-      day = match[3].to_i
-      hour = match[4].to_i
-      minute = match[5].to_i
-      return nil unless valid_original_created_components?(year, month, day, hour, minute, 0)
-
-      return Time.zone.local(year, month, day, hour, minute)
-    end
-
-    match =
-        value.match(
-            /
-              \A(\d{4})-(\d{2})-(\d{2})T(\d{2}):(\d{2})
-              (?::(\d{2})(?:\.(\d+))?)?
-              (Z|[+-]\d{2}:?\d{2})?
-              \z
-            /x)
-    return nil if match.nil?
-
-    year = match[1].to_i
-    month = match[2].to_i
-    day = match[3].to_i
-    hour = match[4].to_i
-    minute = match[5].to_i
-    second = match[6]&.to_i || 0
-    fraction = match[7]
-    offset = match[8]
-    return nil unless valid_original_created_components?(year, month, day, hour, minute, second)
-    return nil if offset.present? && !(valid_original_created_offset?(offset))
-
-    if offset.present?
-      return Time.new(
-          year,
-          month,
-          day,
-          hour,
-          minute,
-          second + Rational(parse_original_created_nanoseconds(fraction), 1_000_000_000),
-          normalise_original_created_offset(offset)).in_time_zone
-    end
-
-    Time.zone.local(
-        year,
-        month,
-        day,
-        hour,
-        minute,
-        second).change(nsec: parse_original_created_nanoseconds(fraction))
-  rescue ArgumentError, TypeError
-    nil
-  end
-
-  def minute_precision_time? value
-    value.sec.zero? && value.nsec.zero?
-  end
-
-  def valid_original_created_components? year, month, day, hour, minute, second
-    return false unless Date.valid_date?(year, month, day)
-    return false unless hour.between?(0, 23)
-    return false unless minute.between?(0, 59)
-    return false unless second.between?(0, 59)
-
-    true
-  end
-
-  def valid_original_created_offset? value
-    match = value.match(/\A([+-])(\d{2}):?(\d{2})\z/)
-    return true if value == 'Z'
-    return false if match.nil?
-
-    hours = match[2].to_i
-    minutes = match[3].to_i
-    hours.between?(0, 23) && minutes.between?(0, 59)
-  end
-
-  def parse_original_created_nanoseconds value
-    return 0 if value.blank?
-
-    digits = value[0, 9].ljust(9, '0')
-    Integer(digits, 10)
-  end
-
-  def normalise_original_created_offset value
-    return '+00:00' if value == 'Z'
-
-    value.match?(/\A[+-]\d{2}:\d{2}\z/) ? value : "#{ value[0, 3] }:#{ value[3, 2] }"
-  end
-
-  def skip_original_created_validation?
-    return false if new_record?
-    return false if will_save_change_to_original_created_from?
-    return false if will_save_change_to_original_created_before?
-
-    true
   end
 end

backend/app/models/post_external_tag.rb

@@ -1,4 +0,0 @@
-class PostExternalTag < ApplicationRecord
-  belongs_to :post
-  belongs_to :external_tag, counter_cache: :post_count
-end

backend/app/models/post_tag.rb

@@ -1,15 +1,29 @@
 class PostTag < ApplicationRecord
+  include Discard::Model
+
+  before_destroy do
+    raise ActiveRecord::ReadOnlyRecord, '消さないでください．'
+  end
+
   belongs_to :post
   belongs_to :tag, counter_cache: :post_count
   belongs_to :created_user, class_name: 'User', optional: true
-
+  belongs_to :deleted_user, class_name: 'User', optional: true
-  has_many :sections, -> { order(:begin_ms) }, class_name: 'PostTagSection',
-                                               foreign_key: [:post_id, :tag_id],
-                                               primary_key: [:post_id, :tag_id],
-                                               dependent: :delete_all,
-                                               inverse_of: :post_tag
 
   validates :post_id, presence: true
   validates :tag_id, presence: true
-  validates :post_id, uniqueness: { scope: :tag_id }
+  validates :post_id, uniqueness: {
+      scope: :tag_id,
+      conditions: -> { where(discarded_at: nil) } }
+
+  def discard_by! deleted_user
+    return self if discarded?
+
+    transaction do
+      update!(discarded_at: Time.current, deleted_user:)
+      Tag.where(id: tag_id).update_all('post_count = GREATEST(post_count - 1, 0)')
+    end
+
+    self
+  end
 end

backend/app/models/post_tag_section.rb

@@ -1,20 +0,0 @@
-class PostTagSection < ApplicationRecord
-  self.primary_key = :post_id, :tag_id, :begin_ms
-
-  belongs_to :post
-  belongs_to :tag
-
-  belongs_to :post_tag, foreign_key: [:post_id, :tag_id],
-                        primary_key: [:post_id, :tag_id],
-                        inverse_of: :sections,
-                        optional: true
-
-  validates :post_id, presence: true
-  validates :tag_id, presence: true
-
-  validates :begin_ms, presence: true,
-                       numericality: { only_integer: true, greater_than_or_equal_to: 0 }
-
-  validates :end_ms, numericality: { only_integer: true, greater_than: :begin_ms },
-                     allow_nil: true
-end

backend/app/models/post_url_sanitisation_rule.rb

@@ -1,163 +0,0 @@
-class PostUrlSanitisationRule < ApplicationRecord
-  include Discard::Model
-
-  class InvalidUrlError < StandardError
-    attr_reader :invalid_rows
-
-    def initialize(invalid_rows)
-      @invalid_rows = invalid_rows
-      ids = invalid_rows.map { _1.fetch(:post_id) }.join(', ')
-      super("post URL sanitisation produced invalid URLs for posts #{ ids }")
-    end
-  end
-
-  class UrlConflictError < StandardError
-    attr_reader :conflicts
-
-    def initialize(conflicts)
-      @conflicts = conflicts
-      urls = conflicts.map { _1.fetch(:url) }.uniq.join(', ')
-      super("post URL sanitisation conflicts detected for #{ urls }")
-    end
-  end
-
-  self.primary_key = :priority
-
-  default_scope -> { kept }
-
-  validates :source_pattern, presence: true, uniqueness: true
-
-  validate :source_pattern_must_be_regexp
-
-  class << self
-    def sanitise(url) = sanitise_with_rules(url, rules)
-
-    def apply!
-      rewrites = nil
-
-      Post.transaction do
-        compiled_rules = rules
-
-        rewrites = Post.order(:id)
-                       .lock('FOR UPDATE')
-                       .pluck(:id, :url)
-                       .map do |post_id, original_url|
-          { post_id:,
-            original_url:,
-            sanitised_url: sanitise_with_rules(original_url, compiled_rules) }
-        end
-
-        invalid_rows = rewrites.filter { invalid_sanitised_url?(_1.fetch(:sanitised_url)) }
-                               .map { { post_id: _1.fetch(:post_id),
-                                        original_url: _1.fetch(:original_url),
-                                        sanitised_url: _1.fetch(:sanitised_url) } }
-        raise InvalidUrlError.new(invalid_rows) if invalid_rows.present?
-
-        conflicts = build_conflicts(rewrites)
-        raise UrlConflictError.new(conflicts) if conflicts.present?
-
-        changed = rewrites.filter { _1.fetch(:original_url) != _1.fetch(:sanitised_url) }
-        return if changed.empty?
-
-        token = SecureRandom.hex(6)
-
-        changed.each do |row|
-          Post.where(id: row.fetch(:post_id))
-              .update_all(url: temporary_url_for(row.fetch(:post_id), token))
-        end
-
-        changed.each do |row|
-          Post.where(id: row.fetch(:post_id))
-              .update_all(url: row.fetch(:sanitised_url))
-        end
-      end
-    rescue ActiveRecord::RecordNotUnique => error
-      conflicts = build_persisted_conflicts(rewrites)
-      raise error if conflicts.empty?
-
-      raise UrlConflictError.new(conflicts), cause: error
-    end
-
-    private
-
-    def rules = kept.order(:priority).map { |r| [Regexp.new(r.source_pattern), r.replacement] }
-
-    def sanitise_with_rules(url, compiled_rules)
-      compiled_rules.reduce(url.dup) do |value, (pattern, replacement)|
-        value.sub(pattern, replacement)
-      end
-    end
-
-    def temporary_url_for(post_id, token) =
-        "https://post-url-sanitising.invalid/#{ token }/#{ post_id }"
-
-    def invalid_sanitised_url?(url)
-      return true if url.blank?
-      return true if url.length > 768
-
-      parsed = URI.parse(url)
-      return true if !(parsed in URI::HTTP)
-      return true if parsed.host.blank?
-
-      false
-    rescue URI::InvalidURIError
-      true
-    end
-
-    def build_conflicts(rewrites)
-      rewrites
-          .group_by { _1.fetch(:sanitised_url).downcase }
-          .values
-          .filter { _1.size > 1 }
-          .flatten
-          .map { { url: _1.fetch(:sanitised_url),
-                   post_id: _1.fetch(:post_id),
-                   original_url: _1.fetch(:original_url) } }
-    end
-
-    def build_persisted_conflicts(rewrites)
-      return [] if rewrites.blank?
-
-      target_rows = rewrites.filter { _1.fetch(:original_url) != _1.fetch(:sanitised_url) }
-      target_keys = target_rows.map { _1.fetch(:sanitised_url).downcase }.uniq
-      return [] if target_keys.empty?
-
-      target_pairs = target_rows.to_h do |row|
-        [row.fetch(:post_id), row.fetch(:sanitised_url).downcase]
-      end
-
-      persisted_rows = Post.order(:id)
-                           .where('LOWER(url) IN (?)', target_keys)
-                           .pluck(:id, :url)
-                           .reject { |post_id, original_url| target_pairs[post_id] == original_url.downcase }
-                           .map { |post_id, original_url|
-        { url: original_url,
-          post_id:,
-          original_url:,
-          conflict_key: original_url.downcase }
-      }
-
-      target_conflicts = target_rows.map { { url: _1.fetch(:sanitised_url),
-                                             post_id: _1.fetch(:post_id),
-                                             original_url: _1.fetch(:original_url),
-                                             conflict_key: _1.fetch(:sanitised_url).downcase } }
-
-      (persisted_rows + target_conflicts)
-          .group_by { _1.fetch(:conflict_key) }
-          .values
-          .filter { _1.size > 1 }
-          .flatten
-          .map { _1.except(:conflict_key) }
-    end
-  end
-
-  private
-
-  def source_pattern_must_be_regexp
-    return if source_pattern.blank?
-
-    Regexp.new(source_pattern)
-  rescue RegexpError
-    errors.add :source_pattern, '変な正規表現だね〜（笑）'
-  end
-end

backend/app/models/post_version.rb

@@ -1,17 +1,10 @@
 class PostVersion < ApplicationRecord
   include VersionRecord
 
-  scope :with_previous_version, -> {
-    joins('LEFT JOIN post_versions prev ON prev.post_id = post_versions.post_id ' +
-          'AND prev.version_no = post_versions.version_no - 1 AND ' +
-          HistoryAccess.sql(PostVersion, 'prev'))
-  }
-
   belongs_to :post
   belongs_to :parent, class_name: 'Post', optional: true
 
   validates :url, presence: true
-  validates :video_ms, numericality: { only_integer: true, greater_than: 0 }, allow_nil: true
 
   validate :validate_original_created_range
 

backend/app/models/region.rb

@@ -1,3 +0,0 @@
-class Region < ApplicationRecord
-  ;
-end

backend/app/models/script.rb

@@ -1,11 +0,0 @@
-class Script < ApplicationRecord
-  has_many :locales, foreign_key: :script_code, primary_key: :code
-
-  # Canonical keys come first so reads and plucks retain ISO 15924 spelling.
-  # Legacy keys preserve the existing enum predicates, scopes and queries.
-  enum :code, { 'Jpan' => 'Jpan', 'Kana' => 'Kana', jpan: 'Jpan', kana: 'Kana' }
-
-  def code= value
-    super(self.class.codes.fetch(value.to_s, value))
-  end
-end

backend/app/models/setting.rb

@@ -1,52 +1,7 @@
 class Setting < ApplicationRecord
-  THEMES = ['system', 'light', 'dark'].freeze
-
-  # These remain in the typed settings schema to preserve existing backend
-  # work. The common `/users/settings` page currently surfaces only `theme`.
-  AUTO_FETCH_MODES = ['auto', 'manual', 'off'].freeze
-  WIKI_EDITOR_MODES = ['split', 'write', 'preview'].freeze
-
-  STRING_ATTRIBUTES = [
-    'theme',
-    'auto_fetch_title',
-    'auto_fetch_thumbnail',
-    'wiki_editor_mode',
-  ].freeze
-  INTEGER_ATTRIBUTES = [].freeze
-  BOOLEAN_ATTRIBUTES = [].freeze
-  EDITABLE_ATTRIBUTES =
-      (STRING_ATTRIBUTES + INTEGER_ATTRIBUTES + BOOLEAN_ATTRIBUTES).freeze
-  TYPE_BY_ATTRIBUTE = {
-    'theme' => :string,
-    'auto_fetch_title' => :string,
-    'auto_fetch_thumbnail' => :string,
-    'wiki_editor_mode' => :string,
-  }.freeze
-
   belongs_to :user
 
   validates :user_id, presence: true
-  validates :user_id, uniqueness: true
+  validates :key, presence: true, length: { maximum: 255 }
-
+  validates :value, presence: true
-  validates :theme, inclusion: { in: THEMES }
-  validates :auto_fetch_title, inclusion: { in: AUTO_FETCH_MODES }
-  validates :auto_fetch_thumbnail, inclusion: { in: AUTO_FETCH_MODES }
-  validates :wiki_editor_mode, inclusion: { in: WIKI_EDITOR_MODES }
-
-  def self.defaults
-    {
-      theme: 'system',
-      auto_fetch_title: 'manual',
-      auto_fetch_thumbnail: 'manual',
-      wiki_editor_mode: 'split',
-    }
-  end
-
-  def self.serializable_attributes
-    EDITABLE_ATTRIBUTES.map(&:to_sym)
-  end
-
-  def serializable_hash(options = nil)
-    super({ only: self.class.serializable_attributes }.merge(options || { }))
-  end
 end

backend/app/models/tag.rb

@@ -2,38 +2,22 @@ require 'set'
 
 
 class Tag < ApplicationRecord
-  before_create { SharedResourceIdAllocator.assign!(self) }
+  include MyDiscard
 
   class NicoTagNormalisationError < ArgumentError
     ;
   end
 
-  class DeprecatedTagNormalisationError < ArgumentError
-    attr_reader :tag_names
-
-    def initialize tag_names
-      @tag_names = Array(tag_names)
-      super('deprecated tags are not allowed')
-    end
-  end
-
-  class SectionLiteralParseError < ArgumentError
-    attr_reader :tag_name, :literal
-
-    def initialize tag_name, literal
-      @tag_name = tag_name
-      @literal = literal
-      super("invalid section literal for tag #{ tag_name }: #{ literal }")
-    end
-  end
-
   has_many :post_tags, inverse_of: :tag
-  has_many :posts, through: :post_tags
+  has_many :active_post_tags, -> { kept }, class_name: 'PostTag', inverse_of: :tag
+  has_many :post_tags_with_discarded, -> { with_discarded }, class_name: 'PostTag'
+  has_many :posts, through: :active_post_tags
+
+  has_many :nico_tag_relations, foreign_key: :nico_tag_id, dependent: :destroy
+  has_many :linked_tags, through: :nico_tag_relations, source: :tag
 
   has_many :reversed_nico_tag_relations,
-           class_name: 'NicoTagRelation',
+           class_name: 'NicoTagRelation', foreign_key: :tag_id, dependent: :destroy
-           foreign_key: :tag_id,
-           dependent: :destroy
   has_many :linked_nico_tags, through: :reversed_nico_tag_relations, source: :nico_tag
 
   has_many :tag_implications, foreign_key: :parent_tag_id, dependent: :destroy
@@ -51,23 +35,31 @@ class Tag < ApplicationRecord
   has_many :materials
 
   has_many :tag_versions
+  has_many :nico_tag_versions
 
-  has_many :tag_names
+  belongs_to :tag_name
-  has_many :tag_descriptions
+  delegate :wiki_page, to: :tag_name
 
   attribute :version_no, :integer, default: 1
 
+  delegate :name, to: :tag_name, allow_nil: true
+  validates :tag_name, presence: true
+
   enum :category, deerjikist:  'deerjikist',
                   meme:        'meme',
                   character:   'character',
                   general:     'general',
                   material:    'material',
+                  nico:        'nico',
                   meta:        'meta'
 
   validates :category, presence: true, inclusion: { in: Tag.categories.keys }
 
+  validate :nico_tag_name_must_start_with_nico
+  validate :tag_name_must_be_canonical
   validate :category_must_be_deerjikist_with_deerjikists
-  validate :tag_names_must_be_valid
+
+  scope :nico_tags, -> { nico }
 
   CATEGORY_PREFIXES = {
       'general:'    => :general,
@@ -81,125 +73,44 @@ class Tag < ApplicationRecord
       'mtr:'        => :material,
       'meta:'       => :meta }.freeze
 
-  SYSTEM_TAG_NAMES = ['タグ希望', 'bot操作', 'ニジラー情報不詳', '動画', 'ニコニコ', 'YouTube'].freeze
+  def name= val
-
+    (self.tag_name ||= build_tag_name).name = val
-  def tag_name language_code
-    if tag_names.loaded?
-      tag_names.find { _1.language_code == language_code && _1.primary_flg }
-    else
-      tag_names.find_by(language_code:, primary_flg: true)
-    end
-  end
-  def name(language_code) = tag_name(language_code)&.name
-
-  def system?
-    tag_name('ja')&.name.in?(SYSTEM_TAG_NAMES)
   end
 
-  def deprecated? = deprecated_at?
+  def has_wiki = wiki_page.present?
-
-  def has_description? language_code:
-    if tag_descriptions.loaded?
-      tag_descriptions.any? { _1.language_code == language_code }
-    else
-      tag_descriptions.exists?(language_code:)
-    end
-  end
 
   def material_id = materials.first&.id
 
   def has_deerjikists = deerjikists.loaded? ? deerjikists.any? : deerjikists.exists?
 
-  def snapshot_aliases language_code
+  def self.tagme = find_or_create_by_tag_name!('タグ希望', category: :meta)
-    if tag_names.loaded?
+  def self.bot = find_or_create_by_tag_name!('bot操作', category: :meta)
-      return tag_names.select { _1.language_code == language_code && !_1.primary? }
+  def self.no_deerjikist = find_or_create_by_tag_name!('ニジラー情報不詳', category: :meta)
-          .sort_by(&:name).map(&:name)
+  def self.video = find_or_create_by_tag_name!('動画', category: :meta)
-    end
+  def self.niconico = find_or_create_by_tag_name!('ニコニコ', category: :meta)
+  def self.youtube = find_or_create_by_tag_name!('YouTube', category: :meta)
 
-    tag_names.where(language_code:, primary_flg: false).order(:name).pluck(:name)
+  def self.normalise_tags! tag_names, with_tagme: true,
-  end
-
-  def snapshot_parent_tag_ids = parents.order(:id).pluck(:id)
-
-  def snapshot_linked_tag_names language_code
-    linked_tags
-        .joins(:tag_names)
-        .where(tag_names: { language_code:, primary_flg: true })
-        .order('tag_names.name')
-        .pluck('tag_names.name')
-  end
-
-  class << self
-    def tagme = find_or_create_by_tag_name!(Locale.nipponese, 'タグ希望', category: :meta)
-    def bot = find_or_create_by_tag_name!(Locale.nipponese, 'bot操作', category: :meta)
-    def no_deerjikist =
-        find_or_create_by_tag_name!(Locale.nipponese, 'ニジラー情報不詳', category: :meta)
-    def video = find_or_create_by_tag_name!(Locale.nipponese, '動画', category: :meta)
-    def niconico = find_or_create_by_tag_name!(Locale.nipponese, 'ニコニコ', category: :meta)
-    def youtube = find_or_create_by_tag_name!(Locale.nipponese, 'YouTube', category: :meta)
-
-    def normalise_tags! locale, tag_names,
-                             with_tagme: true,
                                       with_no_deerjikist: true,
-                             deny_deprecated: false,
+                                      deny_nico: true
-                             with_sections: false,
+    if deny_nico && tag_names.any? { |n| n.downcase.start_with?('nico:') }
-                             created_by_user: nil
-      if tag_names.any? { |n| n.downcase.start_with?('nico:') }
       raise NicoTagNormalisationError
     end
 
-      sections = { }
     tags = tag_names.map do |name|
-        raw_name = name
       pf, cat = CATEGORY_PREFIXES.find { |p, _| name.downcase.start_with?(p) } || ['', nil]
-
+      name = TagName.canonicalise(name.sub(/\A#{ pf }/i, '')).first
-        name = name.sub(/\A#{ pf }/i, '')
+      find_or_create_by_tag_name!(name, category: (cat || :general)).tap do |tag|
-
-        sections_by_tag = []
-        while (match = name.match(/\A(\S*?)\[([^\[\]\s]*)-([^\[\]\s]*)\](\S*)\z/))
-          name = "#{ match[1] }#{ match[4] }"
-          next if match[2].empty? && match[3].empty?
-
-          sections_by_tag << normalise_section_range!(
-              begin_raw: match[2],
-              end_raw: match[3],
-              tag_name: name)
-        end
-
-        if name.include?('[') || name.include?(']')
-          raise SectionLiteralParseError.new(raw_name, raw_name)
-        end
-
-        name = TagName.canonicalise(locale, name).first
-
-        find_or_create_by_tag_name!(locale, name, category: (cat || :general),
-                                   created_by_user:).tap do |tag|
-          if deny_deprecated && tag.deprecated?
-            raise DeprecatedTagNormalisationError, [tag.name(locale.language_code)]
-          end
-
         tag.update!(category: cat) if cat && tag.category != cat
-          next if sections_by_tag.empty?
-
-          sections[tag.id] ||= []
-          sections[tag.id].concat(sections_by_tag)
-          sections[tag.id] = merge_section_ranges(sections[tag.id])
-          sections.delete(tag.id) if sections[tag.id] == [[0, nil]]
       end
     end
 
     tags << Tag.tagme if with_tagme && tags.size < 10 && tags.none?(Tag.tagme)
     tags << Tag.no_deerjikist if with_no_deerjikist && tags.all? { |t| !(t.deerjikist?) }
-      tags.uniq!(&:id)
+    tags.uniq(&:id)
-
-      if with_sections
-        { tags:, sections: }
-      else
-        tags
-      end
   end
 
-    def expand_parent_tags tags
+  def self.expand_parent_tags tags
     return [] if tags.blank?
 
     seen = Set.new
@@ -222,70 +133,18 @@ class Tag < ApplicationRecord
     (result + tags).uniq { |t| t.id }
   end
 
-    def normalise_section_range! begin_raw:, end_raw:, tag_name:
+  def self.find_or_create_by_tag_name! name, category:
-      begin_ms = begin_raw.empty? ? 0 : time_to_ms!(begin_raw, tag_name:)
+    tn = TagName.find_undiscard_or_create_by!(name: name.to_s.strip)
-      end_ms = end_raw.empty? ? nil : time_to_ms!(end_raw, tag_name:)
+    tn = tn.canonical if tn.canonical_id?
 
-      if end_ms
+    Tag.find_undiscard_or_create_by!(tag_name_id: tn.id) do |t|
-        begin_ms, end_ms = end_ms, begin_ms if begin_ms > end_ms
+      t.category = category
-        end_ms = begin_ms + 1 if begin_ms == end_ms
-      end
-
-      [begin_ms, end_ms]
-    end
-
-    def merge_section_ranges ranges
-      sorted_ranges = ranges.sort_by { |begin_ms, end_ms| [begin_ms, end_ms || Float::INFINITY] }
-      merged = []
-
-      sorted_ranges.each do |begin_ms, end_ms|
-        if merged.empty?
-          merged << [begin_ms, end_ms]
-          next
-        end
-
-        last_begin_ms, last_end_ms = merged[-1]
-        if last_end_ms.nil? || begin_ms <= last_end_ms
-          merged[-1] = [last_begin_ms, merge_section_end(last_end_ms, end_ms)]
-        else
-          merged << [begin_ms, end_ms]
-        end
-      end
-
-      merged
-    end
-
-    def merge_section_end left_end_ms, right_end_ms
-      return nil if left_end_ms.nil? || right_end_ms.nil?
-
-      [left_end_ms, right_end_ms].max
-    end
-
-    def find_or_create_by_tag_name! locale, name, category:, created_by_user: nil
-      language_code = locale.language_code
-      name = name.to_s.strip
-
-      ApplicationRecord.transaction do
-        SharedResourceIdAllocator.lock_space!(Tag, 'tags')
-        tn = TagName.lock.find_by(language_code:, name:)
-        next tn.tag if tn
-
-        tag = Tag.new(category:)
-        tag.tag_names.build(language_code:, name:, script_code: locale.script_code,
-                            primary_flg: true, auto_generated: false)
-        TagNameCompletion.for_tag!(tag, created_by_user:)
-        tag.association(:tag_names).reset
-
-        tag
     end
   rescue ActiveRecord::RecordNotUnique
-      existing = TagName.find_by(language_code:, name:)
+    retry
-      raise unless existing
-
-      existing.tag
   end
 
-    def merge_tags! target_tag, source_tags, created_by_user: nil
+  def self.merge_tags! target_tag, source_tags, created_by_user: nil
     target_tag => Tag
 
     affected_post_ids = Set.new
@@ -300,26 +159,30 @@ class Tag < ApplicationRecord
 
         TagVersioning.ensure_snapshot!(source_tag, created_by_user:)
 
-          source_tag.post_tags.find_each do |source_pt|
+        source_tag.post_tags.kept.find_each do |source_pt|
           post_id = source_pt.post_id
           affected_post_ids << post_id
-            source_pt.destroy!
+          source_pt.discard_by!(created_by_user)
-            unless PostTag.exists?(post_id:, tag: target_tag)
+          unless PostTag.kept.exists?(post_id:, tag: target_tag)
             PostTag.create!(post_id:, tag: target_tag)
           end
         end
 
-          if source_tag.tag_descriptions.exists?
+        source_tag_name = source_tag.tag_name
-            source_tag.errors.add(:base, :invalid)
+
-            raise ActiveRecord::RecordInvalid, source_tag
+        if source_tag_name.wiki_page.present?
+          raise ActiveRecord::RecordInvalid.new(source_tag_name)
         end
 
         TagVersioning.record!(source_tag, event_type: :discard, created_by_user:)
-          target_languages = target_tag.tag_names.where(primary_flg: true).pluck(:language_code)
+        source_tag.discard!
-          source_tag.tag_names.where(language_code: target_languages)
+
-              .update_all(primary_flg: false, auto_generated: false, updated_at: Time.current)
+        if source_tag.nico?
-          source_tag.tag_names.update_all(tag_id: target_tag.id, updated_at: Time.current)
+          source_tag_name.discard!
-          source_tag.destroy!
+        else
+          source_tag_name.update_columns(canonical_id: target_tag.tag_name_id,
+                                         updated_at: Time.current)
+        end
 
         TagVersioning.record!(target_tag, event_type: :update, created_by_user:)
       end
@@ -330,47 +193,33 @@ class Tag < ApplicationRecord
       end
 
       # 投稿件数を再集計
-        target_tag.update_columns(post_count: PostTag.where(tag: target_tag).count)
+      target_tag.update_columns(post_count: PostTag.kept.where(tag: target_tag).count)
     end
 
     target_tag.reload
   end
 
-    def time_to_ms! str, tag_name:
+  def snapshot_aliases = tag_name.aliases.kept.order(:name).pluck(:name)
-      match =
-          case str
-          when /\A(?<seconds>\d+)(?:\.(?<ms>\d{1,3}))?\z/
-            { hours: nil, minutes: nil, seconds: Regexp.last_match[:seconds],
-              ms: Regexp.last_match[:ms] }
-          when /\A(?<minutes>\d+):(?<seconds>[0-5]?\d)(?:\.(?<ms>\d{1,3}))?\z/
-            { hours: nil, minutes: Regexp.last_match[:minutes],
-              seconds: Regexp.last_match[:seconds],
-              ms: Regexp.last_match[:ms] }
-          when /\A(?<hours>\d+):(?<minutes>[0-5]?\d):(?<seconds>[0-5]?\d)(?:\.(?<ms>\d{1,3}))?\z/
-            { hours: Regexp.last_match[:hours],
-              minutes: Regexp.last_match[:minutes],
-              seconds: Regexp.last_match[:seconds],
-              ms: Regexp.last_match[:ms] }
-          end
 
-      raise SectionLiteralParseError.new(tag_name, str) unless match
+  def snapshot_parent_tag_ids = parents.order(:id).pluck(:id)
-
-      total_s = match[:seconds].to_i
-      total_s += match[:minutes].to_i * 60 if match[:minutes]
-      total_s += match[:hours].to_i * 3_600 if match[:hours]
-
-      total_s * 1_000 + match[:ms].to_s.ljust(3, '0')[0, 3].to_i
-    end
 
+  def snapshot_linked_tag_names
+    linked_tags.joins(:tag_name).order('tag_names.name').pluck('tag_names.name')
   end
 
   private
 
-  def tag_names_must_be_valid
+  def nico_tag_name_must_start_with_nico
-    tag_names.each do |tag_name|
+    n = name.to_s
-      next if tag_name.valid?
+    if ((nico? && !(n.downcase.start_with?('nico:'))) || 
+        (!(nico?) && n.downcase.start_with?('nico:')))
+      errors.add :name, 'ニコニコ・タグの命名規則に反してゐます．'
+    end
+  end
 
-      tag_name.errors[:name].each { errors.add :name, _1 }
+  def tag_name_must_be_canonical
+    if tag_name&.canonical_id?
+      errors.add :tag_name, 'tag_names へは実体を示す必要があります．'
     end
   end
 

backend/app/models/tag_description.rb

@@ -1,5 +0,0 @@
-class TagDescription < ApplicationRecord
-  before_create { SharedResourceIdAllocator.assign!(self) }
-
-  belongs_to :tag
-end

backend/app/models/tag_description_version.rb

@@ -1,5 +0,0 @@
-class TagDescriptionVersion < ApplicationRecord
-  include VersionRecord
-
-  before_create { SharedResourceIdAllocator.assign!(self) }
-end

backend/app/models/tag_group.rb

@@ -1,27 +0,0 @@
-class TagGroup < ApplicationRecord
-  before_create { SharedResourceIdAllocator.assign!(self) }
-  before_destroy :delete_current_structure!
-
-  belongs_to :created_by_user, class_name: 'User', optional: true
-  belongs_to :updated_by_user, class_name: 'User', optional: true
-
-  has_many :tag_group_tags, dependent: :delete_all
-  has_many :tags, through: :tag_group_tags
-  has_many :tag_group_groups, dependent: :delete_all
-  has_many :child_tag_groups, through: :tag_group_groups
-  has_many :placements, -> { order(:position) },
-           class_name: 'TagGroupPlacement', dependent: :delete_all
-  has_many :versions, -> { order(:version_no) },
-           class_name: 'TagGroupVersion'
-
-  validates :name, presence: true, uniqueness: true, length: { maximum: 255 }
-  validates :version_no, numericality: { only_integer: true, greater_than: 0 }
-
-  private
-
-  def delete_current_structure!
-    placements.reorder(position: :desc).each(&:delete)
-    tag_group_tags.delete_all
-    tag_group_groups.delete_all
-  end
-end

backend/app/models/tag_group_group.rb

@@ -1,7 +0,0 @@
-class TagGroupGroup < ApplicationRecord
-  self.primary_key = [:tag_group_id, :child_tag_group_id]
-
-  belongs_to :tag_group
-  belongs_to :child_tag_group, class_name: 'TagGroup'
-  belongs_to :created_by_user, class_name: 'User', optional: true
-end

backend/app/models/tag_group_placement.rb

@@ -1,9 +0,0 @@
-class TagGroupPlacement < ApplicationRecord
-  self.primary_key = [:tag_group_id, :position]
-
-  belongs_to :tag_group
-  belongs_to :tag, optional: true
-  belongs_to :child_tag_group, class_name: 'TagGroup', optional: true
-  belongs_to :created_by_user, class_name: 'User', optional: true
-  belongs_to :updated_by_user, class_name: 'User', optional: true
-end

backend/app/models/tag_group_tag.rb

@@ -1,7 +0,0 @@
-class TagGroupTag < ApplicationRecord
-  self.primary_key = [:tag_group_id, :tag_id]
-
-  belongs_to :tag_group
-  belongs_to :tag
-  belongs_to :created_by_user, class_name: 'User', optional: true
-end

backend/app/models/tag_group_version.rb

@@ -1,9 +0,0 @@
-class TagGroupVersion < ApplicationRecord
-  include VersionRecord
-
-  before_create { SharedResourceIdAllocator.assign!(self) }
-
-  belongs_to :tag_group
-
-  scope :ordered, -> { order(:version_no) }
-end

backend/app/models/tag_implication.rb

@@ -6,7 +6,6 @@ class TagImplication < ApplicationRecord
   validates :parent_tag_id, presence: true
 
   validate :parent_tag_mustnt_be_itself
-  validate :parent_tag_mustnt_create_cycle
 
   private
 
@@ -15,27 +14,4 @@ class TagImplication < ApplicationRecord
       errors.add :parent_tag_id, '親タグは子タグと同一であってはなりません．'
     end
   end
-
-  def parent_tag_mustnt_create_cycle
-    return if tag_id.blank? || parent_tag_id.blank?
-    return if errors[:parent_tag_id].present?
-
-    seen = { }
-    stack = [parent_tag_id]
-
-    until stack.empty?
-      current_id = stack.pop
-      next if seen[current_id]
-
-      seen[current_id] = true
-
-      if current_id == tag_id
-        errors.add :parent_tag_id, '親タグに子孫タグを指定すると循環します．'
-        errors.add :base, 'タグの親子関係が循環します．'
-        return
-      end
-
-      stack.concat(TagImplication.where(tag_id: current_id).pluck(:parent_tag_id))
-    end
-  end
 end

backend/app/models/tag_name.rb

@@ -1,82 +1,51 @@
 class TagName < ApplicationRecord
-  belongs_to :language, foreign_key: :language_code, primary_key: :code
+  include MyDiscard
-  belongs_to :tag
 
-  validates :name, presence: true,
+  has_one :tag
-                   length: { maximum: 255 },
+  has_one :wiki_page
-                   uniqueness: { scope: :language_code }
 
+  belongs_to :canonical, class_name: 'TagName', optional: true
+  has_many :aliases, class_name: 'TagName', foreign_key: :canonical_id
+
+  validates :name, presence: true, length: { maximum: 255 }, uniqueness: true
+
+  validate :canonical_must_be_canonical
   validate :alias_name_must_not_have_prefix
+  validate :canonical_must_not_be_present_with_tag_or_wiki_page
   validate :name_must_be_sanitised
-  validate :name_mustnt_start_with_nico
-  validate :generated_name_must_be_reserved
 
-  def primary? = primary_flg
+  def self.canonicalise names
-
-  def self.canonicalise locale, names
     names = Array(names).map { |n| n.to_s.strip }.reject(&:blank?)
     return [] if names.blank?
 
-    tns = TagName.where(language_code: locale.language_code, name: names).index_by(&:name)
+    tns = TagName.includes(:canonical).where(name: names).index_by(&:name)
 
-    primary_names = TagName.where(
+    names.map { |name| tns[name]&.canonical&.name || name }.uniq
-        language_code: locale.language_code, tag_id: tns.values.map(&:tag_id),
-        primary_flg: true).pluck(:tag_id, :name).to_h
-
-    names.map { |name|
-      tag_name = tns[name]
-      if !tag_name || tag_name.primary?
-        name
-      else
-        primary_names[tag_name.tag_id] || name
-      end
-    }.uniq
-  end
-
-  def self.generate_name locale, tag, base_tag_name
-    fallback = "Tag_##{ tag.id }"
-    name =
-        case [locale.language_code, base_tag_name&.language_code]
-        when ['ain', 'ja']
-          TagNameTranscription.ainu(base_tag_name.name)
-        when ['ja', 'ain']
-          base_tag_name.name
-        else
-          return fallback
-        end
-    candidate = new(tag:, name:, language_code: locale.language_code,
-                    script_code: locale.script_code,
-                    primary_flg: true, auto_generated: true)
-    candidate.valid? ? name : fallback
-  rescue TagNameTranscription::Unconvertible
-    fallback
   end
 
   private
 
-  def generated_name_must_be_reserved
+  def canonical_must_be_canonical
-    if !auto_generated && name.to_s.start_with?('Tag_#')
+    if canonical&.canonical_id?
-      errors.add :name, 'タグの命名規則に反してゐます．'
+      errors.add :canonical, 'canonical は実体を示す必要があります．'
     end
   end
 
   def alias_name_must_not_have_prefix
-    if !(primary?) && name.to_s.include?(':')
+    if canonical_id? && name.to_s.include?(':')
       errors.add :name, 'エーリアス名にプレフィクスを含むことはできません．'
     end
   end
 
-  def name_must_be_sanitised
+  def canonical_must_not_be_present_with_tag_or_wiki_page
-    return if auto_generated && name == "Tag_##{ tag_id }"
+    if canonical_id? && (tag || wiki_page)
+      errors.add :canonical, 'タグもしくは Wiki の参照がある名前はエーリアスになれません．'
+    end
+  end
 
+  def name_must_be_sanitised
     if name? && name != TagNameSanitisationRule.sanitise(name)
       errors.add :name, '名前に使用できない文字が含まれてゐます．'
     end
   end
-
-  def name_mustnt_start_with_nico
-    if name.to_s.downcase.start_with?('nico:')
-      errors.add :name, 'タグの命名規則に反してゐます．'
-    end
-  end
 end

backend/app/models/tag_name_sanitisation_rule.rb

@@ -10,44 +10,36 @@ class TagNameSanitisationRule < ApplicationRecord
   validate :source_pattern_must_be_regexp
 
   class << self
-    def sanitise(name)
+    def sanitise(name) =
         rules.reduce(name.dup) { |name, (pattern, replacement)| name.gsub(pattern, replacement) }
-    end
 
     def apply!
-      Language.find_each do |language|
+      TagName.find_each do |tn|
-        TagName.where(language:).find_each do |candidate|
-          TagName.transaction do
-            tn = TagName.find_by(id: candidate.id)
-            next unless tn
-
         name = sanitise(tn.name)
         next if name == tn.name
 
-            existing_tn = TagName.find_by(language:, name:)
+        TagName.transaction do
-            source_tag = tn.tag
+          existing_tn = TagName.find_by(name:)
-            target_tag = existing_tn ? existing_tn.tag : source_tag
+          if existing_tn
-            [source_tag, target_tag].uniq(&:id).sort_by(&:id).each(&:lock!)
+            existing_tn = existing_tn.canonical || existing_tn
-            TagVersioning.ensure_snapshot!(source_tag, created_by_user: nil)
+            next if existing_tn.id == tn.id
 
-            if existing_tn && target_tag != source_tag
+            existing_tag = Tag.find_by(tag_name_id: existing_tn.id)
-              Tag.merge_tags!(target_tag, source_tag)
+            source_tag = Tag.find_by(tag_name_id: tn.id)
-              tn.reload.destroy!
+
-            elsif existing_tn
+            if existing_tag
-              if tn.primary?
+              Tag.merge_tags!(existing_tag, source_tag) if tn.tag
-                existing_tn.destroy!
+            elsif source_tag
-                tn.update_columns(name:, updated_at: Time.current)
+              source_tag.update_columns(tag_name_id: existing_tn.id, updated_at: Time.current)
-              else
-                tn.destroy!
             end
-            else
+            tn.discard!
+
+            next
+          end
+
           # TagName 側の自動サニタイズを回避
           tn.update_columns(name:, updated_at: Time.current)
         end
-
-            TagVersioning.record!(target_tag, event_type: :update, created_by_user: nil)
-          end
-        end
       end
     end
 

backend/app/models/tag_version.rb

@@ -11,6 +11,5 @@ class TagVersion < ApplicationRecord
                     meta: 'meta' }, validate: true
 
   validates :name, presence: true
-  validates :names, presence: true
   validates :category, presence: true
 end

backend/app/models/user.rb

@@ -7,16 +7,15 @@ class User < ApplicationRecord
 
   has_many :created_posts,
            class_name: 'Post', foreign_key: :uploaded_user_id, dependent: :nullify
-  has_one :setting, dependent: :destroy
+  has_many :settings
-  has_many :theme_slots, class_name: 'UserThemeSlot', dependent: :destroy
   has_many :user_ips, dependent: :destroy
   has_many :ip_addresses, through: :user_ips
   has_many :user_post_views, dependent: :destroy
   has_many :viewed_posts, through: :user_post_views, source: :post
-  has_many :created_tag_descriptions, class_name: 'TagDescription',
+  has_many :created_wiki_pages,
-           foreign_key: :created_by_user_id, dependent: :nullify
+           class_name: 'WikiPage', foreign_key: :created_user_id, dependent: :nullify
-  has_many :updated_tag_descriptions, class_name: 'TagDescription',
+  has_many :updated_wiki_pages,
-           foreign_key: :updated_by_user_id, dependent: :nullify
+           class_name: 'WikiPage', foreign_key: :updated_user_id, dependent: :nullify
 
   def viewed?(post) = user_post_views.exists?(post_id: post.id)
 

backend/app/models/user_theme_slot.rb

@@ -1,27 +0,0 @@
-class UserThemeSlot < ApplicationRecord
-  BASE_THEMES = ['light', 'dark'].freeze
-  SLOT_NOS = [1, 2, 3].freeze
-
-  belongs_to :user
-
-  validates :user_id, presence: true
-  validates :base_theme, presence: true, inclusion: { in: BASE_THEMES }
-  validates :slot_no, presence: true, inclusion: { in: SLOT_NOS }
-  validates :tokens, presence: true
-  validates :user_id, uniqueness: { scope: [:base_theme, :slot_no] }
-
-  validate :tokens_must_be_object
-
-  def serializable_hash(options = nil)
-    hash = { only: [:base_theme, :slot_no, :tokens, :created_at, :updated_at] }
-    super(hash.merge(options || {}))
-  end
-
-  private
-
-  def tokens_must_be_object
-    return if tokens.is_a?(Hash)
-
-    errors.add(:tokens, 'は JSON object で指定してください．')
-  end
-end

backend/app/models/version_record.rb

@@ -1,25 +1,15 @@
 module VersionRecord
   extend ActiveSupport::Concern
 
-  DEFAULT_EVENT_TYPE_MAP = { create: 'create',
-                             update: 'update',
-                             discard: 'discard',
-                             restore: 'restore' }.freeze
-
-  def require_history_access! = HistoryAccess.require!(self)
-
   def readonly? = persisted?
 
   included do
-    event_type_map = if const_defined?(:EVENT_TYPE_MAP, false)
-                       const_get(:EVENT_TYPE_MAP)
-                     else
-                       DEFAULT_EVENT_TYPE_MAP
-                     end
-
     belongs_to :created_by_user, class_name: 'User', optional: true
 
-    enum :event_type, event_type_map, prefix: true, validate: true
+    enum :event_type, { create: 'create',
+                        update: 'update',
+                        discard: 'discard',
+                        restore: 'restore' }, prefix: true, validate: true
 
     validates :version_no, presence: true, numericality: { only_integer: true, greater_than: 0 }
     validates :event_type, presence: true

backend/app/models/wiki_line.rb

@@ -0,0 +1,15 @@
+class WikiLine < ApplicationRecord
+  has_many :wiki_revision_lines, dependent: :restrict_with_exception
+
+  validates :sha256, presence: true, uniqueness: true, length: { is: 64 }
+  validates :body, presence: true
+
+  def self.upsert_by_body! body
+    sha = Digest::SHA256.hexdigest(body)
+    now = Time.current
+
+    upsert(sha256: sha, body:, created_at: now, updated_at: now)
+
+    find_by!(sha256: sha)
+  end
+end

backend/app/models/wiki_page.rb

@@ -0,0 +1,54 @@
+require 'set'
+
+
+class WikiPage < ApplicationRecord
+  include MyDiscard
+
+  has_many :wiki_revisions, dependent: :destroy
+  belongs_to :created_user, class_name: 'User'
+  belongs_to :updated_user, class_name: 'User'
+
+  has_many :redirected_from_revisions,
+           class_name: 'WikiRevision',
+           foreign_key: :redirect_page_id,
+           dependent: :nullify
+
+  has_many :wiki_versions
+
+  attribute :version_no, :integer, default: 1
+
+  belongs_to :tag_name
+  validates :tag_name, presence: true
+  validates :body, presence: true
+
+  def title = tag_name.name
+
+  def title= val
+    (self.tag_name ||= build_tag_name).name = val
+  end
+
+  def current_revision = wiki_revisions.order(id: :desc).first
+
+  def resolve_redirect limit: 10
+    page = self
+    visited = Set.new
+
+    limit.times do
+      return page if visited.include?(page.id)
+
+      visited.add(page.id)
+
+      rev = page.current_revision
+      return page if !(rev&.redirect?) || !(rev.redirect_page)
+
+      page = rev.redirect_page
+    end
+
+    page
+  end
+
+  def pred_revision_id(revision_id) =
+      wiki_revisions.where('id < ?', revision_id).order(id: :desc).limit(1).pick(:id)
+  def succ_revision_id(revision_id) =
+      wiki_revisions.where('id > ?', revision_id).order(id: :asc).limit(1).pick(:id)
+end

backend/app/models/wiki_revision.rb

@@ -0,0 +1,55 @@
+class WikiRevision < ApplicationRecord
+  belongs_to :wiki_page
+  belongs_to :base_revision, class_name: 'WikiRevision', optional: true
+  belongs_to :created_user, class_name: 'User'
+  belongs_to :redirect_page, class_name: 'WikiPage', optional: true
+
+  has_many :wiki_revision_lines, dependent: :delete_all
+  has_many :wiki_lines, through: :wiki_revision_lines
+
+  enum :kind, content: 0, redirect: 1
+
+  validates :kind, presence: true
+  validates :lines_count, numericality: { only_integer: true, greater_than_or_equal_to: 0 }
+  validates :tree_sha256, length: { is: 64 }, allow_nil: true
+
+  validate :kind_consistency
+
+  def body
+    return unless content?
+
+    wiki_revision_lines
+        .includes(:wiki_line)
+        .order(:position)
+        .map { |rev| rev.wiki_line.body }
+        .join("\n")
+  end
+
+  private
+
+  def kind_consistency
+    if content?
+      if tree_sha256.blank?
+        errors.add(:tree_sha256, '種類がページの場合は必須です．') 
+      end
+
+      if redirect_page_id.present?
+        errors.add(:redirect_page_id, '種類がページの場合は空である必要があります．')
+      end
+    end
+
+    if redirect?
+      if redirect_page_id.blank?
+        errors.add(:redirect_page_id, '種類がリダイレクトの場合は必須です．')
+      end
+
+      if tree_sha256.present?
+        errors.add(:tree_sha256, '種類がリダイレクトの場合は空である必要があります．')
+      end
+
+      if lines_count.to_i > 0
+        errors.add(:lines_count, '種類がリダイレクトの場合は 0 である必要があります．')
+      end
+    end
+  end
+end

backend/app/models/wiki_revision_line.rb

@@ -0,0 +1,8 @@
+class WikiRevisionLine < ApplicationRecord
+  belongs_to :wiki_revision
+  belongs_to :wiki_line
+
+  validates :position, presence: true,
+                       numericality: { only_integer: true, greater_than_or_equal_to: 0 }
+  validates :position, uniqueness: { scope: :wiki_revision_id }
+end

backend/app/models/wiki_version.rb

@@ -0,0 +1,8 @@
+class WikiVersion < ApplicationRecord
+  include VersionRecord
+
+  belongs_to :wiki_page
+
+  validates :title, presence: true
+  validates :body, presence: true
+end

backend/app/representations/counts_version_repr.rb

@@ -1,135 +0,0 @@
-# Counts uses recorded values only; history/diff endpoints retain their own contract.
-class CountsVersionRepr
-  def self.many(rows, resource:, language_code:, at: nil, inclusive: true)
-    new(rows.to_a, resource:, language_code:, at:, inclusive:).serialise
-  end
-
-  def initialize(rows, resource:, language_code:, at:, inclusive:)
-    @rows = rows
-    HistoryAccess.require_many!(rows)
-    @at = at
-    @inclusive = inclusive
-    @resource = resource
-    @language = language_code
-    @users = User.where(id: rows.map(&:created_by_user_id).compact).pluck(:id, :name).to_h
-    @languages = master_names(Language)
-    @scripts = master_names(Script)
-    @tag_versions = {}
-    load_tag_versions if resource == 'posts'
-    load_parent_versions if resource == 'tags'
-  end
-
-  def serialise
-    @rows.map do |row|
-      resource_id = row.public_send(@resource == 'posts' ? :post_id : :tag_id)
-      common = { id: row.id, resource_id:,
-                 version_no: row.version_no, is_latest: row.attributes['counts_latest'].to_i == 1,
-                 created_at: row.created_at.iso8601,
-                 created_by_user: row.created_by_user_id &&
-                     { id: row.created_by_user_id, name: @users[row.created_by_user_id] } }
-      common.merge(@resource == 'posts' ? post(row) : tag(row))
-    end
-  end
-
-  private
-
-  def master_names(model)
-    model.select('code AS counts_code', :name).to_h do |record|
-      [record.attributes['counts_code'], record.name]
-    end
-  end
-
-  def json(value)
-    value.is_a?(String) ? JSON.parse(value) : value
-  end
-
-  def load_tag_versions
-    pairs = @rows.flat_map { Array(json(_1.tags_json)) }
-        .filter_map { |entry| [entry['tag_id'], entry['version_no']] if entry['tag_id'] }
-        .uniq
-    pairs.each_slice(500) do |batch|
-      placeholders = batch.map { '(?, ?)' }.join(', ')
-      scope = HistoryAccess.visible(TagVersion.all)
-      if @at
-        scope = scope.where("created_at #{ @inclusive ? '<=' : '<' } ?", @at)
-      end
-      scope.where(["(tag_id, version_no) IN (#{ placeholders })", *batch.flatten])
-          .select(:tag_id, :version_no, :name, :category, :names).each do |version|
-        @tag_versions[[version.tag_id, version.version_no]] = version
-      end
-    end
-  end
-
-  def names(value)
-    (json(value) || {}).sort.map do |language, entry|
-      aliases = entry.fetch('aliases', []).map do |alias_entry|
-        alias_entry.merge('script_name' => @scripts[alias_entry['script_code']])
-      end
-      { language_code: language, language_name: @languages[language],
-        script_code: entry['script_code'], script_name: @scripts[entry['script_code']],
-        name: entry['name'], auto_generated: entry['auto_generated'], aliases: }
-    end
-  end
-
-  def post(row)
-    snapshots = Array(json(row.tags_json)).map do |entry|
-      version = @tag_versions[[entry['tag_id'], entry['version_no']]]
-      { id: entry['tag_id'] || entry['external_tag_id'],
-        kind: entry['tag_id'] ? 'internal' : 'external',
-        tag_id: entry['tag_id'], external_tag_id: entry['external_tag_id'],
-        version_no: entry['version_no'], name: entry['name'] || version&.name,
-        category: entry['category'] || version&.category,
-        sections: entry.fetch('sections', []), names: version ? names(version.names) : [] }
-    end
-    { title: row.title, url: row.url, thumbnail_base: row.thumbnail_base,
-      tags: snapshots, parents: row.parent_post_ids.to_s.split.map { { id: _1.to_i } },
-      video_ms: row.video_ms,
-      original_created_from: row.original_created_from&.iso8601,
-      original_created_before: row.original_created_before&.iso8601 }
-  end
-
-  def parent_point(row) = @at || row.created_at
-
-  def load_parent_versions
-    @parent_versions = {}
-    requests = @rows.flat_map do |row|
-      row.parent_tag_ids.to_s.split.map { [_1.to_i, parent_point(row)] }
-    end.uniq
-    requests.each_slice(100) do |batch|
-      queries = batch.map do |id, point|
-        scope = HistoryAccess.visible(TagVersion.where(tag_id: id))
-            .where('created_at <= ?', point)
-            .where('NOT EXISTS (SELECT 1 FROM tag_versions newer ' +
-                'WHERE newer.tag_id = tag_versions.tag_id AND newer.created_at <= ? ' +
-                'AND newer.version_no > tag_versions.version_no)', point)
-        quoted = TagVersion.connection.quote(point.iso8601(6))
-        scope.select("tag_id, name, names, #{ quoted } AS reference_point").to_sql
-      end
-      next if queries.empty?
-
-      TagVersion.connection.select_all(queries.map { "(#{ _1 })" }.join(' UNION ALL ')).each do |value|
-        @parent_versions[[value['tag_id'], value['reference_point']]] = value
-      end
-    end
-  end
-
-  def tag(row)
-    recorded_names = names(row.names)
-    selected = recorded_names.find { _1[:language_code] == @language }
-    aliases = recorded_names.flat_map do |entry|
-      entry[:aliases].map {
-        _1.merge('language_code' => entry[:language_code],
-                 'language_name' => entry[:language_name])
-      }
-    end
-    { name: selected&.fetch(:name) || row.name, names: recorded_names,
-      category: row.category, aliases:,
-      parents: row.parent_tag_ids.to_s.split.map do |id|
-        value = @parent_versions[[id.to_i, parent_point(row).iso8601(6)]]
-        snapshots = value && json(value['names'])
-        name = snapshots&.dig(@language, 'name') || value&.fetch('name')
-        { id: id.to_i, name: }
-      end,
-      deprecated: row.deprecated_at&.iso8601 }
-  end
-end

backend/app/representations/deerjikist_repr.rb

@@ -6,12 +6,11 @@ module DeerjikistRepr
 
   module_function
 
-  def base deerjikist, language_code: 'ja'
+  def base deerjikist
-    deerjikist.as_json(BASE).merge(
+    deerjikist.as_json(BASE)
-        tag: deerjikist.tag && TagRepr.base(deerjikist.tag, language_code:))
   end
 
-  def many deerjikists, language_code: 'ja'
+  def many deerjikists
-    deerjikists.map { |d| base(d, language_code:) }
+    deerjikists.map { |d| base(d) }
   end
 end

backend/app/representations/external_tag_repr.rb

@@ -1,20 +0,0 @@
-module ExternalTagRepr
-  module_function
-
-  def base tag
-    { 'id' => tag.id,
-      'name' => "#{ tag.platform }:#{ tag.name }",
-      'category' => 'nico',
-      'post_count' => tag.post_count,
-      'created_at' => tag.created_at,
-      'updated_at' => tag.created_at,
-      'deprecated_at' => nil,
-      'aliases' => [],
-      'parents' => [],
-      'has_wiki' => false,
-      'material_id' => nil,
-      'has_deerjikists' => false }
-  end
-
-  def inline(tag) = base(tag)
-end

backend/app/representations/material_repr.rb

@@ -2,9 +2,7 @@
 
 
 module MaterialRepr
-  BASE = { only: [:id, :url, :version_no, :source_kind, :source_uri,
+  BASE = { only: [:id, :url, :created_at, :updated_at],
-                  :source_path, :source_file_id, :normalized_source_key,
-                  :created_at, :updated_at],
            methods: [:content_type],
            include: { tag: TagRepr::BASE,
                       created_by_user: UserRepr::BASE,
@@ -12,99 +10,15 @@ module MaterialRepr
 
   module_function
 
-  def base material, host:, language_code: 'ja'
+  def base material, host:
     material.as_json(BASE).merge(
-        tag: material.tag && TagRepr.base(material.tag, language_code:),
         file: if material.file.attached?
                 Rails.application.routes.url_helpers.rails_storage_proxy_url(
                     material.file, host:)
-              end,
+              end)
-        thumbnail: thumbnail_url(material, host:),
-        thumbnail_fallback_text: thumbnail_fallback_text(material, language_code:),
-        thumbnail_fallback_kind: thumbnail_fallback_kind(material),
-        media_kind: media_kind(material),
-        file_byte_size: material.file_byte_size,
-        export_paths: export_paths(material),
-        export_items: export_items(material))
   end
 
-  def many materials, host:, language_code: 'ja'
+  def many materials, host:
-    materials.map { |m| base(m, host:, language_code:) }
+    materials.map { |m| base(m, host:) }
-  end
-
-  def list material, host:, language_code: 'ja'
-    { id: material.id,
-      version_no: material.version_no,
-      url: material.url,
-      source_kind: material.source_kind,
-      source_uri: material.source_uri,
-      source_path: material.source_path,
-      source_file_id: material.source_file_id,
-      normalized_source_key: material.normalized_source_key,
-      tag: compact_tag(material.tag, language_code:),
-      thumbnail: thumbnail_url(material, host:),
-      thumbnail_fallback_text: thumbnail_fallback_text(material, language_code:),
-      thumbnail_fallback_kind: thumbnail_fallback_kind(material),
-      media_kind: media_kind(material),
-      content_type: material.content_type,
-      file_byte_size: material.file_byte_size,
-      created_at: material.created_at,
-      updated_at: material.updated_at,
-      export_paths: export_paths(material),
-      export_items: export_items(material) }
-  end
-
-  def list_many materials, host:, language_code: 'ja'
-    materials.map { |m| list(m, host:, language_code:) }
-  end
-
-  def export_paths material
-    material.material_export_items.each_with_object({ }) do |item, hash|
-      hash[item.profile] = item.enabled ? item.export_path : ''
-    end
-  end
-
-  def export_items material
-    material.material_export_items.map do |item|
-      { id: item.id,
-        profile: item.profile,
-        export_path: item.export_path,
-        enabled: item.enabled }
-    end
-  end
-
-  def thumbnail_url material, host:
-    return nil unless material.thumbnail.attached?
-
-    Rails.application.routes.url_helpers.rails_storage_proxy_url(
-        material.thumbnail, host:)
-  end
-
-  def thumbnail_fallback_text material, language_code:
-    material.tag&.name(language_code) || material.created_at&.strftime('%Y-%m-%d')
-  end
-
-  def thumbnail_fallback_kind material
-    material.tag.present? ? 'tag_name' : 'created_at'
-  end
-
-  def media_kind material
-    return 'url_only' unless material.file.attached?
-
-    content_type = material.file.blob.content_type.to_s
-    return 'image' if content_type.start_with?('image/')
-    return 'video' if content_type.start_with?('video/')
-    return 'audio' if content_type.start_with?('audio/')
-
-    'file_other'
-  end
-
-  def compact_tag tag, language_code:
-    return nil unless tag
-
-    { id: tag.id,
-      name: tag.name(language_code),
-      category: tag.category,
-      deprecated_at: tag.deprecated_at }
   end
 end

backend/app/representations/post_compact_repr.rb

@@ -1,19 +0,0 @@
-# frozen_string_literal: true
-
-
-module PostCompactRepr
-  module_function
-
-  def base post, host: nil
-    return nil if post.nil?
-
-    PostRepr
-        .common(post, host:)
-        .slice(
-            'id',
-            'title',
-            'url',
-            'thumbnail',
-            'thumbnail_base')
-  end
-end

backend/app/representations/post_repr.rb

@@ -8,7 +8,6 @@ module PostRepr
       :url,
       :title,
       :thumbnail_base,
-      :video_ms,
       :original_created_from,
       :original_created_before,
       :created_at,
@@ -17,31 +16,17 @@ module PostRepr
 
   module_function
 
-  def base post, current_user = nil, host: nil, language_code: 'ja'
+  def base post, current_user = nil
-    json =
+    json = common(post)
-        if host.present?
+    json['tags'] = tag_json(post.tags)
-          common(post, host:)
-        else
-          common(post)
-        end
-    json['tags'] = tag_json(post, language_code:)
     json['uploaded_user'] = post.uploaded_user && UserRepr.base(post.uploaded_user)
     json['viewed'] = current_user ? current_user.viewed?(post) : false
     json
   end
 
   def detail post, current_user = nil, parent_posts: [], child_posts: [],
-             sibling_posts: { }, related: [], host: nil, language_code: 'ja'
+             sibling_posts: { }, related: []
-    if host.present?
+    base(post, current_user).merge(
-      base(post, current_user, host:, language_code:).merge(
-          'parent_posts' => cards(parent_posts, host:),
-          'child_posts' => cards(child_posts, host:),
-          'sibling_posts' => sibling_posts.transform_keys(&:to_s).transform_values { |posts|
-            cards(posts, host:)
-          },
-          'related' => cards(related, host:))
-    else
-      base(post, current_user, language_code:).merge(
         'parent_posts' => cards(parent_posts),
         'child_posts' => cards(child_posts),
         'sibling_posts' => sibling_posts.transform_keys(&:to_s).transform_values { |posts|
@@ -49,80 +34,33 @@ module PostRepr
         },
         'related' => cards(related))
   end
-  end
 
-  def card post, host: nil
+  def card post
-    if host.present?
-      common(post, host:).merge('parent_posts' => [], 'child_posts' => [])
-    else
     common(post).merge('parent_posts' => [], 'child_posts' => [])
   end
-  end
 
-  def cards posts, host: nil
+  def cards posts
-    if host.present?
-      posts.map { |post| card(post, host:) }
-    else
     posts.map { |post| card(post) }
   end
-  end
 
-  def many posts, current_user = nil, host: nil
+  def many posts, current_user = nil
-    if host.present?
-      posts.map { |p| base(p, current_user, host:) }
-    else
     posts.map { |p| base(p, current_user) }
   end
-  end
 
-  def common post, host: nil
+  def common post
     BASE_FIELDS.to_h { |field| [field.to_s, post.public_send(field)] }
-               .merge(
+               .merge('thumbnail' => thumbnail_url(post))
-                   'thumbnail' =>
-                       if host.present?
-                         thumbnail_url(post, host:)
-                       else
-                         thumbnail_url(post)
-                       end)
   end
 
-  def tag_json post, language_code:
+  def tag_json tags
-    internal_tags =
+    tags.map { |tag| TagRepr.inline(tag) }
-        post
-        .post_tags
-        .reject { _1.tag.deprecated? }
-        .sort_by { TagRepr.name_for(_1.tag, language_code) }
-        .map { |post_tag|
-          TagRepr.inline(post_tag.tag, language_code:).merge(
-              'children' => [],
-              'sections' => post_tag.sections.as_json(only: [:begin_ms, :end_ms]))
-        }
-
-    external_tags =
-        post
-        .external_tags
-        .sort_by { _1.name }
-        .map { ExternalTagRepr.base(_1).merge('children' => [], 'sections' => []) }
-
-    internal_tags + external_tags
   end
 
-  def thumbnail_url post, host: nil
+  def thumbnail_url post
     return nil unless post.thumbnail.attached?
 
-    options = { only_path: false }
+    Rails.application.routes.url_helpers.rails_blob_url(post.thumbnail, only_path: false)
-    options[:host] = host if host.present?
+  rescue
-
-    Rails.application.routes.url_helpers.rails_storage_proxy_url(post.thumbnail, **options)
-  rescue ActionController::UrlGenerationError, ArgumentError, URI::InvalidURIError => e
-    payload = {
-        post_id: post.id,
-        attachment_id: post.thumbnail.attachment&.id,
-        blob_id: post.thumbnail.blob&.id,
-        error_class: e.class,
-        message: e.message }
-
-    Rails.logger.warn("PostRepr.thumbnail_url failed #{ payload.to_json }")
     nil
   end
 end

backend/app/representations/tag_group_repr.rb

@@ -1,207 +0,0 @@
-module TagGroupRepr
-  PREFIX = 'タググループ:'
-
-  module_function
-
-  def name_from_title title
-    prefix = [PREFIX, 'tag_group:'].find { title.downcase.start_with?(_1) }
-    title[prefix.length..] if prefix
-  end
-
-  def many locale, groups
-    ids = groups.map(&:id)
-    return [] if ids.empty?
-
-    tag_members = TagGroupTag.where(tag_group_id: ids)
-        .group_by(&:tag_group_id)
-    group_members = TagGroupGroup.where(tag_group_id: ids)
-        .group_by(&:tag_group_id)
-    placements = TagGroupPlacement.where(tag_group_id: ids).order(:position)
-        .group_by(&:tag_group_id)
-
-    target_ids =
-        tag_members.values.flatten.map(&:tag_id) +
-        placements.values.flatten.filter_map(&:tag_id)
-    child_ids =
-        group_members.values.flatten.map(&:child_tag_group_id) +
-        placements.values.flatten.filter_map(&:child_tag_group_id)
-    tags = Tag.select(:id, :category).where(id: target_ids.uniq).index_by(&:id)
-    names =
-        TagName
-        .where(tag_id: target_ids.uniq,
-               language_code: locale.language_code,
-               primary_flg: true)
-        .pluck(:tag_id, :name)
-        .to_h
-    child_groups = TagGroup.where(id: child_ids.uniq).index_by(&:id)
-
-    groups.map do |group|
-      group_json(group,
-                 tag_members.fetch(group.id, []),
-                 group_members.fetch(group.id, []),
-                 placements.fetch(group.id, []), tags, names, child_groups)
-    end
-  end
-
-  def one locale, group
-    HistoryAccess.require_current!(group, TagGroupVersion)
-    many(locale, [group]).first
-  end
-
-  def tag_group wiki_page
-    { 'id' => wiki_page.id,
-      'title' => "#{ PREFIX }#{ wiki_page.name }",
-      'deprecated_at' => nil,
-      'created_at' => wiki_page.created_at,
-      'updated_at' => wiki_page.updated_at }
-  end
-
-  def current_version group
-    HistoryAccess.visible(TagGroupVersion.all).find_by(tag_group_id: group.id, version_no: group.version_no)
-  end
-
-  def historical locale, version
-    HistoryAccess.require!(version)
-    members = Array(version.members)
-    placements = Array(version.placements)
-    tag_references = members.filter_map { |member| member['tag_id'] } +
-        collect_ids(placements, 'tag_id')
-    group_references = members.filter_map { |member| member['tag_group_id'] } +
-        collect_ids(placements, 'tag_group_id')
-    tag_versions = historical_tag_versions(tag_references)
-    group_versions = historical_group_versions(group_references)
-    tag_version_numbers = members.filter_map do |member|
-      next unless member['tag_id']
-
-      [member.fetch('tag_id'), member.fetch('version_no')]
-    end.to_h
-    group_version_numbers = members.filter_map do |member|
-      next unless member['tag_group_id']
-
-      [member.fetch('tag_group_id'), member.fetch('version_no')]
-    end.to_h
-    historical_members = members.map do |member|
-      historical_member(locale, member, tag_versions, group_versions)
-    end
-    historical_placements = placements.map do |node|
-      historical_placement(locale, node, tag_versions, group_versions,
-                           tag_version_numbers, group_version_numbers)
-    end
-    { 'id' => version.tag_group_id,
-      'name' => version.name,
-      'description' => version.description,
-      'version_no' => version.version_no,
-      'members' => historical_members,
-      'placements' => historical_placements,
-      'created_at' => version.created_at,
-      'updated_at' => version.created_at,
-      'created_by_user' => user_json(version.created_by_user),
-      'updated_by_user' => user_json(version.created_by_user) }
-  end
-
-  def group_json group, tag_members, group_members, rows, tags, names, child_groups
-    tree = placement_tree(rows, tags, names, child_groups)
-    members = tag_members.sort_by(&:tag_id).map do |member|
-      { 'tag' => tag_json(tags.fetch(member.tag_id), names[member.tag_id]) }
-    end
-    members.concat(group_members.sort_by(&:child_tag_group_id).map do |member|
-      { 'tag_group' => group_json_reference(child_groups.fetch(member.child_tag_group_id)) }
-    end)
-    { 'id' => group.id,
-      'name' => group.name,
-      'description' => group.description,
-      'version_no' => group.version_no,
-      'members' => members,
-      'placements' => tree,
-      'created_at' => group.created_at,
-      'updated_at' => group.updated_at,
-      'created_by_user' => user_json(group.created_by_user),
-      'updated_by_user' => user_json(group.updated_by_user) }
-  end
-
-  def user_json user
-    UserRepr.base(user) if user
-  end
-
-  def placement_tree rows, tags, names, child_groups
-    by_parent = rows.group_by(&:parent_position)
-    build = lambda do |parent|
-      by_parent.fetch(parent, []).sort_by(&:position).map do |row|
-        node = if row.tag_id
-          { 'tag' => tag_json(tags.fetch(row.tag_id), names[row.tag_id]) }
-        elsif row.child_tag_group_id
-          { 'tag_group' => group_json_reference(child_groups.fetch(row.child_tag_group_id)) }
-        else
-          { 'section_name' => row.section_name, 'description' => row.description }
-        end
-        node.merge('children' => build.call(row.position))
-      end
-    end
-    build.call(nil)
-  end
-
-  def tag_json tag, name
-    { 'id' => tag.id, 'name' => name, 'category' => tag.category }
-  end
-
-  def group_json_reference group
-    { 'id' => group.id, 'name' => group.name }
-  end
-
-  def collect_ids nodes, key
-    nodes.flat_map do |node|
-      [node[key], *collect_ids(Array(node['children']), key)].compact
-    end
-  end
-
-  def historical_tag_versions ids
-    HistoryAccess.visible(TagVersion.all).where(tag_id: ids.uniq).to_a.index_by do |version|
-      [version.tag_id, version.version_no]
-    end
-  end
-
-  def historical_group_versions ids
-    HistoryAccess.visible(TagGroupVersion.all).where(tag_group_id: ids.uniq).to_a.index_by do |version|
-      [version.tag_group_id, version.version_no]
-    end
-  end
-
-  def historical_member locale, member, tag_versions, group_versions
-    if member['tag_id']
-      id = member.fetch('tag_id')
-      version = tag_versions.fetch([id, member.fetch('version_no')])
-      { 'tag' => { 'id' => id, 'name' => version_name(locale, version),
-                   'category' => version.category } }
-    else
-      id = member.fetch('tag_group_id')
-      version = group_versions.fetch([id, member.fetch('version_no')])
-      { 'tag_group' => { 'id' => id, 'name' => version.name } }
-    end
-  end
-
-  def historical_placement locale, node, tag_versions, group_versions,
-                           tag_version_numbers, group_version_numbers
-    result = if node['tag_id']
-      id = node.fetch('tag_id')
-      version_no = tag_version_numbers.fetch(id)
-      version = tag_versions.fetch([id, version_no])
-      { 'tag' => { 'id' => id, 'name' => version_name(locale, version),
-                   'category' => version.category } }
-    elsif node['tag_group_id']
-      id = node.fetch('tag_group_id')
-      version_no = group_version_numbers.fetch(id)
-      version = group_versions.fetch([id, version_no])
-      { 'tag_group' => { 'id' => id, 'name' => version.name } }
-    else
-      { 'section_name' => node.fetch('section_name'), 'description' => node['description'] }
-    end
-    result.merge('children' => Array(node['children']).map do |child|
-      historical_placement(locale, child, tag_versions, group_versions,
-                           tag_version_numbers, group_version_numbers)
-    end)
-  end
-
-  def version_name locale, version
-    version.names[locale.language_code]&.fetch('name')
-  end
-end

backend/app/representations/tag_repr.rb

@@ -2,31 +2,19 @@
 
 
 module TagRepr
-  BASE = { only: [:id, :category, :post_count, :created_at, :updated_at, :deprecated_at],
+  BASE = { only: [:id, :category, :post_count, :created_at, :updated_at],
-           methods: [:material_id, :has_deerjikists] }.freeze
+           methods: [:name, :has_wiki, :material_id, :has_deerjikists] }.freeze
 
   module_function
 
-  def base tag, language_code: 'ja'
+  def base tag
-    tag.as_json(BASE).merge('has_wiki' => tag.has_description?(language_code:),
+    tag.as_json(BASE).merge(aliases: tag.snapshot_aliases,
-                            name: name_for(tag, language_code),
+                            parents: tag.parents.map { _1.as_json(BASE) })
-                            aliases: tag.snapshot_aliases(language_code),
-                            parents: tag.parents.map { inline(_1, language_code:) })
   end
 
-  def inline tag, language_code: 'ja'
+  def inline tag
-    tag.as_json(BASE).merge('has_wiki' => tag.has_description?(language_code:),
+    tag.as_json(BASE).merge(aliases: [], parents: [])
-                            name: name_for(tag, language_code), aliases: [], parents: [])
   end
 
-  def many tags, language_code: 'ja'
+  def many(tags) = tags.map { |t| base(t) }
-    records = tags.to_a
-    ActiveRecord::Associations::Preloader.new(
-        records:, associations: [:tag_descriptions, { parents: :tag_descriptions }]).call
-    records.map { |tag| base(tag, language_code:) }
-  end
-
-  def name_for tag, language_code
-    tag.name(language_code) || (tag.system? ? tag.name('ja') : nil)
-  end
 end

backend/app/representations/wiki_page_repr.rb

@@ -2,49 +2,15 @@
 
 
 module WikiPageRepr
+  BASE = { methods: [:title] }.freeze
+
   module_function
 
-  def tag_description locale, description
+  def base wiki_page
-    HistoryAccess.require_current!(description, TagDescriptionVersion)
+    wiki_page.as_json(BASE)
-    tag = description.tag
-
-    { 'id' => description.id,
-      'resource_type' => 'tag_description',
-      'tag_id' => tag.id,
-      'language_code' => description.language_code,
-      'tag_name_id' => tag.tag_name(locale.language_code)&.id,
-      'body' => description.body,
-      'created_user_id' => description.created_by_user_id,
-      'updated_user_id' => description.updated_by_user_id,
-      'created_at' => description.created_at,
-      'updated_at' => description.updated_at,
-      'discarded_at' => nil,
-      'version_no' => description.version_no,
-      'title' => tag.name(locale.language_code),
-      'deprecated_at' => tag.deprecated_at }
   end
 
-  def tag_group tag_group
+  def many wiki_pages
-    TagGroupRepr.tag_group(tag_group).merge('resource_type' => 'tag_group')
+    wiki_pages.map { |p| base(p) }
-  end
-
-  def autolink_tag locale, tag_id:, description_id:, name:, language:
-    { resource_type: 'tag_description',
-      resource_id: description_id,
-      tag_id:,
-      tag_description_id: description_id,
-      name:,
-      language_code: language,
-      locale_code: locale.code_before_type_cast }
-  end
-
-  def autolink_group locale, id:, name:, language:
-    { resource_type: 'tag_group',
-      resource_id: id,
-      tag_id: nil,
-      tag_description_id: nil,
-      name: "#{ TagGroupRepr::PREFIX }#{ name }",
-      language_code: language,
-      locale_code: locale.code_before_type_cast }
   end
 end

backend/app/services/certified_tag_name_translations.rb

@@ -1,105 +0,0 @@
-require 'json'
-require 'pathname'
-
-
-class CertifiedTagNameTranslations
-  FORMAT = 'btrc-hub.certified-tag-name-translations.v1'.freeze
-
-  class Error < ArgumentError
-    ;
-  end
-
-  def self.load! path, locale:
-    new(path, locale:).load!
-  end
-
-  def initialize path, locale:
-    @path = path
-    @locale = locale
-  end
-
-  def load!
-    document = read_document
-    unless document.is_a?(Hash) && document['format'] == FORMAT
-      fail!('invalid format or version')
-    end
-
-    source_code = document['source_locale']
-    target_code = document['target_locale']
-    unless target_code.is_a?(String) && target_code == Locale.codes.fetch(@locale.code)
-      fail!("target_locale #{ target_code.inspect } does not match the new Locale")
-    end
-    unless source_code.is_a?(String) && Locale.codes.value?(source_code)
-      fail!("invalid source_locale #{ source_code.inspect }")
-    end
-    source_locale = Locale.find_by(code: source_code)
-    fail!("source Locale #{ source_code.inspect } does not exist") unless source_locale
-    fail!('source_locale and target_locale must differ') if source_code == target_code
-
-    entries = validate_entries(document['entries'], source_code, target_code)
-    resolve_tags(entries, source_locale.language_code)
-  end
-
-  private
-
-  def read_document
-    unless (@path.is_a?(String) && !@path.include?("\0") &&
-            Pathname.new(@path).absolute?)
-      fail!('certified_tag_names_path must be an absolute file path')
-    end
-    fail!('JSON file does not exist or is not a regular file') unless File.file?(@path)
-
-    text = File.read(@path, encoding: Encoding::UTF_8)
-    fail!('JSON file is not valid UTF-8') unless text.valid_encoding?
-    JSON.parse(text)
-  rescue JSON::ParserError, SystemCallError, IOError, EncodingError => error
-    fail!("cannot read JSON: #{ error.message }")
-  end
-
-  def validate_entries entries, source_code, target_code
-    fail!('entries must be an array') unless entries.is_a?(Array)
-
-    sources = {}
-    targets = {}
-    entries.each_with_index.map do |entry, index|
-      fail!("entry #{ index + 1 } must be an object") unless entry.is_a?(Hash)
-      source = entry[source_code]
-      target = entry[target_code]
-      unless source.is_a?(String) && source.present? && target.is_a?(String) && target.present?
-        fail!("entry #{ index + 1 }: missing or empty #{ source_code }/#{ target_code } name")
-      end
-      fail!("duplicate source name #{ source.inspect }") if sources.key?(source)
-      fail!("duplicate target name #{ target.inspect }") if targets.key?(target)
-
-      sources[source] = true
-      targets[target] = true
-      [source, target]
-    end
-  end
-
-  def resolve_tags entries, source_language
-    source_names = entries.map(&:first)
-    # SQL may use a case/accent-insensitive collation; the Ruby keys are exact.
-    matches = TagName.where(language_code: source_language, primary_flg: true,
-                            name: source_names).pluck(:name, :tag_id).group_by(&:first)
-    translations = {}
-    entries.each do |source, target|
-      rows = matches.fetch(source, [])
-      next if rows.empty?
-
-      unless rows.length == 1
-        fail!("source primary name #{ source.inspect } must match exactly one tag")
-      end
-      tag_id = rows.first.last
-      if translations.key?(tag_id)
-        fail!("multiple source primary names resolve to Tag##{ tag_id }: #{ source.inspect }")
-      end
-      translations[tag_id] = target
-    end
-    translations
-  end
-
-  def fail! message
-    raise Error, "Certified tag names #{@path.inspect}: #{ message }"
-  end
-end

backend/app/services/counts_axes.rb

@@ -1,100 +0,0 @@
-class CountsAxes
-  GRAINS = ['year', 'quarter', 'month', 'week', 'day', 'hour'].freeze
-
-  def self.time_expression(expression, grain, timezone)
-    raise ArgumentError unless GRAINS.include?(grain)
-
-    connection = ApplicationRecord.connection
-    # Use the installed TZInfo database, including DST, rather than MySQL timezone tables.
-    first = Time.utc(1900)
-    last = Time.utc(2101)
-    offset = timezone.period_for_utc(first).utc_total_offset
-    clauses = timezone.transitions_up_to(last, first).filter_map do |transition|
-      next if transition.offset.utc_total_offset == offset
-
-      previous = offset
-      offset = transition.offset.utc_total_offset
-      "WHEN #{ expression } < #{ connection.quote(transition.at.to_time) } THEN #{ previous }"
-    end
-    seconds = clauses.empty? ? offset.to_s : "CASE #{ clauses.join(' ') } ELSE #{ offset } END"
-    local = "DATE_ADD(#{ expression }, INTERVAL (#{ seconds }) SECOND)"
-    case grain
-    when 'year' then "DATE_FORMAT(#{ local }, '%Y')"
-    when 'quarter' then "CONCAT(YEAR(#{ local }), '-Q', QUARTER(#{ local }))"
-    when 'month' then "DATE_FORMAT(#{ local }, '%Y-%m')"
-    when 'week' then "DATE_FORMAT(#{ local }, '%x-W%v')"
-    when 'day' then "DATE_FORMAT(#{ local }, '%Y-%m-%d')"
-    else "DATE_FORMAT(#{ local }, '%Y-%m-%d %H:00')"
-    end
-  end
-
-  attr_reader :rows, :columns, :all
-
-  def initialize(population)
-    @population = population
-    @definition = population.definition
-    @rows = parse(population.params.fetch('rows', ''))
-    @columns = parse(population.params.fetch('cols', ''))
-    @all = rows + columns
-    raise ArgumentError if all.length > 8 || all.map { _1['field'] }.uniq.length != all.length
-  end
-
-  def expressions
-    @expressions ||= all.map do |axis|
-      field = axis.fetch('field')
-      case field
-      when 'tags'
-        "CASE WHEN tag_entry.tag_id IS NOT NULL " +
-            "THEN CONCAT('internal:', tag_entry.tag_id) " +
-            "ELSE CONCAT('external:', tag_entry.external_tag_id) END"
-      when 'changed_field' then 'changed.field_key'
-      else
-        expression = @definition.expression(field)
-        @definition.fields[field] == :datetime ? self.class.time_expression(
-            expression, axis.fetch('grain', 'month'), @population.timezone) : expression
-      end
-    end
-  end
-
-  def joins(fields = all.map { _1['field'] })
-    result = ''
-    if fields.include?('tags')
-      result += <<~SQL.squish
-         JOIN JSON_TABLE(CAST(v.tags_json AS JSON), '$[*]' COLUMNS (
-          tag_id BIGINT PATH '$.tag_id' NULL ON EMPTY,
-          external_tag_id BIGINT PATH '$.external_tag_id' NULL ON EMPTY)) tag_entry
-
-      SQL
-    end
-    if fields.include?('changed_field')
-      fields = @definition.changes.map do |field|
-        "SELECT #{ ApplicationRecord.connection.quote(field) } AS field_key"
-      end.join(' UNION ALL ')
-      condition = @definition.changes.map do |field|
-        "(changed.field_key = '#{ field }' AND #{ @population.changed_sql(field) })"
-      end.join(' OR ')
-      result += " JOIN (#{ fields }) changed ON (#{ condition })"
-    end
-    result
-  end
-
-  private
-
-  def parse(value)
-    axes = QueryState.axes(value)
-    raise ArgumentError unless axes.is_a?(Array)
-
-    axes.each do |axis|
-      raise ArgumentError unless axis.is_a?(Hash) && (axis.keys - ['field', 'grain']).empty?
-      raise ArgumentError unless @definition.axes(@population.history).include?(axis['field'])
-      if @definition.fields[axis['field']] == :datetime
-        raise ArgumentError unless GRAINS.include?(axis.fetch('grain', 'month'))
-
-        axis['grain'] ||= 'month'
-      elsif axis.key?('grain')
-        raise ArgumentError
-      end
-    end
-    axes
-  end
-end

backend/app/services/counts_condition.rb

@@ -1,330 +0,0 @@
-# Typed predicates over the same recorded fields used by Counts output.
-# Multi-valued comparisons use EXISTS; inequality means no matching element.
-class CountsCondition
-  CHILD_TYPES = { 'id' => :integer, 'versionNo' => :integer, 'tagId' => :integer,
-                  'tagGroupId' => :integer, 'autoGenerated' => :boolean,
-                  'category' => :enum, 'kind' => :enum, 'sections' => :interval }.freeze
-
-  def self.fields(definition)
-    result = definition.metadata.map { _1.merge(labels: [definition.field_label(_1[:key])]) }
-    visit = lambda do |fields, prefix, labels|
-      fields.each do |field|
-        key = [prefix, field[:key]].compact.join('.')
-        path_labels = labels + [field[:label]]
-        type = if prefix
-                 CHILD_TYPES.fetch(field[:key], :string)
-               elsif key == 'is_latest'
-                 :boolean
-               elsif key == 'parents'
-                 :integer
-               elsif key == 'deprecated'
-                 :datetime
-               elsif key == 'thumbnail'
-                 :string
-               else
-                 definition.fields.fetch(key, :string)
-               end
-        type = :string if field[:children] || type == :json
-        type = :integer if ['parents', 'members'].include?(key)
-        physical = result.find { _1[:key] == key }
-        nullable = prefix.present? || (physical ? physical[:nullable] : key != 'is_latest')
-        nullable = false if field[:children]
-        entry = { key:, type:, labels: path_labels, nullable:,
-                  multiple: prefix.present? || field[:children].present?,
-                  collection: field[:children].present? || type == :interval,
-                  operators: CountsResourceDefinition::OPERATORS.fetch(type) }
-        result.reject! { _1[:key] == key }
-        result << entry
-        visit.call(field[:children] || [], key, path_labels)
-      end
-    end
-    visit.call(definition.output_fields, nil, [])
-    # The raw JSON alias has the same membership semantics as the public tag field.
-    if definition.resource == 'posts'
-      result.find { _1[:key] == 'tags_json' }.merge!(type: :string, multiple: true, collection: true,
-          operators: CountsResourceDefinition::OPERATORS.fetch(:string))
-    end
-    result + [{ key: 'user.id', type: :integer, nullable: true,
-                labels: ['history:actor', 'counts:columns.resourceId'],
-                operators: CountsResourceDefinition::OPERATORS.fetch(:integer) },
-              { key: 'user.name', type: :string, nullable: true,
-                labels: ['history:actor', 'history:name'],
-                operators: CountsResourceDefinition::OPERATORS.fetch(:string) }]
-  end
-
-  def initialize(population)
-    @population = population
-    @definition = population.definition
-    @connection = @definition.model.connection
-    @language = Locale.find_by!(code: population.params.fetch('locale', 'ja')).language_code
-  end
-
-  def apply(scope, entry)
-    raise ArgumentError unless entry.is_a?(Hash)
-    raise ArgumentError if (entry.keys - ['field', 'operator', 'value', 'end', 'blank']).any?
-
-    field = @definition.filter_metadata.find { _1[:key] == entry.fetch('field') }
-    raise ArgumentError unless field && field[:operators].include?(entry.fetch('operator'))
-    raise ArgumentError unless [true, false, nil].include?(entry['blank'])
-    values = [entry['value'], entry['end']]
-    raise ArgumentError unless values.all? { _1.nil? || (_1.is_a?(String) && _1.bytesize <= 8192) }
-    if entry['operator'] != 'range' && !values[1].nil? && values[1] != ''
-      raise ArgumentError
-    end
-
-    source, expression = source_for(field[:key])
-    type = field[:type]
-    blank = blank_sql(expression, type)
-    if entry['blank']
-      sql = if source && field[:collection]
-              "NOT EXISTS (SELECT 1 FROM #{ source })"
-            elsif source
-              "NOT EXISTS (SELECT 1 FROM #{ source } WHERE NOT (#{ blank }))"
-            else blank
-            end
-      return scope.where(Arel.sql(sql))
-    end
-    return scope if values.all? { _1.nil? || _1 == '' }
-
-    parsed = values.map { parse(_1, type) }
-    validate_enum!(field[:key], parsed.first) if type == :enum
-    operator = entry['operator']
-    if type == :interval
-      return interval(scope, source, expression, parsed)
-    end
-    clauses = []
-    binds = []
-    if operator == 'range'
-      ['>=', '<='].each_with_index do |comparison, index|
-        next if parsed[index].nil?
-
-        clauses << "#{ expression } #{ comparison } ?"
-        binds << parsed[index]
-      end
-      raise ArgumentError if parsed.all? && parsed.first > parsed.last
-    elsif !parsed.first.nil?
-      comparison = { 'eq' => '=', 'ne' => '=', 'gt' => '>', 'lt' => '<',
-                     'ge' => '>=', 'le' => '<=', 'like' => 'LIKE' }.fetch(operator)
-      clauses << "#{ expression } #{ comparison } ?"
-      binds << parsed.first
-    end
-    return scope if clauses.empty?
-
-    predicate = clauses.join(' AND ')
-    if source
-      predicate = "EXISTS (SELECT 1 FROM #{ source } WHERE #{ predicate })"
-      predicate = "NOT (#{ predicate })" if operator == 'ne'
-    elsif operator == 'ne'
-      predicate = "(NOT (#{ predicate }) OR #{ blank })"
-    end
-    scope.where(predicate, *binds)
-  end
-
-  private
-
-  def parse(value, type)
-    return nil if value.nil? || value == ''
-
-    case type
-    when :integer, :interval
-      raise ArgumentError unless value.match?(/\A-?[0-9]{1,19}\z/)
-
-      number = Integer(value, 10)
-      raise ArgumentError unless number.between?(-9_223_372_036_854_775_808, 9_223_372_036_854_775_807)
-
-      number
-    when :boolean
-      raise ArgumentError unless ['true', 'false'].include?(value)
-
-      value == 'true' ? 1 : 0
-    when :datetime then @population.timestamp(value)
-    else value
-    end
-  end
-
-  def validate_enum!(key, value)
-    allowed = case key.split('.').last
-              when 'event_type' then VersionRecord::DEFAULT_EVENT_TYPE_MAP.values
-              when 'category', 'tag_category' then TagVersion.categories.keys
-              when 'kind' then ['internal', 'external']
-              when 'source_kind' then Material::SOURCE_KINDS
-              end
-    raise ArgumentError if value && allowed && !allowed.include?(value)
-  end
-
-  def blank_sql(expression, type)
-    if [:string, :enum].include?(type)
-      "(#{ expression } IS NULL OR #{ expression } = '')"
-    else
-      "#{ expression } IS NULL"
-    end
-  end
-
-  def scalar(document, key, type = :string)
-    path = @connection.quote("$.#{ key }")
-    value = "JSON_EXTRACT(#{ document }, #{ path })"
-    text = "JSON_UNQUOTE(NULLIF(#{ value }, CAST('null' AS JSON)))"
-    return "CAST(#{ text } AS SIGNED)" if type == :integer
-    return "CASE #{ text } WHEN 'true' THEN 1 WHEN 'false' THEN 0 END" if type == :boolean
-
-    text
-  end
-
-  def json_table(document, path, name)
-    "JSON_TABLE(COALESCE(#{ document }, JSON_ARRAY()), #{ @connection.quote(path) } " +
-        "COLUMNS (item JSON PATH '$')) #{ name }"
-  end
-
-  def names_source(document, alias_name)
-    source = "JSON_TABLE(COALESCE(JSON_KEYS(#{ document }), JSON_ARRAY()), '$[*]' " +
-        "COLUMNS (language_code TEXT PATH '$')) #{ alias_name }"
-    path = "CONCAT('$.', JSON_QUOTE(#{ alias_name }.language_code))"
-    [source, "JSON_EXTRACT(#{ document }, #{ path })", "#{ alias_name }.language_code"]
-  end
-
-  def point
-    @point ||= @population.history ? @population.at :
-        @population.timestamp(@population.params['at']) || @population.at
-  end
-
-  def name_expression(row)
-    language_path = @connection.quote("$.#{ @language.to_json }.name")
-    name = "JSON_UNQUOTE(JSON_EXTRACT(#{ row }.names, #{ language_path }))"
-    "COALESCE(#{ name }, #{ row }.name)"
-  end
-
-  def parent_source
-    column = @definition.resource == 'posts' ? 'parent_post_ids' : 'parent_tag_ids'
-    ids = "TRIM(REGEXP_REPLACE(COALESCE(v.#{ column }, ''), '[[:space:]]+', ' '))"
-    document = "CONCAT('[', REPLACE(#{ ids }, ' ', ','), ']')"
-    source = json_table(document, '$[*]', 'parent_entry')
-    id = 'CAST(parent_entry.item AS UNSIGNED)'
-    return [source, id] if @definition.resource == 'posts'
-
-    bound = @connection.quote(point)
-    source += ' LEFT JOIN tag_versions parent_version ON parent_version.tag_id = ' + id +
-        " AND parent_version.created_at <= #{ bound } AND " +
-        HistoryAccess.sql(TagVersion, 'parent_version') +
-        ' AND NOT EXISTS (SELECT 1 FROM tag_versions newer_parent ' +
-        'WHERE newer_parent.tag_id = parent_version.tag_id ' +
-        "AND newer_parent.created_at <= #{ bound } " +
-        'AND newer_parent.version_no > parent_version.version_no)'
-    [source, id]
-  end
-
-  def post_tag_source
-    source = json_table('v.tags_json', '$[*]', 'tag_snapshot')
-    source += ' LEFT JOIN tag_versions referenced_tag ON referenced_tag.tag_id = ' +
-        scalar('tag_snapshot.item', 'tag_id', :integer) +
-        ' AND referenced_tag.version_no = ' + scalar('tag_snapshot.item', 'version_no', :integer) +
-        " AND referenced_tag.created_at <= #{ @connection.quote(point) } AND " +
-        HistoryAccess.sql(TagVersion, 'referenced_tag')
-    source
-  end
-
-  def language_field(source, document, parts, inherited_language = nil)
-    return [source, scalar(document, 'name')] if parts.empty?
-    return [source, inherited_language] if parts == ['languageCode'] && inherited_language
-
-    if parts.first == 'aliases'
-      aliases = "JSON_EXTRACT(#{ document }, '$.aliases')"
-      source += ' JOIN ' + json_table(aliases, '$[*]', 'alias_entry')
-      child = parts[1] || 'name'
-      return [source, inherited_language] if child == 'languageCode' && inherited_language
-      columns = { 'scriptCode' => 'script_code', 'name' => 'name' }
-      key = columns.fetch(child) { raise ArgumentError }
-      return [source, scalar('alias_entry.item', key)]
-    end
-    key = { 'name' => 'name', 'scriptCode' => 'script_code',
-            'autoGenerated' => 'auto_generated' }.fetch(parts.first) { raise ArgumentError }
-    [source, scalar(document, key, CHILD_TYPES.fetch(parts.first, :string))]
-  end
-
-  def source_for(key)
-    root, *parts = key.split('.')
-    return [nil, "(#{ @population.latest_sql })"] if root == 'is_latest'
-    if root == 'user' && parts.any?
-      expression = parts.first == 'id' ? 'v.created_by_user_id' :
-          '(SELECT actor.name FROM users actor WHERE actor.id = v.created_by_user_id)'
-      return [nil, expression]
-    end
-    return [nil, 'v.thumbnail_base'] if root == 'thumbnail'
-    return [nil, 'v.deprecated_at'] if root == 'deprecated'
-    if root == 'parents'
-      source, id = parent_source
-      return [source, parts.first == 'name' ? name_expression('parent_version') : id]
-    end
-    if root == 'tags' || root == 'tags_json'
-      return post_field(parts)
-    end
-    if root == 'names' || root == 'aliases'
-      source, document, language = names_source('v.names', 'name_language')
-      parts = ['aliases', *parts] if root == 'aliases'
-      return language_field(source, document, parts, language)
-    end
-    if root == 'members' || root == 'placements'
-      path = '$[*]' + parts.take_while { _1 == 'children' }.map { '.children[*]' }.join
-      source = json_table("v.#{ root }", path, 'structure_entry')
-      if root == 'members' && parts.empty?
-        value = "COALESCE(#{ scalar('structure_entry.item', 'tag_id', :integer) }, " +
-            "#{ scalar('structure_entry.item', 'tag_group_id', :integer) })"
-        return [source, value]
-      end
-      leaf = parts.last || 'sectionName'
-      leaf = 'sectionName' if leaf == 'children'
-      column = leaf.gsub(/[A-Z]/) { "_#{ _1.downcase }" }
-      return [source, scalar('structure_entry.item', column, CHILD_TYPES.fetch(leaf, :string))]
-    end
-    if root == 'export_paths_json'
-      source, value, profile = names_source('v.export_paths_json', 'export_profile')
-      return [source, parts.first == 'profile' ? profile : "JSON_UNQUOTE(#{ value })"]
-    end
-    return [nil, name_expression('v')] if root == 'name' && @definition.resource == 'tags'
-
-    [nil, @definition.expression(root)]
-  end
-
-  def post_field(parts)
-    source = post_tag_source
-    document = 'tag_snapshot.item'
-    case parts.first
-    when 'names'
-      names, entry, language = names_source('referenced_tag.names', 'tag_language')
-      language_field("#{ source } JOIN #{ names }", entry, parts.drop(1), language)
-    when 'sections'
-      sections = "JSON_EXTRACT(#{ document }, '$.sections')"
-      source += ' JOIN ' + json_table(sections, '$[*]', 'section_entry')
-      [source, 'section_entry.item']
-    when 'id'
-      [source, "COALESCE(#{ scalar(document, 'tag_id', :integer) }, " +
-          "#{ scalar(document, 'external_tag_id', :integer) })"]
-    when 'versionNo' then [source, scalar(document, 'version_no', :integer)]
-    when 'kind'
-      [source, "CASE WHEN #{ scalar(document, 'tag_id') } IS NULL " +
-          "THEN 'external' ELSE 'internal' END"]
-    when 'category'
-      [source, "COALESCE(#{ scalar(document, 'category') }, referenced_tag.category)"]
-    else
-      [source, "COALESCE(#{ scalar(document, 'name') }, referenced_tag.name)"]
-    end
-  end
-
-  def interval(scope, source, document, bounds)
-    raise ArgumentError if bounds.compact.any?(&:negative?)
-    raise ArgumentError if bounds.all? && bounds.first > bounds.last
-
-    first = scalar(document, 'begin_ms', :integer)
-    last = scalar(document, 'end_ms', :integer)
-    clauses = []
-    values = []
-    if bounds.first
-      clauses << "(#{ last } IS NULL OR #{ last } > ?)"
-      values << bounds.first
-    end
-    if bounds.last
-      clauses << "#{ first } < ?"
-      values << bounds.last
-    end
-    scope.where("EXISTS (SELECT 1 FROM #{ source } WHERE #{ clauses.join(' AND ') })", *values)
-  end
-end

backend/app/services/counts_defaults.rb

@@ -1,63 +0,0 @@
-# Only confirmed combinations belong here; all others retain the existing behaviour.
-class CountsDefaults
-  POST_COLUMNS = ['resource_id', 'version_no', 'is_latest', 'title', 'url',
-                  'thumbnail_base', 'tags', 'parents', 'original_created_from',
-                  'original_created_before', 'event_type', 'user', 'created_at'].freeze
-
-  def self.entries(resource)
-    result = [{ definition: 'history', mode: 'summary', params: {
-        'rows' => 'created_at:month',
-        'cols' => 'event_type',
-        'sort' => 'a0:desc', 'conditions' => '' } }]
-    result << { definition: 'latest', mode: 'summary', params: {
-        'rows' => '', 'cols' => '', 'conditions' => '', 'tags' => '' } }
-    details = detail_params(resource)
-    if details
-      ['latest', 'history'].each do |definition|
-        result << { definition:, mode: 'details', params: details }
-      end
-    end
-    result
-  end
-
-  def self.detail_params(resource)
-    if resource == 'posts'
-      return { 'columns' => POST_COLUMNS.join(' '), 'order' => POST_COLUMNS.join(' '),
-               'children' => 'tags.name parents.id', 'nested_json' => '0',
-               'conditions' => '', 'tags' => '' }
-    end
-    unless resource == 'tags'
-      return { 'columns' => CountsExportProjection.default_columns(resource).join(' '),
-               'children' => CountsExportProjection.default_children(resource).join(' ') }
-    end
-
-    columns = ['resource_id', 'version_no', 'is_latest', 'name', 'category',
-               'aliases', 'parents', 'deprecated', 'event_type', 'user', 'created_at']
-    { 'columns' => columns.join(' '), 'order' => columns.join(' '),
-      'children' => 'aliases.name parents.name', 'nested_json' => '0' }
-  end
-
-  def self.for(params)
-    settings = entries(params.fetch('resource', 'posts')).find do |entry|
-      entry[:definition] == params.fetch('definition', 'latest') &&
-          entry[:mode] == params.fetch('mode', 'summary')
-    end&.fetch(:params) || {}
-    if settings.key?('sort') && params.key?('rows')
-      actual = QueryState.axes(params['rows'])
-      expected = QueryState.axes(settings.fetch('rows'))
-      compatible = actual.is_a?(Array) && actual.first.is_a?(Hash) &&
-          actual.first['field'] == expected.first['field'] &&
-          actual.first.fetch('grain', 'month') == expected.first['grain']
-      unless compatible
-        settings = settings.except('sort')
-      end
-    end
-    settings
-  end
-
-  def self.metadata(resource)
-    entries(resource).map do |entry|
-      entry.merge(params: entry[:params].map { |key, value| { key:, value: } })
-    end
-  end
-end

backend/app/services/counts_export_download.rb

@@ -1,106 +0,0 @@
-require 'tempfile'
-require 'fileutils'
-
-# Finish generation before reporting readiness; downloads never hold the full file in RAM.
-class CountsExportDownload
-  EXPIRY = 10.minutes
-  MAX_CONCURRENT_EXPORTS = 2
-  MAX_FILE_BYTES = 128.megabytes
-  MAX_STORED_BYTES = 512.megabytes
-  MAX_EXPORT_SECONDS = 120
-  class Busy < StandardError; end
-  class TooLarge < StandardError; end
-  class TimedOut < StandardError; end
-  class StorageFull < StandardError; end
-
-  def self.directory
-    Rails.root.join(ENV.fetch('COUNTS_EXPORT_DIRECTORY', 'tmp/counts_exports'))
-  end
-  def self.verifier = Rails.application.message_verifier('counts_export_download')
-
-  def self.prepare(exporter, owner, progress: nil, check: nil, identifier: nil)
-    FileUtils.mkdir_p(directory, mode: 0o700)
-    authority = HistoryAccess.authority
-    slot = acquire_slot
-
-    begin
-      generate(exporter, owner, progress:, check:, identifier:, authority:)
-    ensure
-      slot.close
-    end
-  end
-
-  def self.acquire_slot
-    MAX_CONCURRENT_EXPORTS.times do |index|
-      path = directory.join(".slot-#{ index }.lock")
-      handle = File.open(path, File::RDWR | File::CREAT, 0o600)
-      return handle if handle.flock(File::LOCK_EX | File::LOCK_NB)
-
-      handle.close
-    end
-    raise Busy
-  end
-
-  def self.generate(exporter, owner, progress: nil, check: nil, identifier: nil, authority: HistoryAccess.authority)
-    FileUtils.mkdir_p(directory, mode: 0o700)
-    Dir.glob(directory.join('counts-*.export')).each do |path|
-      File.unlink(path) if File.mtime(path) < Time.current - EXPIRY
-    rescue Errno::ENOENT
-      # Another request may have removed an expired download.
-    end
-    stored = Dir.glob(directory.join('counts-*')).sum do |path|
-      File.size(path)
-    rescue Errno::ENOENT
-      0
-    end
-    reserved = MAX_CONCURRENT_EXPORTS * MAX_FILE_BYTES
-    raise StorageFull if stored + reserved > MAX_STORED_BYTES
-
-    prefix = identifier ? "counts-#{ identifier }-" : 'counts-'
-    file = Tempfile.create([prefix, '.part'], directory)
-    complete = false
-    bytes = 0
-    deadline = Process.clock_gettime(Process::CLOCK_MONOTONIC) + MAX_EXPORT_SECONDS
-    path = file.path
-    begin
-      file.binmode
-      exporter.body(progress:).each do |chunk|
-        check&.call
-        raise TimedOut if Process.clock_gettime(Process::CLOCK_MONOTONIC) > deadline
-
-        bytes += chunk.bytesize
-        raise TooLarge if bytes > MAX_FILE_BYTES
-
-        file.write(chunk)
-      end
-      file.flush
-      file.fsync
-      file.close
-      finished_path = path.sub(/\.part\z/, '.export')
-      File.rename(path, finished_path)
-      path = finished_path
-      token = verifier.generate({ 'file' => File.basename(path),
-                          'resource' => exporter.resource, 'format' => exporter.format,
-                          'owner' => owner, **authority }, expires_in: EXPIRY, purpose: 'counts_export')
-      complete = true
-      token
-    ensure
-      file.close unless file.closed?
-      File.unlink(path) if !complete && File.exist?(path)
-    end
-  end
-
-  def self.find(token, owner)
-    metadata = verifier.verified(token, purpose: 'counts_export')
-    return unless metadata.is_a?(Hash) && metadata['owner'] == owner
-    return unless HistoryAccess.authorised_artifact?(metadata)
-    return unless metadata['file']&.match?(/\Acounts-[a-zA-Z0-9_-]+\.export\z/)
-    return unless CountsResourceDefinition::DEFINITIONS.key?(metadata['resource'])
-    return unless ['json', 'csv'].include?(metadata['format'])
-
-    path = directory.join(metadata['file'])
-    return unless path.file?
-
-    metadata.merge('path' => path.to_s)
-  end
-end

backend/app/services/counts_export_jobs.rb

@@ -1,255 +0,0 @@
-require 'json'
-require 'fileutils'
-
-# A durable, bounded export spool. Web and supervised workers share the same POSIX volume.
-class CountsExportJobs
-  class Cancelled < StandardError; end
-  ACTIVE = ['queued', 'running', 'cancelling'].freeze
-
-  def self.directory = CountsExportDownload.directory.join('jobs')
-  def self.verifier = Rails.application.message_verifier('counts_export_job')
-
-  def self.synchronise
-    FileUtils.mkdir_p(directory, mode: 0o700)
-    File.open(directory.join('.registry.lock'), File::RDWR | File::CREAT, 0o600) do |lock|
-      lock.flock(File::LOCK_EX)
-      yield
-    end
-  end
-
-  def self.write(state)
-    path = directory.join("#{ state.fetch('id') }.json")
-    temporary = "#{ path }.#{ SecureRandom.hex(8) }.tmp"
-    File.open(temporary, 'w', 0o600) do |file|
-      file.write(JSON.generate(state))
-      file.flush
-      file.fsync
-    end
-    File.rename(temporary, path)
-  ensure
-    File.unlink(temporary) if temporary && File.exist?(temporary)
-  end
-
-  def self.read(id)
-    JSON.parse(File.read(directory.join("#{ id }.json")))
-  rescue Errno::ENOENT
-    nil
-  end
-
-  def self.remove_download(state)
-    name = state['download_file']
-    return unless name&.match?(/\Acounts-[a-zA-Z0-9_-]+\.export\z/)
-
-    File.unlink(CountsExportDownload.directory.join(name))
-  rescue Errno::ENOENT
-    nil
-  end
-
-  def self.remove_partial(id)
-    Dir.glob(CountsExportDownload.directory.join("counts-#{ id }-*")).each do |path|
-      File.unlink(path)
-    rescue Errno::ENOENT
-      nil
-    end
-  end
-
-  def self.clean
-    Dir.glob(directory.join('*.tmp')).each do |path|
-      File.unlink(path) if File.mtime(path) < Time.current - CountsExportDownload::EXPIRY
-    end
-    Dir.glob(directory.join('*.run')).each do |path|
-      id = File.basename(path, '.run')
-      next if directory.join("#{ id }.json").exist?
-
-      File.open(path, File::RDWR) do |lock|
-        File.unlink(path) if lock.flock(File::LOCK_EX | File::LOCK_NB)
-      end
-    end
-    Dir.glob(directory.join('*.json')).each do |path|
-      state = JSON.parse(File.read(path))
-      next if state.fetch('expires_at') > Time.current.to_f
-
-      run = directory.join("#{ state.fetch('id') }.run")
-      File.open(run, File::RDWR | File::CREAT, 0o600) do |lock|
-        unless lock.flock(File::LOCK_EX | File::LOCK_NB)
-          state['status'] = 'cancelling'
-          write(state)
-          next
-        end
-        remove_download(state)
-        remove_partial(state.fetch('id'))
-        File.unlink(path)
-        File.unlink(run)
-      end
-    end
-  end
-
-  def self.start(payload, owner, language)
-    payload = payload.dup
-    time_key = payload.key?('definition') ? 'execution_at' : 'at'
-    if payload[time_key].nil? || payload[time_key] == ''
-      payload[time_key] = Time.current.utc.iso8601(6)
-    end
-    exporter = CountsExporter.new(payload, language_code: language)
-    synchronise do
-      clean
-      active = Dir.glob(directory.join('*.json')).count do |path|
-        ACTIVE.include?(JSON.parse(File.read(path))['status'])
-      end
-      raise CountsExportDownload::Busy if active >= CountsExportDownload::MAX_CONCURRENT_EXPORTS
-
-      id = SecureRandom.hex(24)
-      state = { 'id' => id, 'owner' => owner, 'payload' => payload,
-          'language' => language, 'resource' => exporter.resource,
-          'format' => exporter.format, 'status' => 'queued',
-          'processed' => 0, 'total' => nil,
-          **HistoryAccess.authority,
-          'expires_at' => (Time.current + CountsExportDownload::EXPIRY).to_f }
-      write(state)
-      verifier.generate({ 'id' => id, 'owner' => owner },
-          expires_in: CountsExportDownload::EXPIRY, purpose: 'counts_export_job')
-    end
-  end
-
-  def self.authorised_id(token, owner)
-    value = verifier.verified(token, purpose: 'counts_export_job')
-    return unless value.is_a?(Hash) && value['owner'] == owner
-    return unless value['id']&.match?(/\A[a-f0-9]{48}\z/)
-
-    value['id']
-  end
-
-  def self.status(token, owner, cancel: false)
-    id = authorised_id(token, owner)
-    return unless id
-
-    synchronise do
-      state = read(id)
-      return unless state && state['owner'] == owner && state['expires_at'] > Time.current.to_f
-
-      return unless HistoryAccess.authorised_artifact?(state)
-
-      if cancel && ['queued', 'running', 'ready'].include?(state['status'])
-        state['status'] = state['status'] == 'running' ? 'cancelling' : 'cancelled'
-        remove_download(state)
-        state.delete('token')
-        write(state)
-      end
-      state.slice('status', 'processed', 'total', 'format', 'token', 'error')
-    end
-  end
-
-  def self.checkpoint(id, processed = nil, total = nil)
-    synchronise do
-      state = read(id)
-      unless state && state['status'] == 'running' && state['expires_at'] > Time.current.to_f
-        raise Cancelled
-      end
-      if processed
-        state.merge!('processed' => processed, 'total' => total)
-        write(state)
-      end
-    end
-  end
-
-  def self.work_one
-    FileUtils.mkdir_p(directory, mode: 0o700)
-    Dir.glob(directory.join('*.json')).sort.each do |path|
-      id = File.basename(path, '.json')
-      File.open(directory.join("#{ id }.run"), File::RDWR | File::CREAT, 0o600) do |lock|
-        next unless lock.flock(File::LOCK_EX | File::LOCK_NB)
-
-        state = synchronise do
-          candidate = read(id)
-          next unless candidate && ACTIVE.include?(candidate['status'])
-
-          if candidate['status'] != 'queued'
-            candidate['status'] = candidate['status'] == 'cancelling' ? 'cancelled' : 'failed'
-            remove_partial(id)
-            candidate['error'] = 'interrupted'
-            write(candidate)
-            next
-          end
-          candidate['status'] = 'running'
-          write(candidate)
-          candidate
-        end
-        next unless state
-
-        perform(state)
-        return true
-      end
-    end
-    synchronise { clean }
-    false
-  end
-
-  def self.perform(state)
-    actor = state['actor_id'] && User.find_by(id: state['actor_id'])
-    actor = nil unless state['admin'] && actor&.admin? && !actor.banned?
-    HistoryAccess.set(user: actor) do
-      unless HistoryAccess.authorised_artifact?(state)
-        synchronise do
-          current = read(state.fetch('id'))
-          current['status'] = 'failed'
-          write(current)
-        end
-        return
-      end
-      perform_authorised(state)
-    end
-  end
-
-  def self.perform_authorised(state)
-    id = state.fetch('id')
-    token = nil
-    begin
-      checkpoint(id)
-      exporter = CountsExporter.new(state.fetch('payload'), language_code: state.fetch('language'))
-      last = Process.clock_gettime(Process::CLOCK_MONOTONIC)
-      progress = lambda do |processed, total|
-        now = Process.clock_gettime(Process::CLOCK_MONOTONIC)
-        if processed.zero? || now - last >= 0.25 || processed == total
-          checkpoint(id, processed, total)
-          last = now
-        end
-      end
-      check = lambda do
-        now = Process.clock_gettime(Process::CLOCK_MONOTONIC)
-        if now - last >= 0.25
-          checkpoint(id)
-          last = now
-        end
-      end
-      token = CountsExportDownload.prepare(exporter, state.fetch('owner'),
-          progress:, check:, identifier: id)
-      synchronise do
-        current = read(id)
-        if (!current || current['status'] != 'running' ||
-            current['expires_at'] <= Time.current.to_f)
-          file = CountsExportDownload.find(token, state.fetch('owner'))
-          File.unlink(file.fetch('path')) if file
-          raise Cancelled
-        end
-        file = CountsExportDownload.find(token, state.fetch('owner'))
-        current.merge!('status' => 'ready', 'token' => token,
-            'download_file' => File.basename(file.fetch('path')))
-        current['total'] = current['processed']
-        write(current)
-      end
-    rescue StandardError => error
-      remove_partial(id)
-      synchronise do
-        current = read(id)
-        if current
-          current['status'] = error.is_a?(Cancelled) ? 'cancelled' : 'failed'
-          current['error'] = error.class.name
-          write(current)
-        end
-      end
-      unless error.is_a?(Cancelled)
-        Rails.logger.error("counts_export_job #{ id }: #{ error.class }")
-      end
-    end
-  end
-end

backend/app/services/counts_export_projection.rb

@@ -1,265 +0,0 @@
-require 'set'
-
-# Export identifiers are stable; UI translations are supplied separately as column labels.
-class CountsExportProjection
-  ROOT_KEYS = { 'event_type' => 'event_type', 'resource_id' => 'resource_id', 'version_no' => 'version_no',
-                'is_latest' => 'is_latest', 'thumbnail' => 'thumbnail_url',
-                'thumbnail_base' => 'thumbnail_base', 'title' => 'title', 'url' => 'url',
-                'tags' => 'tags', 'parents' => 'parents', 'video_ms' => 'video_ms',
-                'original_created_from' => 'original_created_from',
-                'original_created_before' => 'original_created_before',
-                'user' => 'created_by_user', 'created_at' => 'created_at',
-                'name' => 'name', 'names' => 'names', 'category' => 'category',
-                'aliases' => 'aliases', 'deprecated' => 'deprecated' }.freeze
-  CHILD_KEYS = { 'id' => 'id', 'versionNo' => 'version_no', 'kind' => 'kind',
-                 'name' => 'name', 'category' => 'category', 'sections' => 'sections',
-                 'names' => 'names', 'aliases' => 'aliases',
-                 'languageCode' => 'language', 'scriptCode' => 'script',
-                 'autoGenerated' => 'auto_generated' }.freeze
-  NAME_FIELDS = ['languageCode', 'scriptCode', 'name', 'aliases', 'autoGenerated'].freeze
-  CHILDREN = {
-      'tags' => ['id', 'versionNo', 'kind', 'name', 'category', 'sections', 'names'],
-      'names' => NAME_FIELDS,
-      'aliases' => ['languageCode', 'name', 'scriptCode'],
-      'parents' => ['id', 'name'] }.freeze
-
-  def self.default_columns(resource)
-    fields = { 'posts' => ['title', 'url', 'tags', 'parents', 'video_ms'],
-        'tags' => ['name', 'category', 'aliases', 'parents'],
-        'nico_tags' => ['name', 'linked_tags'],
-        'tag_descriptions' => ['language_code', 'body', 'source_language_code'],
-        'tag_groups' => ['name', 'description', 'members'],
-        'materials' => ['url', 'tag_id', 'source_kind'] }.fetch(resource)
-    ['resource_id', 'version_no'] + fields + ['event_type', 'user', 'created_at']
-  end
-
-  def self.default_children(resource)
-    { 'posts' => ['tags.name', 'parents.id'],
-        'tags' => ['aliases.name', 'parents.id'],
-        'tag_groups' => ['members.tagId'] }.fetch(resource, [])
-  end
-
-  attr_reader :columns, :headers
-
-  def initialize(params, resource:, summary:, tags:)
-    if params.key?('definition')
-      params = CountsDefaults.for(params.merge('resource' => resource)).merge(params)
-    end
-    @params = params
-    @labels = JSON.parse(params.fetch('labels', '{}'))
-    raise ArgumentError unless @labels.is_a?(Hash) && @labels.length <= 100
-    raise ArgumentError unless @labels.all? { |key, value| valid_label?(key, value) }
-    @summary = summary
-    allowed = if summary
-                tags ? ['kind', 'tag_id', 'name', 'deprecated', 'count'] : ['group', 'count']
-              else
-                CountsQuery::COLUMNS.fetch(resource)
-              end
-    defaults = summary ? allowed : self.class.default_columns(resource)
-    @columns = QueryState.list(params.fetch('columns', defaults.join(' ')))
-    raise ArgumentError if (@columns - allowed).any? || @columns.uniq != @columns
-
-    @headers = JSON.parse(params.fetch('headers', JSON.generate(@columns)))
-    unless (@headers.is_a?(Array) && @headers.length == @columns.length &&
-        @headers.all? { _1.is_a?(String) && _1.length <= 200 })
-      raise ArgumentError
-    end
-    @order = QueryState.list(params.fetch('order', ''))
-    @sorts = QueryState.list(params.fetch('child_sort', '')).map do |entry|
-      parts = entry.split(':', -1)
-      raise ArgumentError unless parts.length == 2
-
-      path, direction = parts
-      raise ArgumentError unless ['asc', 'desc'].include?(direction)
-
-      [path, direction]
-    end
-    @selected = params.key?('children') ? QueryState.list(params['children']).to_set :
-        self.class.default_children(resource).to_set
-    valid = CHILDREN.flat_map do |key, fields|
-      paths(fields, key)
-    end
-    raise ArgumentError if @selected && (@selected.to_a - valid).any?
-    prune = lambda do |fields, prefix|
-      fields.each do |key|
-        path = "#{ prefix }.#{ key }"
-        children = fields_for(key)
-        next if children.empty?
-
-        prune.call(children, path)
-        @selected.delete(path) unless children.any? { @selected.include?("#{ path }.#{ _1 }") }
-      end
-    end
-    CHILDREN.each { |key, fields| prune.call(fields, key) }
-    pairs = @columns.zip(@headers).select do |key, _header|
-      !CHILDREN.key?(key) || CHILDREN[key].any? { @selected.include?("#{ key }.#{ _1 }") }
-    end
-    @columns = pairs.map(&:first)
-    @headers = pairs.map(&:last)
-    allowed_sorts = valid.map do |path|
-      parts = path.split('.')
-      field = parts.pop
-      "#{ parts.join('.') }/#{ field }"
-    end
-    if ((@sorts.map(&:first) - allowed_sorts).any? ||
-        @sorts.map(&:first).uniq.length != @sorts.length)
-      raise ArgumentError
-    end
-  end
-
-  def metadata
-    @columns.each_with_index.map do |key, index|
-      { key: @summary ? key : ROOT_KEYS.fetch(key), label: @headers[index] }
-    end
-  end
-
-  def project(row)
-    @columns.to_h do |key|
-      if @summary
-        value = key == 'group' ? { key: row[:key], name: row[:name] } : row[key.to_sym]
-        [key, value]
-      else
-        value = case key
-                when 'thumbnail' then row[:thumbnail_base]
-                else row[ROOT_KEYS.fetch(key).to_sym]
-                end
-        value = nested(value || [], CHILDREN.fetch(key), key) if CHILDREN.key?(key)
-        [ROOT_KEYS.fetch(key), value]
-      end
-    end
-  end
-
-  def csv_record(record, row)
-    record.map do |key, value|
-      key == 'group' ? row[:name] || row[:key] : csv_display(value, key)
-    end
-  end
-
-  private
-
-  def valid_label?(key, value)
-    key.is_a?(String) && value.is_a?(String) && key.length <= 100 && value.length <= 300
-  end
-
-  def csv_display(value, key)
-    return @labels.fetch('bot', '') if key == 'created_by_user' && value.nil?
-    return @labels.fetch("event_type:#{ value }", value) if key == 'event_type'
-    return value ? @labels.fetch('latest', 'true') : '' if key == 'is_latest'
-    return @labels.fetch("kind:#{ value }", value) if key == 'kind'
-    return @labels.fetch("category:#{ value }", value) if key == 'category'
-    if ['language', 'script'].include?(key) && value.is_a?(Hash)
-      return @labels.fetch("#{ key }:#{ value[:code] }",
-                           value[:name] || @labels.fetch('notRegistered', ''))
-    end
-    return @labels.fetch("boolean:#{ value }", value) if value == true || value == false
-    return value.map { csv_display(_1, key) } if value.is_a?(Array)
-    if value.is_a?(Hash)
-      return value.to_h do |field, item|
-        [@labels.fetch(field.to_s, field), csv_display(item, field.to_s)]
-      end
-    end
-
-    value
-  end
-
-  def fields_for(key)
-    return NAME_FIELDS if key == 'names'
-    return ['name', 'scriptCode'] if key == 'aliases'
-
-    []
-  end
-
-  def paths(fields, prefix)
-    fields.flat_map do |key|
-      path = "#{ prefix }.#{ key }"
-      [path, *paths(fields_for(key), path)]
-    end
-  end
-
-  def fetch(row, key) = row.key?(key) ? row[key] : row[key.to_sym]
-
-  def named_code(row, kind)
-    code = fetch(row, "#{ kind }_code")
-    name = fetch(row, "#{ kind }_name")
-    { code:, name: name.present? && name.to_s.downcase != code.to_s.downcase ? name : nil }
-  end
-
-  def stable(value)
-    case value
-    when Hash
-      value.stringify_keys.sort.to_h.transform_values { stable(_1) }
-    when Array
-      value.map { stable(_1) }
-    else
-      value
-    end
-  end
-
-  def sort_value(row, key)
-    if key == 'languageCode'
-      language = named_code(row, 'language')
-      return @labels.fetch("language:#{ language[:code] }", language[:name])
-    end
-    return named_code(row, 'script')[:name] if key == 'scriptCode'
-
-    fetch(row, CHILD_KEYS.fetch(key))
-  end
-
-  def comparison(left, right)
-    if left.is_a?(Array) && right.is_a?(Array) && left.length != right.length
-      return left.length <=> right.length
-    end
-    return left <=> right if left.is_a?(Numeric) && right.is_a?(Numeric)
-    if [true, false].include?(left) && [true, false].include?(right)
-      return (left ? 1 : 0) <=> (right ? 1 : 0)
-    end
-
-    a = left.is_a?(Hash) || left.is_a?(Array) ? JSON.generate(stable(left)) : left.to_s
-    b = right.is_a?(Hash) || right.is_a?(Array) ? JSON.generate(stable(right)) : right.to_s
-    a <=> b
-  end
-
-  def sorted(rows, prefix, fields)
-    sorts = @sorts.filter_map do |path, direction|
-      parent, key = path.split('/', 2)
-      next unless parent == prefix
-      raise ArgumentError unless fields.include?(key)
-
-      [key, direction]
-    end
-    rows.each_with_index.sort do |(left, a), (right, b)|
-      result = 0
-      sorts.each do |key, direction|
-        first = sort_value(left, key)
-        last = sort_value(right, key)
-        result = if first.nil? || last.nil?
-                   first.nil? == last.nil? ? 0 : (first.nil? ? 1 : -1)
-                 else
-                   comparison(first, last) * (direction == 'asc' ? 1 : -1)
-                 end
-        break unless result.zero?
-      end
-      result.zero? ? a <=> b : result
-    end.map(&:first)
-  end
-
-  def nested(rows, fields, prefix)
-    keys = fields.select { !@selected || @selected.include?("#{ prefix }.#{ _1 }") }
-    ordered = (@order + keys.map { "#{ prefix }.#{ _1 }" }).uniq
-        .filter_map { |path| keys.find { "#{ prefix }.#{ _1 }" == path } }
-    sorted(rows, prefix, fields).map do |row|
-      ordered.to_h do |key|
-        stable = CHILD_KEYS.fetch(key)
-        value = case key
-                when 'languageCode' then named_code(row, 'language')
-                when 'scriptCode' then named_code(row, 'script')
-                else fetch(row, stable)
-                end
-        if fields_for(key).any?
-          value = nested(value || [], fields_for(key), "#{ prefix }.#{ key }")
-        end
-        [stable, value]
-      end
-    end
-  end
-end

backend/app/services/counts_exporter.rb

@@ -1,152 +0,0 @@
-require 'csv'
-
-# One read view and a fixed point for the entire download; only one page is held in memory.
-class CountsExporter
-  EXTRA_KEYS = ['basis', 'format', 'resource', 'headers', 'children', 'order',
-                'summary_columns', 'nested_json', 'child_sort', 'labels'].freeze
-  attr_reader :format, :resource
-
-  def initialize(params, language_code:)
-    if params.key?('definition')
-      @table_export = CountsTableExport.new(params)
-      @format = @table_export.format
-      @resource = @table_export.resource
-      return
-    end
-    allowed = (CountsQuery::KEYS + CountsStateQuery::KEYS + EXTRA_KEYS).uniq
-    QueryState.validate!(params, allowed)
-    raise ArgumentError unless params.values.all? { _1.is_a?(String) }
-
-    @params = params
-    @language = language_code
-    @format = params.fetch('format', 'json')
-    @resource = params.fetch('resource', 'posts')
-    @mode = params.fetch('mode', 'summary')
-    @history = params.fetch('basis', 'state') == 'history'
-    raise ArgumentError unless ['json', 'csv'].include?(format)
-    raise ArgumentError unless ['posts', 'tags'].include?(resource)
-    raise ArgumentError unless ['summary', 'details'].include?(@mode)
-    raise ArgumentError unless ['state', 'history'].include?(params.fetch('basis', 'state'))
-
-    @query_params = params.except(*EXTRA_KEYS, 'columns', 'page', 'limit')
-        .merge('mode' => @mode, 'page' => '1', 'limit' => '100')
-    point = CountsStateQuery.new(@query_params.slice('at').merge('resource' => resource),
-                                 language_code: @language).at
-    @query_params['at'] = point.iso8601(6)
-    @query_params['axis'] ||= !@history && resource == 'posts' ? 'tags' : 'month'
-    @projection = CountsExportProjection.new(params, resource:,
-                                             summary: @mode == 'summary',
-                                             tags: !@history && @query_params['axis'] == 'tags')
-    if @history
-      @query_params.delete('kind')
-      history_query
-    else
-      @query_params.delete('latest')
-      @state_query = CountsStateQuery.new(@query_params.merge('resource' => resource),
-                                          language_code: @language).validate_query!
-    end
-  end
-
-  def body(progress: nil)
-    return @table_export.body(progress:) if @table_export
-
-    Enumerator.new do |output|
-      ApplicationRecord.transaction(isolation: :repeatable_read) do
-        processed = 0
-        first_row = true
-        started = false
-        pages do |page|
-          total = @mode == 'details' ? page[:count] : nil
-          progress&.call(processed, total)
-          unless started
-            meta = { resource:, mode: @mode, basis: @history ? 'history' : 'state',
-                     at: @query_params['at'], boundary: 'inclusive',
-                     filters: @query_params.except('sort', 'page', 'limit', 'mode'),
-                     sort: export_sort,
-                     child_sort: sort_metadata(@params.fetch('child_sort', '')),
-                     columns: @projection.metadata,
-                     children: @params['children']&.split(' '),
-                     field_order: @QueryState.list(params.fetch('order', '')), timezone: 'UTC',
-                     count: @mode == 'summary' ? page[:group_count] : page[:count],
-                     totals: page[:totals], coverage: page[:coverage] }
-            output << (format == 'json' ? "{\"metadata\":#{ meta.to_json },\"data\":[" : "\uFEFF")
-            if format == 'csv'
-              output << CSV.generate_line(
-                  @projection.headers.map { csv_value(_1) }, row_sep: "\r\n")
-            end
-            started = true
-          end
-          (page[:versions] || page[:state_rows] || page[:groups] || []).each do |row|
-            record = @projection.project(row)
-            if format == 'json'
-              output << ',' unless first_row
-              output << record.to_json
-            else
-              values = @projection.csv_record(record, row).map { csv_value(_1) }
-              output << CSV.generate_line(values, row_sep: "\r\n")
-            end
-            first_row = false
-            processed += 1
-            progress&.call(processed, total)
-          end
-        end
-        progress&.call(processed, processed)
-        output << ']}' if format == 'json'
-      end
-    end
-  end
-
-  private
-
-  def export_sort
-    sort_metadata(@query_params.fetch('sort', default_sort))
-  end
-
-  def sort_metadata(value)
-    value.split(' ').map do |entry|
-      key, direction = entry.split(':', 2)
-      { key:, direction: }
-    end
-  end
-
-  def default_sort
-    return 'created_at:desc' if @mode == 'details'
-
-    !@history && @query_params['axis'] == 'tags' ? 'count:desc' : 'group:asc'
-  end
-
-  def history_query
-    scope = resource == 'posts' ? PostVersion.with_previous_version : TagVersion.all
-    CountsQuery.new(scope, resource, @query_params)
-  end
-
-  def pages
-    unless @history
-      @state_query.each_page { yield _1 }
-      return
-    end
-    query = history_query
-    total = @mode == 'summary' ? nil : query.count
-    loop do
-      result = if @mode == 'summary'
-                 query.summary(include_count: @query_params['page'] == '1')
-               else
-                 { count: total,
-                   versions: CountsVersionRepr.many(query.details, resource:,
-                                                   language_code: @language,
-                                                   at: Time.iso8601(@query_params['at'])) }
-               end
-      total ||= result[:group_count]
-      yield result
-      break if @query_params['page'].to_i * 100 >= total
-
-      @query_params['page'] = (@query_params['page'].to_i + 1).to_s
-      query = history_query
-    end
-  end
-
-  def csv_value(value)
-    text = value.is_a?(Hash) || value.is_a?(Array) ? JSON.generate(value) : value.to_s
-    text.match?(/\A\s*[=+@-]/) ? "'#{ text }" : text
-  end
-end

backend/app/services/counts_pivot_layout.rb

@@ -1,28 +0,0 @@
-# Coordinates carry their aggregation depth; SQL NULL remains an ordinary axis value.
-class CountsPivotLayout
-  def self.depths(keys, sorts, measures: true)
-    return [] if keys.length < 2 || sorts.first&.first == 'count' ||
-        sorts.first&.first&.match?(/\Am[0-9]+\z/)
-
-    relevant = sorts.map(&:first).select { keys.include?(_1) || (measures && (_1 == 'count' ||
-        _1.match?(/\Am[0-9]+\z/))) }
-    order = (relevant + keys).uniq
-    (1...keys.length).select { order.first(_1).sort == keys.first(_1).sort }
-  end
-
-  def self.headers(values, depths, next_values: nil, grand: false)
-    result = []
-    values.each_with_index do |tuple, index|
-      result << { values: tuple, kind: 'data' }
-      following = values[index + 1] || next_values
-      depths.reverse_each do |depth|
-        prefix = tuple.first(depth)
-        next if following && following.first(depth) == prefix
-
-        result << { values: prefix, kind: 'subtotal' }
-      end
-    end
-    result << { values: [], kind: 'grand_total' } if grand
-    result
-  end
-end

backend/app/services/counts_pivot_totals.rb

@@ -1,59 +0,0 @@
-# Re-aggregate the population, never add up already grouped cells (multi-value axes overlap).
-class CountsPivotTotals
-  def initialize(population, axes, row_headers, column_headers)
-    @population = population
-    @axes = axes
-    @rows = row_headers.group_by { _1[:values].length }
-    @columns = column_headers.group_by { _1[:values].length }
-  end
-
-  def call
-    queries = @rows.flat_map do |row_depth, headers|
-      @columns.filter_map do |column_depth, columns|
-        next if row_depth == @axes.rows.length && column_depth == @axes.columns.length
-
-        indexes = (0...row_depth).to_a +
-            (0...column_depth).map { @axes.rows.length + _1 }
-        expressions = indexes.map { @axes.expressions[_1] }
-        scope = @population.scope.except(:select, :order)
-        joins = @axes.joins(indexes.map { @axes.all[_1].fetch('field') })
-        scope = scope.joins(joins) unless joins.empty?
-        if row_depth.positive?
-          tuples = headers.map { _1[:values] }.uniq
-          predicate = tuples.map { Array.new(row_depth) { |i|
-            "(#{ expressions[i] }) <=> ?"
-          }.join(' AND ') }.map { "(#{ _1 })" }.join(' OR ')
-          scope = scope.where(predicate, *tuples.flatten)
-        end
-        if column_depth.positive?
-          tuples = columns.map { _1[:values] }.uniq
-          predicate = tuples.map { Array.new(column_depth) { |i|
-            "(#{ expressions[row_depth + i] }) <=> ?"
-          }.join(' AND ') }.map { "(#{ _1 })" }.join(' OR ')
-          scope = scope.where(predicate, *tuples.flatten)
-        end
-        grouped = expressions.each_with_index.map { |expression, index|
-          "#{ expression } AS v#{ index }"
-        }
-        # Fixed-width result coordinates, independent of NULL and of translated labels.
-        padding = (expressions.length...@axes.all.length).map { "NULL AS v#{ _1 }" }
-        fields = ["#{ row_depth } AS row_depth", "#{ column_depth } AS column_depth"] +
-            grouped + padding + ['COUNT(DISTINCT v.id) AS count']
-        scope = scope.group(*expressions.map { Arel.sql(_1) }) if expressions.any?
-        scope.optimizer_hints('MAX_EXECUTION_TIME(10000)')
-            .select(Arel.sql(fields.join(', '))).to_sql
-      end
-    end
-    return [] if queries.empty?
-
-    sql = queries.map { "(#{ _1 })" }.join(' UNION ALL ')
-    @population.definition.model.connection.select_all(
-        "SELECT /*+ MAX_EXECUTION_TIME(10000) */ * FROM (#{ sql }) totals").map do |record|
-      row_depth = record['row_depth'].to_i
-      column_depth = record['column_depth'].to_i
-      { row: (0...row_depth).map { record["v#{ _1 }"] },
-        column: (0...column_depth).map { record["v#{ row_depth + _1 }"] },
-        count: record['count'].to_i }
-    end
-  end
-end

backend/app/services/counts_population.rb

@@ -1,168 +0,0 @@
-# Population first, then filters. Previous versions are joined before period filtering.
-class CountsPopulation
-  attr_reader :definition, :params, :at, :history, :scope, :timezone
-
-  def initialize(params)
-    @params = params
-    @definition = CountsResourceDefinition.new(params.fetch('resource', 'posts'))
-    @history = params.fetch('definition', 'latest') == 'history'
-    raise ArgumentError unless ['latest', 'history'].include?(params.fetch('definition', 'latest'))
-
-    @timezone = TZInfo::Timezone.get(params.fetch('timezone', 'UTC'))
-    @at = timestamp(params['execution_at']) || Time.current.utc
-    raise ArgumentError if at > Time.current + 1.second
-
-    @scope = definition.model.all
-    make_population
-    @scope = HistoryAccess.visible(@scope, 'v')
-    apply_filters
-  rescue TZInfo::InvalidTimezoneIdentifier
-    raise ArgumentError
-  end
-
-  def timestamp(value, end_date: false)
-    return nil if value.nil? || value == ''
-    raise ArgumentError unless value.is_a?(String)
-
-    if value.match?(/\A\d{4}-\d{2}-\d{2}\z/)
-      date = Date.iso8601(value)
-      date += 1 if end_date
-      timezone.local_to_utc(Time.utc(date.year, date.month, date.day))
-    else
-      raise ArgumentError unless value.match?(/(?:Z|[+-]\d{2}:\d{2})\z/)
-
-      time = Time.iso8601(value).utc
-      raise ArgumentError unless time.year.between?(1900, 2100)
-
-      time
-    end
-  rescue TZInfo::AmbiguousTime, TZInfo::PeriodNotFound, Date::Error
-    raise ArgumentError
-  end
-
-  def latest_sql(row = 'v', cutoff = nil)
-    cutoff ||= history ? at : timestamp(params['at']) || at
-    keys = definition.keys.map { "newer.#{ _1 } = #{ row }.#{ _1 }" }.join(' AND ')
-    bound = definition.model.connection.quote(cutoff)
-    <<~SQL.squish
-      NOT EXISTS (SELECT 1 FROM #{ definition.table } newer
-        WHERE #{ keys } AND newer.version_no > #{ row }.version_no
-          AND newer.#{ definition.time_column } <= #{ bound })
-    SQL
-  end
-
-  def changed_sql(key)
-    column = definition.column(key)
-    "(previous.id IS NOT NULL AND NOT (v.#{ column } <=> previous.#{ column }))"
-  end
-
-  private
-
-  def make_population
-    table = definition.table
-    @scope = @scope.from("#{ table } v").select('v.*')
-    keys = definition.keys.map { "previous.#{ _1 } = v.#{ _1 }" }.join(' AND ')
-    @scope = @scope.joins("LEFT JOIN #{ table } previous ON #{ keys } " +
-        'AND previous.version_no = v.version_no - 1 AND ' +
-        HistoryAccess.sql(definition.model, 'previous'))
-    @scope = @scope.where("v.#{ definition.time_column } <= ?", at)
-    unless history
-      point = timestamp(params['at']) || at
-      raise ArgumentError if point > at
-
-      @scope = @scope.where("v.#{ definition.time_column } <= ?", point)
-          .where(Arel.sql(latest_sql('v', point))).where.not('v.event_type' => 'discard')
-      return
-    end
-    first = timestamp(params['from'])
-    last = timestamp(params['through'], end_date: true)
-    raise ArgumentError if first && last && first > last
-
-    @scope = @scope.where('v.created_at >= ?', first) if first
-    if last
-      operator = params['through'].length == 10 ? '<' : '<='
-      @scope = @scope.where("v.created_at #{ operator } ?", last)
-    end
-  end
-
-  def apply_filters
-    ['resource_id', 'user'].each do |key|
-      value = params[key == 'user' ? 'user_id' : key]
-      next if value.nil? || value == ''
-      raise ArgumentError unless value.match?(/\A[0-9]+\z/)
-
-      @scope = @scope.where("#{ definition.expression(key) } = ?", value.to_i)
-    end
-    apply_tags if params['tags'].present? || params['tag'].present?
-    conditions = QueryState.records(params, 'conditions')
-    raise ArgumentError unless conditions.is_a?(Array) && conditions.length <= 100
-
-    conditions.each { apply_condition(_1) }
-    drills = QueryState.records(params, 'drill')
-    raise ArgumentError unless drills.is_a?(Array) && drills.length <= 100
-
-    drills.each do |entry|
-      key = entry.fetch('field')
-      if key == 'tags'
-        @scope = @scope.where(snapshot_membership(entry.fetch('value')))
-      elsif key == 'changed_field'
-        raise ArgumentError unless history && definition.changes.include?(entry['value'])
-
-        @scope = @scope.where(Arel.sql(changed_sql(entry['value'])))
-      elsif definition.fields[key] == :datetime && entry['grain']
-        @scope = @scope.where(definition.expression(key) => nil) if entry['value'].nil?
-        if entry['value']
-          expression = CountsAxes.time_expression(definition.expression(key),
-                                                  entry['grain'], timezone)
-          @scope = @scope.where("#{ expression } = ?", entry['value'])
-        end
-      else
-        raise ArgumentError unless definition.axes(history).include?(key)
-
-        @scope = @scope.where(definition.expression(key) => entry['value'])
-      end
-    end
-  end
-
-  def apply_condition(entry)
-    @condition ||= CountsCondition.new(self)
-    @scope = @condition.apply(@scope, entry)
-  end
-
-  def snapshot_membership(value)
-    kind, identifier = value.to_s.split(':', 2)
-    raise ArgumentError unless ['internal', 'external'].include?(kind)
-    raise ArgumentError unless identifier&.match?(/\A[1-9]\d*\z/)
-
-    key = kind == 'internal' ? 'tag_id' : 'external_tag_id'
-    definition.model.sanitize_sql_array([
-        "JSON_CONTAINS(v.tags_json, JSON_OBJECT('#{ key }', CAST(? AS UNSIGNED)))",
-        identifier.to_i])
-  end
-
-  def apply_tags
-    raise ArgumentError unless definition.resource == 'posts'
-
-    if params['tag'].present?
-      @scope = @scope.where(snapshot_membership(
-          "#{ params.fetch('kind', 'internal') }:#{ params['tag'] }"))
-    end
-    return unless params['tags'].present?
-
-    locale = Locale.find_by!(code: params.fetch('locale', 'ja'))
-    refs = PostTagSearch.parse_references(params, params['tags'].split)
-    raw = params['tags']
-    literals = PostTagSearch.literals(locale, raw, references: refs)
-    predicates = literals.map do |literal|
-      internal, external = PostTagSearch.references(locale, literal)
-      parts = Array(internal).map { snapshot_membership("internal:#{ _1 }") } +
-          Array(external).map { snapshot_membership("external:#{ _1 }") }
-      expression = parts.empty? ? 'FALSE' : "(#{ parts.join(' OR ') })"
-      literal[:negative] ? "NOT #{ expression }" : expression
-    end
-    return if predicates.empty?
-
-    any = params['match'] == 'any' || raw.split.any? { _1.downcase == 'type:or' }
-    @scope = @scope.where(Arel.sql("(#{ predicates.join(any ? ' OR ' : ' AND ') })"))
-  end
-end

backend/app/services/counts_query.rb

@@ -1,207 +0,0 @@
-# Read-only history queries. SQL identifiers and expressions are server-owned.
-class CountsQuery
-  AXES = ['month', 'user'].freeze
-  COMMON_COLUMNS = ['resource_id', 'version_no', 'is_latest', 'event_type', 'user', 'created_at'].freeze
-  COLUMNS = {
-      'posts' => COMMON_COLUMNS + [
-          'thumbnail', 'thumbnail_base', 'title', 'url', 'tags', 'parents', 'video_ms',
-          'original_created_from', 'original_created_before'],
-      'tags' => COMMON_COLUMNS + [
-          'name', 'names', 'category', 'aliases', 'parents', 'deprecated'] }.freeze
-  KEYS = [
-      'locale', 'mode', 'axis', 'page', 'limit', 'from', 'before', 'latest',
-      'user_id', 'resource_id', 'tag', 'columns', 'sort', 'at'].freeze
-
-  attr_reader :page, :limit, :mode, :axis
-
-  def initialize(scope, resource, params, latest_expression: nil)
-    @table = scope.klass.table_name
-    @resource = resource
-    @id_column = resource == 'posts' ? 'post_id' : 'tag_id'
-    @params = params
-    @latest_expression = latest_expression
-    scope = HistoryAccess.visible(scope)
-    @scope = scope.except(:select, :order).joins(
-        "LEFT JOIN users counts_actor ON counts_actor.id = #{ @table }.created_by_user_id")
-    raise ArgumentError unless (params.keys - KEYS).empty?
-    raise ArgumentError unless params.values.all? { _1.is_a?(String) }
-
-    @mode = params.fetch('mode', 'summary')
-    @axis = params.fetch('axis', 'month')
-    raise ArgumentError unless ['summary', 'details'].include?(mode) && AXES.include?(axis)
-
-    @page = integer('page', 1)
-    @limit = integer('limit', 20)
-    raise ArgumentError if limit > 100 || page > 1_000_000
-
-    columns = QueryState.list(params.fetch('columns', COLUMNS.fetch(resource).join(' ')))
-    if columns.empty? || columns.uniq != columns || (columns - COLUMNS.fetch(resource)).any?
-      raise ArgumentError
-    end
-    @sort = parse_sort
-    apply_filters
-  end
-
-  def relation = @scope
-
-  def count = @scope.count
-
-  def latest_expression
-    return @latest_expression if @latest_expression
-
-    at = timestamp('at')
-    cutoff = at ? "AND newer.created_at <= #{ @scope.klass.connection.quote(at) }" : ''
-    <<~SQL.squish
-      NOT EXISTS (SELECT 1 FROM #{ @table } newer
-        WHERE newer.#{ @id_column } = #{ @table }.#{ @id_column }
-          AND newer.version_no > #{ @table }.version_no #{ cutoff })
-    SQL
-  end
-
-  def details
-    @scope.select("#{ @table }.*", Arel.sql("(#{ latest_expression }) AS counts_latest"))
-        .order(Arel.sql(order_sql(detail_expressions, ["#{ @table }.id DESC"])))
-        .limit(limit).offset((page - 1) * limit)
-  end
-
-  def summary(include_count: true)
-    expression = axis == 'month' ? "DATE_FORMAT(#{ @table }.created_at, '%Y-%m')" :
-                                  "#{ @table }.created_by_user_id"
-    group_order = axis == 'user' ? 'MAX(counts_actor.name)' : expression
-    sql = @scope.group(Arel.sql(expression)).select(Arel.sql(
-        "#{ expression } AS group_key, COUNT(*) AS record_count, " +
-        "#{ group_order } AS group_name")).to_sql
-    connection = @scope.klass.connection
-    source = "FROM (#{ sql }) counts_groups"
-    groups = include_count ? connection.select_value("SELECT COUNT(*) #{ source }") : nil
-    expressions = { 'group' => ['group_name'], 'count' => ['record_count'] }
-    order = order_sql(expressions, ['group_key IS NULL ASC', 'group_key ASC'])
-    rows = connection.select_all(
-        "SELECT group_key, group_name, record_count #{ source } " +
-        "ORDER BY #{ order } LIMIT #{ limit } OFFSET #{ (page - 1) * limit }")
-    { group_count: groups&.to_i,
-      groups: rows.map do |row|
-        { key: row['group_key'],
-          name: axis == 'user' ? row['group_name'] : nil,
-          count: row['record_count'].to_i }
-      end }
-  end
-
-  private
-
-  def parse_sort
-    value = @params.fetch('sort', mode == 'details' ? 'created_at:desc' : 'group:asc')
-    return [] if value.empty?
-
-    values = QueryState.list(value).map { _1.split(':', -1) }
-    allowed = mode == 'details' ? COLUMNS.fetch(@resource) - ['thumbnail'] : ['group', 'count']
-    invalid = values.any? do |key, direction, *extra|
-      extra.any? || !allowed.include?(key) || !['asc', 'desc'].include?(direction)
-    end
-    if (values.empty? || values.length > allowed.length || invalid ||
-        values.map(&:first).uniq.length != values.length)
-      raise ArgumentError
-    end
-    values
-  end
-
-  # NULL always sorts last, including descending keys. The final key is unique.
-  def order_sql(expressions, final)
-    (@sort.flat_map do |key, direction|
-      expressions.fetch(key).flat_map do |expression|
-        ["(#{ expression }) IS NULL ASC", "#{ expression } #{ direction.upcase }"]
-      end
-    end + final).join(', ')
-  end
-
-  def detail_expressions
-    common = { 'resource_id' => ["#{ @table }.#{ @id_column }"],
-               'version_no' => ["#{ @table }.version_no"],
-               'is_latest' => ["(#{ latest_expression })"],
-               'user' => ['counts_actor.name', "#{ @table }.created_by_user_id"],
-               'created_at' => ["#{ @table }.created_at"] }
-    if @resource == 'posts'
-      fields = [
-          'title', 'url', 'thumbnail_base', 'video_ms',
-          'original_created_from', 'original_created_before']
-      common.merge(fields.to_h { [_1, ["#{ @table }.#{ _1 }"]] }).merge(
-          'tags' => json_order('post_versions.tags_json'),
-          'parents' => list_order('post_versions.parent_post_ids'))
-    else
-      name = "JSON_UNQUOTE(JSON_EXTRACT(tag_versions.names, #{ locale_name_path }))"
-      common.merge(
-          'name' => ["COALESCE(#{ name }, tag_versions.name)"],
-          'names' => json_order('tag_versions.names'),
-          'category' => ['tag_versions.category'],
-          'aliases' => json_order(
-              "JSON_EXTRACT(tag_versions.names, '$.*.aliases[*]')"),
-          'parents' => list_order('tag_versions.parent_tag_ids'),
-          'deprecated' => ['tag_versions.deprecated_at'])
-    end
-  end
-
-  def locale_name_path
-    language = Locale.find_by(code: @params.fetch('locale', 'ja'))&.language_code || 'ja'
-    path = "$.#{ JSON.generate(language) }.name"
-    @scope.klass.connection.quote(path)
-  end
-
-  # Multi-valued parent cells sort by cardinality, then binary snapshot text.
-  def json_order(column)
-    [
-        "COALESCE(JSON_LENGTH(#{ column }), 0)",
-        "COALESCE(CAST(#{ column } AS CHAR), '[]') COLLATE utf8mb4_bin"]
-  end
-
-  def list_order(column)
-    [
-        "CASE WHEN #{ column } = '' THEN 0 ELSE " +
-            "1 + LENGTH(#{ column }) - LENGTH(REPLACE(#{ column }, ' ', '')) END",
-        "#{ column } COLLATE utf8mb4_bin"]
-  end
-
-  def integer(key, default = nil)
-    value = @params[key].presence
-    return default if value.nil? && !default.nil?
-    raise ArgumentError unless value&.match?(/\A[1-9][0-9]{0,9}\z/)
-
-    value.to_i
-  end
-
-  def timestamp(key)
-    value = @params[key].presence
-    return nil unless value
-    pattern = /\A\d{4}-\d{2}-\d{2}T\d{2}:\d{2}(?::\d{2}(?:\.\d{1,6})?)?Z\z/
-    raise ArgumentError unless value.match?(pattern)
-
-    Date.iso8601(value.first(10))
-    Time.iso8601(value.length == 17 ? value.sub(/Z\z/, ':00Z') : value)
-  end
-
-  def apply_filters
-    at = timestamp('at')
-    first = timestamp('from')
-    last = timestamp('before')
-    raise ArgumentError if first && last && first >= last
-    raise ArgumentError unless ['0', '1'].include?(@params.fetch('latest', '1'))
-
-    @scope = @scope.where("#{ @table }.created_at <= ?", at) if at
-    @scope = @scope.where(Arel.sql(latest_expression)) unless @params['latest'] == '0'
-    @scope = @scope.where("#{ @table }.created_at >= ?", first) if first
-    @scope = @scope.where("#{ @table }.created_at < ?", last) if last
-    if @params['resource_id'].present?
-      @scope = @scope.where(@table => { @id_column => integer('resource_id') })
-    end
-    if @params['user_id'].present?
-      user = @params['user_id'] == 'none' ? nil : integer('user_id')
-      @scope = @scope.where(@table => { created_by_user_id: user })
-    end
-    return unless @params['tag'].present?
-    raise ArgumentError unless @resource == 'posts'
-
-    id = integer('tag')
-    @scope = @scope.where(
-        "JSON_CONTAINS(post_versions.tags_json, JSON_OBJECT('tag_id', CAST(? AS UNSIGNED))) OR " +
-        "JSON_CONTAINS(prev.tags_json, JSON_OBJECT('tag_id', CAST(? AS UNSIGNED)))", id, id)
-  end
-end

backend/app/services/counts_resource_definition.rb

@@ -1,146 +0,0 @@
-# Server-owned version identities, attributes and field types. No client SQL is accepted.
-class CountsResourceDefinition
-  COMMON = { 'resource_id' => :integer, 'version_no' => :integer,
-             'event_type' => :enum, 'user' => :integer, 'created_at' => :datetime }.freeze
-  DEFINITIONS = {
-      'posts' => [PostVersion, ['post_id'], {
-          'title' => :string, 'url' => :string, 'thumbnail_base' => :string,
-          'tags_json' => :json, 'parent_post_ids' => :string, 'video_ms' => :integer,
-          'original_created_from' => :datetime, 'original_created_before' => :datetime }],
-      'tags' => [TagVersion, ['tag_id'], {
-          'name' => :string, 'names' => :json, 'category' => :enum, 'aliases' => :string,
-          'parent_tag_ids' => :string, 'deprecated_at' => :datetime }],
-      'nico_tags' => [NicoTagVersion, ['tag_id'], {
-          'name' => :string, 'linked_tags' => :string }],
-      'tag_descriptions' => [TagDescriptionVersion, ['tag_id', 'language_code'], {
-          'language_code' => :enum, 'body' => :string,
-          'source_language_code' => :enum, 'source_version_no' => :integer }],
-      'tag_groups' => [TagGroupVersion, ['tag_group_id'], {
-          'name' => :string, 'description' => :string,
-          'members' => :json, 'placements' => :json }],
-      'materials' => [MaterialVersion, ['material_id'], {
-          'url' => :string, 'parent_id' => :integer, 'tag_id' => :integer,
-          'tag_name' => :string, 'tag_category' => :enum,
-          'discarded_at' => :datetime, 'updated_at' => :datetime,
-          'updated_by_user_id' => :integer, 'export_paths_json' => :json,
-          'file_blob_id' => :integer, 'file_filename' => :string,
-          'file_content_type' => :string, 'file_byte_size' => :integer,
-          'file_checksum' => :string, 'file_sha256' => :string,
-          'source_kind' => :enum, 'source_uri' => :string, 'source_path' => :string,
-          'source_file_id' => :string, 'normalized_source_key' => :string }] }.freeze
-  OPERATORS = {
-      string: ['eq', 'ne', 'like'], enum: ['eq', 'ne'], boolean: ['eq', 'ne'],
-      integer: ['eq', 'ne', 'gt', 'lt', 'ge', 'le', 'range'],
-      datetime: ['eq', 'ne', 'gt', 'lt', 'ge', 'le', 'range'],
-      interval: ['range'] }.freeze
-  LABELS = {
-      'resource_id' => 'counts:columns.resourceId', 'version_no' => 'counts:columns.version',
-      'is_latest' => 'counts:columns.latest', 'event_type' => 'counts:eventType',
-      'user' => 'history:actor', 'created_at' => 'counts:columns.changedAt',
-      'thumbnail' => 'counts:columns.thumbnailDisplay',
-      'thumbnail_base' => 'counts:columns.thumbnailURL',
-      'title' => 'counts:columns.postTitle', 'url' => 'counts:columns.postURL',
-      'tags' => 'counts:columns.tags', 'tags_json' => 'counts:columns.tags',
-      'video_ms' => 'shared:labels.duration', 'name' => 'counts:columns.name',
-      'names' => 'counts:columns.names', 'category' => 'common:categoryLabel',
-      'aliases' => 'tags:aliases', 'deprecated' => 'counts:columns.deprecated',
-      'original_created_from' => 'counts:columns.originalFrom',
-      'original_created_before' => 'counts:columns.originalBefore',
-      'language_code' => 'counts:columns.language' }.freeze
-  CHILD_LABELS = {
-      'id' => 'counts:columns.resourceId', 'versionNo' => 'history:version',
-      'kind' => 'counts:columns.kind', 'name' => 'history:name',
-      'category' => 'common:categoryLabel', 'sections' => 'counts:columns.sections',
-      'names' => 'counts:columns.names', 'aliases' => 'tags:aliases',
-      'languageCode' => 'counts:columns.language',
-      'scriptCode' => 'history:nameChanges.script',
-      'autoGenerated' => 'history:nameChanges.automatic',
-      'tagId' => 'counts:attributes.tag_id', 'tagGroupId' => 'counts:resources.tag_groups',
-      'sectionName' => 'counts:attributes.section_name',
-      'description' => 'counts:attributes.description',
-      'children' => 'counts:attributes.children', 'profile' => 'counts:attributes.profile',
-      'path' => 'counts:attributes.path' }.freeze
-
-  attr_reader :model, :keys, :fields, :resource
-
-  def initialize(resource)
-    @resource = resource
-    @model, @keys, attributes = DEFINITIONS.fetch(resource) { raise ArgumentError }
-    @fields = COMMON.merge(attributes)
-  end
-
-  def table = model.table_name
-  def time_column = 'created_at'
-  def column(key)
-    raise ArgumentError unless fields.key?(key)
-
-    { 'resource_id' => keys.first, 'user' => 'created_by_user_id' }.fetch(key, key)
-  end
-
-  def expression(key, row = 'v') = "#{ row }.#{ column(key) }"
-  def changes = fields.keys - COMMON.keys - ['updated_at', 'updated_by_user_id']
-  def axes(history)
-    fields.reject { |_key, type| type == :json }.keys +
-        (resource == 'posts' ? ['tags'] : []) + (history ? ['changed_field'] : [])
-  end
-
-  def metadata
-    fields.map do |key, type|
-      { key:, type:, nullable: model.columns_hash.fetch(column(key)).null,
-        operators: OPERATORS.fetch(type, []) }
-    end
-  end
-
-  def field_label(key) = LABELS.fetch(key, "counts:attributes.#{ key }")
-
-  def output_fields
-    content = case resource
-              when 'posts'
-                ['thumbnail', 'title', 'url', 'thumbnail_base', 'tags', 'parents', 'video_ms',
-                 'original_created_from', 'original_created_before']
-              when 'tags' then ['name', 'names', 'category', 'aliases', 'parents', 'deprecated']
-              else fields.keys - COMMON.keys
-              end
-    keys = ['resource_id', 'version_no', 'is_latest'] + (content - ['thumbnail']) +
-        ['event_type', 'user', 'created_at']
-    keys.unshift('thumbnail') if content.include?('thumbnail')
-    keys.map do |key|
-      label = if key == 'parents'
-                resource == 'tags' ? 'tags:parents' : 'shared:labels.parentPosts'
-              else field_label(key)
-              end
-      field = { key:, label: }
-      children = child_fields(key)
-      field[:children] = children if children.any?
-      field
-    end
-  end
-
-  def child_fields(key, depth = 4)
-    keys = case key
-           when 'tags' then ['id', 'versionNo', 'kind', 'name', 'category', 'sections', 'names']
-           when 'names' then ['languageCode', 'scriptCode', 'name', 'aliases', 'autoGenerated']
-           when 'aliases'
-             resource == 'tags' ? ['languageCode', 'name', 'scriptCode'] : ['name', 'scriptCode']
-           when 'parents' then resource == 'tags' ? ['id', 'name'] : ['id']
-           when 'members' then ['tagId', 'tagGroupId', 'versionNo']
-           when 'export_paths_json' then ['profile', 'path']
-           when 'placements', 'children'
-             scalar = ['sectionName', 'description', 'tagId', 'tagGroupId']
-             scalar + (depth > 0 ? ['children'] : [])
-           else []
-           end
-    keys.map do |child|
-      label = key == 'members' && child == 'tagId' ?
-          'counts:columns.resourceId' : CHILD_LABELS.fetch(child)
-      field = { key: child, label: }
-      children = child_fields(child, child == 'children' ? depth - 1 : depth)
-      # Language is attached to top-level aliases, not to aliases inside a language entry.
-      children = children.reject { _1[:key] == 'languageCode' } if child == 'aliases'
-      field[:children] = children if children.any?
-      field
-    end
-  end
-
-  def filter_metadata = @filter_metadata ||= CountsCondition.fields(self)
-end