コミットを比較

...
17 コミット
作成者 SHA1 メッセージ 日付
みてるぞ 5493673996 #434 2026-10-26 22:57:39 +09:00
みてるぞ 1154babe45 #434 2026-10-26 21:53:09 +09:00
みてるぞ 2563b3fef3 #434 2026-10-26 21:24:32 +09:00
みてるぞ d5bb9c8508 #434 2026-10-26 20:57:56 +09:00
みてるぞ f44a436e20 #434 2026-10-26 19:31:00 +09:00
みてるぞ c61d9d76e3 #434 2026-10-26 19:16:19 +09:00
みてるぞ 88711d2bd8 #434 2026-10-26 18:59:25 +09:00
みてるぞ b4139f8370 #434 2026-10-26 18:45:28 +09:00
みてるぞ f91f9d0eb5 #434 2026-10-26 12:44:22 +09:00
みてるぞ 3e8f209542 #434 2026-10-26 12:19:43 +09:00
みてるぞ 8642225b94 #434 2026-10-25 23:08:48 +09:00
みてるぞ 96a5bb8601 #434 2026-10-25 22:16:20 +09:00
みてるぞ e71663bd22 #434 2026-10-25 16:47:05 +09:00
みてるぞ 52a8d62be5 #434 2026-10-25 16:25:04 +09:00
みてるぞ 3e05757820 #434 2026-10-25 16:00:37 +09:00
みてるぞ d411b3b40f #434 2026-10-25 14:53:35 +09:00
みてるぞ 304090f717 #434 2026-10-25 14:14:55 +09:00
102個のファイルの変更、8204行の追加、220行の削除
+30 -4
ファイルの表示
@@ -1,5 +1,18 @@
# AGENTS.md # AGENTS.md
## Mandatory security and URL contracts
- URL クエリへの JSON 格納を禁止する。Base64・16進数等による迂回も禁止。
単純な配列は半角スペース区切り、構造化条件は番号付きの個別キーで表す。
未指定と明示的な空選択を区別し、自由入力を区切り文字で分割しない。
Frontend / Backend で同じ文法・上限・検証を使い、新しい URL 状態の追加前に確認する。
- 一度除票されたリソースの除票以前(除票版を含む)の履歴は admin 以外へ公開しない。
恢復しても解除しない。すべての後続除票を確定版番号と正しい複合キーで判定する。
本文・属性・ID・版番号・関連参照・旧値を API / SSR / キャッシュ / 出力から漏らさない。
サーバーが確定した current_user の role のみを認可に使用する。
秘匿版も安全な全体件数には含めるが、公開明細・内訳・検索件数とは分離する。
秘匿属性を条件・軸・ソート・差分件数から推測できる実装を禁止する。
## Project overview ## Project overview
BTRC Hub / タグ広場 is a split Rails API and React frontend repository. BTRC Hub / タグ広場 is a split Rails API and React frontend repository.
@@ -573,10 +586,10 @@ and layout reuse, follow `frontend/AGENTS.md`.
- When new user-facing wording appears necessary, ask the user for the exact - When new user-facing wording appears necessary, ask the user for the exact
wording and placement before implementing it. wording and placement before implementing it.
- Do not invent replacement copy when removing unrequested wording. - Do not invent replacement copy when removing unrequested wording.
- Do not create, modify, or run tests unless the user explicitly asks for - 明示的な個別指示がない限り、テストの追加・修正・削除・実行を一切行わない。
test work. When the user asks for tests, keep working within the permitted fixture、mock、snapshot、テスト補助コードも対象とする。
test-file scope and rerun them until they pass or the remaining failure is 既存のテスト差分を勝手に修正・差戻ししない。
clearly blocked. この規約はディレクトリ固有の自動的なテスト実行推奨より優先する。
- Test-only work includes adding, updating, deleting, reorganising, or fixing - Test-only work includes adding, updating, deleting, reorganising, or fixing
SyntaxError in tests. During test-only work, do not modify production code. SyntaxError in tests. During test-only work, do not modify production code.
- During test-only work, do not change production constants, behaviour, API - During test-only work, do not change production constants, behaviour, API
@@ -631,6 +644,19 @@ and layout reuse, follow `frontend/AGENTS.md`.
## Frontend rules ## Frontend rules
- 原則として、ユーザ向け UI 文言を TSX / TypeScript に直接記述しない。
新しい文言は既存の i18n システムで管理し、適切な既存キーを先に確認する。
不必要な重複定義を避け、新規機能固有の文言は適切な namespace にまとめる。
- 対応 locale の辞書を整合させる。未翻訳項目には既存の日本語 fallback と
`_untranslated` 規約を使い、language metadata を維持する。
翻訳を捏造せず、未翻訳文字列を正式な翻訳として登録しない。
- UI 文言の追加・変更時はビルド、静的検査、実画面の確認等で、辞書と
SSR / hydration の整合及び翻訳キーの非露出を検証する。
i18n の検証を理由にテスト関連作業を開始しない。
- API の内部識別子、パラメータ、ユーザ入力データ、外部 URL は機械的に
翻訳しない。ボタン、項目名、CSV ヘッダ、エラー、空状態、ツールチップ、
アクセシビリティ用ラベルも UI 文言として扱う。
- Use `frontend/src/lib/api.ts` for API calls so headers and camelCase conversion stay consistent. - Use `frontend/src/lib/api.ts` for API calls so headers and camelCase conversion stay consistent.
- Add or reuse TanStack Query keys through `frontend/src/lib/queryKeys.ts`; - Add or reuse TanStack Query keys through `frontend/src/lib/queryKeys.ts`;
avoid ad hoc query key arrays. avoid ad hoc query key arrays.
+15
ファイルの表示
@@ -1,5 +1,20 @@
# backend/AGENTS.md # backend/AGENTS.md
## Mandatory security and URL contracts
- URL クエリへの JSON 格納を禁止する。Base64・16進数等による迂回も禁止。
単純な配列は半角スペース区切り、構造化条件は番号付きの個別キーで表す。
未指定と明示的な空選択を区別し、自由入力を区切り文字で分割しない。
Frontend / Backend で同じ文法・上限・検証を使い、新しい URL 状態の追加前に確認する。
- 一度除票されたリソースの除票以前(除票版を含む)の履歴は admin 以外へ公開しない。
恢復しても解除しない。すべての後続除票を確定版番号と正しい複合キーで判定する。
本文・属性・ID・版番号・関連参照・旧値を API / SSR / キャッシュ / 出力から漏らさない。
サーバーが確定した current_user の role のみを認可に使用する。
秘匿版も安全な全体件数には含めるが、公開明細・内訳・検索件数とは分離する。
秘匿属性を条件・軸・ソート・差分件数から推測できる実装を禁止する。
- 履歴読み取りは HistoryAccess の共通認可を必須とする。書込用 relation を公開しない。
export は生成時と取得時に認可し、権限変更・除票後に古いファイルを再利用しない。
## Scope ## Scope
These rules apply to work under `backend/`. These rules apply to work under `backend/`.
+22 -22
ファイルの表示
@@ -1,4 +1,5 @@
class ApplicationController < ActionController::API class ApplicationController < ActionController::API
include RequestAccess
include ActionController::Cookies include ActionController::Cookies
include ActionController::RequestForgeryProtection include ActionController::RequestForgeryProtection
@@ -8,34 +9,46 @@ class ApplicationController < ActionController::API
rescue_from ActiveRecord::RecordInvalid, with: :render_record_invalid rescue_from ActiveRecord::RecordInvalid, with: :render_record_invalid
rescue_from ActiveRecord::RecordNotUnique, with: :render_record_not_unique rescue_from ActiveRecord::RecordNotUnique, with: :render_record_not_unique
rescue_from(ActiveRecord::RecordNotFound) { head :not_found }
rescue_from(UnsupportedLocale) { head :not_found } rescue_from(UnsupportedLocale) { head :not_found }
before_action :reject_banned_ip_address! before_action :reject_banned_ip_address!
before_action :authenticate_user before_action :authenticate_user
before_action :reject_banned_user! before_action :reject_banned_user!
before_action :validate_explicit_locale! before_action :validate_explicit_locale!
before_action :enforce_query_contract!
around_action :with_history_authority
rescue_from(HistoryAccess::Forbidden) { head :not_found }
protect_from_forgery with: :exception protect_from_forgery with: :exception
self.forgery_protection_origin_check = true self.forgery_protection_origin_check = true
rescue_from ActionController::InvalidAuthenticityToken, with: :reject_invalid_csrf rescue_from ActionController::InvalidAuthenticityToken, with: :reject_invalid_csrf
after_action :prevent_authenticated_caching after_action :prevent_authenticated_caching
def current_user = @current_user
private private
def enforce_query_contract!
pairs = URI.decode_www_form(request.query_string)
return head :bad_request if pairs.length > 1000 || pairs.map(&:first).uniq.length != pairs.length
return head :bad_request if request.query_parameters.values.any? { !_1.is_a?(String) }
if (controller_name.start_with?('counts_') &&
(request.query_parameters.keys & ['headers', 'labels']).any?)
return head :bad_request
end
rescue ArgumentError
head :bad_request
end
def with_history_authority(&action)
response.headers['Cache-Control'] = 'private, no-store'
HistoryAccess.set(user: current_user, &action)
end
def reject_invalid_csrf def reject_invalid_csrf
response.headers['X-CSRF-Invalid'] = '1' response.headers['X-CSRF-Invalid'] = '1'
response.headers['Cache-Control'] = 'private, no-store' response.headers['Cache-Control'] = 'private, no-store'
head :forbidden head :forbidden
end end
def authenticate_user
code = session[:inheritance_code]
return if code.blank?
@current_user = User.find_by(inheritance_code: code)
end
def prevent_authenticated_caching def prevent_authenticated_caching
response.headers['Cache-Control'] = 'private, no-store' if current_user response.headers['Cache-Control'] = 'private, no-store' if current_user
end end
@@ -79,19 +92,6 @@ class ApplicationController < ActionController::API
render_validation_error base: ['すでに存在してゐます.'] render_validation_error base: ['すでに存在してゐます.']
end end
def reject_banned_ip_address!
ip_address = IpAddress.find_by(ip_address: IPAddr.new(request.remote_ip).hton)
return unless ip_address&.banned?
head :forbidden
end
def reject_banned_user!
return unless current_user&.banned?
head :forbidden
end
def render_validation_error record = nil, fields: { }, base: [], status: :unprocessable_entity def render_validation_error record = nil, fields: { }, base: [], status: :unprocessable_entity
errors = { } errors = { }
+21
ファイルの表示
@@ -0,0 +1,21 @@
module CountsListing
extend ActiveSupport::Concern
def index
locale = resolve_locale!
resource = is_a?(CountsPostsController) ? 'posts' : 'tags'
query = CountsQuery.new(version_scope, resource, request.query_parameters.except('locale')
.merge('locale' => locale.code))
result = { population_count_scope: 'all_recorded_versions', population_count: HistoryAccess.population_count(query.relation.klass), count: query.count, page: query.page, limit: query.limit,
at: request.query_parameters['at'].presence || Time.current.utc.iso8601(6) }
if query.mode == 'summary'
result.merge!(query.summary)
else
result[:versions] = CountsVersionRepr.many(
query.details, resource:, language_code: locale.language_code)
end
render json: result
rescue ArgumentError, TypeError
head :bad_request
end
end
+52
ファイルの表示
@@ -0,0 +1,52 @@
# Permanent storage URLs must not bypass a resource's later discard boundary.
module HistoryStorageAccess
extend ActiveSupport::Concern
included do
include RequestAccess
prepend_before_action :authorise_history_storage!
end
private
def authorise_history_storage!
response.headers['Cache-Control'] = 'private, no-store'
return unless request.get? || request.head?
reject_banned_ip_address!
return if performed?
authenticate_user
reject_banned_user!
return if performed?
signed = params[:signed_blob_id] || params[:signed_id]
blob = if signed
ActiveStorage::Blob.find_signed(signed)
elsif params[:encoded_key]
payload = ActiveStorage.verifier.verified(params[:encoded_key], purpose: :blob_key)
key = payload && (payload[:key] || payload['key'])
if key
original = key.start_with?('variants/') ? key.split('/')[1] : key
ActiveStorage::Blob.find_by(key: original)
end
end
return head :not_found unless blob
allowed = HistoryAccess.set(user: current_user) { HistoryAccess.blob_visible?(blob) }
head :not_found unless allowed
rescue ActiveSupport::MessageVerifier::InvalidSignature
head :not_found
end
# Override the engine's public-forever caching before a streaming response commits.
def http_cache_forever(**_options)
response.headers['Cache-Control'] = 'private, no-store'
yield
end
def expires_in(*arguments, **options)
super
response.headers['Cache-Control'] = 'private, no-store'
end
end
+28
ファイルの表示
@@ -0,0 +1,28 @@
# Canonical read-side session and BAN boundary, shared by API and storage controllers.
module RequestAccess
extend ActiveSupport::Concern
def current_user = @current_user
private
def authenticate_user
code = session[:inheritance_code]
return if code.blank?
@current_user = User.find_by(inheritance_code: code)
end
def reject_banned_ip_address!
ip_address = IpAddress.find_by(ip_address: IPAddr.new(request.remote_ip).hton)
return unless ip_address&.banned?
head :forbidden
end
def reject_banned_user!
return unless current_user&.banned?
head :forbidden
end
end
+83
ファイルの表示
@@ -0,0 +1,83 @@
class CountsExportsController < ApplicationController
wrap_parameters false
def create_job
locale = resolve_locale!
payload = request.request_parameters.merge(request.query_parameters)
session[:counts_export_owner] ||= SecureRandom.hex(32)
job = CountsExportJobs.start(payload, session[:counts_export_owner], locale.language_code)
response.headers['Cache-Control'] = 'private, no-store'
render json: { job: }, status: :accepted
rescue CountsExportDownload::Busy
head :too_many_requests
rescue ArgumentError, TypeError, KeyError, JSON::ParserError
head :bad_request
end
def job_status
job_state(cancel: false)
end
def cancel_job
job_state(cancel: true)
end
def prepare
locale = resolve_locale!
payload = request.request_parameters.merge(request.query_parameters)
exporter = CountsExporter.new(payload, language_code: locale.language_code)
session[:counts_export_owner] ||= SecureRandom.hex(32)
token = CountsExportDownload.prepare(exporter, session[:counts_export_owner])
response.headers['Cache-Control'] = 'private, no-store'
render json: { token: }
rescue CountsExportDownload::Busy
head :too_many_requests
rescue CountsExportDownload::TooLarge
head 413
rescue CountsExportDownload::TimedOut
head :gateway_timeout
rescue CountsExportDownload::StorageFull, Errno::ENOSPC
head :insufficient_storage
rescue ArgumentError, TypeError, KeyError, JSON::ParserError
head :bad_request
end
def download
file = CountsExportDownload.find(params[:token], session[:counts_export_owner])
return head :not_found unless file
response.headers['Cache-Control'] = 'private, no-store'
send_file file.fetch('path'),
filename: "counts-#{ file.fetch('resource') }.#{ file.fetch('format') }",
type: file['format'] == 'json' ? 'application/json' : 'text/csv',
disposition: 'attachment'
end
def job_state(cancel:)
result = CountsExportJobs.status(params[:token], session[:counts_export_owner], cancel:)
return head :not_found unless result
response.headers['Cache-Control'] = 'private, no-store'
render json: result
end
private :job_state
def index
locale = resolve_locale!
payload = request.request_parameters.except('authenticity_token')
.merge(request.query_parameters)
exporter = CountsExporter.new(payload,
language_code: locale.language_code)
session[:counts_export_owner] ||= SecureRandom.hex(32)
token = CountsExportDownload.prepare(exporter, session[:counts_export_owner])
file = CountsExportDownload.find(token, session[:counts_export_owner])
return head :not_found unless file
send_file file.fetch('path'),
filename: "counts-#{ exporter.resource }.#{ exporter.format }",
type: exporter.format == 'json' ? 'application/json' : 'text/csv',
disposition: 'attachment'
rescue ArgumentError, TypeError, KeyError, JSON::ParserError
head :bad_request
end
end
+3
ファイルの表示
@@ -0,0 +1,3 @@
class CountsPostsController < PostVersionsController
include CountsListing
end
+10
ファイルの表示
@@ -0,0 +1,10 @@
class CountsStatesController < ApplicationController
def index
locale = resolve_locale!
result = CountsStateQuery.new(request.query_parameters,
language_code: locale.language_code).call
render json: result
rescue ArgumentError, TypeError
head :bad_request
end
end
+21
ファイルの表示
@@ -0,0 +1,21 @@
class CountsTablesController < ApplicationController
def definition
resolve_locale!
definition = CountsResourceDefinition.new(params[:resource])
render json: { fields: definition.metadata,
axes: { latest: definition.axes(false), history: definition.axes(true) },
grains: CountsAxes::GRAINS, defaults: CountsDefaults.metadata(params[:resource]) }
rescue ArgumentError
head :bad_request
end
def index
resolve_locale!
render json: CountsTableQuery.new(request.query_parameters).call
rescue CountsTableQuery::TooManyColumns
render json: { error: 'too_many_columns', max_columns: CountsTableQuery::MAX_COLUMNS },
status: :unprocessable_entity
rescue ArgumentError, TypeError, KeyError, JSON::ParserError
head :bad_request
end
end
+3
ファイルの表示
@@ -0,0 +1,3 @@
class CountsTagsController < TagVersionsController
include CountsListing
end
+2 -2
ファイルの表示
@@ -8,7 +8,7 @@ class MaterialVersionsController < ApplicationController
offset = (page - 1) * limit offset = (page - 1) * limit
q = MaterialVersion.includes(:created_by_user) q = HistoryAccess.visible(MaterialVersion.includes(:created_by_user))
q = q.where(material_id: params[:material_id]) if params[:material_id].present? q = q.where(material_id: params[:material_id]) if params[:material_id].present?
q = q.where(tag_name: params[:tag].to_s.strip) if params[:tag].present? q = q.where(tag_name: params[:tag].to_s.strip) if params[:tag].present?
q = q.where(event_type: params[:event_type]) if params[:event_type].present? q = q.where(event_type: params[:event_type]) if params[:event_type].present?
@@ -20,7 +20,7 @@ class MaterialVersionsController < ApplicationController
.limit(limit) .limit(limit)
.offset(offset) .offset(offset)
render json: { versions: serialise_versions(versions), count: } render json: { versions: serialise_versions(versions), count:, population_count: HistoryAccess.population_count(MaterialVersion) }
end end
private private
+15 -17
ファイルの表示
@@ -14,21 +14,7 @@ class PostVersionsController < ApplicationController
offset = (page - 1) * limit offset = (page - 1) * limit
q = PostVersion.joins(<<~SQL.squish) q = HistoryAccess.visible(version_scope)
LEFT JOIN
post_versions prev
ON
prev.post_id = post_versions.post_id
AND prev.version_no = post_versions.version_no - 1
SQL
.select('post_versions.*',
'prev.title AS prev_title',
'prev.url AS prev_url',
'prev.thumbnail_base AS prev_thumbnail_base',
'prev.tags_json AS prev_tags_json',
'prev.video_ms AS prev_video_ms',
'prev.original_created_from AS prev_original_created_from',
'prev.original_created_before AS prev_original_created_before')
q = q.where('post_versions.post_id = ?', post_id) if post_id q = q.where('post_versions.post_id = ?', post_id) if post_id
if external_tag_id || (tag_id && !(Tag.exists?(id: tag_id))) if external_tag_id || (tag_id && !(Tag.exists?(id: tag_id)))
q = q.where('JSON_CONTAINS(post_versions.tags_json,' + q = q.where('JSON_CONTAINS(post_versions.tags_json,' +
@@ -46,11 +32,23 @@ class PostVersionsController < ApplicationController
.limit(limit) .limit(limit)
.offset(offset) .offset(offset)
render json: { versions: serialise_versions(versions, locale.language_code), count: } render json: { versions: serialise_versions(versions, locale.language_code), count:, population_count: HistoryAccess.population_count(PostVersion) }
end end
private private
def version_scope
PostVersion.with_previous_version
.select('post_versions.*',
'prev.title AS prev_title',
'prev.url AS prev_url',
'prev.thumbnail_base AS prev_thumbnail_base',
'prev.tags_json AS prev_tags_json',
'prev.video_ms AS prev_video_ms',
'prev.original_created_from AS prev_original_created_from',
'prev.original_created_before AS prev_original_created_before')
end
def serialise_versions rows, language_code def serialise_versions rows, language_code
rows = rows.to_a rows = rows.to_a
user_ids = rows.map(&:created_by_user_id).compact.uniq user_ids = rows.map(&:created_by_user_id).compact.uniq
@@ -103,7 +101,7 @@ class PostVersionsController < ApplicationController
} }
missing_ids = ids - names.keys missing_ids = ids - names.keys
NicoTagVersion HistoryAccess.visible(NicoTagVersion.all)
.where(tag_id: missing_ids) .where(tag_id: missing_ids)
.order(:tag_id, version_no: :desc) .order(:tag_id, version_no: :desc)
.pluck(:tag_id, :name) .pluck(:tag_id, :name)
+9 -55
ファイルの表示
@@ -38,7 +38,7 @@ class PostsController < ApplicationController
offset = (page - 1) * limit offset = (page - 1) * limit
pt_max_sql = pt_max_sql =
PostVersion HistoryAccess.visible(PostVersion.all)
.select('post_id, MAX(created_at) AS max_updated_at') .select('post_id, MAX(created_at) AS max_updated_at')
.group('post_id') .group('post_id')
.to_sql .to_sql
@@ -344,6 +344,7 @@ class PostsController < ApplicationController
unless force unless force
base_version = post.post_versions.find_by!(version_no: base_version_no) base_version = post.post_versions.find_by!(version_no: base_version_no)
HistoryAccess.require!(base_version)
base_snapshot = post_snapshot_from_version(base_version) base_snapshot = post_snapshot_from_version(base_version)
current_snapshot = post_snapshot_from_record(post, locale) current_snapshot = post_snapshot_from_record(post, locale)
end end
@@ -449,24 +450,8 @@ class PostsController < ApplicationController
private private
def tag_filter_references literals def tag_filter_references literals
raw = params[:tag_refs] PostTagSearch.parse_references(params.to_unsafe_h, literals)
return nil if raw.blank? rescue ArgumentError, JSON::ParserError, TypeError
references = JSON.parse(raw)
valid = references.is_a?(Array) && references.all? do |reference|
next false unless reference.is_a?(Hash) && reference['name'].is_a?(String)
ids = [reference['tag_id'], reference['external_tag_id']].compact
((ids.any? || reference['missing'] == true) &&
ids.all? { _1.is_a?(Integer) && _1.positive? } &&
(!reference.key?('missing') || (reference['missing'] == true && ids.empty?)) &&
(!reference.key?('negative') || [true, false].include?(reference['negative'])))
end
raise ActionController::BadRequest unless valid
return nil unless references.map { _1['name'] } == literals
references
rescue JSON::ParserError, TypeError
raise ActionController::BadRequest raise ActionController::BadRequest
end end
@@ -482,17 +467,7 @@ class PostsController < ApplicationController
def filter_posts_by_tags locale, tag_names, match_type def filter_posts_by_tags locale, tag_names, match_type
references = tag_filter_references(tag_names) references = tag_filter_references(tag_names)
literals = tag_names.each_with_index.map do |raw_name, index| literals = PostTagSearch.literals(locale, tag_names.join(' '), references:)
name = raw_name.sub(/\Anot:/i, '')
name = TagName.canonicalise(locale, name).first unless references
reference = references&.[](index)
negative = if reference&.key?('negative')
reference['negative']
else
raw_name.downcase.start_with?('not:')
end
{ name:, negative:, reference: }
end
return Post.all if literals.empty? return Post.all if literals.empty?
@@ -523,31 +498,10 @@ class PostsController < ApplicationController
end end
def tagged_post_ids_for locale, name, reference: nil def tagged_post_ids_for locale, name, reference: nil
if reference internal, external = PostTagSearch.references(locale, { name:, reference: })
matches = Post.none Post.where(id: PostTag.where(tag_id: Array(internal)).select(:post_id))
if reference['tag_id'] .or(Post.where(id: PostExternalTag.where(external_tag_id: Array(external))
matches = matches.or(Post.where(id: PostTag.where( .select(:post_id))).select(:id)
tag_id: reference['tag_id']).select(:post_id)))
end
if reference['external_tag_id']
matches = matches.or(Post.where(id: PostExternalTag.where(
external_tag_id: reference['external_tag_id']).select(:post_id)))
end
return matches.select(:id)
end
posts_by_internal_tags =
Post
.joins(tags: :tag_names)
.where(tag_names: { language_code: locale.language_code, name: })
.select(:id)
posts_by_external_tags =
Post
.joins(:external_tags)
.where("CONCAT(external_tags.platform, ':', external_tags.name) = ?", name)
(posts_by_internal_tags + posts_by_external_tags).uniq(&:id)
end end
def sync_post_tags! post, desired_tags, sections def sync_post_tags! post, desired_tags, sections
+2 -1
ファイルの表示
@@ -42,7 +42,7 @@ class SitemapEntriesController < ApplicationController
when 'descriptions' when 'descriptions'
TagDescription.where(language_code: locale.language_code, tag_id: named_tags) TagDescription.where(language_code: locale.language_code, tag_id: named_tags)
.where.not(body: [nil, '']) .where.not(body: [nil, ''])
.where(TagDescriptionVersion .where(HistoryAccess.visible(TagDescriptionVersion.all)
.where('tag_description_versions.tag_id = tag_descriptions.tag_id') .where('tag_description_versions.tag_id = tag_descriptions.tag_id')
.where('tag_description_versions.language_code = tag_descriptions.language_code') .where('tag_description_versions.language_code = tag_descriptions.language_code')
.where('tag_description_versions.version_no = tag_descriptions.version_no') .where('tag_description_versions.version_no = tag_descriptions.version_no')
@@ -50,6 +50,7 @@ class SitemapEntriesController < ApplicationController
when 'tag_groups' when 'tag_groups'
TagGroup.joins(:versions) TagGroup.joins(:versions)
.where('tag_group_versions.version_no = tag_groups.version_no') .where('tag_group_versions.version_no = tag_groups.version_no')
.where(Arel.sql(HistoryAccess.sql(TagGroupVersion)))
when 'theatres' when 'theatres'
Theatre.kept Theatre.kept
end end
+1
ファイルの表示
@@ -49,6 +49,7 @@ class TagDescriptionsController < ApplicationController
private private
def representation description def representation description
HistoryAccess.require_current!(description, TagDescriptionVersion)
description.as_json(only: [ description.as_json(only: [
:tag_id, :tag_id,
:language_code, :language_code,
+1 -1
ファイルの表示
@@ -1,7 +1,7 @@
class TagGroupsController < ApplicationController class TagGroupsController < ApplicationController
def index def index
locale = resolve_locale! locale = resolve_locale!
groups = TagGroup.includes(:created_by_user, :updated_by_user).order(:id) groups = HistoryAccess.current(TagGroup.all, TagGroupVersion).includes(:created_by_user, :updated_by_user).order(:id)
render json: TagGroupRepr.many(locale, groups.to_a) render json: TagGroupRepr.many(locale, groups.to_a)
end end
+17 -12
ファイルの表示
@@ -10,17 +10,7 @@ class TagVersionsController < ApplicationController
offset = (page - 1) * limit offset = (page - 1) * limit
q = TagVersion.joins(<<~SQL.squish) q = HistoryAccess.visible(version_scope)
LEFT JOIN
tag_versions prev
ON
prev.tag_id = tag_versions.tag_id
AND prev.version_no = tag_versions.version_no - 1
SQL
.select('tag_versions.*', 'prev.name AS prev_name', 'prev.category AS prev_category',
'prev.deprecated_at AS prev_deprecated_at',
'prev.aliases AS prev_aliases', 'prev.names AS prev_names',
'prev.parent_tag_ids AS prev_parent_tag_ids')
q = q.where('tag_versions.tag_id = ?', tag_id) if tag_id q = q.where('tag_versions.tag_id = ?', tag_id) if tag_id
count = q.except(:select, :order, :limit, :offset).count count = q.except(:select, :order, :limit, :offset).count
@@ -29,11 +19,26 @@ class TagVersionsController < ApplicationController
.limit(limit) .limit(limit)
.offset(offset) .offset(offset)
render json: { versions: serialise_versions(versions, locale.language_code), count: } render json: { versions: serialise_versions(versions, locale.language_code), count:, population_count: HistoryAccess.population_count(TagVersion) }
end end
private private
def version_scope
TagVersion.joins(<<~SQL.squish)
LEFT JOIN
tag_versions prev
ON
prev.tag_id = tag_versions.tag_id
AND prev.version_no = tag_versions.version_no - 1
AND #{ HistoryAccess.sql(TagVersion, 'prev') }
SQL
.select('tag_versions.*', 'prev.name AS prev_name', 'prev.category AS prev_category',
'prev.deprecated_at AS prev_deprecated_at',
'prev.aliases AS prev_aliases', 'prev.names AS prev_names',
'prev.parent_tag_ids AS prev_parent_tag_ids')
end
def serialise_versions rows, language_code def serialise_versions rows, language_code
rows = rows.to_a rows = rows.to_a
tag_ids = rows.map(&:tag_id).uniq tag_ids = rows.map(&:tag_id).uniq
+14 -14
ファイルの表示
@@ -8,12 +8,12 @@ class WikiPagesController < ApplicationController
title = params[:title].to_s.strip title = params[:title].to_s.strip
descriptions = descriptions =
TagDescription HistoryAccess.current(TagDescription.all, TagDescriptionVersion)
.where(language_code: locale.language_code) .where(language_code: locale.language_code)
.includes(tag: :tag_names) .includes(tag: :tag_names)
descriptions = descriptions.order(updated_at: :desc, id: :asc) descriptions = descriptions.order(updated_at: :desc, id: :asc)
tag_groups = TagGroup.order(updated_at: :desc, id: :asc) tag_groups = HistoryAccess.current(TagGroup.all, TagGroupVersion).order(updated_at: :desc, id: :asc)
if title.present? if title.present?
pattern = "%#{ ApplicationRecord.sanitize_sql_like(title) }%" pattern = "%#{ ApplicationRecord.sanitize_sql_like(title) }%"
descriptions = descriptions =
@@ -41,13 +41,13 @@ class WikiPagesController < ApplicationController
return head :not_found unless language.is_a?(String) return head :not_found unless language.is_a?(String)
return head :not_found unless Language.exists?(code: language) return head :not_found unless Language.exists?(code: language)
names = TagDescription.joins(tag: :tag_names) names = HistoryAccess.current(TagDescription.all, TagDescriptionVersion).joins(tag: :tag_names)
.where(language_code: locale.language_code, tag_names: { language_code: language }) .where(language_code: locale.language_code, tag_names: { language_code: language })
.pluck('tag_descriptions.tag_id', 'tag_descriptions.id', 'tag_names.name') .pluck('tag_descriptions.tag_id', 'tag_descriptions.id', 'tag_names.name')
candidates = names.map do |tag_id, description_id, name| candidates = names.map do |tag_id, description_id, name|
WikiPageRepr.autolink_tag(locale, tag_id:, description_id:, name:, language:) WikiPageRepr.autolink_tag(locale, tag_id:, description_id:, name:, language:)
end end
candidates.concat(TagGroup.order(:id).pluck(:id, :name).map do |id, name| candidates.concat(HistoryAccess.current(TagGroup.all, TagGroupVersion).order(:id).pluck(:id, :name).map do |id, name|
WikiPageRepr.autolink_group(locale, id:, name:, language:) WikiPageRepr.autolink_group(locale, id:, name:, language:)
end) end)
render json: candidates render json: candidates
@@ -286,12 +286,12 @@ class WikiPagesController < ApplicationController
tag_group = id && TagGroup.find_by(id:) tag_group = id && TagGroup.find_by(id:)
if tag_group if tag_group
return render json: tag_group_changes(TagGroupVersion.where(tag_group_id: tag_group.id)) return render json: tag_group_changes(HistoryAccess.visible(TagGroupVersion.all).where(tag_group_id: tag_group.id))
end end
json = [] json = []
unless id unless id
json.concat(tag_group_changes(TagGroupVersion.all)) json.concat(tag_group_changes(HistoryAccess.visible(TagGroupVersion.all)))
json.concat(description_changes(locale, json.concat(description_changes(locale,
TagDescription.where(language_code: locale.language_code))) TagDescription.where(language_code: locale.language_code)))
json = json.each_with_index.sort_by do |row, index| json = json.each_with_index.sort_by do |row, index|
@@ -327,7 +327,7 @@ class WikiPagesController < ApplicationController
def render_group_representation representation, version, body: nil, status: :ok def render_group_representation representation, version, body: nil, status: :ok
body ||= TagGroupMarkdownRenderer.render(representation) body ||= TagGroupMarkdownRenderer.render(representation)
versions = TagGroupVersion.where(tag_group_id: version.tag_group_id) versions = HistoryAccess.visible(TagGroupVersion.all).where(tag_group_id: version.tag_group_id)
pred = versions.find_by(version_no: version.version_no - 1)&.id pred = versions.find_by(version_no: version.version_no - 1)&.id
succ = versions.find_by(version_no: version.version_no + 1)&.id succ = versions.find_by(version_no: version.version_no + 1)&.id
updated_at = version.created_at updated_at = version.created_at
@@ -346,11 +346,11 @@ class WikiPagesController < ApplicationController
end end
def current_group_version group def current_group_version group
TagGroupVersion.find_by(tag_group_id: group.id, version_no: group.version_no) HistoryAccess.visible(TagGroupVersion.all).find_by(tag_group_id: group.id, version_no: group.version_no)
end end
def group_version group, version_id def group_version group, version_id
TagGroupVersion.find_by(tag_group_id: group.id, id: version_id) HistoryAccess.visible(TagGroupVersion.all).find_by(tag_group_id: group.id, id: version_id)
end end
def wiki_diff from_body, to_body def wiki_diff from_body, to_body
@@ -373,7 +373,7 @@ class WikiPagesController < ApplicationController
def tag_group_changes versions def tag_group_changes versions
rows = versions.includes(:created_by_user).order(id: :desc).limit(200).to_a rows = versions.includes(:created_by_user).order(id: :desc).limit(200).to_a
groups = TagGroup.where(id: rows.map(&:tag_group_id).uniq).index_by(&:id) groups = TagGroup.where(id: rows.map(&:tag_group_id).uniq).index_by(&:id)
predecessors = TagGroupVersion.where(tag_group_id: groups.keys, predecessors = HistoryAccess.visible(TagGroupVersion.all).where(tag_group_id: groups.keys,
version_no: rows.map { _1.version_no - 1 }) version_no: rows.map { _1.version_no - 1 })
.pluck(:tag_group_id, :version_no, :id) .pluck(:tag_group_id, :version_no, :id)
.to_h { |group_id, number, version_id| [[group_id, number], version_id] } .to_h { |group_id, number, version_id| [[group_id, number], version_id] }
@@ -406,18 +406,18 @@ class WikiPagesController < ApplicationController
end end
def description_versions description def description_versions description
TagDescriptionVersion.where(tag_id: description.tag_id, HistoryAccess.visible(TagDescriptionVersion.all).where(tag_id: description.tag_id,
language_code: description.language_code) language_code: description.language_code)
end end
def tag_description_by_title locale, title def tag_description_by_title locale, title
TagDescription.joins(tag: :tag_names) HistoryAccess.current(TagDescription.all, TagDescriptionVersion).joins(tag: :tag_names)
.find_by(language_code: locale.language_code, .find_by(language_code: locale.language_code,
tag_names: { language_code: locale.language_code, name: title }) tag_names: { language_code: locale.language_code, name: title })
end end
def description_changes locale, descriptions def description_changes locale, descriptions
versions = TagDescriptionVersion.where(language_code: locale.language_code, versions = HistoryAccess.visible(TagDescriptionVersion.all).where(language_code: locale.language_code,
tag_id: descriptions.select(:tag_id)) tag_id: descriptions.select(:tag_id))
.order(id: :desc).limit(200).to_a .order(id: :desc).limit(200).to_a
return [] if versions.empty? return [] if versions.empty?
@@ -425,7 +425,7 @@ class WikiPagesController < ApplicationController
by_tag = descriptions.where(tag_id: versions.map(&:tag_id)) by_tag = descriptions.where(tag_id: versions.map(&:tag_id))
.includes(tag: :tag_names).index_by(&:tag_id) .includes(tag: :tag_names).index_by(&:tag_id)
users = User.where(id: versions.map(&:created_by_user_id)).index_by(&:id) users = User.where(id: versions.map(&:created_by_user_id)).index_by(&:id)
predecessors = TagDescriptionVersion.where( predecessors = HistoryAccess.visible(TagDescriptionVersion.all).where(
language_code: locale.language_code, tag_id: by_tag.keys, language_code: locale.language_code, tag_id: by_tag.keys,
version_no: versions.map { |version| version.version_no - 1 }) version_no: versions.map { |version| version.version_no - 1 })
.pluck(:tag_id, :version_no, :id) .pluck(:tag_id, :version_no, :id)
+151
ファイルの表示
@@ -0,0 +1,151 @@
require 'digest'
# Request/worker authority comes only from a server-resolved user. Default is public.
class HistoryAccess < ActiveSupport::CurrentAttributes
attribute :user
class Forbidden < StandardError; end
KEYS = { 'post_versions' => ['post_id'], 'tag_versions' => ['tag_id'],
'nico_tag_versions' => ['tag_id'],
'tag_description_versions' => ['tag_id', 'language_code'],
'tag_group_versions' => ['tag_group_id'], 'material_versions' => ['material_id'] }.freeze
def self.revision
values = KEYS.keys.map do |table|
table.classify.constantize.unscoped.where(event_type: 'discard').maximum(:id)
end
Digest::SHA256.hexdigest(values.to_json)
end
def self.authority
{ 'policy_revision' => revision, 'admin' => admin?, 'actor_id' => user&.id }
end
def self.authorised_artifact?(metadata)
return false unless metadata['policy_revision'].is_a?(String)
if metadata['admin']
admin? && user.id == metadata['actor_id']
else
metadata['policy_revision'] == revision
end
end
def self.admin? = user&.admin? == true
def self.sql(model, row = model.table_name, references: true)
return 'TRUE' if admin?
table = model.table_name
keys = KEYS.fetch(table)
matches = keys.map { "boundary.#{ _1 } = #{ row }.#{ _1 }" }.join(' AND ')
clauses = ["NOT EXISTS (SELECT 1 FROM #{ table } boundary WHERE #{ matches } " +
"AND boundary.event_type = 'discard' AND boundary.version_no >= #{ row }.version_no)"]
clauses += reference_sql(table, row) if references
clauses.map { "(#{ _1 })" }.join(' AND ')
end
def self.discard_reference(table, key, expression)
"NOT EXISTS (SELECT 1 FROM #{ table } blocked WHERE blocked.event_type = 'discard' " +
"AND #{ expression.call("blocked.#{ key }") })"
end
def self.reference_sql(table, row)
case table
when 'post_versions'
[['tag_versions', 'tag_id', 'tag_id'],
['nico_tag_versions', 'tag_id', 'external_tag_id']].map do |target, key, field|
discard_reference(target, key, ->(id) {
"JSON_CONTAINS(#{ row }.tags_json, JSON_OBJECT('#{ field }', #{ id }))"
})
end + [discard_reference('post_versions', 'post_id', ->(id) {
"FIND_IN_SET(#{ id }, REPLACE(#{ row }.parent_post_ids, ' ', ',')) > 0"
})]
when 'tag_versions'
[discard_reference('tag_versions', 'tag_id', ->(id) {
"FIND_IN_SET(#{ id }, REPLACE(#{ row }.parent_tag_ids, ' ', ',')) > 0"
})]
when 'tag_description_versions'
[discard_reference('tag_versions', 'tag_id', ->(id) { "#{ row }.tag_id = #{ id }" })]
when 'material_versions'
[discard_reference('tag_versions', 'tag_id', ->(id) { "#{ row }.tag_id = #{ id }" }),
discard_reference('material_versions', 'material_id', ->(id) {
"#{ row }.parent_id = #{ id }"
})]
when 'tag_group_versions'
[['tag_versions', 'tag_id'], ['tag_group_versions', 'tag_group_id']].map do |target, field|
discard_reference(target, field, ->(id) {
['members', 'placements'].map do |column|
"JSON_CONTAINS(COALESCE(JSON_EXTRACT(#{ row }.#{ column }, " +
"'$**.#{ field }'), JSON_ARRAY()), CAST(#{ id } AS CHAR))"
end.join(' OR ')
})
end
else []
end
end
def self.visible(scope, row = scope.klass.table_name)
scope.where(Arel.sql(sql(scope.klass, row)))
end
def self.current(scope, version_model)
return scope if admin?
table = scope.klass.table_name
keys = KEYS.fetch(version_model.table_name)
matches = keys.map do |key|
column = scope.klass.column_names.include?(key) ? key : 'id'
"current_snapshot.#{ key } = #{ table }.#{ column }"
end.join(' AND ')
boundary = "#{ matches } AND current_snapshot.version_no >= #{ table }.version_no " +
"AND current_snapshot.event_type = 'discard'"
scope = scope.where(Arel.sql("NOT EXISTS (SELECT 1 FROM #{ version_model.table_name } " +
"current_snapshot WHERE #{ boundary })"))
condition = "#{ matches } AND current_snapshot.version_no = #{ table }.version_no"
scope.where(Arel.sql("NOT EXISTS (SELECT 1 FROM #{ version_model.table_name } " +
"current_snapshot WHERE #{ condition } AND NOT (#{ sql(version_model, 'current_snapshot') }))"))
end
def self.require_current!(record, version_model)
return if admin?
raise Forbidden unless current(record.class.where(id: record.id), version_model).exists?
end
def self.blob_visible?(blob, seen = [])
return true if admin?
return false if seen.include?(blob.id)
seen = seen + [blob.id]
private_versions = MaterialVersion.where(file_blob_id: blob.id)
.where(Arel.sql("NOT (#{ sql(MaterialVersion) })"))
return false if private_versions.exists?
blob.attachments.each do |attachment|
model, key = { 'Post' => [PostVersion, 'post_id'],
'Material' => [MaterialVersion, 'material_id'] }[attachment.record_type]
if model && model.where(key => attachment.record_id, event_type: 'discard').exists?
return false
end
if attachment.record_type == 'ActiveStorage::VariantRecord'
original = ActiveStorage::VariantRecord.find_by(id: attachment.record_id)&.blob
return false unless original && blob_visible?(original, seen)
end
end
true
end
def self.require_many!(records)
return if admin? || records.empty?
ids = records.map(&:id).uniq
raise Forbidden unless visible(records.first.class.where(id: ids)).count == ids.length
end
def self.require!(record)
return record if admin? || visible(record.class.where(id: record.id)).exists?
raise Forbidden
end
# This number deliberately ignores time, identity and content filters to prevent count probes.
def self.population_count(model) = model.unscoped.count
end
+6
ファイルの表示
@@ -1,6 +1,12 @@
class PostVersion < ApplicationRecord class PostVersion < ApplicationRecord
include VersionRecord include VersionRecord
scope :with_previous_version, -> {
joins('LEFT JOIN post_versions prev ON prev.post_id = post_versions.post_id ' +
'AND prev.version_no = post_versions.version_no - 1 AND ' +
HistoryAccess.sql(PostVersion, 'prev'))
}
belongs_to :post belongs_to :post
belongs_to :parent, class_name: 'Post', optional: true belongs_to :parent, class_name: 'Post', optional: true
+2
ファイルの表示
@@ -1,3 +1,5 @@
class TagDescriptionVersion < ApplicationRecord class TagDescriptionVersion < ApplicationRecord
include VersionRecord
before_create { SharedResourceIdAllocator.assign!(self) } before_create { SharedResourceIdAllocator.assign!(self) }
end end
+2
ファイルの表示
@@ -6,6 +6,8 @@ module VersionRecord
discard: 'discard', discard: 'discard',
restore: 'restore' }.freeze restore: 'restore' }.freeze
def require_history_access! = HistoryAccess.require!(self)
def readonly? = persisted? def readonly? = persisted?
included do included do
+135
ファイルの表示
@@ -0,0 +1,135 @@
# Counts uses recorded values only; history/diff endpoints retain their own contract.
class CountsVersionRepr
def self.many(rows, resource:, language_code:, at: nil, inclusive: true)
new(rows.to_a, resource:, language_code:, at:, inclusive:).serialise
end
def initialize(rows, resource:, language_code:, at:, inclusive:)
@rows = rows
HistoryAccess.require_many!(rows)
@at = at
@inclusive = inclusive
@resource = resource
@language = language_code
@users = User.where(id: rows.map(&:created_by_user_id).compact).pluck(:id, :name).to_h
@languages = master_names(Language)
@scripts = master_names(Script)
@tag_versions = {}
load_tag_versions if resource == 'posts'
load_parent_versions if resource == 'tags'
end
def serialise
@rows.map do |row|
resource_id = row.public_send(@resource == 'posts' ? :post_id : :tag_id)
common = { id: row.id, resource_id:,
version_no: row.version_no, is_latest: row.attributes['counts_latest'].to_i == 1,
created_at: row.created_at.iso8601,
created_by_user: row.created_by_user_id &&
{ id: row.created_by_user_id, name: @users[row.created_by_user_id] } }
common.merge(@resource == 'posts' ? post(row) : tag(row))
end
end
private
def master_names(model)
model.select('code AS counts_code', :name).to_h do |record|
[record.attributes['counts_code'], record.name]
end
end
def json(value)
value.is_a?(String) ? JSON.parse(value) : value
end
def load_tag_versions
pairs = @rows.flat_map { Array(json(_1.tags_json)) }
.filter_map { |entry| [entry['tag_id'], entry['version_no']] if entry['tag_id'] }
.uniq
pairs.each_slice(500) do |batch|
placeholders = batch.map { '(?, ?)' }.join(', ')
scope = HistoryAccess.visible(TagVersion.all)
if @at
scope = scope.where("created_at #{ @inclusive ? '<=' : '<' } ?", @at)
end
scope.where(["(tag_id, version_no) IN (#{ placeholders })", *batch.flatten])
.select(:tag_id, :version_no, :name, :category, :names).each do |version|
@tag_versions[[version.tag_id, version.version_no]] = version
end
end
end
def names(value)
(json(value) || {}).sort.map do |language, entry|
aliases = entry.fetch('aliases', []).map do |alias_entry|
alias_entry.merge('script_name' => @scripts[alias_entry['script_code']])
end
{ language_code: language, language_name: @languages[language],
script_code: entry['script_code'], script_name: @scripts[entry['script_code']],
name: entry['name'], auto_generated: entry['auto_generated'], aliases: }
end
end
def post(row)
snapshots = Array(json(row.tags_json)).map do |entry|
version = @tag_versions[[entry['tag_id'], entry['version_no']]]
{ id: entry['tag_id'] || entry['external_tag_id'],
kind: entry['tag_id'] ? 'internal' : 'external',
tag_id: entry['tag_id'], external_tag_id: entry['external_tag_id'],
version_no: entry['version_no'], name: entry['name'] || version&.name,
category: entry['category'] || version&.category,
sections: entry.fetch('sections', []), names: version ? names(version.names) : [] }
end
{ title: row.title, url: row.url, thumbnail_base: row.thumbnail_base,
tags: snapshots, parents: row.parent_post_ids.to_s.split.map { { id: _1.to_i } },
video_ms: row.video_ms,
original_created_from: row.original_created_from&.iso8601,
original_created_before: row.original_created_before&.iso8601 }
end
def parent_point(row) = @at || row.created_at
def load_parent_versions
@parent_versions = {}
requests = @rows.flat_map do |row|
row.parent_tag_ids.to_s.split.map { [_1.to_i, parent_point(row)] }
end.uniq
requests.each_slice(100) do |batch|
queries = batch.map do |id, point|
scope = HistoryAccess.visible(TagVersion.where(tag_id: id))
.where('created_at <= ?', point)
.where('NOT EXISTS (SELECT 1 FROM tag_versions newer ' +
'WHERE newer.tag_id = tag_versions.tag_id AND newer.created_at <= ? ' +
'AND newer.version_no > tag_versions.version_no)', point)
quoted = TagVersion.connection.quote(point.iso8601(6))
scope.select("tag_id, name, names, #{ quoted } AS reference_point").to_sql
end
next if queries.empty?
TagVersion.connection.select_all(queries.map { "(#{ _1 })" }.join(' UNION ALL ')).each do |value|
@parent_versions[[value['tag_id'], value['reference_point']]] = value
end
end
end
def tag(row)
recorded_names = names(row.names)
selected = recorded_names.find { _1[:language_code] == @language }
aliases = recorded_names.flat_map do |entry|
entry[:aliases].map {
_1.merge('language_code' => entry[:language_code],
'language_name' => entry[:language_name])
}
end
{ name: selected&.fetch(:name) || row.name, names: recorded_names,
category: row.category, aliases:,
parents: row.parent_tag_ids.to_s.split.map do |id|
value = @parent_versions[[id.to_i, parent_point(row).iso8601(6)]]
snapshots = value && json(value['names'])
name = snapshots&.dig(@language, 'name') || value&.fetch('name')
{ id: id.to_i, name: }
end,
deprecated: row.deprecated_at&.iso8601 }
end
end
+5 -3
ファイルの表示
@@ -44,6 +44,7 @@ module TagGroupRepr
end end
def one locale, group def one locale, group
HistoryAccess.require_current!(group, TagGroupVersion)
many(locale, [group]).first many(locale, [group]).first
end end
@@ -56,10 +57,11 @@ module TagGroupRepr
end end
def current_version group def current_version group
TagGroupVersion.find_by(tag_group_id: group.id, version_no: group.version_no) HistoryAccess.visible(TagGroupVersion.all).find_by(tag_group_id: group.id, version_no: group.version_no)
end end
def historical locale, version def historical locale, version
HistoryAccess.require!(version)
members = Array(version.members) members = Array(version.members)
placements = Array(version.placements) placements = Array(version.placements)
tag_references = members.filter_map { |member| member['tag_id'] } + tag_references = members.filter_map { |member| member['tag_id'] } +
@@ -153,13 +155,13 @@ module TagGroupRepr
end end
def historical_tag_versions ids def historical_tag_versions ids
TagVersion.where(tag_id: ids.uniq).to_a.index_by do |version| HistoryAccess.visible(TagVersion.all).where(tag_id: ids.uniq).to_a.index_by do |version|
[version.tag_id, version.version_no] [version.tag_id, version.version_no]
end end
end end
def historical_group_versions ids def historical_group_versions ids
TagGroupVersion.where(tag_group_id: ids.uniq).to_a.index_by do |version| HistoryAccess.visible(TagGroupVersion.all).where(tag_group_id: ids.uniq).to_a.index_by do |version|
[version.tag_group_id, version.version_no] [version.tag_group_id, version.version_no]
end end
end end
+1
ファイルの表示
@@ -5,6 +5,7 @@ module WikiPageRepr
module_function module_function
def tag_description locale, description def tag_description locale, description
HistoryAccess.require_current!(description, TagDescriptionVersion)
tag = description.tag tag = description.tag
{ 'id' => description.id, { 'id' => description.id,
+100
ファイルの表示
@@ -0,0 +1,100 @@
class CountsAxes
GRAINS = ['year', 'quarter', 'month', 'week', 'day', 'hour'].freeze
def self.time_expression(expression, grain, timezone)
raise ArgumentError unless GRAINS.include?(grain)
connection = ApplicationRecord.connection
# Use the installed TZInfo database, including DST, rather than MySQL timezone tables.
first = Time.utc(1900)
last = Time.utc(2101)
offset = timezone.period_for_utc(first).utc_total_offset
clauses = timezone.transitions_up_to(last, first).filter_map do |transition|
next if transition.offset.utc_total_offset == offset
previous = offset
offset = transition.offset.utc_total_offset
"WHEN #{ expression } < #{ connection.quote(transition.at.to_time) } THEN #{ previous }"
end
seconds = clauses.empty? ? offset.to_s : "CASE #{ clauses.join(' ') } ELSE #{ offset } END"
local = "DATE_ADD(#{ expression }, INTERVAL (#{ seconds }) SECOND)"
case grain
when 'year' then "DATE_FORMAT(#{ local }, '%Y')"
when 'quarter' then "CONCAT(YEAR(#{ local }), '-Q', QUARTER(#{ local }))"
when 'month' then "DATE_FORMAT(#{ local }, '%Y-%m')"
when 'week' then "DATE_FORMAT(#{ local }, '%x-W%v')"
when 'day' then "DATE_FORMAT(#{ local }, '%Y-%m-%d')"
else "DATE_FORMAT(#{ local }, '%Y-%m-%d %H:00')"
end
end
attr_reader :rows, :columns, :all
def initialize(population)
@population = population
@definition = population.definition
@rows = parse(population.params.fetch('rows', ''))
@columns = parse(population.params.fetch('cols', ''))
@all = rows + columns
raise ArgumentError if all.length > 8 || all.map { _1['field'] }.uniq.length != all.length
end
def expressions
@expressions ||= all.map do |axis|
field = axis.fetch('field')
case field
when 'tags'
"CASE WHEN tag_entry.tag_id IS NOT NULL " +
"THEN CONCAT('internal:', tag_entry.tag_id) " +
"ELSE CONCAT('external:', tag_entry.external_tag_id) END"
when 'changed_field' then 'changed.field_key'
else
expression = @definition.expression(field)
@definition.fields[field] == :datetime ? self.class.time_expression(
expression, axis.fetch('grain', 'month'), @population.timezone) : expression
end
end
end
def joins(fields = all.map { _1['field'] })
result = ''
if fields.include?('tags')
result += <<~SQL.squish
JOIN JSON_TABLE(CAST(v.tags_json AS JSON), '$[*]' COLUMNS (
tag_id BIGINT PATH '$.tag_id' NULL ON EMPTY,
external_tag_id BIGINT PATH '$.external_tag_id' NULL ON EMPTY)) tag_entry
SQL
end
if fields.include?('changed_field')
fields = @definition.changes.map do |field|
"SELECT #{ ApplicationRecord.connection.quote(field) } AS field_key"
end.join(' UNION ALL ')
condition = @definition.changes.map do |field|
"(changed.field_key = '#{ field }' AND #{ @population.changed_sql(field) })"
end.join(' OR ')
result += " JOIN (#{ fields }) changed ON (#{ condition })"
end
result
end
private
def parse(value)
axes = QueryState.axes(value)
raise ArgumentError unless axes.is_a?(Array)
axes.each do |axis|
raise ArgumentError unless axis.is_a?(Hash) && (axis.keys - ['field', 'grain']).empty?
raise ArgumentError unless @definition.axes(@population.history).include?(axis['field'])
if @definition.fields[axis['field']] == :datetime
raise ArgumentError unless GRAINS.include?(axis.fetch('grain', 'month'))
axis['grain'] ||= 'month'
elsif axis.key?('grain')
raise ArgumentError
end
end
axes
end
end
+53
ファイルの表示
@@ -0,0 +1,53 @@
# Only confirmed combinations belong here; all others retain the existing behaviour.
class CountsDefaults
POST_COLUMNS = ['resource_id', 'version_no', 'is_latest', 'title', 'url',
'thumbnail_base', 'tags', 'parents', 'original_created_from',
'original_created_before', 'event_type', 'user', 'created_at'].freeze
def self.entries(resource)
result = [{ definition: 'history', mode: 'summary', params: {
'rows' => 'created_at:month',
'cols' => 'event_type',
'sort' => 'a0:desc', 'conditions' => '' } }]
result << { definition: 'latest', mode: 'summary', params: {
'rows' => '', 'cols' => '', 'conditions' => '', 'tags' => '' } }
if resource == 'posts'
result += [{ definition: 'latest', mode: 'details', params: {
'columns' => POST_COLUMNS.join(' '), 'order' => POST_COLUMNS.join(' '),
'children' => 'tags.name parents.id', 'nested_json' => '0',
'conditions' => '', 'tags' => '' } }]
end
if resource == 'tags'
columns = ['resource_id', 'version_no', 'is_latest', 'name', 'category',
'aliases', 'parents', 'deprecated', 'event_type', 'user', 'created_at']
result << { definition: 'latest', mode: 'details', params: {
'columns' => columns.join(' '), 'order' => columns.join(' '),
'children' => 'aliases.name parents.name', 'nested_json' => '0' } }
end
result
end
def self.for(params)
settings = entries(params.fetch('resource', 'posts')).find do |entry|
entry[:definition] == params.fetch('definition', 'latest') &&
entry[:mode] == params.fetch('mode', 'summary')
end&.fetch(:params) || {}
if settings.key?('sort') && params.key?('rows')
actual = QueryState.axes(params['rows'])
expected = QueryState.axes(settings.fetch('rows'))
compatible = actual.is_a?(Array) && actual.first.is_a?(Hash) &&
actual.first['field'] == expected.first['field'] &&
actual.first.fetch('grain', 'month') == expected.first['grain']
unless compatible
settings = settings.except('sort')
end
end
settings
end
def self.metadata(resource)
entries(resource).map do |entry|
entry.merge(params: entry[:params].map { |key, value| { key:, value: } })
end
end
end
+106
ファイルの表示
@@ -0,0 +1,106 @@
require 'tempfile'
require 'fileutils'
# Finish generation before reporting readiness; downloads never hold the full file in RAM.
class CountsExportDownload
EXPIRY = 10.minutes
MAX_CONCURRENT_EXPORTS = 2
MAX_FILE_BYTES = 128.megabytes
MAX_STORED_BYTES = 512.megabytes
MAX_EXPORT_SECONDS = 120
class Busy < StandardError; end
class TooLarge < StandardError; end
class TimedOut < StandardError; end
class StorageFull < StandardError; end
def self.directory
Rails.root.join(ENV.fetch('COUNTS_EXPORT_DIRECTORY', 'tmp/counts_exports'))
end
def self.verifier = Rails.application.message_verifier('counts_export_download')
def self.prepare(exporter, owner, progress: nil, check: nil, identifier: nil)
FileUtils.mkdir_p(directory, mode: 0o700)
authority = HistoryAccess.authority
slot = acquire_slot
begin
generate(exporter, owner, progress:, check:, identifier:, authority:)
ensure
slot.close
end
end
def self.acquire_slot
MAX_CONCURRENT_EXPORTS.times do |index|
path = directory.join(".slot-#{ index }.lock")
handle = File.open(path, File::RDWR | File::CREAT, 0o600)
return handle if handle.flock(File::LOCK_EX | File::LOCK_NB)
handle.close
end
raise Busy
end
def self.generate(exporter, owner, progress: nil, check: nil, identifier: nil, authority: HistoryAccess.authority)
FileUtils.mkdir_p(directory, mode: 0o700)
Dir.glob(directory.join('counts-*.export')).each do |path|
File.unlink(path) if File.mtime(path) < Time.current - EXPIRY
rescue Errno::ENOENT
# Another request may have removed an expired download.
end
stored = Dir.glob(directory.join('counts-*')).sum do |path|
File.size(path)
rescue Errno::ENOENT
0
end
reserved = MAX_CONCURRENT_EXPORTS * MAX_FILE_BYTES
raise StorageFull if stored + reserved > MAX_STORED_BYTES
prefix = identifier ? "counts-#{ identifier }-" : 'counts-'
file = Tempfile.create([prefix, '.part'], directory)
complete = false
bytes = 0
deadline = Process.clock_gettime(Process::CLOCK_MONOTONIC) + MAX_EXPORT_SECONDS
path = file.path
begin
file.binmode
exporter.body(progress:).each do |chunk|
check&.call
raise TimedOut if Process.clock_gettime(Process::CLOCK_MONOTONIC) > deadline
bytes += chunk.bytesize
raise TooLarge if bytes > MAX_FILE_BYTES
file.write(chunk)
end
file.flush
file.fsync
file.close
finished_path = path.sub(/\.part\z/, '.export')
File.rename(path, finished_path)
path = finished_path
token = verifier.generate({ 'file' => File.basename(path),
'resource' => exporter.resource, 'format' => exporter.format,
'owner' => owner, **authority }, expires_in: EXPIRY, purpose: 'counts_export')
complete = true
token
ensure
file.close unless file.closed?
File.unlink(path) if !complete && File.exist?(path)
end
end
def self.find(token, owner)
metadata = verifier.verified(token, purpose: 'counts_export')
return unless metadata.is_a?(Hash) && metadata['owner'] == owner
return unless HistoryAccess.authorised_artifact?(metadata)
return unless metadata['file']&.match?(/\Acounts-[a-zA-Z0-9_-]+\.export\z/)
return unless CountsResourceDefinition::DEFINITIONS.key?(metadata['resource'])
return unless ['json', 'csv'].include?(metadata['format'])
path = directory.join(metadata['file'])
return unless path.file?
metadata.merge('path' => path.to_s)
end
end
+255
ファイルの表示
@@ -0,0 +1,255 @@
require 'json'
require 'fileutils'
# A durable, bounded export spool. Web and supervised workers share the same POSIX volume.
class CountsExportJobs
class Cancelled < StandardError; end
ACTIVE = ['queued', 'running', 'cancelling'].freeze
def self.directory = CountsExportDownload.directory.join('jobs')
def self.verifier = Rails.application.message_verifier('counts_export_job')
def self.synchronise
FileUtils.mkdir_p(directory, mode: 0o700)
File.open(directory.join('.registry.lock'), File::RDWR | File::CREAT, 0o600) do |lock|
lock.flock(File::LOCK_EX)
yield
end
end
def self.write(state)
path = directory.join("#{ state.fetch('id') }.json")
temporary = "#{ path }.#{ SecureRandom.hex(8) }.tmp"
File.open(temporary, 'w', 0o600) do |file|
file.write(JSON.generate(state))
file.flush
file.fsync
end
File.rename(temporary, path)
ensure
File.unlink(temporary) if temporary && File.exist?(temporary)
end
def self.read(id)
JSON.parse(File.read(directory.join("#{ id }.json")))
rescue Errno::ENOENT
nil
end
def self.remove_download(state)
name = state['download_file']
return unless name&.match?(/\Acounts-[a-zA-Z0-9_-]+\.export\z/)
File.unlink(CountsExportDownload.directory.join(name))
rescue Errno::ENOENT
nil
end
def self.remove_partial(id)
Dir.glob(CountsExportDownload.directory.join("counts-#{ id }-*")).each do |path|
File.unlink(path)
rescue Errno::ENOENT
nil
end
end
def self.clean
Dir.glob(directory.join('*.tmp')).each do |path|
File.unlink(path) if File.mtime(path) < Time.current - CountsExportDownload::EXPIRY
end
Dir.glob(directory.join('*.run')).each do |path|
id = File.basename(path, '.run')
next if directory.join("#{ id }.json").exist?
File.open(path, File::RDWR) do |lock|
File.unlink(path) if lock.flock(File::LOCK_EX | File::LOCK_NB)
end
end
Dir.glob(directory.join('*.json')).each do |path|
state = JSON.parse(File.read(path))
next if state.fetch('expires_at') > Time.current.to_f
run = directory.join("#{ state.fetch('id') }.run")
File.open(run, File::RDWR | File::CREAT, 0o600) do |lock|
unless lock.flock(File::LOCK_EX | File::LOCK_NB)
state['status'] = 'cancelling'
write(state)
next
end
remove_download(state)
remove_partial(state.fetch('id'))
File.unlink(path)
File.unlink(run)
end
end
end
def self.start(payload, owner, language)
payload = payload.dup
time_key = payload.key?('definition') ? 'execution_at' : 'at'
if payload[time_key].nil? || payload[time_key] == ''
payload[time_key] = Time.current.utc.iso8601(6)
end
exporter = CountsExporter.new(payload, language_code: language)
synchronise do
clean
active = Dir.glob(directory.join('*.json')).count do |path|
ACTIVE.include?(JSON.parse(File.read(path))['status'])
end
raise CountsExportDownload::Busy if active >= CountsExportDownload::MAX_CONCURRENT_EXPORTS
id = SecureRandom.hex(24)
state = { 'id' => id, 'owner' => owner, 'payload' => payload,
'language' => language, 'resource' => exporter.resource,
'format' => exporter.format, 'status' => 'queued',
'processed' => 0, 'total' => nil,
**HistoryAccess.authority,
'expires_at' => (Time.current + CountsExportDownload::EXPIRY).to_f }
write(state)
verifier.generate({ 'id' => id, 'owner' => owner },
expires_in: CountsExportDownload::EXPIRY, purpose: 'counts_export_job')
end
end
def self.authorised_id(token, owner)
value = verifier.verified(token, purpose: 'counts_export_job')
return unless value.is_a?(Hash) && value['owner'] == owner
return unless value['id']&.match?(/\A[a-f0-9]{48}\z/)
value['id']
end
def self.status(token, owner, cancel: false)
id = authorised_id(token, owner)
return unless id
synchronise do
state = read(id)
return unless state && state['owner'] == owner && state['expires_at'] > Time.current.to_f
return unless HistoryAccess.authorised_artifact?(state)
if cancel && ['queued', 'running', 'ready'].include?(state['status'])
state['status'] = state['status'] == 'running' ? 'cancelling' : 'cancelled'
remove_download(state)
state.delete('token')
write(state)
end
state.slice('status', 'processed', 'total', 'format', 'token', 'error')
end
end
def self.checkpoint(id, processed = nil, total = nil)
synchronise do
state = read(id)
unless state && state['status'] == 'running' && state['expires_at'] > Time.current.to_f
raise Cancelled
end
if processed
state.merge!('processed' => processed, 'total' => total)
write(state)
end
end
end
def self.work_one
FileUtils.mkdir_p(directory, mode: 0o700)
Dir.glob(directory.join('*.json')).sort.each do |path|
id = File.basename(path, '.json')
File.open(directory.join("#{ id }.run"), File::RDWR | File::CREAT, 0o600) do |lock|
next unless lock.flock(File::LOCK_EX | File::LOCK_NB)
state = synchronise do
candidate = read(id)
next unless candidate && ACTIVE.include?(candidate['status'])
if candidate['status'] != 'queued'
candidate['status'] = candidate['status'] == 'cancelling' ? 'cancelled' : 'failed'
remove_partial(id)
candidate['error'] = 'interrupted'
write(candidate)
next
end
candidate['status'] = 'running'
write(candidate)
candidate
end
next unless state
perform(state)
return true
end
end
synchronise { clean }
false
end
def self.perform(state)
actor = state['actor_id'] && User.find_by(id: state['actor_id'])
actor = nil unless state['admin'] && actor&.admin? && !actor.banned?
HistoryAccess.set(user: actor) do
unless HistoryAccess.authorised_artifact?(state)
synchronise do
current = read(state.fetch('id'))
current['status'] = 'failed'
write(current)
end
return
end
perform_authorised(state)
end
end
def self.perform_authorised(state)
id = state.fetch('id')
token = nil
begin
checkpoint(id)
exporter = CountsExporter.new(state.fetch('payload'), language_code: state.fetch('language'))
last = Process.clock_gettime(Process::CLOCK_MONOTONIC)
progress = lambda do |processed, total|
now = Process.clock_gettime(Process::CLOCK_MONOTONIC)
if processed.zero? || now - last >= 0.25 || processed == total
checkpoint(id, processed, total)
last = now
end
end
check = lambda do
now = Process.clock_gettime(Process::CLOCK_MONOTONIC)
if now - last >= 0.25
checkpoint(id)
last = now
end
end
token = CountsExportDownload.prepare(exporter, state.fetch('owner'),
progress:, check:, identifier: id)
synchronise do
current = read(id)
if (!current || current['status'] != 'running' ||
current['expires_at'] <= Time.current.to_f)
file = CountsExportDownload.find(token, state.fetch('owner'))
File.unlink(file.fetch('path')) if file
raise Cancelled
end
file = CountsExportDownload.find(token, state.fetch('owner'))
current.merge!('status' => 'ready', 'token' => token,
'download_file' => File.basename(file.fetch('path')))
current['total'] = current['processed']
write(current)
end
rescue StandardError => error
remove_partial(id)
synchronise do
current = read(id)
if current
current['status'] = error.is_a?(Cancelled) ? 'cancelled' : 'failed'
current['error'] = error.class.name
write(current)
end
end
unless error.is_a?(Cancelled)
Rails.logger.error("counts_export_job #{ id }: #{ error.class }")
end
end
end
end
+265
ファイルの表示
@@ -0,0 +1,265 @@
require 'set'
# Export identifiers are stable; UI translations are supplied separately as column labels.
class CountsExportProjection
ROOT_KEYS = { 'event_type' => 'event_type', 'resource_id' => 'resource_id', 'version_no' => 'version_no',
'is_latest' => 'is_latest', 'thumbnail' => 'thumbnail_url',
'thumbnail_base' => 'thumbnail_base', 'title' => 'title', 'url' => 'url',
'tags' => 'tags', 'parents' => 'parents', 'video_ms' => 'video_ms',
'original_created_from' => 'original_created_from',
'original_created_before' => 'original_created_before',
'user' => 'created_by_user', 'created_at' => 'created_at',
'name' => 'name', 'names' => 'names', 'category' => 'category',
'aliases' => 'aliases', 'deprecated' => 'deprecated' }.freeze
CHILD_KEYS = { 'id' => 'id', 'versionNo' => 'version_no', 'kind' => 'kind',
'name' => 'name', 'category' => 'category', 'sections' => 'sections',
'names' => 'names', 'aliases' => 'aliases',
'languageCode' => 'language', 'scriptCode' => 'script',
'autoGenerated' => 'auto_generated' }.freeze
NAME_FIELDS = ['languageCode', 'scriptCode', 'name', 'aliases', 'autoGenerated'].freeze
CHILDREN = {
'tags' => ['id', 'versionNo', 'kind', 'name', 'category', 'sections', 'names'],
'names' => NAME_FIELDS,
'aliases' => ['languageCode', 'name', 'scriptCode'],
'parents' => ['id', 'name'] }.freeze
def self.default_columns(resource)
fields = { 'posts' => ['title', 'url', 'tags', 'parents', 'video_ms'],
'tags' => ['name', 'category', 'aliases', 'parents'],
'nico_tags' => ['name', 'linked_tags'],
'tag_descriptions' => ['language_code', 'body', 'source_language_code'],
'tag_groups' => ['name', 'description', 'members'],
'materials' => ['url', 'tag_id', 'source_kind'] }.fetch(resource)
['resource_id', 'version_no'] + fields + ['event_type', 'user', 'created_at']
end
def self.default_children(resource)
{ 'posts' => ['tags.name', 'parents.id'],
'tags' => ['aliases.name', 'parents.id'],
'tag_groups' => ['members.tagId'] }.fetch(resource, [])
end
attr_reader :columns, :headers
def initialize(params, resource:, summary:, tags:)
if params.key?('definition')
params = CountsDefaults.for(params.merge('resource' => resource)).merge(params)
end
@params = params
@labels = JSON.parse(params.fetch('labels', '{}'))
raise ArgumentError unless @labels.is_a?(Hash) && @labels.length <= 100
raise ArgumentError unless @labels.all? { |key, value| valid_label?(key, value) }
@summary = summary
allowed = if summary
tags ? ['kind', 'tag_id', 'name', 'deprecated', 'count'] : ['group', 'count']
else
CountsQuery::COLUMNS.fetch(resource)
end
defaults = summary ? allowed : self.class.default_columns(resource)
@columns = QueryState.list(params.fetch('columns', defaults.join(' ')))
raise ArgumentError if (@columns - allowed).any? || @columns.uniq != @columns
@headers = JSON.parse(params.fetch('headers', JSON.generate(@columns)))
unless (@headers.is_a?(Array) && @headers.length == @columns.length &&
@headers.all? { _1.is_a?(String) && _1.length <= 200 })
raise ArgumentError
end
@order = QueryState.list(params.fetch('order', ''))
@sorts = QueryState.list(params.fetch('child_sort', '')).map do |entry|
parts = entry.split(':', -1)
raise ArgumentError unless parts.length == 2
path, direction = parts
raise ArgumentError unless ['asc', 'desc'].include?(direction)
[path, direction]
end
@selected = params.key?('children') ? QueryState.list(params['children']).to_set :
self.class.default_children(resource).to_set
valid = CHILDREN.flat_map do |key, fields|
paths(fields, key)
end
raise ArgumentError if @selected && (@selected.to_a - valid).any?
prune = lambda do |fields, prefix|
fields.each do |key|
path = "#{ prefix }.#{ key }"
children = fields_for(key)
next if children.empty?
prune.call(children, path)
@selected.delete(path) unless children.any? { @selected.include?("#{ path }.#{ _1 }") }
end
end
CHILDREN.each { |key, fields| prune.call(fields, key) }
pairs = @columns.zip(@headers).select do |key, _header|
!CHILDREN.key?(key) || CHILDREN[key].any? { @selected.include?("#{ key }.#{ _1 }") }
end
@columns = pairs.map(&:first)
@headers = pairs.map(&:last)
allowed_sorts = valid.map do |path|
parts = path.split('.')
field = parts.pop
"#{ parts.join('.') }/#{ field }"
end
if ((@sorts.map(&:first) - allowed_sorts).any? ||
@sorts.map(&:first).uniq.length != @sorts.length)
raise ArgumentError
end
end
def metadata
@columns.each_with_index.map do |key, index|
{ key: @summary ? key : ROOT_KEYS.fetch(key), label: @headers[index] }
end
end
def project(row)
@columns.to_h do |key|
if @summary
value = key == 'group' ? { key: row[:key], name: row[:name] } : row[key.to_sym]
[key, value]
else
value = case key
when 'thumbnail' then row[:thumbnail_base]
else row[ROOT_KEYS.fetch(key).to_sym]
end
value = nested(value || [], CHILDREN.fetch(key), key) if CHILDREN.key?(key)
[ROOT_KEYS.fetch(key), value]
end
end
end
def csv_record(record, row)
record.map do |key, value|
key == 'group' ? row[:name] || row[:key] : csv_display(value, key)
end
end
private
def valid_label?(key, value)
key.is_a?(String) && value.is_a?(String) && key.length <= 100 && value.length <= 300
end
def csv_display(value, key)
return @labels.fetch('bot', '') if key == 'created_by_user' && value.nil?
return @labels.fetch("event_type:#{ value }", value) if key == 'event_type'
return value ? @labels.fetch('latest', 'true') : '' if key == 'is_latest'
return @labels.fetch("kind:#{ value }", value) if key == 'kind'
return @labels.fetch("category:#{ value }", value) if key == 'category'
if ['language', 'script'].include?(key) && value.is_a?(Hash)
return @labels.fetch("#{ key }:#{ value[:code] }",
value[:name] || @labels.fetch('notRegistered', ''))
end
return @labels.fetch("boolean:#{ value }", value) if value == true || value == false
return value.map { csv_display(_1, key) } if value.is_a?(Array)
if value.is_a?(Hash)
return value.to_h do |field, item|
[@labels.fetch(field.to_s, field), csv_display(item, field.to_s)]
end
end
value
end
def fields_for(key)
return NAME_FIELDS if key == 'names'
return ['name', 'scriptCode'] if key == 'aliases'
[]
end
def paths(fields, prefix)
fields.flat_map do |key|
path = "#{ prefix }.#{ key }"
[path, *paths(fields_for(key), path)]
end
end
def fetch(row, key) = row.key?(key) ? row[key] : row[key.to_sym]
def named_code(row, kind)
code = fetch(row, "#{ kind }_code")
name = fetch(row, "#{ kind }_name")
{ code:, name: name.present? && name.to_s.downcase != code.to_s.downcase ? name : nil }
end
def stable(value)
case value
when Hash
value.stringify_keys.sort.to_h.transform_values { stable(_1) }
when Array
value.map { stable(_1) }
else
value
end
end
def sort_value(row, key)
if key == 'languageCode'
language = named_code(row, 'language')
return @labels.fetch("language:#{ language[:code] }", language[:name])
end
return named_code(row, 'script')[:name] if key == 'scriptCode'
fetch(row, CHILD_KEYS.fetch(key))
end
def comparison(left, right)
if left.is_a?(Array) && right.is_a?(Array) && left.length != right.length
return left.length <=> right.length
end
return left <=> right if left.is_a?(Numeric) && right.is_a?(Numeric)
if [true, false].include?(left) && [true, false].include?(right)
return (left ? 1 : 0) <=> (right ? 1 : 0)
end
a = left.is_a?(Hash) || left.is_a?(Array) ? JSON.generate(stable(left)) : left.to_s
b = right.is_a?(Hash) || right.is_a?(Array) ? JSON.generate(stable(right)) : right.to_s
a <=> b
end
def sorted(rows, prefix, fields)
sorts = @sorts.filter_map do |path, direction|
parent, key = path.split('/', 2)
next unless parent == prefix
raise ArgumentError unless fields.include?(key)
[key, direction]
end
rows.each_with_index.sort do |(left, a), (right, b)|
result = 0
sorts.each do |key, direction|
first = sort_value(left, key)
last = sort_value(right, key)
result = if first.nil? || last.nil?
first.nil? == last.nil? ? 0 : (first.nil? ? 1 : -1)
else
comparison(first, last) * (direction == 'asc' ? 1 : -1)
end
break unless result.zero?
end
result.zero? ? a <=> b : result
end.map(&:first)
end
def nested(rows, fields, prefix)
keys = fields.select { !@selected || @selected.include?("#{ prefix }.#{ _1 }") }
ordered = (@order + keys.map { "#{ prefix }.#{ _1 }" }).uniq
.filter_map { |path| keys.find { "#{ prefix }.#{ _1 }" == path } }
sorted(rows, prefix, fields).map do |row|
ordered.to_h do |key|
stable = CHILD_KEYS.fetch(key)
value = case key
when 'languageCode' then named_code(row, 'language')
when 'scriptCode' then named_code(row, 'script')
else fetch(row, stable)
end
if fields_for(key).any?
value = nested(value || [], fields_for(key), "#{ prefix }.#{ key }")
end
[stable, value]
end
end
end
end
+152
ファイルの表示
@@ -0,0 +1,152 @@
require 'csv'
# One read view and a fixed point for the entire download; only one page is held in memory.
class CountsExporter
EXTRA_KEYS = ['basis', 'format', 'resource', 'headers', 'children', 'order',
'summary_columns', 'nested_json', 'child_sort', 'labels'].freeze
attr_reader :format, :resource
def initialize(params, language_code:)
if params.key?('definition')
@table_export = CountsTableExport.new(params)
@format = @table_export.format
@resource = @table_export.resource
return
end
allowed = (CountsQuery::KEYS + CountsStateQuery::KEYS + EXTRA_KEYS).uniq
QueryState.validate!(params, allowed)
raise ArgumentError unless params.values.all? { _1.is_a?(String) }
@params = params
@language = language_code
@format = params.fetch('format', 'json')
@resource = params.fetch('resource', 'posts')
@mode = params.fetch('mode', 'summary')
@history = params.fetch('basis', 'state') == 'history'
raise ArgumentError unless ['json', 'csv'].include?(format)
raise ArgumentError unless ['posts', 'tags'].include?(resource)
raise ArgumentError unless ['summary', 'details'].include?(@mode)
raise ArgumentError unless ['state', 'history'].include?(params.fetch('basis', 'state'))
@query_params = params.except(*EXTRA_KEYS, 'columns', 'page', 'limit')
.merge('mode' => @mode, 'page' => '1', 'limit' => '100')
point = CountsStateQuery.new(@query_params.slice('at').merge('resource' => resource),
language_code: @language).at
@query_params['at'] = point.iso8601(6)
@query_params['axis'] ||= !@history && resource == 'posts' ? 'tags' : 'month'
@projection = CountsExportProjection.new(params, resource:,
summary: @mode == 'summary',
tags: !@history && @query_params['axis'] == 'tags')
if @history
@query_params.delete('kind')
history_query
else
@query_params.delete('latest')
@state_query = CountsStateQuery.new(@query_params.merge('resource' => resource),
language_code: @language).validate_query!
end
end
def body(progress: nil)
return @table_export.body(progress:) if @table_export
Enumerator.new do |output|
ApplicationRecord.transaction(isolation: :repeatable_read) do
processed = 0
first_row = true
started = false
pages do |page|
total = @mode == 'details' ? page[:count] : nil
progress&.call(processed, total)
unless started
meta = { resource:, mode: @mode, basis: @history ? 'history' : 'state',
at: @query_params['at'], boundary: 'inclusive',
filters: @query_params.except('sort', 'page', 'limit', 'mode'),
sort: export_sort,
child_sort: sort_metadata(@params.fetch('child_sort', '')),
columns: @projection.metadata,
children: @params['children']&.split(' '),
field_order: @QueryState.list(params.fetch('order', '')), timezone: 'UTC',
count: @mode == 'summary' ? page[:group_count] : page[:count],
totals: page[:totals], coverage: page[:coverage] }
output << (format == 'json' ? "{\"metadata\":#{ meta.to_json },\"data\":[" : "\uFEFF")
if format == 'csv'
output << CSV.generate_line(
@projection.headers.map { csv_value(_1) }, row_sep: "\r\n")
end
started = true
end
(page[:versions] || page[:state_rows] || page[:groups] || []).each do |row|
record = @projection.project(row)
if format == 'json'
output << ',' unless first_row
output << record.to_json
else
values = @projection.csv_record(record, row).map { csv_value(_1) }
output << CSV.generate_line(values, row_sep: "\r\n")
end
first_row = false
processed += 1
progress&.call(processed, total)
end
end
progress&.call(processed, processed)
output << ']}' if format == 'json'
end
end
end
private
def export_sort
sort_metadata(@query_params.fetch('sort', default_sort))
end
def sort_metadata(value)
value.split(' ').map do |entry|
key, direction = entry.split(':', 2)
{ key:, direction: }
end
end
def default_sort
return 'created_at:desc' if @mode == 'details'
!@history && @query_params['axis'] == 'tags' ? 'count:desc' : 'group:asc'
end
def history_query
scope = resource == 'posts' ? PostVersion.with_previous_version : TagVersion.all
CountsQuery.new(scope, resource, @query_params)
end
def pages
unless @history
@state_query.each_page { yield _1 }
return
end
query = history_query
total = @mode == 'summary' ? nil : query.count
loop do
result = if @mode == 'summary'
query.summary(include_count: @query_params['page'] == '1')
else
{ count: total,
versions: CountsVersionRepr.many(query.details, resource:,
language_code: @language,
at: Time.iso8601(@query_params['at'])) }
end
total ||= result[:group_count]
yield result
break if @query_params['page'].to_i * 100 >= total
@query_params['page'] = (@query_params['page'].to_i + 1).to_s
query = history_query
end
end
def csv_value(value)
text = value.is_a?(Hash) || value.is_a?(Array) ? JSON.generate(value) : value.to_s
text.match?(/\A\s*[=+@-]/) ? "'#{ text }" : text
end
end
+28
ファイルの表示
@@ -0,0 +1,28 @@
# Coordinates carry their aggregation depth; SQL NULL remains an ordinary axis value.
class CountsPivotLayout
def self.depths(keys, sorts, measures: true)
return [] if keys.length < 2 || sorts.first&.first == 'count' ||
sorts.first&.first&.match?(/\Am[0-9]+\z/)
relevant = sorts.map(&:first).select { keys.include?(_1) || (measures && (_1 == 'count' ||
_1.match?(/\Am[0-9]+\z/))) }
order = (relevant + keys).uniq
(1...keys.length).select { order.first(_1).sort == keys.first(_1).sort }
end
def self.headers(values, depths, next_values: nil, grand: false)
result = []
values.each_with_index do |tuple, index|
result << { values: tuple, kind: 'data' }
following = values[index + 1] || next_values
depths.reverse_each do |depth|
prefix = tuple.first(depth)
next if following && following.first(depth) == prefix
result << { values: prefix, kind: 'subtotal' }
end
end
result << { values: [], kind: 'grand_total' } if grand
result
end
end
+59
ファイルの表示
@@ -0,0 +1,59 @@
# Re-aggregate the population, never add up already grouped cells (multi-value axes overlap).
class CountsPivotTotals
def initialize(population, axes, row_headers, column_headers)
@population = population
@axes = axes
@rows = row_headers.group_by { _1[:values].length }
@columns = column_headers.group_by { _1[:values].length }
end
def call
queries = @rows.flat_map do |row_depth, headers|
@columns.filter_map do |column_depth, columns|
next if row_depth == @axes.rows.length && column_depth == @axes.columns.length
indexes = (0...row_depth).to_a +
(0...column_depth).map { @axes.rows.length + _1 }
expressions = indexes.map { @axes.expressions[_1] }
scope = @population.scope.except(:select, :order)
joins = @axes.joins(indexes.map { @axes.all[_1].fetch('field') })
scope = scope.joins(joins) unless joins.empty?
if row_depth.positive?
tuples = headers.map { _1[:values] }.uniq
predicate = tuples.map { Array.new(row_depth) { |i|
"(#{ expressions[i] }) <=> ?"
}.join(' AND ') }.map { "(#{ _1 })" }.join(' OR ')
scope = scope.where(predicate, *tuples.flatten)
end
if column_depth.positive?
tuples = columns.map { _1[:values] }.uniq
predicate = tuples.map { Array.new(column_depth) { |i|
"(#{ expressions[row_depth + i] }) <=> ?"
}.join(' AND ') }.map { "(#{ _1 })" }.join(' OR ')
scope = scope.where(predicate, *tuples.flatten)
end
grouped = expressions.each_with_index.map { |expression, index|
"#{ expression } AS v#{ index }"
}
# Fixed-width result coordinates, independent of NULL and of translated labels.
padding = (expressions.length...@axes.all.length).map { "NULL AS v#{ _1 }" }
fields = ["#{ row_depth } AS row_depth", "#{ column_depth } AS column_depth"] +
grouped + padding + ['COUNT(DISTINCT v.id) AS count']
scope = scope.group(*expressions.map { Arel.sql(_1) }) if expressions.any?
scope.optimizer_hints('MAX_EXECUTION_TIME(10000)')
.select(Arel.sql(fields.join(', '))).to_sql
end
end
return [] if queries.empty?
sql = queries.map { "(#{ _1 })" }.join(' UNION ALL ')
@population.definition.model.connection.select_all(
"SELECT /*+ MAX_EXECUTION_TIME(10000) */ * FROM (#{ sql }) totals").map do |record|
row_depth = record['row_depth'].to_i
column_depth = record['column_depth'].to_i
{ row: (0...row_depth).map { record["v#{ _1 }"] },
column: (0...column_depth).map { record["v#{ row_depth + _1 }"] },
count: record['count'].to_i }
end
end
end
+210
ファイルの表示
@@ -0,0 +1,210 @@
# Population first, then filters. Previous versions are joined before period filtering.
class CountsPopulation
attr_reader :definition, :params, :at, :history, :scope, :timezone
def initialize(params)
@params = params
@definition = CountsResourceDefinition.new(params.fetch('resource', 'posts'))
@history = params.fetch('definition', 'latest') == 'history'
raise ArgumentError unless ['latest', 'history'].include?(params.fetch('definition', 'latest'))
@timezone = TZInfo::Timezone.get(params.fetch('timezone', 'UTC'))
@at = timestamp(params['execution_at']) || Time.current.utc
raise ArgumentError if at > Time.current + 1.second
@scope = definition.model.all
make_population
@scope = HistoryAccess.visible(@scope, 'v')
apply_filters
rescue TZInfo::InvalidTimezoneIdentifier
raise ArgumentError
end
def timestamp(value, end_date: false)
return nil if value.nil? || value == ''
raise ArgumentError unless value.is_a?(String)
if value.match?(/\A\d{4}-\d{2}-\d{2}\z/)
date = Date.iso8601(value)
date += 1 if end_date
timezone.local_to_utc(Time.utc(date.year, date.month, date.day))
else
raise ArgumentError unless value.match?(/(?:Z|[+-]\d{2}:\d{2})\z/)
time = Time.iso8601(value).utc
raise ArgumentError unless time.year.between?(1900, 2100)
time
end
rescue TZInfo::AmbiguousTime, TZInfo::PeriodNotFound, Date::Error
raise ArgumentError
end
def latest_sql(row = 'v', cutoff = nil)
cutoff ||= history ? at : timestamp(params['at']) || at
keys = definition.keys.map { "newer.#{ _1 } = #{ row }.#{ _1 }" }.join(' AND ')
bound = definition.model.connection.quote(cutoff)
<<~SQL.squish
NOT EXISTS (SELECT 1 FROM #{ definition.table } newer
WHERE #{ keys } AND newer.version_no > #{ row }.version_no
AND newer.#{ definition.time_column } <= #{ bound })
SQL
end
def changed_sql(key)
column = definition.column(key)
"(previous.id IS NOT NULL AND NOT (v.#{ column } <=> previous.#{ column }))"
end
private
def make_population
table = definition.table
@scope = @scope.from("#{ table } v").select('v.*')
keys = definition.keys.map { "previous.#{ _1 } = v.#{ _1 }" }.join(' AND ')
@scope = @scope.joins("LEFT JOIN #{ table } previous ON #{ keys } " +
'AND previous.version_no = v.version_no - 1 AND ' +
HistoryAccess.sql(definition.model, 'previous'))
@scope = @scope.where("v.#{ definition.time_column } <= ?", at)
unless history
point = timestamp(params['at']) || at
raise ArgumentError if point > at
@scope = @scope.where("v.#{ definition.time_column } <= ?", point)
.where(Arel.sql(latest_sql('v', point))).where.not('v.event_type' => 'discard')
return
end
first = timestamp(params['from'])
last = timestamp(params['through'], end_date: true)
raise ArgumentError if first && last && first > last
@scope = @scope.where('v.created_at >= ?', first) if first
if last
operator = params['through'].length == 10 ? '<' : '<='
@scope = @scope.where("v.created_at #{ operator } ?", last)
end
end
def apply_filters
['resource_id', 'user'].each do |key|
value = params[key == 'user' ? 'user_id' : key]
next if value.nil? || value == ''
raise ArgumentError unless value.match?(/\A[0-9]+\z/)
@scope = @scope.where("#{ definition.expression(key) } = ?", value.to_i)
end
apply_tags if params['tags'].present? || params['tag'].present?
conditions = QueryState.records(params, 'conditions')
raise ArgumentError unless conditions.is_a?(Array) && conditions.length <= 100
conditions.each { apply_condition(_1) }
drills = QueryState.records(params, 'drill')
raise ArgumentError unless drills.is_a?(Array) && drills.length <= 100
drills.each do |entry|
key = entry.fetch('field')
if key == 'tags'
@scope = @scope.where(snapshot_membership(entry.fetch('value')))
elsif key == 'changed_field'
raise ArgumentError unless history && definition.changes.include?(entry['value'])
@scope = @scope.where(Arel.sql(changed_sql(entry['value'])))
elsif definition.fields[key] == :datetime && entry['grain']
@scope = @scope.where(definition.expression(key) => nil) if entry['value'].nil?
if entry['value']
expression = CountsAxes.time_expression(definition.expression(key),
entry['grain'], timezone)
@scope = @scope.where("#{ expression } = ?", entry['value'])
end
else
raise ArgumentError unless definition.axes(history).include?(key)
@scope = @scope.where(definition.expression(key) => entry['value'])
end
end
end
def apply_condition(entry)
raise ArgumentError unless entry.is_a?(Hash)
raise ArgumentError if (entry.keys - ['field', 'operator', 'value', 'end', 'blank']).any?
key = entry.fetch('field')
type = definition.fields.fetch(key) { raise ArgumentError }
operator = entry.fetch('operator')
raise ArgumentError unless CountsResourceDefinition::OPERATORS.fetch(type, []).include?(operator)
raise ArgumentError unless [true, false, nil].include?(entry['blank'])
expression = definition.expression(key)
blank = type == :string || type == :enum ?
"(#{ expression } IS NULL OR #{ expression } = '')" : "#{ expression } IS NULL"
if entry['blank']
@scope = @scope.where(Arel.sql(blank))
return
end
values = [entry['value'], entry['end']]
raise ArgumentError unless values.all? { _1.nil? || _1.is_a?(String) }
return if values.all? { _1.nil? || _1 == '' }
parsed = values.map do |value|
next nil if value.nil? || value == ''
case type
when :integer
raise ArgumentError unless value.match?(/\A-?\d+\z/)
value.to_i
when :datetime then timestamp(value)
else value
end
end
if operator == 'range'
@scope = @scope.where("#{ expression } >= ?", parsed[0]) if parsed[0]
@scope = @scope.where("#{ expression } <= ?", parsed[1]) if parsed[1]
return
end
return unless parsed[0]
sql_operator = { 'eq' => '=', 'ne' => '<>', 'gt' => '>', 'lt' => '<',
'ge' => '>=', 'le' => '<=', 'like' => 'LIKE' }.fetch(operator)
sql = "#{ expression } #{ sql_operator } ?"
sql = "(#{ sql } OR #{ blank })" if operator == 'ne'
@scope = @scope.where(sql, parsed[0])
end
def snapshot_membership(value)
kind, identifier = value.to_s.split(':', 2)
raise ArgumentError unless ['internal', 'external'].include?(kind)
raise ArgumentError unless identifier&.match?(/\A[1-9]\d*\z/)
key = kind == 'internal' ? 'tag_id' : 'external_tag_id'
definition.model.sanitize_sql_array([
"JSON_CONTAINS(v.tags_json, JSON_OBJECT('#{ key }', CAST(? AS UNSIGNED)))",
identifier.to_i])
end
def apply_tags
raise ArgumentError unless definition.resource == 'posts'
if params['tag'].present?
@scope = @scope.where(snapshot_membership(
"#{ params.fetch('kind', 'internal') }:#{ params['tag'] }"))
end
return unless params['tags'].present?
locale = Locale.find_by!(code: params.fetch('locale', 'ja'))
refs = PostTagSearch.parse_references(params, params['tags'].split)
raw = params['tags']
literals = PostTagSearch.literals(locale, raw, references: refs)
predicates = literals.map do |literal|
internal, external = PostTagSearch.references(locale, literal)
parts = Array(internal).map { snapshot_membership("internal:#{ _1 }") } +
Array(external).map { snapshot_membership("external:#{ _1 }") }
expression = parts.empty? ? 'FALSE' : "(#{ parts.join(' OR ') })"
literal[:negative] ? "NOT #{ expression }" : expression
end
return if predicates.empty?
any = params['match'] == 'any' || raw.split.any? { _1.downcase == 'type:or' }
@scope = @scope.where(Arel.sql("(#{ predicates.join(any ? ' OR ' : ' AND ') })"))
end
end
+207
ファイルの表示
@@ -0,0 +1,207 @@
# Read-only history queries. SQL identifiers and expressions are server-owned.
class CountsQuery
AXES = ['month', 'user'].freeze
COMMON_COLUMNS = ['resource_id', 'version_no', 'is_latest', 'event_type', 'user', 'created_at'].freeze
COLUMNS = {
'posts' => COMMON_COLUMNS + [
'thumbnail', 'thumbnail_base', 'title', 'url', 'tags', 'parents', 'video_ms',
'original_created_from', 'original_created_before'],
'tags' => COMMON_COLUMNS + [
'name', 'names', 'category', 'aliases', 'parents', 'deprecated'] }.freeze
KEYS = [
'locale', 'mode', 'axis', 'page', 'limit', 'from', 'before', 'latest',
'user_id', 'resource_id', 'tag', 'columns', 'sort', 'at'].freeze
attr_reader :page, :limit, :mode, :axis
def initialize(scope, resource, params, latest_expression: nil)
@table = scope.klass.table_name
@resource = resource
@id_column = resource == 'posts' ? 'post_id' : 'tag_id'
@params = params
@latest_expression = latest_expression
scope = HistoryAccess.visible(scope)
@scope = scope.except(:select, :order).joins(
"LEFT JOIN users counts_actor ON counts_actor.id = #{ @table }.created_by_user_id")
raise ArgumentError unless (params.keys - KEYS).empty?
raise ArgumentError unless params.values.all? { _1.is_a?(String) }
@mode = params.fetch('mode', 'summary')
@axis = params.fetch('axis', 'month')
raise ArgumentError unless ['summary', 'details'].include?(mode) && AXES.include?(axis)
@page = integer('page', 1)
@limit = integer('limit', 20)
raise ArgumentError if limit > 100 || page > 1_000_000
columns = QueryState.list(params.fetch('columns', COLUMNS.fetch(resource).join(' ')))
if columns.empty? || columns.uniq != columns || (columns - COLUMNS.fetch(resource)).any?
raise ArgumentError
end
@sort = parse_sort
apply_filters
end
def relation = @scope
def count = @scope.count
def latest_expression
return @latest_expression if @latest_expression
at = timestamp('at')
cutoff = at ? "AND newer.created_at <= #{ @scope.klass.connection.quote(at) }" : ''
<<~SQL.squish
NOT EXISTS (SELECT 1 FROM #{ @table } newer
WHERE newer.#{ @id_column } = #{ @table }.#{ @id_column }
AND newer.version_no > #{ @table }.version_no #{ cutoff })
SQL
end
def details
@scope.select("#{ @table }.*", Arel.sql("(#{ latest_expression }) AS counts_latest"))
.order(Arel.sql(order_sql(detail_expressions, ["#{ @table }.id DESC"])))
.limit(limit).offset((page - 1) * limit)
end
def summary(include_count: true)
expression = axis == 'month' ? "DATE_FORMAT(#{ @table }.created_at, '%Y-%m')" :
"#{ @table }.created_by_user_id"
group_order = axis == 'user' ? 'MAX(counts_actor.name)' : expression
sql = @scope.group(Arel.sql(expression)).select(Arel.sql(
"#{ expression } AS group_key, COUNT(*) AS record_count, " +
"#{ group_order } AS group_name")).to_sql
connection = @scope.klass.connection
source = "FROM (#{ sql }) counts_groups"
groups = include_count ? connection.select_value("SELECT COUNT(*) #{ source }") : nil
expressions = { 'group' => ['group_name'], 'count' => ['record_count'] }
order = order_sql(expressions, ['group_key IS NULL ASC', 'group_key ASC'])
rows = connection.select_all(
"SELECT group_key, group_name, record_count #{ source } " +
"ORDER BY #{ order } LIMIT #{ limit } OFFSET #{ (page - 1) * limit }")
{ group_count: groups&.to_i,
groups: rows.map do |row|
{ key: row['group_key'],
name: axis == 'user' ? row['group_name'] : nil,
count: row['record_count'].to_i }
end }
end
private
def parse_sort
value = @params.fetch('sort', mode == 'details' ? 'created_at:desc' : 'group:asc')
return [] if value.empty?
values = QueryState.list(value).map { _1.split(':', -1) }
allowed = mode == 'details' ? COLUMNS.fetch(@resource) - ['thumbnail'] : ['group', 'count']
invalid = values.any? do |key, direction, *extra|
extra.any? || !allowed.include?(key) || !['asc', 'desc'].include?(direction)
end
if (values.empty? || values.length > allowed.length || invalid ||
values.map(&:first).uniq.length != values.length)
raise ArgumentError
end
values
end
# NULL always sorts last, including descending keys. The final key is unique.
def order_sql(expressions, final)
(@sort.flat_map do |key, direction|
expressions.fetch(key).flat_map do |expression|
["(#{ expression }) IS NULL ASC", "#{ expression } #{ direction.upcase }"]
end
end + final).join(', ')
end
def detail_expressions
common = { 'resource_id' => ["#{ @table }.#{ @id_column }"],
'version_no' => ["#{ @table }.version_no"],
'is_latest' => ["(#{ latest_expression })"],
'user' => ['counts_actor.name', "#{ @table }.created_by_user_id"],
'created_at' => ["#{ @table }.created_at"] }
if @resource == 'posts'
fields = [
'title', 'url', 'thumbnail_base', 'video_ms',
'original_created_from', 'original_created_before']
common.merge(fields.to_h { [_1, ["#{ @table }.#{ _1 }"]] }).merge(
'tags' => json_order('post_versions.tags_json'),
'parents' => list_order('post_versions.parent_post_ids'))
else
name = "JSON_UNQUOTE(JSON_EXTRACT(tag_versions.names, #{ locale_name_path }))"
common.merge(
'name' => ["COALESCE(#{ name }, tag_versions.name)"],
'names' => json_order('tag_versions.names'),
'category' => ['tag_versions.category'],
'aliases' => json_order(
"JSON_EXTRACT(tag_versions.names, '$.*.aliases[*]')"),
'parents' => list_order('tag_versions.parent_tag_ids'),
'deprecated' => ['tag_versions.deprecated_at'])
end
end
def locale_name_path
language = Locale.find_by(code: @params.fetch('locale', 'ja'))&.language_code || 'ja'
path = "$.#{ JSON.generate(language) }.name"
@scope.klass.connection.quote(path)
end
# Multi-valued parent cells sort by cardinality, then binary snapshot text.
def json_order(column)
[
"COALESCE(JSON_LENGTH(#{ column }), 0)",
"COALESCE(CAST(#{ column } AS CHAR), '[]') COLLATE utf8mb4_bin"]
end
def list_order(column)
[
"CASE WHEN #{ column } = '' THEN 0 ELSE " +
"1 + LENGTH(#{ column }) - LENGTH(REPLACE(#{ column }, ' ', '')) END",
"#{ column } COLLATE utf8mb4_bin"]
end
def integer(key, default = nil)
value = @params[key].presence
return default if value.nil? && !default.nil?
raise ArgumentError unless value&.match?(/\A[1-9][0-9]{0,9}\z/)
value.to_i
end
def timestamp(key)
value = @params[key].presence
return nil unless value
pattern = /\A\d{4}-\d{2}-\d{2}T\d{2}:\d{2}(?::\d{2}(?:\.\d{1,6})?)?Z\z/
raise ArgumentError unless value.match?(pattern)
Date.iso8601(value.first(10))
Time.iso8601(value.length == 17 ? value.sub(/Z\z/, ':00Z') : value)
end
def apply_filters
at = timestamp('at')
first = timestamp('from')
last = timestamp('before')
raise ArgumentError if first && last && first >= last
raise ArgumentError unless ['0', '1'].include?(@params.fetch('latest', '1'))
@scope = @scope.where("#{ @table }.created_at <= ?", at) if at
@scope = @scope.where(Arel.sql(latest_expression)) unless @params['latest'] == '0'
@scope = @scope.where("#{ @table }.created_at >= ?", first) if first
@scope = @scope.where("#{ @table }.created_at < ?", last) if last
if @params['resource_id'].present?
@scope = @scope.where(@table => { @id_column => integer('resource_id') })
end
if @params['user_id'].present?
user = @params['user_id'] == 'none' ? nil : integer('user_id')
@scope = @scope.where(@table => { created_by_user_id: user })
end
return unless @params['tag'].present?
raise ArgumentError unless @resource == 'posts'
id = integer('tag')
@scope = @scope.where(
"JSON_CONTAINS(post_versions.tags_json, JSON_OBJECT('tag_id', CAST(? AS UNSIGNED))) OR " +
"JSON_CONTAINS(prev.tags_json, JSON_OBJECT('tag_id', CAST(? AS UNSIGNED)))", id, id)
end
end
+65
ファイルの表示
@@ -0,0 +1,65 @@
# Server-owned version identities, attributes and field types. No client SQL is accepted.
class CountsResourceDefinition
COMMON = { 'resource_id' => :integer, 'version_no' => :integer,
'event_type' => :enum, 'user' => :integer, 'created_at' => :datetime }.freeze
DEFINITIONS = {
'posts' => [PostVersion, ['post_id'], {
'title' => :string, 'url' => :string, 'thumbnail_base' => :string,
'tags_json' => :json, 'parent_post_ids' => :string, 'video_ms' => :integer,
'original_created_from' => :datetime, 'original_created_before' => :datetime }],
'tags' => [TagVersion, ['tag_id'], {
'name' => :string, 'names' => :json, 'category' => :enum, 'aliases' => :string,
'parent_tag_ids' => :string, 'deprecated_at' => :datetime }],
'nico_tags' => [NicoTagVersion, ['tag_id'], {
'name' => :string, 'linked_tags' => :string }],
'tag_descriptions' => [TagDescriptionVersion, ['tag_id', 'language_code'], {
'language_code' => :enum, 'body' => :string,
'source_language_code' => :enum, 'source_version_no' => :integer }],
'tag_groups' => [TagGroupVersion, ['tag_group_id'], {
'name' => :string, 'description' => :string,
'members' => :json, 'placements' => :json }],
'materials' => [MaterialVersion, ['material_id'], {
'url' => :string, 'parent_id' => :integer, 'tag_id' => :integer,
'tag_name' => :string, 'tag_category' => :enum,
'discarded_at' => :datetime, 'updated_at' => :datetime,
'updated_by_user_id' => :integer, 'export_paths_json' => :json,
'file_blob_id' => :integer, 'file_filename' => :string,
'file_content_type' => :string, 'file_byte_size' => :integer,
'file_checksum' => :string, 'file_sha256' => :string,
'source_kind' => :enum, 'source_uri' => :string, 'source_path' => :string,
'source_file_id' => :string, 'normalized_source_key' => :string }] }.freeze
OPERATORS = {
string: ['eq', 'ne', 'like'], enum: ['eq', 'ne'], boolean: ['eq', 'ne'],
integer: ['eq', 'ne', 'gt', 'lt', 'ge', 'le', 'range'],
datetime: ['eq', 'ne', 'gt', 'lt', 'ge', 'le', 'range'] }.freeze
attr_reader :model, :keys, :fields, :resource
def initialize(resource)
@resource = resource
@model, @keys, attributes = DEFINITIONS.fetch(resource) { raise ArgumentError }
@fields = COMMON.merge(attributes)
end
def table = model.table_name
def time_column = 'created_at'
def column(key)
raise ArgumentError unless fields.key?(key)
{ 'resource_id' => keys.first, 'user' => 'created_by_user_id' }.fetch(key, key)
end
def expression(key, row = 'v') = "#{ row }.#{ column(key) }"
def changes = fields.keys - COMMON.keys - ['updated_at', 'updated_by_user_id']
def axes(history)
fields.reject { |_key, type| type == :json }.keys +
(resource == 'posts' ? ['tags'] : []) + (history ? ['changed_field'] : [])
end
def metadata
fields.map do |key, type|
{ key:, type:, nullable: model.columns_hash.fetch(column(key)).null,
operators: OPERATORS.fetch(type, []) }
end
end
end
+370
ファイルの表示
@@ -0,0 +1,370 @@
# Restores one recorded version per resource before applying record filters.
class CountsStateQuery
KEYS = ['locale', 'resource', 'mode', 'at', 'from', 'before', 'grain',
'kind', 'tag', 'page', 'limit', 'sort', 'axis', 'user_id', 'resource_id'].freeze
MAX_PERIODS = 366
attr_reader :at
def initialize(params, language_code:)
raise ArgumentError unless (params.keys - KEYS).empty?
raise ArgumentError unless params.values.all? { _1.is_a?(String) }
@params = params
@connection = ApplicationRecord.connection
@language = language_code
@resource = params.fetch('resource', 'posts')
@mode = params.fetch('mode', 'summary')
@inclusive = ['summary', 'details'].include?(@mode)
@axis = params.fetch('axis', @resource == 'posts' ? 'tags' : 'month')
@kind = params.fetch('kind', 'internal')
@grain = params.fetch('grain', 'month')
raise ArgumentError unless ['posts', 'tags'].include?(@resource)
raise ArgumentError unless ['summary', 'details', 'state', 'trend', 'snapshot'].include?(@mode)
raise ArgumentError unless ['internal', 'external', 'all'].include?(@kind)
raise ArgumentError unless ['day', 'month', 'year'].include?(@grain)
raise ArgumentError unless ['tags', 'month', 'user'].include?(@axis)
if @mode == 'summary' && @axis == 'tags' && @resource != 'posts'
raise ArgumentError
end
@page = integer('page', 1)
@limit = integer('limit', 20)
@tag = integer('tag') if params['tag'].present?
raise ArgumentError if @limit > 100 || @page > 1_000_000
raise ArgumentError if @tag && @kind == 'all'
raise ArgumentError if @mode == 'trend' && !@tag
@now = Time.current.utc
@at = timestamp('at') || @now
raise ArgumentError if @at > @now
end
def call(metadata: true)
@metadata = metadata
# Keep totals, rows, and detail pages on one repeatable-read database snapshot.
ApplicationRecord.transaction do
unless @history_loaded
@post_start = HistoryAccess.visible(PostVersion.all).minimum(:created_at)
@tag_start = HistoryAccess.visible(TagVersion.all).minimum(:created_at)
@history_loaded = true
end
result = { population_count: HistoryAccess.population_count(
@resource == 'posts' ? PostVersion : TagVersion), page: @page, limit: @limit, at: @at.iso8601(6),
coverage: { basis: 'recorded_snapshots', exact: false,
post_history_from: @post_start&.iso8601(6),
tag_history_from: @tag_start&.iso8601(6) } }
case @mode
when 'trend'
result.merge(trend)
when 'summary'
result.merge(metadata ? { totals: totals } : {}).merge(summary)
when 'details', 'snapshot'
result.merge(metadata ? { totals: totals } : {}).merge(details)
else
result.merge(metadata ? { totals: totals } : {}).merge(listing)
end
end
end
def each_page
return enum_for(:each_page) unless block_given?
@page = 1
first = call
yield first
total = ['summary', 'state', 'trend'].include?(@mode) ? first[:group_count] : first[:count]
while @page * @limit < total
@page += 1
yield call(metadata: false)
end
end
def validate_query!
filtered_scope(@resource)
case @mode
when 'summary'
if @axis == 'tags'
order_sql({ 'name' => 'name', 'count' => 'post_count', 'tag_id' => 'tag_id',
'kind' => 'kind', 'deprecated' => 'deprecated_at' }, 'count:desc', [])
else
record_query('summary')
end
when 'details', 'snapshot'
record_query('details')
end
self
end
private
def quote(value) = @connection.quote(value)
def integer(key, default = nil)
value = @params[key]
return default if value.blank? && default
raise ArgumentError unless value&.match?(/\A[1-9][0-9]{0,9}\z/)
value.to_i
end
def timestamp(key)
value = @params[key].presence
return nil unless value
pattern = /\A\d{4}-\d{2}-\d{2}T\d{2}:\d{2}(?::\d{2}(?:\.\d{1,6})?)?Z\z/
raise ArgumentError unless value.match?(pattern)
Date.iso8601(value.first(10))
Time.iso8601(value.length == 17 ? value.sub(/Z\z/, ':00Z') : value).utc
end
# Version numbers order equal-timestamp writes. Do not filter tags before this selection.
def latest_condition(table, row, boundary)
key = table == 'post_versions' ? 'post_id' : 'tag_id'
<<~SQL.squish
#{ row }.created_at #{ @inclusive ? '<=' : '<' } #{ boundary } AND NOT EXISTS (
SELECT 1 FROM #{ table } newer
WHERE newer.#{ key } = #{ row }.#{ key }
AND newer.created_at #{ @inclusive ? '<=' : '<' } #{ boundary }
AND newer.version_no > #{ row }.version_no)
SQL
end
def state_sql(table, boundary, live: false)
model = { 'post_versions' => PostVersion, 'tag_versions' => TagVersion,
'nico_tag_versions' => NicoTagVersion }.fetch(table)
sql = "SELECT v.* FROM #{ table } v WHERE #{ latest_condition(table, 'v', boundary) } " +
"AND (#{ HistoryAccess.sql(model, 'v') })"
live ? "#{ sql } AND v.event_type <> 'discard'" : sql
end
def membership(row)
key = @kind == 'external' ? 'external_tag_id' : 'tag_id'
"JSON_CONTAINS(#{ row }.tags_json, JSON_OBJECT('#{ key }', #{ @tag }))"
end
# Deprecation keeps a tag in existence. Only an explicit discard removes it.
def tag_visible(boundary)
return '1 = 1' if @kind == 'external'
<<~SQL.squish
NOT EXISTS (SELECT 1 FROM tag_versions t
WHERE t.tag_id = #{ @tag } AND t.event_type = 'discard'
AND #{ latest_condition('tag_versions', 't', boundary) })
SQL
end
def ctes
boundary = quote(@at)
<<~SQL
WITH post_at AS (#{ filtered_scope('posts').to_sql }),
tag_at AS (#{ state_sql('tag_versions', boundary) }),
nico_at AS (#{ state_sql('nico_tag_versions', boundary) }),
pairs AS (
SELECT DISTINCT p.post_id,
CASE WHEN entry.tag_id IS NULL THEN 'external' ELSE 'internal' END AS kind,
COALESCE(entry.tag_id, entry.external_tag_id) AS tag_id
FROM post_at p
JOIN JSON_TABLE(CAST(p.tags_json AS JSON), '$[*]' COLUMNS (
tag_id BIGINT PATH '$.tag_id' NULL ON EMPTY ERROR ON ERROR,
external_tag_id BIGINT PATH '$.external_tag_id' NULL ON EMPTY ERROR ON ERROR
)) entry
LEFT JOIN tag_at t ON t.tag_id = entry.tag_id
WHERE COALESCE(entry.tag_id, entry.external_tag_id) IS NOT NULL
AND (t.event_type IS NULL OR t.event_type <> 'discard')
)
SQL
end
def totals
row = @connection.select_one(<<~SQL)
#{ ctes }
SELECT (SELECT COUNT(*) FROM post_at) AS posts,
(SELECT COUNT(*) FROM (#{ filtered_scope('tags').to_sql }) counted_tags) AS tags,
(SELECT COUNT(*) FROM tag_at WHERE event_type <> 'discard'
AND deprecated_at IS NOT NULL) AS deprecated_tags,
(SELECT COUNT(*) FROM pairs) AS assignments,
(SELECT COUNT(*) FROM pairs WHERE kind = 'external') AS external_assignments
SQL
row.transform_values!(&:to_i)
row.merge('posts' => available?(@post_start, @at) ? row['posts'] : nil,
'assignments' => available?(@post_start, @at) ? row['assignments'] : nil,
'external_assignments' => available?(@post_start, @at) ?
row['external_assignments'] : nil,
'tags' => available?(@tag_start, @at) ? row['tags'] : nil,
'deprecated_tags' => available?(@tag_start, @at) ? row['deprecated_tags'] : nil)
end
def available?(first, boundary)
first && (@inclusive ? first <= boundary : first < boundary)
end
def listing
path = quote("$.#{ JSON.generate(@language) }.name")
catalog = <<~SQL
, tag_counts AS (
SELECT membership.kind, membership.tag_id, COUNT(*) AS post_count
FROM pairs membership GROUP BY membership.kind, membership.tag_id
), catalog AS (
SELECT 'internal' AS kind, tag_id,
COALESCE(JSON_UNQUOTE(JSON_EXTRACT(names, #{ path })), name) AS name,
deprecated_at FROM tag_at WHERE event_type <> 'discard'
UNION ALL
SELECT tc.kind, tc.tag_id, external_version.name, NULL AS deprecated_at
FROM tag_counts tc
LEFT JOIN tag_at internal_version
ON tc.kind = 'internal' AND internal_version.tag_id = tc.tag_id
LEFT JOIN nico_at external_version
ON tc.kind = 'external' AND external_version.tag_id = tc.tag_id
WHERE tc.kind = 'external' OR internal_version.tag_id IS NULL
)
SQL
filters = []
filters << "c.kind = #{ quote(@kind) }" unless @kind == 'all'
filters << "c.tag_id = #{ @tag }" if @tag
where = filters.any? ? "WHERE #{ filters.join(' AND ') }" : ''
source = 'FROM catalog c LEFT JOIN tag_counts n ON n.kind = c.kind AND n.tag_id = c.tag_id'
prefix = "#{ ctes } #{ catalog }"
total = if @metadata
@connection.select_value("#{ prefix } SELECT COUNT(*) #{ source } #{ where }").to_i
end
order = order_sql({ 'name' => 'c.name', 'count' => 'COALESCE(n.post_count, 0)',
'tag_id' => 'c.tag_id', 'kind' => 'c.kind',
'deprecated' => 'c.deprecated_at' }, 'count:desc',
['c.kind ASC', 'c.tag_id ASC'])
rows = @connection.select_all(<<~SQL)
#{ prefix }
SELECT c.kind, c.tag_id, c.name, c.deprecated_at,
COALESCE(n.post_count, 0) AS post_count #{ source } #{ where }
ORDER BY #{ order } LIMIT #{ @limit } OFFSET #{ (@page - 1) * @limit }
SQL
{ count: total, group_count: total, state_rows: rows.map do |row|
{ kind: row['kind'], tag_id: row['tag_id'], name: row['name'],
deprecated: row['deprecated_at']&.iso8601,
count: available?(@post_start, @at) ? row['post_count'].to_i : nil,
at: @at.iso8601(6) }
end }
end
# All record filters apply after selecting the latest recorded version at the point.
def restored_scope(resource)
model = resource == 'posts' ? PostVersion : TagVersion
table = model.table_name
scope = model.where(Arel.sql(latest_condition(table, table, quote(@at))))
.where.not(event_type: 'discard')
if @tag
scope = if resource == 'posts'
scope.where("#{ membership(table) } AND #{ tag_visible(quote(@at)) }")
elsif @kind == 'internal'
scope.where(tag_id: @tag)
else
scope.none
end
end
scope
end
def filtered_scope(resource)
params = @params.slice('from', 'before', 'user_id', 'resource_id')
.merge('mode' => 'details', 'latest' => '0', 'sort' => '')
CountsQuery.new(restored_scope(resource), resource, params,
latest_expression: '1').relation
end
def record_query(mode)
params = @params.slice('locale', 'sort', 'page', 'limit')
.merge('mode' => mode, 'latest' => '0', 'axis' => @axis == 'tags' ? 'month' : @axis,
'page' => @page.to_s, 'limit' => @limit.to_s)
CountsQuery.new(filtered_scope(@resource), @resource, params, latest_expression: '1')
end
def summary
return listing if @axis == 'tags'
query = record_query('summary')
{ count: @metadata ? query.count : nil }.merge(query.summary(include_count: @metadata))
end
def details
query = record_query('details')
{ count: @metadata ? query.count : nil,
versions: CountsVersionRepr.many(query.details, resource: @resource,
language_code: @language, at: @at,
inclusive: @inclusive) }
end
def next_boundary(time)
case @grain
when 'day' then time.beginning_of_day + 1.day
when 'month' then time.beginning_of_month.next_month
else time.beginning_of_year.next_year
end
end
def periods
@before = timestamp('before') || @now.beginning_of_month
@from = timestamp('from') || @before.prev_year
raise ArgumentError if @from >= @before || @before > @now
rows = []
boundary = next_boundary(@from)
while boundary <= @before
raise ArgumentError if rows.length >= MAX_PERIODS
format = { 'day' => '%Y-%m-%d', 'month' => '%Y-%m', 'year' => '%Y' }.fetch(@grain)
rows << [boundary, (boundary - 1.second).strftime(format)]
boundary = next_boundary(boundary)
end
rows
end
def trend
points = periods
metadata = { count: points.length, group_count: points.length, state_rows: [],
from: @from.iso8601, before: @before.iso8601, grain: @grain }
return metadata if points.empty?
values = points.map do |boundary, period|
"SELECT CAST(#{ quote(boundary) } AS DATETIME(6)) AS boundary, " +
"#{ quote(period) } AS period"
end.join(' UNION ALL ')
order = order_sql({ 'period' => 'boundary', 'count' => 'post_count' },
'period:asc', ['boundary ASC'])
known = @post_start ? "periods.boundary > #{ quote(@post_start) }" : 'FALSE'
rows = @connection.select_all(<<~SQL)
WITH periods AS (#{ values }), series AS (
SELECT periods.boundary, periods.period,
CASE WHEN #{ known } THEN COUNT(DISTINCT p.post_id) ELSE NULL END AS post_count
FROM periods
LEFT JOIN post_versions p ON p.event_type <> 'discard'
AND #{ membership('p') }
AND #{ latest_condition('post_versions', 'p', 'periods.boundary') }
AND #{ tag_visible('periods.boundary') }
GROUP BY periods.boundary, periods.period
)
SELECT * FROM series ORDER BY #{ order }
LIMIT #{ @limit } OFFSET #{ (@page - 1) * @limit }
SQL
metadata.merge(state_rows: rows.map do |row|
{ kind: @kind, tag_id: @tag, name: nil, period: row['period'],
at: row['boundary'].utc.iso8601(6), count: row['post_count']&.to_i }
end)
end
def order_sql(expressions, default, final)
raw = @params.fetch('sort', default)
sorts = raw.empty? ? [] : QueryState.list(raw).map { _1.split(':', -1) }
raise ArgumentError if sorts.map(&:first).uniq.length != sorts.length
raise ArgumentError if sorts.length > expressions.length
order = sorts.flat_map do |key, direction, *extra|
if extra.any? || !expressions.key?(key) || !['asc', 'desc'].include?(direction)
raise ArgumentError
end
expression = expressions.fetch(key)
["(#{ expression }) IS NULL ASC", "#{ expression } #{ direction.upcase }"]
end
(order + final).join(', ')
end
end
+308
ファイルの表示
@@ -0,0 +1,308 @@
require 'csv'
# Export uses the exact same population and grouped cells as the interactive table.
class CountsTableExport
EXTRA = ['format', 'columns', 'headers', 'labels', 'children', 'child_sort',
'order', 'nested_json', 'summary_columns', 'basis', 'axis', 'latest'].freeze
attr_reader :format, :resource
def initialize(params)
QueryState.validate!(params, CountsTableQuery::KEYS + EXTRA + ['measures'])
raise ArgumentError unless params.values.all? { _1.is_a?(String) }
params = CountsDefaults.for(params).merge(params)
unless params.key?('children')
params = params.merge('children' => CountsExportProjection.default_children(
params.fetch('resource', 'posts')).join(' '))
end
@params = params
@labels = JSON.parse(params.fetch('labels', '{}'))
raise ArgumentError unless @labels.is_a?(Hash)
@selected = params.key?('children') ? QueryState.list(params['children']) : nil
@order = QueryState.list(params.fetch('order', ''))
@child_sort = QueryState.list(params.fetch('child_sort', '')).map { _1.split(':', -1) }
@child_sort.each do |path, direction, *extra|
raise ArgumentError if extra.any? || !['asc', 'desc'].include?(direction)
raise ArgumentError unless path&.match?(/\A[a-zA-Z_]+(?:\.[a-zA-Z]+)*\/[a-zA-Z]+\z/)
end
@format = params.fetch('format', 'json')
@resource = params.fetch('resource', 'posts')
raise ArgumentError unless ['json', 'csv'].include?(format)
@query = CountsTableQuery.new(params.select { |key, _value| CountsTableQuery::KEYS.include?(key) ||
key == 'measures' || QueryState.valid_key?(key) }
.merge('page' => '1', 'limit' => '100'))
@summary = @query.mode == 'summary'
default_columns = @summary ? @query.axes.all.map { _1.fetch('field') } + ['count'] :
CountsExportProjection.default_columns(resource)
@columns = QueryState.list(params.fetch('columns', default_columns.join(' ')))
default_headers = @columns.map { @labels.fetch(_1, _1) }
@headers = JSON.parse(params.fetch('headers', default_headers.to_json))
raise ArgumentError unless @headers.is_a?(Array) && @headers.all? { _1.is_a?(String) }
if @summary && @headers.any? && @headers.length != @query.axes.all.length + 1
raise ArgumentError
end
unless @summary
if ['posts', 'tags'].include?(resource)
@projection = CountsExportProjection.new(params, resource:, summary: false, tags: false)
else
allowed = @query.population.definition.fields.keys + ['is_latest']
raise ArgumentError if (@columns - allowed).any?
raise ArgumentError unless @columns.length == @headers.length && @columns.uniq == @columns
end
end
end
def body(progress: nil)
Enumerator.new do |output|
ApplicationRecord.transaction(isolation: :repeatable_read) do
@language_names = Language.pluck(:code, :name).to_h
processed = 0
started = false
first = true
@query.each_page do |page|
total = @summary ? nil : page[:count]
progress&.call(processed, total)
unless started
metadata = page.except(:versions, :cells, :axis_labels, :column_values,
:row_headers, :column_headers, :pivot_totals).merge(
resource:, filters: @params.select { |key, _value| CountsTableQuery::KEYS.include?(key) ||
key == 'measures' || QueryState.valid_key?(key) },
fields: @query.population.definition.metadata,
columns: @columns, column_labels: @headers,
order: @params['order'], children: @params['children'],
sort: @params.fetch('sort', ''), child_sort: @params['child_sort'])
output << (format == 'json' ? "{\"metadata\":#{ metadata.to_json },\"data\":[" : "\uFEFF")
if format == 'csv'
headings = @summary ? pivot_headers(page) : [@headers]
headings.each do |headers|
output << CSV.generate_line(headers.map { safe(_1) }, row_sep: "\r\n")
end
end
started = true
end
if @summary
pivot_rows(page).each do |header|
records = page.fetch(:column_headers).map { pivot_cell(header, _1, page) }
if format == 'csv'
labels = pivot_row_labels(header, page)
values = labels + records.map { _1[:count] }
output << CSV.generate_line(values.map { safe(_1) }, row_sep: "\r\n")
processed += records.length
progress&.call(processed, nil)
else
records.each do |record|
output << ',' unless first
output << record.to_json
first = false
processed += 1
progress&.call(processed, nil)
end
end
end
next
end
page[:versions].each do |row|
record = project(row)
if format == 'json'
output << ',' unless first
output << record.to_json
else
values = if @summary
row[:values].each_with_index.map do |value, index|
summary_value(value, index, page)
end + [row[:count]]
elsif @projection
@projection.csv_record(record, row)
else
record.map { |key, value| csv_logical(key, value) }
end
output << CSV.generate_line(values.map { safe(_1) }, row_sep: "\r\n")
end
first = false
processed += 1
progress&.call(processed, total)
end
end
progress&.call(processed, processed)
output << ']}' if format == 'json'
end
end
end
private
def pivot_rows(page)
page.fetch(:row_headers).reject do |header|
header[:kind] == 'grand_total' && @query.page * @query.limit < page[:group_count]
end
end
def total_label(header)
key = header[:kind] == 'grand_total' ?
(@query.axes.all.length == 1 ? 'singleTotal' : 'grandTotal') : 'subtotal'
@labels.fetch(key, '')
end
def pivot_row_labels(header, page)
values = header[:values].each_with_index.map { |value, index|
summary_value(value, index, page)
}
if header[:kind] == 'data' && @previous_pivot_row&.fetch(:kind) == 'data'
values.each_index do |index|
if header[:values].first(index + 1) == @previous_pivot_row[:values].first(index + 1)
values[index] = ''
end
end
end
@previous_pivot_row = header
values << total_label(header) unless header[:kind] == 'data'
leading = [@query.axes.rows.length, @query.axes.columns.any? ? 1 : 0].max
Array.new(leading) { values[_1] || '' }
end
def pivot_headers(page)
row_count = @query.axes.rows.length
cols = @query.axes.columns
leading = [row_count, cols.any? ? 1 : 0].max
columns = page.fetch(:column_headers)
Array.new([cols.length, 1].max) do |level|
prefix = Array.new(leading) { level.zero? ? @headers[_1].to_s : '' }
prefix = [''] if row_count.zero? && cols.any?
prefix + columns.each_with_index.map do |header, index|
values = header[:values]
if cols.empty?
header[:kind] == 'data' ? @labels.fetch('count', '') : total_label(header)
elsif level == values.length && header[:kind] != 'data'
total_label(header)
elsif level < values.length
previous = index.positive? ? columns[index - 1][:values] : nil
if level < cols.length - 1 && previous && previous.first(level + 1) == values.first(level + 1)
''
else
summary_value(values[level], row_count + level, page)
end
else
''
end
end
end
end
def pivot_cell(row, column, page)
@pivot_lookup ||= {}
# Discard the previous page's lookup; memory remains bounded by one page and its rollups.
unless @pivot_page.equal?(page)
@pivot_page = page
@pivot_lookup = page.fetch(:cells).to_h do |cell|
values = cell[:values]
[[values.first(@query.axes.rows.length), values.drop(@query.axes.rows.length)], cell[:count]]
end
page.fetch(:pivot_totals).each { @pivot_lookup[[_1[:row], _1[:column]]] = _1[:count] }
end
kinds = [row[:kind], column[:kind]]
kind = if kinds.include?('grand_total')
'grand_total'
elsif kinds.include?('subtotal')
'subtotal'
else
'data'
end
{ row: row[:values], column: column[:values], kind:,
row_kind: row[:kind], column_kind: column[:kind],
row_depth: row[:values].length, column_depth: column[:values].length,
count: @pivot_lookup.fetch([row[:values], column[:values]], 0) }
end
def summary_value(value, index, page)
field = @query.axes.all.fetch(index).fetch('field')
return @labels.fetch('bot', '') if field == 'user' && value.nil?
names = page[:axis_labels].fetch("a#{ index }", {})
name = field == 'user' ? names[value.to_s] : names.fetch(value.to_s, value)
if field == 'user'
name = @labels.fetch('notRegistered', '') if name.nil? || name == ''
pattern = @labels.fetch('actorFormat', '{{name}} (#{{id}})')
return pattern.gsub(/\{\{(name|id)\}\}/) do
Regexp.last_match(1) == 'name' ? name.to_s : value.to_s
end
end
@labels.fetch("#{ field }:#{ value }", name)
end
def project(row)
return { row: row[:values].first(@query.axes.rows.length),
column: row[:values].drop(@query.axes.rows.length), count: row[:count] } if @summary
return @projection.project(row) if @projection
@columns.to_h do |column|
key = column == 'user' ? 'created_by_user' : column
value = row[key]
value = project_nested(value, key) if ['members', 'placements', 'export_paths_json'].include?(key)
[key, value]
end
end
def camel(key)
key.gsub(/_([a-z])/) { Regexp.last_match(1).upcase }
end
def project_nested(value, path)
rows = path == 'export_paths_json' ? (value || {}).map { |profile, target|
{ 'profile' => profile, 'path' => target }
} : Array(value)
sorts = @child_sort.filter_map do |key, direction|
parent, field = key.split('/', 2)
[field, direction] if parent == path
end
rows = rows.each_with_index.sort do |(first, a), (last, b)|
compared = 0
sorts.each do |field, direction|
left = first.find { |key, _| camel(key) == field }&.last
right = last.find { |key, _| camel(key) == field }&.last
compared = if left.nil? || right.nil?
left.nil? == right.nil? ? 0 : (left.nil? ? 1 : -1)
else
(left.to_s <=> right.to_s) * (direction == 'asc' ? 1 : -1)
end
break unless compared.zero?
end
compared.zero? ? a <=> b : compared
end.map(&:first)
rows.map do |row|
fields = row.keys.select { !@selected || @selected.include?("#{ path }.#{ camel(_1) }") }
ordered = (@order + fields.map { "#{ path }.#{ camel(_1) }" }).uniq
.filter_map { |item| fields.find { "#{ path }.#{ camel(_1) }" == item } }
ordered.to_h do |key|
item = row[key]
item = project_nested(item, "#{ path }.#{ camel(key) }") if item.is_a?(Array)
[key, item]
end
end
end
def csv_logical(key, value)
return @labels.fetch('bot', '') if key == 'created_by_user' && value.nil?
return value ? @labels.fetch('latest', '') : '' if key == 'is_latest'
if value.is_a?(Time) || value.is_a?(ActiveSupport::TimeWithZone)
return @query.population.timezone.utc_to_local(value.utc).iso8601
end
if key == 'language_code' || key == 'source_language_code'
return @language_names[value]
end
if value.is_a?(Hash)
return value.to_h { |field, item| [@labels.fetch(camel(field.to_s), ''), csv_logical(field, item)] }
end
return value.map { csv_logical(key, _1) } if value.is_a?(Array)
@labels.fetch("#{ key }:#{ value }", value)
end
def safe(value)
return value if value.is_a?(Numeric)
text = value.is_a?(Hash) || value.is_a?(Array) ? JSON.generate(value) : value.to_s
text.match?(/\A\s*[=+@-]/) ? "'#{ text }" : text
end
end
+316
ファイルの表示
@@ -0,0 +1,316 @@
# Common entrypoint for six version resources; legacy endpoints keep their contracts.
class CountsTableQuery
MAX_COLUMNS = 100
class TooManyColumns < ArgumentError; end
KEYS = ['resource', 'definition', 'mode', 'at', 'execution_at', 'from', 'through',
'timezone', 'rows', 'cols', 'conditions', 'drill', 'tags', 'tag_refs', 'match',
'tag', 'kind', 'resource_id', 'user_id', 'page', 'limit', 'sort', 'locale', 'measures', 'column_order'].freeze
attr_reader :population, :axes, :page, :limit, :mode
def initialize(params)
QueryState.validate!(params, KEYS + ['measures'])
raise ArgumentError unless params.values.all? { _1.is_a?(String) }
params = CountsDefaults.for(params).slice(*KEYS).merge(params)
@params = params
@population = CountsPopulation.new(params)
@definition = population.definition
@connection = @definition.model.connection
@axes = CountsAxes.new(population)
@page = Integer(params.fetch('page', '1'), 10)
@limit = Integer(params.fetch('limit', '20'), 10)
@mode = params.fetch('mode', 'summary')
raise ArgumentError unless page.between?(1, 1_000_000) && limit.between?(1, 100)
raise ArgumentError unless ['summary', 'details'].include?(mode)
@sort = QueryState.list(params.fetch('sort', '')).map { _1.split(':', -1) }
allowed = mode == 'summary' ? axes.all.each_index.map { "a#{ _1 }" } + ['count'] :
@definition.fields.keys + ['is_latest'] +
(@definition.resource == 'posts' ? ['tags', 'parents'] : []) +
(@definition.resource == 'tags' ? ['parents', 'deprecated'] : [])
raise ArgumentError if @sort.map(&:first).uniq.length != @sort.length
@sort.each do |key, direction, *extra|
measure = mode == 'summary' && key&.match?(/\Am[0-9]{1,2}\z/)
measure_values(key) if measure
if extra.any? || (!allowed.include?(key) && !measure) || !['asc', 'desc'].include?(direction)
raise ArgumentError
end
end
end
def call
ApplicationRecord.transaction do
metadata.merge(mode == 'summary' ? summary : details)
end
end
def metadata
{ population_count_scope: 'all_recorded_versions', population_count: HistoryAccess.population_count(@definition.model),
page:, limit:, execution_at: population.at.iso8601(6), at: @params['at'].presence,
definition: population.history ? 'history' : 'latest',
timezone: population.timezone.identifier, rows: axes.rows, cols: axes.columns,
fields: @definition.metadata,
count: population.scope.except(:select).optimizer_hints('MAX_EXECUTION_TIME(10000)').count }
end
def summary
expressions = axes.expressions
selected = expressions.each_with_index.map do |expression, index|
"#{ expression } AS a#{ index }"
end
source = population.scope
source = source.joins(axes.joins) unless axes.joins.empty?
sql = if expressions.empty?
source.except(:select).select('COUNT(DISTINCT v.id) AS count').to_sql
else
source.except(:select).group(*expressions.map { Arel.sql(_1) })
.select(Arel.sql((selected + ['COUNT(DISTINCT v.id) AS count']).join(', ')))
.to_sql
end
prefix = "WITH cells AS (#{ sql })"
column_values = limited_columns(prefix)
row_keys = axes.rows.each_index.map { "a#{ _1 }" }
# Page whole row tuples so a pivot row is never split across display pages.
row_selection = row_keys.empty? ? '1 AS singleton' : row_keys.join(', ')
row_group = row_keys.empty? ? 'GROUP BY singleton' : "GROUP BY #{ row_keys.join(', ') }"
measure_joins = []
extras = []
row_sort = @sort.each_with_index.filter_map do |(key, direction), index|
if key == 'count' || key.match?(/\Am[0-9]+\z/)
values = key == 'count' ? [] : measure_values(key)
relation = measure_relation(values)
matches = row_keys.map { "grouped.#{ _1 } <=> sort#{ index }.#{ _1 }" }
condition = matches.empty? ? 'TRUE' : matches.join(' AND ')
measure_joins << "LEFT JOIN (#{ relation }) sort#{ index } ON #{ condition }"
extras << "sort#{ index }.count AS sort#{ index }"
"sort#{ index } IS NULL ASC, sort#{ index } #{ direction.upcase }"
elsif row_keys.include?(key)
"#{ key } IS NULL ASC, #{ key } #{ direction.upcase }"
end
end
order_rows = row_sort + row_keys.map { "#{ _1 } IS NULL, #{ _1 }" }
order_rows = ['singleton ASC'] if order_rows.empty?
grouped = "SELECT #{ row_selection } FROM cells #{ row_group }"
selection = (['grouped.*'] + extras).join(', ')
group_sql = "SELECT #{ selection } FROM (#{ grouped }) grouped #{ measure_joins.join(' ') }"
total = @connection.select_value(<<~SQL).to_i
#{ prefix }
SELECT /*+ MAX_EXECUTION_TIME(10000) */ COUNT(*) FROM (#{ group_sql }) row_groups
SQL
join = row_keys.map { "cells.#{ _1 } <=> page_rows.#{ _1 }" }.join(' AND ')
join = 'TRUE' if row_keys.empty?
order_cols = @sort.filter_map do |key, direction|
next unless key.match?(/\Aa[0-9]+\z/) && !row_keys.include?(key)
"cells.#{ key } #{ direction.upcase }"
end
order = ['page_rows.position'] + order_cols +
axes.all.each_index.map { "cells.a#{ _1 }" }
records = @connection.select_all(<<~SQL)
#{ prefix }, row_groups AS (#{ group_sql }),
numbered AS (SELECT row_groups.*,
ROW_NUMBER() OVER (ORDER BY #{ order_rows.join(', ') }) AS position FROM row_groups),
page_rows AS (SELECT * FROM numbered
WHERE position > #{ (page - 1) * limit } AND position <= #{ page * limit })
SELECT /*+ MAX_EXECUTION_TIME(10000) */ cells.* FROM cells JOIN page_rows ON #{ join }
ORDER BY #{ order.join(', ') }
SQL
next_row = @connection.select_one(<<~SQL)
#{ prefix }, row_groups AS (#{ group_sql }),
numbered AS (SELECT row_groups.*,
ROW_NUMBER() OVER (ORDER BY #{ order_rows.join(', ') }) AS position FROM row_groups)
SELECT /*+ MAX_EXECUTION_TIME(10000) */ * FROM numbered
WHERE position = #{ page * limit + 1 }
SQL
row_values = records.map { |record| row_keys.map { record[_1] } }.uniq
row_depths = CountsPivotLayout.depths(row_keys, @sort)
column_keys = axes.columns.each_index.map { "a#{ axes.rows.length + _1 }" }
column_sort = @sort
if @params.key?('column_order') && column_keys.any? &&
!(@sort.first&.first == 'count' || @sort.first&.first&.match?(/\Am[0-9]+\z/))
column_sort = [[column_keys.first, 'asc']] + @sort
end
column_depths = CountsPivotLayout.depths(column_keys, column_sort, measures: false)
row_headers = CountsPivotLayout.headers(row_values, row_depths,
next_values: next_row && row_keys.map { next_row[_1] }, grand: axes.rows.any?)
if axes.rows.empty? && axes.columns.any?
row_headers = [{ values: [], kind: 'grand_total' }]
end
column_headers = CountsPivotLayout.headers(column_values, column_depths,
grand: axes.columns.any?)
if axes.columns.empty? && axes.rows.any?
column_headers = [{ values: [], kind: 'grand_total' }]
end
totals = CountsPivotTotals.new(population, axes, row_headers, column_headers).call
{ group_count: total, column_values:, row_headers:, column_headers:, pivot_totals: totals,
cells: records.map do |row|
{ values: axes.all.each_index.map { row["a#{ _1 }"] }, count: row['count'].to_i }
end, axis_labels: axis_labels(records, column_values) }
end
def details
query = population.scope.select(Arel.sql(
"(#{ population.latest_sql }) AS counts_latest"))
order = sort_sql { detail_expression(_1) }
query = query.order(Arel.sql((order + ['v.id DESC']).join(', ')))
.limit(limit).offset((page - 1) * limit)
rows = query.to_a
if ['posts', 'tags'].include?(@definition.resource)
point = population.history ? population.at :
population.timestamp(@params['at']) || population.at
values = CountsVersionRepr.many(rows, resource: @definition.resource,
language_code: locale.language_code,
at: point)
values.each_with_index { |value, index| value[:event_type] = rows[index].event_type }
else
users = User.where(id: rows.map(&:created_by_user_id)).pluck(:id, :name).to_h
values = rows.map do |row|
attributes = @definition.fields.keys.to_h do |key|
[key, row.public_send(@definition.column(key))]
end
attributes.merge('id' => row.id, 'resource_id' => row[@definition.keys.first],
'is_latest' => row.attributes['counts_latest'].to_i == 1,
'created_by_user' => row.created_by_user_id &&
{ id: row.created_by_user_id, name: users[row.created_by_user_id] })
end
end
{ versions: values }
end
def each_page
total = nil
loop do
result = call
total ||= mode == 'summary' ? result[:group_count] : result[:count]
yield result
break if page * limit >= total
@page += 1
end
end
private
def locale = @locale ||= Locale.find_by!(code: @params.fetch('locale', 'ja'))
def limited_columns(prefix)
keys = axes.columns.each_index.map { "a#{ axes.rows.length + _1 }" }
return [[]] if keys.empty?
configured = @sort.select { |key, _direction| keys.include?(key) }
directions = (configured + keys.map { [_1, 'asc'] }).uniq { _1.first }
order = directions.map { |key, direction|
"#{ key } IS NULL ASC, #{ key } #{ direction.upcase }"
}.join(', ')
custom = QueryState.records(@params, 'column_order')
if custom.any?
raise ArgumentError unless custom.map { _1['value'] }.uniq.length == custom.length
cases = custom.each_with_index.map do |record, index|
"WHEN #{ keys.first } <=> #{ @connection.quote(record['value']) } THEN #{ index }"
end.join(' ')
order = "CASE #{ cases } ELSE #{ custom.length } END, #{ order }"
end
values = @connection.select_rows(<<~SQL)
#{ prefix }
SELECT /*+ MAX_EXECUTION_TIME(10000) */ DISTINCT #{ keys.join(', ') } FROM cells
ORDER BY #{ order } LIMIT #{ MAX_COLUMNS + 1 }
SQL
raise TooManyColumns if values.length > MAX_COLUMNS
values
end
def measure_relation(values)
row_expressions = axes.expressions.first(axes.rows.length)
source = population.scope.except(:select, :order)
fields = axes.rows + axes.columns.first(values.length)
joins = axes.joins(fields.map { _1.fetch('field') })
source = source.joins(joins) unless joins.empty?
values.each_with_index do |value, index|
expression = axes.expressions[axes.rows.length + index]
source = source.where("(#{ expression }) <=> ?", value)
end
selected = row_expressions.each_with_index.map { |expression, index|
"#{ expression } AS a#{ index }"
} + ['COUNT(DISTINCT v.id) AS count']
source = source.group(*row_expressions.map { Arel.sql(_1) }) if row_expressions.any?
source.select(Arel.sql(selected.join(', '))).to_sql
end
def measure_values(key)
records = QueryState.records(@params, 'measures').select { _1['key'] == key }
raise ArgumentError if records.empty?
raise ArgumentError unless records.map { _1['axis'] }.sort == (0...records.length).to_a
value = records.sort_by { _1['axis'] }.map { _1['value'] }
unless (value.is_a?(Array) && value.length <= axes.columns.length &&
value.all? { _1.nil? || _1.is_a?(String) || _1.is_a?(Numeric) })
raise ArgumentError
end
value
end
def detail_expression(key)
return "(#{ population.latest_sql })" if key == 'is_latest'
aliases = { 'tags' => 'tags_json', 'parents' => @definition.resource == 'posts' ?
'parent_post_ids' : 'parent_tag_ids', 'deprecated' => 'deprecated_at' }
field = aliases.fetch(key, key)
expression = @definition.expression(field)
return expression unless @definition.fields[field] == :json
"CAST(#{ expression } AS CHAR) COLLATE utf8mb4_bin"
end
def sort_sql
@sort.flat_map do |key, direction|
expression = yield key
["(#{ expression }) IS NULL ASC", "#{ expression } #{ direction.upcase }"]
end
end
def axis_labels(records, column_values)
result = {}
axes.all.each_with_index do |axis, index|
values = records.map { _1["a#{ index }"] }
if index >= axes.rows.length
values += column_values.map { _1[index - axes.rows.length] }
end
values = values.compact.uniq
labels = case axis['field']
when 'user'
User.where(id: values).pluck(:id, :name).to_h.transform_keys(&:to_s)
when 'language_code', 'source_language_code'
Language.where(code: values).pluck(:code, :name).to_h
when 'tags'
tag_labels(values)
else {}
end
result["a#{ index }"] = labels
end
result
end
def tag_labels(values)
internal = values.grep(/\Ainternal:/).map { _1.split(':').last.to_i }
external = values.grep(/\Aexternal:/).map { _1.split(':').last.to_i }
point = population.timestamp(@params['at']) || population.at
labels = {}
sources = [[TagVersion, internal, 'internal'], [NicoTagVersion, external, 'external']]
sources.each do |model, ids, kind|
next if ids.empty?
table = model.table_name
model.where(tag_id: ids).where('created_at <= ?', point)
.where("NOT EXISTS (SELECT 1 FROM #{ table } newer " +
"WHERE newer.tag_id = #{ table }.tag_id AND newer.created_at <= ? " +
"AND newer.version_no > #{ table }.version_no)", point).each do |row|
names = row.respond_to?(:names) ? row.names : nil
name = names&.dig(locale.language_code, 'name') || row.name
labels["#{ kind }:#{ row.tag_id }"] = name
end
end
labels
end
end
+42
ファイルの表示
@@ -0,0 +1,42 @@
# Shared literal parsing and reference resolution for current and snapshot post search.
class PostTagSearch
def self.parse_references(raw, literals)
return nil unless raw.key?('tag_refs')
references = QueryState.records(raw, 'tag_refs')
valid = references.is_a?(Array) && references.all? do |reference|
next false unless reference.is_a?(Hash) && reference['name'].is_a?(String)
ids = [reference['tag_id'], reference['external_tag_id']].compact
((ids.any? || reference['missing'] == true) &&
ids.all? { _1.is_a?(Integer) && _1.positive? } &&
(!reference.key?('missing') || (reference['missing'] == true && ids.empty?)) &&
(!reference.key?('negative') || [true, false].include?(reference['negative'])))
end
raise ArgumentError unless valid
references.map { _1['name'] } == literals ? references : nil
end
def self.literals(locale, raw, references: nil)
raw.split.each_with_index.filter_map do |token, index|
next if token.downcase == 'type:or'
reference = references&.[](index)
name = token.sub(/\Anot:/i, '')
name = TagName.canonicalise(locale, name).first unless reference
negative = reference&.key?('negative') ? reference['negative'] :
token.downcase.start_with?('not:')
{ name:, negative:, reference: }
end
end
def self.references(locale, literal)
reference = literal[:reference]
return [reference['tag_id'], reference['external_tag_id']] if reference
[TagName.where(language_code: locale.language_code, name: literal[:name])
.distinct.pluck(:tag_id),
ExternalTag.where("CONCAT(platform, ':', name) = ?", literal[:name]).pluck(:id)]
end
end
+88
ファイルの表示
@@ -0,0 +1,88 @@
# Public query grammar shared with frontend/src/lib/queryState.ts.
class QueryState
SCHEMAS = {
'conditions' => ['field', 'operator', 'value', 'end', 'blank'],
'drill' => ['field', 'grain', 'value', 'value_type'],
'tag_refs' => ['name', 'tag_id', 'external_tag_id', 'negative', 'missing'],
'column_order' => ['value', 'value_type'],
'measures' => ['key', 'axis', 'value', 'value_type'] }.freeze
def self.list(value)
return [] if value.nil? || value == ''
raise ArgumentError unless value.is_a?(String) && value.bytesize <= 32_768
values = value.split(/ /, -1)
raise ArgumentError if values.length > 200 || values.any?(&:empty?) || values.uniq != values
values
end
def self.valid_key?(key)
root, index, field, extra = key.split('.', -1)
!extra && SCHEMAS.key?(root) && index&.match?(/\A(?:0|[1-9][0-9]?)\z/) &&
SCHEMAS[root].include?(field)
end
def self.validate!(params, allowed)
raise ArgumentError if params.length > 1000
params.each do |key, value|
raise ArgumentError unless allowed.include?(key) || valid_key?(key)
raise ArgumentError unless value.is_a?(String) && value.bytesize <= 131_072
end
SCHEMAS.each_key { records(params, _1) }
end
def self.records(params, key)
records = {}
params.each do |name, value|
next unless name.start_with?("#{ key }.")
raise ArgumentError unless params.key?(key) && valid_key?(name)
_root, index, field = name.split('.')
record = records[index.to_i] ||= {}
if ['blank', 'negative', 'missing'].include?(field)
raise ArgumentError unless ['0', '1'].include?(value)
value = value == '1'
elsif ['tag_id', 'external_tag_id', 'axis'].include?(field)
raise ArgumentError unless value.match?(/\A[0-9]{1,18}\z/)
value = value.to_i
end
record[field] = value
end
raise ArgumentError if params.key?(key) && params[key] != ''
raise ArgumentError unless records.keys.sort == (0...records.length).to_a
records.sort.map do |_index, record|
type = record.delete('value_type')
if ['drill', 'measures', 'column_order'].include?(key)
raise ArgumentError unless ['null', 'string', 'number', 'boolean'].include?(type)
raise ArgumentError unless type == 'null' || record.key?('value')
value = record['value']
record['value'] = case type
when 'null' then nil
when 'number'
raise ArgumentError unless value&.match?(/\A-?[0-9]{1,18}\z/)
Integer(value, 10)
when 'boolean'
raise ArgumentError unless ['true', 'false'].include?(value)
value == 'true'
else value
end
end
record
end
end
def self.axes(value)
list(value).map do |token|
field, grain, extra = token.split(':', -1)
raise ArgumentError if extra || !field.match?(/\A[a-z_]+\z/) || grain == ''
grain ? { 'field' => field, 'grain' => grain } : { 'field' => field }
end
end
end
+7
ファイルの表示
@@ -0,0 +1,7 @@
Rails.application.config.to_prepare do
[ActiveStorage::Blobs::ProxyController, ActiveStorage::Blobs::RedirectController,
ActiveStorage::Representations::ProxyController,
ActiveStorage::Representations::RedirectController, ActiveStorage::DiskController].each do |controller|
controller.include(HistoryStorageAccess) unless controller < HistoryStorageAccess
end
end
+13
ファイルの表示
@@ -1,4 +1,17 @@
Rails.application.routes.draw do Rails.application.routes.draw do
get 'counts/definitions/:resource', to: 'counts_tables#definition'
get 'counts/query', to: 'counts_tables#index'
post 'counts/export/jobs', to: 'counts_exports#create_job'
get 'counts/export/jobs/:token', to: 'counts_exports#job_status'
delete 'counts/export/jobs/:token', to: 'counts_exports#cancel_job'
post 'counts/export/prepare', to: 'counts_exports#prepare'
get 'counts/export/files/:token', to: 'counts_exports#download'
get 'counts/export', to: 'counts_exports#index'
post 'counts/export', to: 'counts_exports#index'
get 'counts/state', to: 'counts_states#index'
get 'counts/posts', to: 'counts_posts#index'
get 'counts/tags', to: 'counts_tags#index'
resources :locales, only: [:index] resources :locales, only: [:index]
resources :sitemap_entries, only: [:index, :show], param: :resource resources :sitemap_entries, only: [:index, :show], param: :resource
+11
ファイルの表示
@@ -0,0 +1,11 @@
namespace :counts do
desc 'Process the persistent Counts export queue (run under a process supervisor)'
task export_worker: :environment do
stopping = false
['TERM', 'INT'].each { |signal| Signal.trap(signal) { stopping = true } }
until stopping
worked = Rails.application.executor.wrap { CountsExportJobs.work_one }
sleep(1) unless worked
end
end
end
+96
ファイルの表示
@@ -0,0 +1,96 @@
require 'rails_helper'
RSpec.describe 'Counts API', type: :request do
let!(:actor) { create(:user, :member) }
let!(:post) { create(:post) }
let!(:tag) { create(:tag, primary_name: 'counts tag', category: :general) }
before do
snapshot = { tag_id: tag.id, version_no: 1, name: 'counts tag',
category: 'general', sections: [] }
[1, 2, 3].each do |number|
time = Time.utc(2024, number == 3 ? 2 : 1, number, 0, 0)
PostVersion.create!(post:, version_no: number,
event_type: number == 1 ? 'create' : 'update',
tags: '', url: post.url, title: "Snapshot #{ number }", parent_post_ids: '',
tags_json: number == 1 ? [snapshot] : [],
created_by_user: number == 3 ? nil : actor, created_at: time)
TagVersion.create!(tag:, version_no: number,
event_type: number == 1 ? 'create' : 'update',
name: 'counts tag', category: 'general', aliases: '', parent_tag_ids: '',
names: { ja: { name: 'counts tag', script_code: 'Jpan',
auto_generated: false, aliases: [] } },
created_by_user: number == 3 ? nil : actor, created_at: time)
end
end
['posts', 'tags'].each do |resource|
context resource do
let(:path) { "/counts/#{ resource }" }
let(:id) { resource == 'posts' ? post.id : tag.id }
let(:filters) { { resource_id: id, user_id: actor.id,
from: '2024-01-01T00:00Z', before: '2024-02-01T00:00Z' } }
it 'uses the same AND filters for SQL totals, groups and details' do
get path, params: filters.merge(axis: 'event_type')
expect(response).to have_http_status(:ok)
summary = response.parsed_body
expect(summary['count']).to eq(2)
expect(summary['groups'].sum { _1['count'] }).to eq(2)
expect(summary['groups'].map { _1['key'] }).to eq(['create', 'update'])
get path, params: filters.merge(mode: 'details')
expect(response.parsed_body['count']).to eq(2)
expect(response.parsed_body['versions'].map { _1['version_no'] }).to eq([2, 1])
end
it 'paginates details and groups independently from totals' do
get path, params: { resource_id: id, mode: 'details', limit: 1, page: 2 }
expect(response.parsed_body['count']).to eq(3)
expect(response.parsed_body['versions'].sole['version_no']).to eq(2)
get path, params: { resource_id: id, axis: 'month', limit: 1, page: 2 }
expect(response.parsed_body['group_count']).to eq(2)
expect(response.parsed_body['groups'].sole).to include('key' => '2024-02', 'count' => 1)
end
it 'groups null actors and supports their drill-down filter' do
get path, params: { resource_id: id, axis: 'user' }
expect(response.parsed_body['groups']).to include(
{ 'key' => nil, 'name' => nil, 'count' => 1 })
get path, params: { resource_id: id, mode: 'details', user_id: 'none' }
expect(response.parsed_body['count']).to eq(1)
end
it 'uses an inclusive start and exclusive end' do
get path, params: { resource_id: id,
from: '2024-01-01T00:00Z', before: '2024-01-02T00:00Z' }
expect(response.parsed_body['count']).to eq(1)
end
it 'rejects unsupported or malformed query values' do
[
{ axis: 'SQL' }, { limit: 101 }, { page: '1x' }, { user_id: '-1' },
{ event_type: 'invalid' }, { columns: 'password' }, { order: 'id' },
{ from: '2024-02-30T00:00Z' }, { from: '2024-01-01' },
{ user_id: ['1'] }, { mode: 'invalid' }].each do |invalid|
get path, params: invalid
expect(response).to have_http_status(:bad_request)
end
end
it 'retains the existing banned-user boundary' do
actor.update!(banned_at: Time.current)
sign_in_as(actor)
get path
expect(response).to have_http_status(:forbidden)
end
end
end
it 'matches removed tags from the previous snapshot, not current post_tags' do
get '/counts/posts', params: { tag: tag.id, mode: 'details' }
expect(response.parsed_body['count']).to eq(2)
expect(response.parsed_body['versions'].map { _1['version_no'] }).to eq([2, 1])
expect(response.parsed_body['versions'].first['tags'].sole['type']).to eq('removed')
end
end
+53
ファイルの表示
@@ -0,0 +1,53 @@
# Counts エクスポートワーカー
Counts の画面からのエクスポートは永続ファイルキューへ登録する。
Active Job の既定のプロセス内 async アダプターは使用しない。
追加 gem、外部キュー、DB マイグレーションは不要。
## 起動・配置
backend で `bundle exec rake counts:export_worker` を、systemd 等の監視対象として
別プロセスで起動する。Web プロセス内では起動しない。
ワーカー未起動の間は待機状態となる。1 プロセスは同時に1ジョブを処理し、
全体の実行枠は既存の2枠を共有する。ワーカーの起動・配備は実施していない。
Web とワーカーは同一リリース、Rails secret、アプリケーション環境、DB 接続先を
使用する。DB コネクションはワーカープロセスごとに必要。
`COUNTS_EXPORT_DIRECTORY` を同じ永続 POSIX 共有ディレクトリに設定する。
省略時は `backend/tmp/counts_exports`。
全プロセスで同一の所有者とアクセス権を使用する。
複数ホストでは、flock・同一ファイルシステム内の atomic rename・fsync が正しく
機能する共有ボリュームが必須。オブジェクトストレージや各コンテナ独立の tmp は不可。
ジョブ JSON と完成ファイルが格納されるため、このディレクトリを静的公開しない。
## API と進捗
- POST `/counts/export/jobs`:要求検証・実行時点の固定・署名付きジョブトークンの発行。
- GET `/counts/export/jobs/:token`:状態、実際の処理済みレコード数、判明している総数。
- DELETE `/counts/export/jobs/:token`:取消を永続記録。
- GET `/counts/export/files/:token`:生成完了後の署名付きファイルトークンによる取得。
ジョブ・ファイルのトークンは作成元のセッションに紐付く。
既存の BAN・CSRF 制御を使用する。フロントは約800ms間隔で状態を取得する。
明細の総数は同じ読み取りトランザクション内の SQL 件数を使用する。
集計結果の出力セル総数は生成終了まで確定させず、不確定進捗として扱う。
完成状態はファイルの生成完了であり、ブラウザの保存完了を意味しない。
## 取消・障害・制限
通常は約250ms間隔の生成チェックポイントで取消を検知する。
DB 問い合わせ・ファイル I/O がブロックしている間は即時に中断できない。
取消済みへの遷移は生成処理の終了と一時ファイル削除後に行う。
既に始まった HTTP 転送やブラウザ側の保存は取り消せない。
ワーカーが異常終了したジョブは、次のワーカーが実行ロックを取得した際に失敗状態にし、
そのジョブの残存生成ファイルを削除する。別の読み取り状態で自動再実行しない。
原子的に書き換える状態ファイルとプロセス間ロックにより、二重処理・取消との競合を防ぐ。
既存の同時実行2、1ファイル128MiB、保存領域512MiB、生成120秒、期限10分を維持する。
生成時間制限もチェックポイント方式であり、実行中 SQL の強制停止ではない。
期限切れのジョブ・ファイル・残存状態書込ファイルは登録時またはワーカーの巡回時に削除する。
ワーカーと Web の両方を停止した間は巡回削除も停止する。
これらの処理は静的確認のみで、実環境のキュー処理・取消・転送・共有ボリュームでの
排他動作は未確認。
+32 -4
ファイルの表示
@@ -1,5 +1,20 @@
# frontend/AGENTS.md # frontend/AGENTS.md
## Mandatory security and URL contracts
- URL クエリへの JSON 格納を禁止する。Base64・16進数等による迂回も禁止。
単純な配列は半角スペース区切り、構造化条件は番号付きの個別キーで表す。
未指定と明示的な空選択を区別し、自由入力を区切り文字で分割しない。
Frontend / Backend で同じ文法・上限・検証を使い、新しい URL 状態の追加前に確認する。
- 一度除票されたリソースの除票以前(除票版を含む)の履歴は admin 以外へ公開しない。
恢復しても解除しない。すべての後続除票を確定版番号と正しい複合キーで判定する。
本文・属性・ID・版番号・関連参照・旧値を API / SSR / キャッシュ / 出力から漏らさない。
サーバーが確定した current_user の role のみを認可に使用する。
秘匿版も安全な全体件数には含めるが、公開明細・内訳・検索件数とは分離する。
秘匿属性を条件・軸・ソート・差分件数から推測できる実装を禁止する。
- 秘匿は React / CSS では実装しない。認可済み API のみを使う。
認証・権限変更時は取得中リクエストと履歴キャッシュを破棄し、SSR の管理者データを共有しない。
## Scope ## Scope
These rules apply to work under `frontend/`. These rules apply to work under `frontend/`.
@@ -33,9 +48,9 @@ npm run lint
If either command cannot be run or fails, report the exact command and failure. If either command cannot be run or fails, report the exact command and failure.
Do not create, modify, or run tests unless the user explicitly asks for test 明示的な個別指示がない限り、テストの追加・修正・削除・実行を一切行わない。
work. When the user asks for tests, keep working and rerun them until they fixture、mock、snapshot、テスト補助コード及び既存のテスト差分にも触れない。
pass or the remaining failure is clearly blocked. ビルド・lint・静的確認を使い、未確認事項はそのまま報告する。
## TypeScript ## TypeScript
@@ -128,6 +143,19 @@ pass or the remaining failure is clearly blocked.
## Tailwind and UI ## Tailwind and UI
- 原則として、ユーザ向け UI 文言を TSX / TypeScript に直接記述しない。
新しい文言は既存の i18n システムで管理し、適切な既存キーを先に確認する。
不必要な重複定義を避け、新規機能固有の文言は適切な namespace にまとめる。
- 対応 locale の辞書を整合させる。未翻訳項目には既存の日本語 fallback と
`_untranslated` 規約を使い、language metadata を維持する。
翻訳を捏造せず、未翻訳文字列を正式な翻訳として登録しない。
- UI 文言の追加・変更時はビルド、静的検査、実画面の確認等で、辞書と
SSR / hydration の整合及び翻訳キーの非露出を検証する。
i18n の検証を理由にテスト関連作業を開始しない。
- API の内部識別子、パラメータ、ユーザ入力データ、外部 URL は機械的に
翻訳しない。ボタン、項目名、CSV ヘッダ、エラー、空状態、ツールチップ、
アクセシビリティ用ラベルも UI 文言として扱う。
- Tailwind scans `src/**/*.{html,js,ts,jsx,tsx,mdx}`. - Tailwind scans `src/**/*.{html,js,ts,jsx,tsx,mdx}`.
- Use `cn` from `src/lib/utils.ts` for conditional class names and class merging. - Use `cn` from `src/lib/utils.ts` for conditional class names and class merging.
- In JavaScript, JSX, TypeScript, and TSX, use `cn` from `@/lib/utils` - In JavaScript, JSX, TypeScript, and TSX, use `cn` from `@/lib/utils`
@@ -144,7 +172,7 @@ pass or the remaining failure is clearly blocked.
heavy borders, or nested panel decoration unless the surrounding screen heavy borders, or nested panel decoration unless the surrounding screen
already uses them. already uses them.
- Keep operational screens dense and direct; trim explanatory copy and use - Keep operational screens dense and direct; trim explanatory copy and use
short Japanese labels that fit the control. short localised labels that fit the control.
- Preserve existing Japanese tone and orthography in nearby UI text, including - Preserve existing Japanese tone and orthography in nearby UI text, including
old-kana wording where the file already uses it. old-kana wording where the file already uses it.
- Do not add user-facing copy, helper text, descriptions, notes, tooltips, - Do not add user-facing copy, helper text, descriptions, notes, tooltips,
生成ファイル
+15
ファイルの表示
@@ -11,6 +11,7 @@
"dependencies": { "dependencies": {
"@dnd-kit/core": "^6.3.1", "@dnd-kit/core": "^6.3.1",
"@dnd-kit/modifiers": "^9.0.0", "@dnd-kit/modifiers": "^9.0.0",
"@dnd-kit/sortable": "^10.0.0",
"@dnd-kit/utilities": "^3.2.2", "@dnd-kit/utilities": "^3.2.2",
"@fontsource-variable/noto-sans-jp": "^5.2.9", "@fontsource-variable/noto-sans-jp": "^5.2.9",
"@mdx-js/react": "^3.1.1", "@mdx-js/react": "^3.1.1",
@@ -574,6 +575,20 @@
"react": ">=16.8.0" "react": ">=16.8.0"
} }
}, },
"node_modules/@dnd-kit/sortable": {
"version": "10.0.0",
"resolved": "https://registry.npmjs.org/@dnd-kit/sortable/-/sortable-10.0.0.tgz",
"integrity": "sha512-+xqhmIIzvAYMGfBYYnbKuNicfSsk4RksY2XdmJhT+HAC01nix6fHCztU68jooFiMUB01Ky3F0FyOvhG/BZrWkg==",
"license": "MIT",
"dependencies": {
"@dnd-kit/utilities": "^3.2.2",
"tslib": "^2.0.0"
},
"peerDependencies": {
"@dnd-kit/core": "^6.3.0",
"react": ">=16.8.0"
}
},
"node_modules/@dnd-kit/utilities": { "node_modules/@dnd-kit/utilities": {
"version": "3.2.2", "version": "3.2.2",
"resolved": "https://registry.npmjs.org/@dnd-kit/utilities/-/utilities-3.2.2.tgz", "resolved": "https://registry.npmjs.org/@dnd-kit/utilities/-/utilities-3.2.2.tgz",
+1
ファイルの表示
@@ -18,6 +18,7 @@
"dependencies": { "dependencies": {
"@dnd-kit/core": "^6.3.1", "@dnd-kit/core": "^6.3.1",
"@dnd-kit/modifiers": "^9.0.0", "@dnd-kit/modifiers": "^9.0.0",
"@dnd-kit/sortable": "^10.0.0",
"@dnd-kit/utilities": "^3.2.2", "@dnd-kit/utilities": "^3.2.2",
"@fontsource-variable/noto-sans-jp": "^5.2.9", "@fontsource-variable/noto-sans-jp": "^5.2.9",
"@mdx-js/react": "^3.1.1", "@mdx-js/react": "^3.1.1",
+89 -1
ファイルの表示
@@ -62,6 +62,15 @@ const responseFor = url => {
return { locales: [{ code: 'ja', name: '日本語' }, { code: 'ain-Kana', name: 'アイヌ' }] } return { locales: [{ code: 'ja', name: '日本語' }, { code: 'ain-Kana', name: 'アイヌ' }] }
if (path === '/users/settings') return { theme: 'system', wiki_editor_mode: 'split' } if (path === '/users/settings') return { theme: 'system', wiki_editor_mode: 'split' }
if (path === '/users/theme_slots') return [] if (path === '/users/theme_slots') return []
if (path === '/counts/posts' || path === '/counts/tags') {
const resource = path.endsWith ('posts') ? 'posts' : 'tags'
return { count: 1, page: 1, limit: 20, group_count: 1,
groups: [{ key: url.searchParams.get ('axis') === 'user' ? 1 : '2024-01',
name: null, count: 1 }],
versions: resource === 'posts'
? [{ post_id: 1, version_no: 1, title: { current: `SSR count ${locale}` } }]
: [{ tag_id: 1, version_no: 1, name: { current: `SSR count ${locale}` } }] }
}
if (path === '/posts') return { posts: [post (locale)], count: 1 } if (path === '/posts') return { posts: [post (locale)], count: 1 }
if (path === '/gekanator/posts') return { posts: [post (locale)] } if (path === '/gekanator/posts') return { posts: [post (locale)] }
if (path === '/gekanator/questions') return { questions: [] } if (path === '/gekanator/questions') return { questions: [] }
@@ -300,6 +309,25 @@ after (async () => {
if (errors.length) throw new AggregateError (errors, 'SSR teardown failed') if (errors.length) throw new AggregateError (errors, 'SSR teardown failed')
}) })
test ('counts SSR renders history data and retains URL query state', async () => {
for (const locale of ['ja', 'ain-Kana']) {
for (const resource of ['posts', 'tags']) {
const column = resource === 'posts' ? 'title' : 'name'
const query = `mode=details&columns=${column}&user_id=4`
const result = await renderPage (`/${locale}/counts/${resource}?${query}`, origin, apiURL)
assert.equal (result.status, 200)
assert.match (result.html, new RegExp (`SSR count ${locale}`))
assert.match (result.html, new RegExp (`/${locale}/${resource}/1`))
assert.doesNotMatch (result.html, /counts:|columns\.changedAt|fields\.actor/)
assert.match (result.html, /aria-expanded="false"/)
if (locale === 'ain-Kana')
assert.match (result.html, /lang="ja"[^>]*>集計実行/)
assert.ok (requests.some (request => request.url.includes (`/counts/${resource}`)
&& request.url.includes ('user_id=4') && request.url.includes (`locale=${locale}`)))
}
}
})
test ('every canonical screen route returns page HTML without JavaScript', async t => { test ('every canonical screen route returns page HTML without JavaScript', async t => {
const cookie = await authenticatedCookie () const cookie = await authenticatedCookie ()
for (const route of routes) { for (const route of routes) {
@@ -582,7 +610,8 @@ test ('hydrate every route with stored browser preferences', {
}` }, sessionId) }` }, sessionId)
const evaluate = async expression => (await send ('Runtime.evaluate', { const evaluate = async expression => (await send ('Runtime.evaluate', {
expression, returnByValue: true }, sessionId)).result.value expression, returnByValue: true }, sessionId)).result.value
for (const route of routes.filter (entry => !entry.handle?.redirectTo)) { for (const route of routes.filter (entry => !entry.handle?.redirectTo
&& (process.env.SSR_COUNTS_ONLY !== '1' || entry.path.startsWith ('counts')))) {
for (const locale of ['ja', 'ain-Kana']) { for (const locale of ['ja', 'ain-Kana']) {
await t.test (`hydrate ${locale} ${route.path}`, async () => { await t.test (`hydrate ${locale} ${route.path}`, async () => {
const path = route.path.replace (':identifier', 'home').replace (/:id|:title/g, '1') const path = route.path.replace (':identifier', 'home').replace (/:id|:title/g, '1')
@@ -608,6 +637,65 @@ test ('hydrate every route with stored browser preferences', {
}) })
} }
} }
for (const width of [375, 1240]) {
await send ('Emulation.setDeviceMetricsOverride', {
width, height: 900, deviceScaleFactor: 1, mobile: width < 768 }, sessionId)
for (const resource of ['posts', 'tags']) {
await t.test (`counts layout ${width} ${resource}`, async () => {
await send ('Page.navigate', {
url: `${httpOrigin}/ja/counts/${resource}` }, sessionId)
let ready = false
for (let attempt = 0; attempt < 100; ++attempt) {
await new Promise (resolve => setTimeout (resolve, 75))
try {
ready = await evaluate (`document.querySelector('tbody tr')!=null`)
} catch {}
if (ready) break
}
assert.ok (ready, 'history query results must render')
const layout = await evaluate (`(()=>{
const nav=document.getElementById('counts-targets');
const main=document.querySelector('main').getBoundingClientRect();
return {visible:nav.getClientRects().length>0,
right:nav.getBoundingClientRect().right,left:main.left,width:main.width};
})()`)
assert.ok (width >= 768 ? layout.visible && layout.right <= layout.left
: !layout.visible && layout.width >= width - 16,
'the sidebar sits beside the table on desktop and collapses on mobile')
const toggle = `Array.from(document.querySelectorAll('button'))
.find(button=>button.textContent.includes('集計条件'))`
assert.equal (await evaluate (`(${toggle}).getAttribute('aria-expanded')`), 'false')
await evaluate (`(${toggle}).click()`)
const positions = await evaluate (`(()=>{
const output=document.querySelector('[data-counts-output]').getBoundingClientRect();
const filters=document.querySelector('[data-counts-filters]').getBoundingClientRect();
return {left:output.left,right:filters.left,top:output.top,bottom:filters.top};
})()`)
assert.ok (width >= 768 ? positions.left < positions.right
: positions.top < positions.bottom, 'output precedes filters in the layout')
assert.ok (await evaluate ('document.documentElement.scrollWidth<=innerWidth'))
if (resource === 'posts' && process.env.SSR_COUNTS_ONLY === '1') {
const { writeFile } = await import ('node:fs/promises')
const shot = await send ('Page.captureScreenshot', { format: 'png' }, sessionId)
await writeFile (`/tmp/counts-${width}-conditions.png`,
Buffer.from (shot.data, 'base64'))
}
await evaluate (`(${toggle}).click()`)
for (const dark of [false, true]) {
await evaluate (`document.documentElement.classList.toggle('dark',${dark})`)
assert.ok (await evaluate ('document.documentElement.scrollWidth<=innerWidth'),
'counts controls must fit the viewport in both themes')
if (resource === 'posts' && process.env.SSR_COUNTS_ONLY === '1') {
const { writeFile } = await import ('node:fs/promises')
const shot = await send ('Page.captureScreenshot', { format: 'png' }, sessionId)
await writeFile (`/tmp/counts-${width}-${dark ? 'dark' : 'light'}.png`,
Buffer.from (shot.data, 'base64'))
}
}
})
}
}
if (process.env.SSR_COUNTS_ONLY === '1') return
probeActive = true probeActive = true
for (const width of [375, 1240]) { for (const width of [375, 1240]) {
await send ('Emulation.setDeviceMetricsOverride', { await send ('Emulation.setDeviceMetricsOverride', {
+31 -4
ファイルの表示
@@ -1,6 +1,6 @@
import { useQuery, useQueryClient } from '@tanstack/react-query' import { useQuery, useQueryClient } from '@tanstack/react-query'
import { AnimatePresence, LayoutGroup, MotionConfig, motion } from 'framer-motion' import { AnimatePresence, LayoutGroup, MotionConfig, motion } from 'framer-motion'
import { createElement, Suspense, lazy, useEffect, import { createElement, Suspense, lazy, useCallback, useEffect, useRef,
useLayoutEffect, useMemo, useState } from 'react' useLayoutEffect, useMemo, useState } from 'react'
import { useTranslation } from 'react-i18next' import { useTranslation } from 'react-i18next'
import { createBrowserRouter, import { createBrowserRouter,
@@ -77,6 +77,8 @@ const MaterialListPage = lazyPage (() => import ('@/pages/materials/MaterialList
const MaterialNewPage = lazyPage (() => import ('@/pages/materials/MaterialNewPage')) const MaterialNewPage = lazyPage (() => import ('@/pages/materials/MaterialNewPage'))
const MaterialSyncSuppressionsPage = lazyPage ( const MaterialSyncSuppressionsPage = lazyPage (
() => import ('@/pages/materials/MaterialSyncSuppressionsPage')) () => import ('@/pages/materials/MaterialSyncSuppressionsPage'))
const CountsPage = lazyPage (() => import ('@/pages/CountsPage'))
const CountsResourcePage = lazyPage (() => import ('@/pages/CountsResourcePage'))
const MorePage = lazyPage (() => import ('@/pages/MorePage')) const MorePage = lazyPage (() => import ('@/pages/MorePage'))
const GekanatorUnavailablePage = lazyPage (() => import ('@/pages/GekanatorUnavailablePage')) const GekanatorUnavailablePage = lazyPage (() => import ('@/pages/GekanatorUnavailablePage'))
const NicoTagListPage = lazyPage (() => import ('@/pages/tags/NicoTagListPage')) const NicoTagListPage = lazyPage (() => import ('@/pages/tags/NicoTagListPage'))
@@ -194,6 +196,20 @@ export const pageRoutes = (
{ id: 'gekanator', path: 'gekanator', element: <GekanatorRoute user={user}/>, { id: 'gekanator', path: 'gekanator', element: <GekanatorRoute user={user}/>,
handle: { locale: 'ja', sitemap: ['static'], preload: locale => locale === 'ja' handle: { locale: 'ja', sitemap: ['static'], preload: locale => locale === 'ja'
? GekanatorPage.preload () : GekanatorUnavailablePage.preload () } }, ? GekanatorPage.preload () : GekanatorUnavailablePage.preload () } },
{ path: 'counts', element: <CountsPage/>,
handle: { preload: CountsPage.preload } },
{ path: 'counts/posts', element: <CountsResourcePage resource="posts"/>,
handle: { preload: CountsResourcePage.preload } },
{ path: 'counts/tags', element: <CountsResourcePage resource="tags"/>,
handle: { preload: CountsResourcePage.preload } },
{ path: 'counts/nico_tags', element: <CountsResourcePage resource="nico_tags"/>,
handle: { preload: CountsResourcePage.preload } },
{ path: 'counts/tag_descriptions', element: <CountsResourcePage resource="tag_descriptions"/>,
handle: { preload: CountsResourcePage.preload } },
{ path: 'counts/tag_groups', element: <CountsResourcePage resource="tag_groups"/>,
handle: { preload: CountsResourcePage.preload } },
{ path: 'counts/materials', element: <CountsResourcePage resource="materials"/>,
handle: { preload: CountsResourcePage.preload } },
{ path: 'more', element: <MorePage/>, { path: 'more', element: <MorePage/>,
handle: { preload: MorePage.preload, sitemap: ['static'] } }, handle: { preload: MorePage.preload, sitemap: ['static'] } },
{ path: '*', element: <NotFound/>, handle: { notFound: true } }] { path: '*', element: <NotFound/>, handle: { notFound: true } }]
@@ -251,8 +267,19 @@ const RoutedApp = ({ locales }: { locales: AvailableLocale[] }) => {
? resolve ({ location, params: route.params, locale, targetLocale }) ? resolve ({ location, params: route.params, locale, targetLocale })
: switchLocalePath (location, targetLocale) : switchLocalePath (location, targetLocale)
} }
const [user, setUser] = useState<User | null> ( const [user, setUserState] = useState<User | null> (
() => client.getQueryData<User> (sessionKeys.current) ?? null) () => client.getQueryData<User> (sessionKeys.current) ?? null)
const authority = useRef (user)
const setUser = useCallback<Dispatch<SetStateAction<User | null>>> (action => {
const next = typeof action === 'function' ? action (authority.current) : action
if (next?.id !== authority.current?.id || next?.role !== authority.current?.role)
{
client.cancelQueries ()
client.removeQueries ({ predicate: query => query.queryKey[0] !== 'session' })
}
authority.current = next
setUserState (next)
}, [client])
const [status, setStatus] = useState (200) const [status, setStatus] = useState (200)
const behaviourSettings = useClientBehaviourSettings () const behaviourSettings = useClientBehaviourSettings ()
const animationMode = behaviourSettings.animation ?? 'normal' const animationMode = behaviourSettings.animation ?? 'normal'
@@ -301,7 +328,7 @@ const RoutedApp = ({ locales }: { locales: AvailableLocale[] }) => {
} }
}) () }) ()
return () => { cancelled = true } return () => { cancelled = true }
}, []) }, [setUser])
useEffect (() => { useEffect (() => {
if (!(user)) if (!(user))
@@ -393,7 +420,7 @@ const LocaleRoute: FC = () => {
void i18n.changeLanguage (activeLocale) void i18n.changeLanguage (activeLocale)
}, [activeLocale, i18n]) }, [activeLocale, i18n])
const redirect = legacyLocalePath (location) const redirect = legacyLocalePath (location, pathname => resolvePageRoute (pathname) != null)
if (!(isLocaleCode (locale)) && redirect == null) if (!(isLocaleCode (locale)) && redirect == null)
return <NotFound/> return <NotFound/>
if (isError && data == null) if (isError && data == null)
+22
ファイルの表示
@@ -0,0 +1,22 @@
import { motion } from 'framer-motion'
import { useClientMotion } from '@/lib/useClientMotion'
import { cn } from '@/lib/utils'
import type { ReactNode } from 'react'
const AnimatedCollapse = ({ open, children, id, className }: {
open: boolean; children: ReactNode; id?: string; className?: string },
) => {
const motionSettings = useClientMotion ()
return <motion.div id={id} initial={false} aria-hidden={!open} inert={!open}
animate={{ height: open ? 'auto' : 0, opacity: open ? 1 : 0 }}
transition={motionSettings.transition ('collapse')}
className={cn ('min-h-0 overflow-hidden', className)}>
{children}
</motion.div>
}
export default AnimatedCollapse
+12 -8
ファイルの表示
@@ -8,13 +8,13 @@ import type { ComponentPropsWithoutRef, FC, FocusEvent } from 'react'
const pad = (n: number): string => n.toString ().padStart (2, '0') const pad = (n: number): string => n.toString ().padStart (2, '0')
const toDateTimeLocalValue = (value: Date) => { const toDateTimeLocalValue = (value: Date, seconds = false) => {
const y = value.getFullYear () const y = value.getFullYear ()
const m = pad (value.getMonth () + 1) const m = pad (value.getMonth () + 1)
const day = pad (value.getDate ()) const day = pad (value.getDate ())
const h = pad (value.getHours ()) const h = pad (value.getHours ())
const min = pad (value.getMinutes ()) const min = pad (value.getMinutes ())
return `${ y }-${ m }-${ day }T${ h }:${ min }` return `${ y }-${ m }-${ day }T${ h }:${ min }${seconds ? `:${pad (value.getSeconds ())}` : ''}`
} }
@@ -37,15 +37,17 @@ type Props = Omit<ComponentPropsWithoutRef<'input'>, 'onChange'> & {
onChange?: (isoUTC: string | null) => void onChange?: (isoUTC: string | null) => void
className?: string className?: string
onBlur?: (ev: FocusEvent<HTMLInputElement>) => void onBlur?: (ev: FocusEvent<HTMLInputElement>) => void
invalid?: boolean } invalid?: boolean
precision?: 'minute' | 'second' }
const DateTimeField: FC<Props> = ({ value, onChange, className, onBlur, invalid, ...rest }) => { const DateTimeField: FC<Props> = props => {
const { value, onChange, className, onBlur, invalid, precision = 'minute', ...rest } = props
const [local, setLocal] = useState ('') const [local, setLocal] = useState ('')
useEffect (() => { useEffect (() => {
setLocal (value ? toDateTimeLocalValue (new Date (value)) : '') setLocal (value ? toDateTimeLocalValue (new Date (value), precision === 'second') : '')
}, [value]) }, [value, precision])
return ( return (
<input <input
@@ -61,13 +63,15 @@ const DateTimeField: FC<Props> = ({ value, onChange, className, onBlur, invalid,
'focus:ring-2 focus:ring-blue-200']), 'focus:ring-2 focus:ring-blue-200']),
className)} className)}
type="datetime-local" type="datetime-local"
step={60} step={precision === 'second' ? 1 : 60}
value={local} value={local}
aria-invalid={invalid} aria-invalid={invalid}
onChange={ev => { onChange={ev => {
const v = ev.target.value const v = ev.target.value
setLocal (v) setLocal (v)
onChange?.(v ? toMinutePrecisionIsoUtc (v) : null) const iso = v ? (precision === 'second'
? new Date (v).toISOString () : toMinutePrecisionIsoUtc (v)) : null
onChange?.(iso)
}} }}
onBlur={onBlur}/>) onBlur={onBlur}/>)
} }
+35
ファイルの表示
@@ -0,0 +1,35 @@
import { useId, useRef } from 'react'
import { CircleHelp } from 'lucide-react'
import type { ReactNode } from 'react'
const HelpPopover = ({ label, children }: {
label: { text: string; language: string }; children: ReactNode },
) => {
const id = useId ()
const trigger = useRef<HTMLButtonElement> (null)
const panel = useRef<HTMLDivElement> (null)
return <>
<button ref={trigger} type="button" popoverTarget={id} aria-label={label.text}
lang={label.language} className="inline-flex align-middle"
onClick={() => {
const rect = trigger.current?.getBoundingClientRect ()
if (panel.current != null && rect != null)
{
panel.current.style.top = `${rect.bottom + 4}px`
panel.current.style.left = `${Math.min (rect.left, window.innerWidth - 288)}px`
}
}}><CircleHelp size={14}/></button>
<div ref={panel} id={id} popover="auto" tabIndex={-1}
role="dialog" aria-label={label.text} lang={label.language}
onToggle={event => {
if (event.newState === 'open') panel.current?.focus ()
}} className="fixed m-0 w-72 max-w-[90vw] rounded border bg-white p-3
text-sm text-slate-900 shadow-lg dark:border-slate-600 dark:bg-slate-900
dark:text-slate-100">{children}</div>
</>
}
export default HelpPopover
+79
ファイルの表示
@@ -0,0 +1,79 @@
import { DndContext, DragOverlay, KeyboardSensor, MouseSensor, TouchSensor,
closestCenter, useSensor, useSensors } from '@dnd-kit/core'
import { SortableContext, arrayMove, horizontalListSortingStrategy,
sortableKeyboardCoordinates } from '@dnd-kit/sortable'
import { useRef, useState } from 'react'
import { createPortal } from 'react-dom'
import { useClientMotion } from '@/lib/useClientMotion'
import type { ReactNode } from 'react'
const SortableColumns = ({ ids, fixed = [], name, onChange, children }: {
ids: string[]; fixed?: string[]; name: (id: string) => { text: string; language: string }
onChange: (ids: string[]) => void; children: (ids: string[]) => ReactNode },
) => {
const [preview, setPreview] = useState<string[] | null> (null)
const [active, setActive] = useState<string | null> (null)
const latest = useRef<string[] | null> (null)
const lastOver = useRef<string | null> (null)
const motion = useClientMotion ()
const sensors = useSensors (
useSensor (MouseSensor, { activationConstraint: { distance: 4 } }),
useSensor (TouchSensor, { activationConstraint: { delay: 350, tolerance: 8 } }),
useSensor (KeyboardSensor, { coordinateGetter: sortableKeyboardCoordinates }))
const move = (current: string[], from: string, to: string) => {
if (fixed.includes (from) || fixed.includes (to)) return current
const source = current.indexOf (from)
const target = current.indexOf (to)
return source < 0 || target < 0 ? current : arrayMove (current, source, target)
}
const reset = () => {
latest.current = null
lastOver.current = null
setPreview (null)
setActive (null)
}
return <DndContext sensors={sensors} collisionDetection={closestCenter}
accessibility={{ announcements: {
onDragStart: event => name (String (event.active.id)).text,
onDragOver: event => event.over == null ? undefined : name (String (event.over.id)).text,
onDragMove: () => undefined, onDragEnd: () => undefined,
onDragCancel: () => undefined },
screenReaderInstructions: { draggable: active == null ? '' : name (active).text } }}
onDragStart={event => {
latest.current = ids
setPreview (ids)
setActive (String (event.active.id))
}} onDragOver={event => {
if (event.over == null || event.active.id === event.over.id) return
const target = String (event.over.id)
if (target === lastOver.current) return
latest.current = move (latest.current ?? ids, String (event.active.id), target)
lastOver.current = target
setPreview (latest.current)
}} onDragCancel={reset} onDragEnd={event => {
if (event.over != null)
{
const target = String (event.over.id)
const current = latest.current ?? ids
onChange (target === lastOver.current || target === String (event.active.id)
? current : move (current, String (event.active.id), target))
}
reset ()
}}>
<SortableContext items={preview ?? ids} strategy={horizontalListSortingStrategy}>
{children (preview ?? ids)}
</SortableContext>
{typeof document !== 'undefined' && createPortal (<DragOverlay
dropAnimation={motion.reduced ? null : {
duration: motion.duration ('drop'), easing: motion.easing }}>
{active != null && <div lang={name (active).language}
className="rounded border border-blue-500 bg-white p-2 text-slate-900 shadow-lg
dark:border-blue-400 dark:bg-slate-800 dark:text-slate-100">{name (active).text}</div>}
</DragOverlay>, document.body)}
</DndContext>
}
export default SortableColumns
+58
ファイルの表示
@@ -0,0 +1,58 @@
import { defaultAnimateLayoutChanges, useSortable } from '@dnd-kit/sortable'
import { CSS } from '@dnd-kit/utilities'
import { useClientMotion } from '@/lib/useClientMotion'
import { cn } from '@/lib/utils'
import type { CSSProperties, ReactNode } from 'react'
type Name = { text: string; language: string }
const SortableItem = ({ id, name, fixed, children, moveGroup, handleId, renderShell }: {
id: string; name: Name; fixed?: boolean; children: ReactNode
moveGroup: (direction: number) => void; handleId: string
renderShell?: (shell: { ref: (node: HTMLElement | null) => void
style: CSSProperties; handle: ReactNode; dragging: boolean }) => ReactNode },
) => {
const motion = useClientMotion ()
const item = useSortable ({ id, disabled: fixed,
transition: { duration: motion.duration ('reorder'), easing: motion.easing },
animateLayoutChanges: options => !motion.reduced
&& defaultAnimateLayoutChanges ({ ...options, wasDragging: true }) })
const handle = fixed ? <span className="w-6 shrink-0"/> : <button type="button"
id={handleId} ref={item.setActivatorNodeRef} {...item.attributes} {...item.listeners}
aria-label={name.text} lang={name.language}
aria-keyshortcuts={renderShell == null
? 'Space ArrowUp ArrowDown Enter Escape Alt+ArrowLeft Alt+ArrowRight'
: 'Space ArrowLeft ArrowRight Enter Escape'}
className="mt-1 w-6 shrink-0 cursor-grab touch-none rounded
focus-visible:outline-none focus-visible:ring-2 focus-visible:ring-blue-500
active:cursor-grabbing dark:focus-visible:ring-blue-400"
onClick={event => event.stopPropagation ()}
onKeyDown={event => {
if (event.altKey && !item.isDragging
&& ['ArrowLeft', 'ArrowRight'].includes (event.key))
{
event.preventDefault ()
event.stopPropagation ()
moveGroup (event.key === 'ArrowLeft' ? -1 : 1)
}
else item.listeners?.onKeyDown?.(event)
}}><span aria-hidden>⠿</span></button>
const style = { transform: CSS.Transform.toString (item.transform), transition: item.transition }
if (renderShell != null)
return renderShell ({ ref: item.setNodeRef, style, handle, dragging: item.isDragging })
return <div ref={item.setNodeRef} style={style}
className={cn ('relative max-w-full rounded border border-slate-300 bg-white',
'text-slate-900 dark:border-slate-600 dark:bg-slate-900 dark:text-slate-100',
item.isDragging && 'opacity-30 ring-2 ring-blue-500 dark:ring-blue-400')}>
<div className="flex min-w-0 items-start gap-1 px-1">
{handle}
<div className="min-w-0 flex-1 py-1">{children}</div>
</div>
</div>
}
export default SortableItem
+136
ファイルの表示
@@ -0,0 +1,136 @@
import { DndContext, DragOverlay, KeyboardSensor, MouseSensor, TouchSensor,
closestCenter, useDroppable, useSensor, useSensors } from '@dnd-kit/core'
import { SortableContext, arrayMove, rectSortingStrategy,
sortableKeyboardCoordinates } from '@dnd-kit/sortable'
import { useId, useRef, useState } from 'react'
import { createPortal } from 'react-dom'
import SortableItem from '@/components/common/SortableItem'
import { useClientMotion } from '@/lib/useClientMotion'
import { cn } from '@/lib/utils'
import type { ReactNode } from 'react'
type Groups = Record<string, string[]>
type Name = { text: string; language: string }
const Group = ({ id, children }: { id: string; children: ReactNode }) => {
const drop = useDroppable ({ id: `group:${id}` })
return <div ref={drop.setNodeRef} className={cn (
'flex min-h-10 flex-wrap items-start gap-1 rounded p-1',
drop.isOver && 'ring-1 ring-blue-500 dark:ring-blue-400')}>{children}</div>
}
const SortableList = ({ groups, name, renderItem, renderGroup, onChange, fixed = [] }: {
groups: Groups; name: (id: string) => Name
renderItem: (id: string, group: string) => ReactNode
renderGroup: (id: string, content: ReactNode) => ReactNode
onChange: (groups: Groups) => void; fixed?: string[] },
) => {
const [preview, setPreview] = useState<Groups | null> (null)
const previewRef = useRef<Groups | null> (null)
const lastOver = useRef<string | null> (null)
const scope = useId ()
const [active, setActive] = useState<string | null> (null)
const motion = useClientMotion ()
const current = preview ?? groups
const sensors = useSensors (
useSensor (MouseSensor, { activationConstraint: { distance: 4 } }),
useSensor (TouchSensor, { activationConstraint: { delay: 350, tolerance: 8 } }),
useSensor (KeyboardSensor, { coordinateGetter: sortableKeyboardCoordinates }))
const owner = (id: string, value: Groups) => Object.keys (value)
.find (group => value[group].includes (id))
const relocate = (value: Groups, id: string, over: string): Groups => {
const source = owner (id, value)
const target = over.startsWith ('group:') ? over.slice (6) : owner (over, value)
if (source == null || target == null || fixed.includes (id) || fixed.includes (over))
return value
if (source === target)
{
const from = value[source].indexOf (id)
const to = over.startsWith ('group:')
? value[target].length - 1 : value[target].indexOf (over)
if (from === to || to < 0) return value
return { ...value, [source]: arrayMove (value[source], from, to) }
}
const destination = value[target].filter (item => item !== id)
const position = destination.indexOf (over)
destination.splice (position < 0 ? destination.length : position, 0, id)
return { ...value, [source]: value[source].filter (item => item !== id),
[target]: destination }
}
const moveGroup = (id: string, direction: number) => {
const keys = Object.keys (groups)
const group = owner (id, groups)
const target = keys[keys.indexOf (group ?? '') + direction]
if (target != null)
{
onChange (relocate (groups, id, `group:${target}`))
requestAnimationFrame (() => document.getElementById (`${scope}-${id}`)?.focus ())
}
}
const reset = () => {
previewRef.current = null
lastOver.current = null
setPreview (null)
setActive (null)
}
return <DndContext sensors={sensors} collisionDetection={closestCenter}
accessibility={{ announcements: {
onDragStart: event => name (String (event.active.id)).text,
onDragOver: event => event.over == null ? undefined : (
event.over.id.toString ().startsWith ('group:') ? undefined
: name (String (event.over.id)).text),
onDragMove: () => undefined, onDragEnd: () => undefined,
onDragCancel: () => undefined },
screenReaderInstructions: { draggable: active == null ? '' : name (active).text } }}
onDragStart={event => {
previewRef.current = groups
lastOver.current = null
setPreview (groups)
setActive (String (event.active.id))
}} onDragOver={event => {
if (event.over == null || event.active.id === event.over.id) return
const target = String (event.over.id)
if (lastOver.current === target) return
const next = relocate (previewRef.current ?? groups, String (event.active.id), target)
previewRef.current = next
lastOver.current = target
setPreview (next)
}} onDragCancel={reset} onDragEnd={event => {
if (event.over != null)
{
const id = String (event.active.id)
const target = String (event.over.id)
const latest = previewRef.current ?? groups
const next = (target === id || target === lastOver.current)
? latest : relocate (latest, id, target)
onChange (next)
}
reset ()
}}>
{Object.entries (current).map (([group, ids]) => <div key={group}>
{renderGroup (group, <SortableContext items={ids} strategy={rectSortingStrategy}>
<Group id={group}>{ids.map (id => <SortableItem key={id} id={id}
name={name (id)} fixed={fixed.includes (id)} handleId={`${scope}-${id}`}
moveGroup={direction => moveGroup (id, direction)}>
{renderItem (id, group)}
</SortableItem>)}</Group>
</SortableContext>)}
</div>)}
{typeof document !== 'undefined' && createPortal (
<DragOverlay dropAnimation={motion.reduced ? null : {
duration: motion.duration ('drop'), easing: motion.easing }}>
{active == null ? null : <div lang={name (active).language}
className="rounded border border-blue-500 bg-white px-3 py-2 text-sm
text-slate-900 shadow-lg dark:border-blue-400 dark:bg-slate-800 dark:text-slate-100">
{name (active).text}</div>}
</DragOverlay>, document.body)}
</DndContext>
}
export default SortableList
+7 -4
ファイルの表示
@@ -12,6 +12,7 @@ import type { Tag } from '@/types'
type Props = { type Props = {
includeNico?: boolean includeNico?: boolean
includeUnused?: boolean
describedBy?: string describedBy?: string
invalid?: boolean invalid?: boolean
value: string value: string
@@ -20,6 +21,7 @@ type Props = {
const TagInput: FC<Props> = ({ const TagInput: FC<Props> = ({
includeNico = false, includeNico = false,
includeUnused = false,
describedBy, describedBy,
invalid, invalid,
value, value,
@@ -57,11 +59,11 @@ const TagInput: FC<Props> = ({
{ {
const tags = await apiGet<Tag[]> ('/tags/autocomplete', { const tags = await apiGet<Tag[]> ('/tags/autocomplete', {
locale, signal: controller.signal, locale, signal: controller.signal,
params: { q: request.query, nico: includeNico ? '1' : '0' } }) params: { q: request.query.replace (/^not:/i, ''), nico: includeNico ? '1' : '0' } })
if (active) if (active)
{ {
setResult ({ locale, includeNico, request, setResult ({ locale, includeNico, request,
tags: tags.filter (tag => tag.postCount > 0) }) tags: includeUnused ? tags : tags.filter (tag => tag.postCount > 0) })
} }
} }
catch catch
@@ -74,7 +76,7 @@ const TagInput: FC<Props> = ({
active = false active = false
controller.abort () controller.abort ()
} }
}, [request, locale, includeNico]) }, [request, locale, includeNico, includeUnused])
useEffect (() => { useEffect (() => {
setActiveIndex (-1) setActiveIndex (-1)
@@ -85,7 +87,8 @@ const TagInput: FC<Props> = ({
// TODO: TagSearch からのコピペのため,共通化を考へる. // TODO: TagSearch からのコピペのため,共通化を考へる.
const handleTagSelect = (tag: Tag) => { const handleTagSelect = (tag: Tag) => {
const parts = value?.split (' ') const parts = value?.split (' ')
parts[parts.length - 1] = tag.name const negative = /^not:/i.test (parts[parts.length - 1])
parts[parts.length - 1] = `${negative ? 'not:' : ''}${tag.name}`
setValue (parts.join (' ') + ' ') setValue (parts.join (' ') + ' ')
setRequest ({ query: '' }) setRequest ({ query: '' })
setActiveIndex (-1) setActiveIndex (-1)
+128
ファイルの表示
@@ -0,0 +1,128 @@
import { useEffect, useState } from 'react'
import DateTimeField from '@/components/common/DateTimeField'
import { Button } from '@/components/ui/button'
import { countsEventTypes, countsFieldLabel, countsFilterFields,
useCountsDefinition }
from '@/lib/countsDefinitions'
import useDialogue from '@/lib/dialogues/useDialogue'
import { inputClass } from '@/lib/utils'
import { useUITranslation } from '@/lib/useUITranslation'
import { readCountState, encodeCountState } from '@/lib/queryState'
import type { DialogueFormControls } from '@/lib/dialogues/useDialogue'
import type { CountsCondition, CountsDefinition } from '@/lib/countsDefinitions'
const ConditionEditor = (
{ definition, initial, controls, save }: {
definition: CountsDefinition; initial?: CountsCondition
controls: DialogueFormControls
save: (condition: CountsCondition) => void },
) => {
const { label, message } = useUITranslation ('counts')
const fields = countsFilterFields (definition)
const [condition, setCondition] = useState<CountsCondition> (() => {
const next = { ...initial, field: initial?.field ?? 'resource_id',
operator: initial?.operator ?? 'eq' }
if (next.operator !== 'range') delete next.end
return next
})
const { field, operator } = condition
const operators = definition.fields.find (item => item.key === field)?.operators ?? []
useEffect (() => {
controls.setActions ([{ label: label (initial == null ? 'add' : 'shared:actions.edit'),
onSelect: () => save (condition) }])
}, [controls, condition, initial, label, save])
return <div className="space-y-2">
<select value={field} aria-label={message ('field').text} className={inputClass ()}
onChange={event => {
const field = event.target.value
const operator = definition.fields.find (item => item.key === field)?.operators[0]
if (operator != null) setCondition ({ field, operator })
}}>
{Object.keys (fields).filter (key => fields[key] !== 'json').map (key => (
<option key={key} value={key}>{message (countsFieldLabel (key)).text}</option>))}
</select>
<select value={operator} aria-label={message ('operator').text} className={inputClass ()}
onChange={event => setCondition (previous => {
const operator = event.target.value
const next = { ...previous, operator }
if (operator !== 'range') delete next.end
if ((operator === 'like') !== (previous.operator === 'like')) delete next.value
return next
})}>
{operators.map (key => <option key={key} value={key}>
{message (`operators.${key}`).text}</option>)}
</select>
</div>
}
const CountsAdditionalConditions = ({ params, update }: {
params: URLSearchParams
update: (key: string, value: string) => void },
) => {
const { label, message } = useUITranslation ('counts')
const definition = useCountsDefinition ()
const dialogue = useDialogue ()
const conditions = readCountState<CountsCondition[]> (params, 'conditions', [])
const write = (next: CountsCondition[]) => update ('conditions', encodeCountState ('conditions', next))
const edit = (index?: number) => {
const initial = index == null ? undefined : conditions[index]
const save = (condition: CountsCondition) => {
write (index == null ? [...conditions, condition]
: conditions.map ((item, position) => position === index ? condition : item))
}
dialogue.form ({ title: label ('additionalConditions'), footerLayout: 'inline',
body: controls => <ConditionEditor definition={definition}
initial={initial} controls={controls} save={save}/> })
}
const patch = (index: number, change: Partial<CountsCondition>) => write (
conditions.map ((item, position) => position === index ? { ...item, ...change } : item))
return <fieldset className="space-y-1">
<legend className="flex items-center gap-2">{label ('additionalConditions')}
<Button type="button" variant="outline" size="sm" onClick={() => edit ()}>
{label ('add')}</Button></legend>
{conditions.map ((condition, index) => {
const datetime = countsFilterFields (definition)[condition.field] === 'datetime'
return <div key={index} className="flex flex-wrap items-center gap-1">
<button type="button" onClick={() => edit (index)}>
{label (countsFieldLabel (condition.field))}
{' · '}{label (`operators.${condition.operator}`)}
</button>
{(condition.operator === 'range' ? ['value', 'end'] as const : ['value'] as const)
.map (key => condition.field === 'event_type' ? (
<select key={key} value={condition[key] ?? ''} disabled={condition.blank}
className={inputClass (false, 'w-36 py-1')}
aria-label={message ('value').text}
onChange={event => patch (index, { [key]: event.target.value })}>
<option value=""/>
{condition[key] && !countsEventTypes.some (event => event === condition[key])
&& <option value={condition[key]}>{condition[key]}</option>}
{countsEventTypes.map (event => <option key={event} value={event}
lang={message (`events.${event}`).language}>
{message (`events.${event}`).text}</option>)}
</select>) : (datetime ? <DateTimeField precision="second" key={key}
value={condition[key] ?? ''} disabled={condition.blank}
aria-label={message (key === 'end' ? 'through' : 'value').text}
onChange={value => patch (index, { [key]: value ?? '' })}/> : (
<input key={key} value={condition[key] ?? ''} disabled={condition.blank}
className={inputClass (false, 'w-36 py-1')}
aria-label={message ('value').text}
onChange={event => patch (index, { [key]: event.target.value })}/>)))}
<label className="inline-flex items-center gap-1">
<input type="checkbox" checked={condition.blank ?? false}
onChange={event => patch (index, { blank: event.target.checked })}/>
{label ('blank')}</label>
<button type="button" aria-label={message ('remove').text}
onClick={() => write (conditions.filter (
(_, position) => index !== position))}>×</button>
</div>
})}
</fieldset>
}
export default CountsAdditionalConditions
+95
ファイルの表示
@@ -0,0 +1,95 @@
import LocalisedText from '@/components/LocalisedText'
import SortableList from '@/components/common/SortableList'
import { countsAxisFields, countsAxisLabel, countsFilterFields,
useCountsDefinition }
from '@/lib/countsDefinitions'
import { useClientMotion } from '@/lib/useClientMotion'
import { cn, inputClass } from '@/lib/utils'
import { useLocale } from '@/lib/useLocale'
import { useUITranslation } from '@/lib/useUITranslation'
import { readCountState, encodeCountState } from '@/lib/queryState'
import type { CountsAxis } from '@/lib/countsDefinitions'
const CountsAxes = ({ params, update }: {
params: URLSearchParams; update: (key: string, value: string) => void },
) => {
const { label, message } = useUITranslation ('counts')
const definition = useCountsDefinition ()
const locale = useLocale ()
const motion = useClientMotion ()
const available = countsAxisFields (definition, params.get ('definition') === 'history')
const rows = readCountState<CountsAxis[]> (params, 'rows', [])
const cols = readCountState<CountsAxis[]> (params, 'cols', [])
const axes = [...rows, ...cols]
const groups = {
available: available.map (field => field.key).filter (id => !axes.some (
axis => axis.field === id)),
rows: rows.map (axis => axis.field), cols: cols.map (axis => axis.field) }
const name = (id: string) => {
const axis = axes.find (item => item.field === id)
return axis == null ? message (available.find (field => field.key === id)?.label ?? 'axis')
: countsAxisLabel (axis, definition, locale)
}
const commit = (next: Record<string, string[]>) => {
for (const group of ['rows', 'cols'] as const)
update (group, encodeCountState (group, next[group].map (field => axes.find (
axis => axis.field === field) ?? { field,
...(countsFilterFields (definition)[field] === 'datetime' ? { grain: 'month' } : { }) })))
}
const place = (id: string, target?: 'rows' | 'cols') => {
const next = Object.fromEntries (Object.entries (groups).map (([key, ids]) => [
key, ids.filter (value => value !== id)]))
if (target != null) next[target].push (id)
else next.available.push (id)
commit (next)
}
return <SortableList groups={groups} name={name} onChange={commit}
renderGroup={(id, content) => <fieldset className="mb-2 min-w-0">
<legend>{label (id === 'available' ? 'axisItems' : id)}</legend>
<div className="rounded border dark:border-slate-700">{content}</div>
</fieldset>} renderItem={(id, group) => {
const axis = axes.find (axis => axis.field === id)
return <div className={cn ('flex items-center gap-2 text-sm',
group !== 'available' && 'flex-wrap')}>
<span className="min-w-0 flex-1 break-words">
<LocalisedText language={name (id).language}>{name (id).text}</LocalisedText>
</span>
{group === 'available' && <span className="ml-auto inline-flex shrink-0 gap-1">
{(['rows', 'cols'] as const).map (target => <button key={target} type="button"
className="h-7 w-7 rounded-full border border-slate-300 bg-slate-100
text-xs text-slate-700 hover:bg-blue-100 hover:text-blue-900
focus-visible:outline-none focus-visible:ring-2 focus-visible:ring-blue-500
dark:border-slate-600 dark:bg-slate-800 dark:text-slate-100
dark:hover:bg-blue-900 dark:hover:text-blue-100 dark:focus-visible:ring-blue-400"
style={{ transition: `background-color ${motion.duration ('feedback')}ms ${
motion.easing}, color ${motion.duration ('feedback')}ms ${motion.easing}` }}
aria-label={message ('axisPlace', {
field: name (id).text, target: message (target).text }).text}
lang={message ('axisPlace').language} onClick={() => place (id, target)}>
{label (target)}</button>)}
</span>}
{group !== 'available' && <>
{countsFilterFields (definition)[id] === 'datetime' && <select
className={inputClass (false, 'w-auto py-0')}
value={axis?.grain ?? 'month'} aria-label={message ('grain').text}
onChange={event => update (group, encodeCountState (group,
(group === 'rows' ? rows : cols).map (item => item.field === id
? { ...item, grain: event.target.value } : item)))}>
{definition.grains.map (grain => <option key={grain} value={grain}
lang={message (`grains.${grain}`).language}>
{message (`grains.${grain}`).text}</option>)}
</select>}
<button type="button" aria-label={message ('remove').text}
lang={message ('remove').language} className="rounded px-1
focus-visible:outline-none focus-visible:ring-2 focus-visible:ring-blue-500
dark:focus-visible:ring-blue-400" onClick={() => place (id)}>×</button>
</>}
</div>
}}/>
}
export default CountsAxes
+88
ファイルの表示
@@ -0,0 +1,88 @@
import { useEffect, useLayoutEffect, useRef } from 'react'
import { createPortal } from 'react-dom'
import { useUITranslation } from '@/lib/useUITranslation'
export type CountsMenuAction = 'copy' | 'copyHeaders' | 'row' | 'column' | 'all'
const CountsCellMenu = (
{ x, y, act, close }: {
x: number
y: number
act: (action: CountsMenuAction) => void
close: () => void },
) => {
const root = useRef<HTMLDivElement> (null)
const { label, message } = useUITranslation ('counts')
useLayoutEffect (() => {
const menu = root.current
if (menu == null) return
const rect = menu.getBoundingClientRect ()
menu.style.left = `${Math.max (4, Math.min (x, window.innerWidth - rect.width - 4))}px`
menu.style.top = `${Math.max (4, Math.min (y, window.innerHeight - rect.height - 4))}px`
menu.querySelector ('button')?.focus ({ preventScroll: true })
}, [x, y])
useEffect (() => {
const outside = (event: Event) => {
if (!root.current?.contains (event.target as Node)) close ()
}
const escape = (event: KeyboardEvent) => {
if (event.key === 'Escape')
{
event.preventDefault ()
event.stopPropagation ()
close ()
}
}
document.addEventListener ('pointerdown', outside, true)
document.addEventListener ('keydown', escape, true)
window.addEventListener ('resize', close)
document.addEventListener ('scroll', outside, true)
return () => {
document.removeEventListener ('pointerdown', outside, true)
document.removeEventListener ('keydown', escape, true)
window.removeEventListener ('resize', close)
document.removeEventListener ('scroll', outside, true)
}
}, [close])
const actions: CountsMenuAction[] = ['copy', 'copyHeaders', 'row', 'column', 'all']
return createPortal (
<div ref={root} role="menu" aria-label={message ('selection.table').text}
className="fixed z-50 min-w-40 rounded border border-slate-400 bg-white
py-1 text-sm text-slate-900 shadow-lg dark:border-slate-600
dark:bg-slate-800 dark:text-slate-100"
style={{ left: x, top: y }} onContextMenu={event => event.preventDefault ()}
onPointerDown={event => event.stopPropagation ()}
onKeyDown={event => {
event.stopPropagation ()
if ((event.ctrlKey || event.metaKey) && event.key.toLowerCase () === 'c')
{
event.preventDefault ()
act ('copy')
return
}
const buttons = Array.from (root.current?.querySelectorAll ('button') ?? [])
const current = buttons.indexOf (document.activeElement as HTMLButtonElement)
if (['ArrowDown', 'ArrowUp', 'Home', 'End'].includes (event.key))
{
event.preventDefault ()
const next = event.key === 'Home' ? 0 : (event.key === 'End' ? buttons.length - 1
: (current + (event.key === 'ArrowDown' ? 1 : -1) + buttons.length)
% buttons.length)
buttons[next]?.focus ()
}
if (event.key === 'Tab') close ()
}}>
{actions.map (action => (
<button key={action} type="button" role="menuitem"
className="block w-full px-3 py-1 text-left hover:bg-slate-200
hover:text-slate-900 focus:bg-slate-200 focus:text-slate-900
dark:hover:bg-slate-700 dark:hover:text-slate-100
dark:focus:bg-slate-700 dark:focus:text-slate-100"
onClick={() => act (action)}>{label (`selection.${action}`)}</button>))}
</div>, document.body)
}
export default CountsCellMenu
+211
ファイルの表示
@@ -0,0 +1,211 @@
import { useEffect, useId, useRef, useState } from 'react'
import CountsAdditionalConditions from '@/components/counts/CountsAdditionalConditions'
import CountsAxes from '@/components/counts/CountsAxes'
import CountsFieldTree from '@/components/counts/CountsFieldTree'
import AnimatedCollapse from '@/components/common/AnimatedCollapse'
import DateTimeField from '@/components/common/DateTimeField'
import FormField from '@/components/common/FormField'
import HelpPopover from '@/components/common/HelpPopover'
import TagInput from '@/components/common/TagInput'
import { Button } from '@/components/ui/button'
import { countsUseLatest, fieldsForCounts, normaliseCountsParams, readCountSort,
selectedCountChildren, selectedCountColumns } from '@/lib/counts'
import { countsNestedIsScalar } from '@/lib/countsDisplay'
import { useCountsDefinition } from '@/lib/countsDefinitions'
import { useClientMotion } from '@/lib/useClientMotion'
import { inputClass } from '@/lib/utils'
import { useUITranslation } from '@/lib/useUITranslation'
import { readCountState, setCountState } from '@/lib/queryState'
import type { ReactNode } from 'react'
import type { CountsResource } from '@/lib/counts'
const CountsControls = (
{ resource, applied, open, setOpen, execute, displayJSON, displayRowNumbers,
displayThumbnail, reorder, actions }: {
resource: CountsResource; applied: URLSearchParams; open: boolean
setOpen: (open: boolean) => void
execute: (params: URLSearchParams) => void
displayJSON: (enabled: boolean) => void
displayRowNumbers: (enabled: boolean) => void
displayThumbnail: (enabled: boolean) => void
reorder: (value: string) => void; actions?: ReactNode },
) => {
const { label, message } = useUITranslation ('counts')
const definition = useCountsDefinition ()
const id = useId ()
const motion = useClientMotion ()
const [draft, setDraft] = useState (() => new URLSearchParams (applied))
const order = applied.get ('order')
const nestedJSON = applied.get ('nested_json')
const rowNumbers = applied.get ('row_numbers')
const thumbnail = selectedCountColumns (resource, applied, definition).includes ('thumbnail')
const previousThumbnail = useRef (thumbnail)
const mode = draft.get ('mode') ?? 'summary'
const columns = selectedCountColumns (resource, draft, definition)
const details = mode === 'details'
const hasNested = details && fieldsForCounts (resource, definition).some (field =>
columns.includes (field.key) && field.children != null
&& !countsNestedIsScalar (field.children, field.key, draft)
&& field.children.some (child => selectedCountChildren (draft)
.has (`${field.key}.${child.key}`)))
// Preserve absent columns until a real presentation change or an explicit edit.
useEffect (() => {
if (previousThumbnail.current === thumbnail) return
previousThumbnail.current = thumbnail
setDraft (previous => {
const next = new URLSearchParams (previous)
const columns = selectedCountColumns (resource, previous, definition)
.filter (column => column !== 'thumbnail')
next.set ('columns', (thumbnail ? ['thumbnail', ...columns] : columns).join (' '))
return next
})
}, [thumbnail, resource, definition])
useEffect (() => {
setDraft (previous => {
const next = new URLSearchParams (previous)
const display = { order, nested_json: nestedJSON, row_numbers: rowNumbers }
for (const [key, value] of Object.entries (display))
{
if (value == null) next.delete (key)
else next.set (key, value)
}
return next
})
}, [order, nestedJSON, rowNumbers])
const history = draft.get ('definition') === 'history'
const update = (key: string, value: string) => setDraft (previous => {
const next = new URLSearchParams (previous)
if (['rows', 'cols'].includes (key) && next.get ('sort') !== '') next.delete ('sort')
setCountState (next, key, value)
if (key !== 'mode') return next
const resolved = normaliseCountsParams (resource, next, definition)
const axes = [...readCountState<unknown[]> (resolved, 'rows', []),
...readCountState<unknown[]> (resolved, 'cols', [])]
const allowed = value === 'summary'
? ['count', ...axes.map ((_, index) => `a${index}`)]
: fieldsForCounts (resource, definition).map (field => field.key)
if (readCountSort (resolved.get ('sort')).some (sort => !allowed.includes (sort.key)
&& !(value === 'summary' && /^m[0-9]{1,2}$/.test (sort.key))))
next.delete ('sort')
return normaliseCountsParams (resource, next, definition)
})
return <form className="shrink-0" onSubmit={event => {
event.preventDefault ()
const next = new URLSearchParams (draft)
next.delete ('page')
next.delete ('execution_at')
execute (normaliseCountsParams (resource, next, definition))
}}>
{!history && <div className="flex flex-wrap items-center gap-2 border-b pb-1
dark:border-slate-700">
<div className="flex flex-wrap items-center gap-1 text-sm">
{label ('referenceAt')}<HelpPopover label={message ('referenceAt')}>
{label ('referenceHelp')}</HelpPopover>
<div className="flex min-w-0 max-w-full items-center gap-1">
<DateTimeField precision="second" disabled={countsUseLatest (draft)}
className="min-w-0 flex-1"
value={draft.get ('at') ?? ''}
aria-label={message ('referenceAt').text} lang={message ('referenceAt').language}
onChange={value => update ('at', value ?? '')}/>
<label className="inline-flex shrink-0 items-center gap-1">
<input type="checkbox" checked={countsUseLatest (draft)}
onChange={event => update ('current', event.target.checked ? '1' : '0')}/>
{label ('latest')}</label>
</div>
</div>
</div>}
<div className="flex flex-wrap items-center gap-2 border-b py-1 dark:border-slate-700">
<Button type="submit" size="sm">{label ('run')}</Button>
<Button type="button" size="sm" variant="outline" aria-expanded={open}
aria-controls={id} onClick={() => setOpen (!open)}>
{label ('conditions')}<span aria-hidden className="inline-block" style={{
transform: open ? 'rotate(90deg)' : 'rotate(0deg)',
transition: `transform ${motion.duration ('collapse')}ms ${motion.easing}` }}>▸</span>
</Button>
<select className={inputClass (false, 'w-auto py-1')}
aria-label={message ('mode').text} lang={message ('mode').language}
value={mode} onChange={event => update ('mode', event.target.value)}>
{['summary', 'details'].map (value => <option key={value} value={value}>
{message (value).text}</option>)}
</select>
{actions}
</div>
<fieldset className="flex flex-wrap items-center gap-x-3 gap-y-1 border-b py-1
dark:border-slate-700">
<legend className="sr-only">{label ('displaySettings')}</legend>
<label className="inline-flex items-center gap-1 text-sm">
<input type="checkbox" checked={draft.get ('row_numbers') === '1'}
onChange={event => {
update ('row_numbers', event.target.checked ? '1' : '0')
displayRowNumbers (event.target.checked)
}}/>{label ('rowNumbers')}
</label>
{details && resource === 'posts' && <label
className="inline-flex items-center gap-1 text-sm">
<input type="checkbox" checked={thumbnail}
onChange={event => displayThumbnail (event.target.checked)}/>
{label ('columns.thumbnailDisplay')}
</label>}
{hasNested && <label className="inline-flex items-center gap-1 text-sm">
<input type="checkbox" checked={draft.get ('nested_json') === '1'}
onChange={event => {
update ('nested_json', event.target.checked ? '1' : '0')
displayJSON (event.target.checked)
}}/>{label ('nested.asJSON')}
</label>}
</fieldset>
<AnimatedCollapse id={id} open={open}>
<div className="max-h-[35dvh] overflow-y-auto overscroll-contain
border-b py-2 dark:border-slate-700 md:max-h-[40dvh]">
<div className="grid gap-3 md:grid-cols-2">
<fieldset className="min-w-0" data-counts-output>
{mode === 'summary' ? <CountsAxes params={draft} update={update}/>
: <>
<legend className="mb-1 font-semibold">{label ('output')}</legend>
<CountsFieldTree resource={resource} params={draft} update={(key, value) => {
update (key, value)
if (key === 'order') reorder (value)
}}/>
</>}
</fieldset>
<fieldset className="min-w-0 space-y-2" data-counts-filters>
<legend className="font-semibold">{label ('filters')}</legend>
{history && <div className="flex min-w-0 max-w-full items-center gap-1">
<DateTimeField precision="second" value={draft.get ('from') ?? ''}
aria-label={message ('from').text} lang={message ('from').language}
className="min-w-0 flex-1" onChange={value => update ('from', value ?? '')}/>
<span className="shrink-0">{label ('periodSeparator')}</span>
<DateTimeField precision="second" value={draft.get ('through') ?? ''}
aria-label={message ('through').text} lang={message ('through').language}
className="min-w-0 flex-1" onChange={value => update ('through', value ?? '')}/>
</div>}
<div className="flex flex-wrap gap-2">
{['resource_id', 'user_id', 'limit'].map (key => <FormField key={key}
label={label (key === 'resource_id' ? 'columns.resourceId'
: (key === 'user_id' ? 'fields.actor' : 'fields.limit'))}>
{() => <input value={draft.get (key) ?? (key === 'limit' ? '20' : '')}
aria-label={message (key === 'resource_id' ? 'columns.resourceId'
: (key === 'user_id' ? 'fields.actor' : 'fields.limit')).text}
className={inputClass (false, 'w-28 py-1')}
onChange={event => update (key, event.target.value)}/>}
</FormField>)}
</div>
{resource === 'posts' && <FormField label={label ('columns.tags')}>
{() => <TagInput includeNico includeUnused value={draft.get ('tags') ?? ''}
setValue={value => update ('tags', value)}/>}
</FormField>}
<CountsAdditionalConditions params={draft} update={update}/>
</fieldset>
</div>
</div>
</AnimatedCollapse>
</form>
}
export default CountsControls
+130
ファイルの表示
@@ -0,0 +1,130 @@
import { useEffect, useRef, useState } from 'react'
import LocalisedText from '@/components/LocalisedText'
import { apiDelete, apiGet, apiPost, isApiError, startAPIDownload } from '@/lib/api'
import { useUITranslation } from '@/lib/useUITranslation'
import type { DialogueFormControls } from '@/lib/dialogues/useDialogue'
import type { LocaleCode } from '@/lib/locales'
export type CountsExportHandle = { cancel: () => Promise<boolean> }
type ExportStatus = {
status: 'starting' | 'queued' | 'running' | 'cancelling' | 'cancelled' | 'ready' | 'failed'
processed: number; total: number | null; token?: string }
const CountsExportProgress = (
{ payload, locale, handle, controls }: {
payload: Record<string, string>; locale: LocaleCode
handle: CountsExportHandle; controls: DialogueFormControls },
) => {
const { label, message } = useUITranslation ('counts')
const [state, setState] = useState<ExportStatus> ({
status: 'starting', processed: 0, total: null })
const [cancelError, setCancelError] = useState (false)
const terminal = useRef (false)
const cancelled = useRef (false)
const job = useRef<string | null> (null)
const creation = useRef<Promise<{ job: string }> | null> (null)
const downloaded = useRef (false)
useEffect (() => {
handle.cancel = async () => {
if (terminal.current) return true
cancelled.current = true
if (job.current != null)
{
try
{
setCancelError (false)
const result = await apiDelete<ExportStatus> (
`/counts/export/jobs/${encodeURIComponent (job.current)}`, { locale })
setState (result)
}
catch
{
setCancelError (true)
}
}
return false
}
}, [handle, locale])
useEffect (() => {
let active = true
let timer: ReturnType<typeof setTimeout> | undefined
const poll = async (token: string) => {
try
{
const response = await apiGet<ExportStatus> (
`/counts/export/jobs/${encodeURIComponent (token)}`, { locale })
if (!active) return
const result = (cancelled.current && response.status === 'ready')
? await apiDelete<ExportStatus> (
`/counts/export/jobs/${encodeURIComponent (token)}`, { locale }) : response
if (!active) return
setState (result)
terminal.current = ['ready', 'failed', 'cancelled'].includes (result.status)
if (result.status === 'ready' && result.token != null && !downloaded.current)
{
downloaded.current = true
startAPIDownload ('/counts/export', result.token, () => {
if (active) setState (previous => ({ ...previous, status: 'failed' }))
})
}
if (!terminal.current) timer = setTimeout (() => poll (token), 800)
}
catch (error)
{
if (active)
{
if (isApiError (error) && error.response?.status === 404)
{
terminal.current = true
setState (previous => ({ ...previous, status: 'failed' }))
return
}
setCancelError (true)
timer = setTimeout (() => poll (token), 2000)
}
}
}
creation.current ??= apiPost<{ job: string }> ('/counts/export/jobs', payload, { locale })
creation.current.then (async result => {
if (!active) return
job.current = result.job
if (cancelled.current) await handle.cancel ()
await poll (result.job)
}).catch (() => {
if (active)
{
terminal.current = true
setState (previous => ({ ...previous, status: 'failed' }))
}
})
return () => {
active = false
if (timer != null) clearTimeout (timer)
}
}, [handle, locale, payload])
useEffect (() => {
controls.setActions ([])
}, [controls])
const percent = state.total == null ? null : (state.total === 0 ? 100
: Math.min (100, Math.floor (state.processed / state.total * 100)))
const phase = message (`exportProgress.${state.status}`)
return <div className="space-y-3 py-2">
<p>{label ('exportProgress.format', {
format: message (`exportFormats.${payload.format}`).text })}</p>
<p role="status" aria-live="polite">
<LocalisedText language={phase.language}>{phase.text}</LocalisedText>
</p>
<progress className="h-3 w-full" max={100} value={percent ?? undefined}
aria-label={phase.text} lang={phase.language}/>
<p aria-live="polite">{percent == null
? label ('exportProgress.processed', { count: state.processed })
: label ('exportProgress.measured', {
processed: state.processed, total: state.total ?? 0, percent })}</p>
{cancelError && <p role="alert">{label ('common:fetchError')}</p>}
</div>
}
export default CountsExportProgress
+138
ファイルの表示
@@ -0,0 +1,138 @@
import { useId, useState } from 'react'
import AnimatedCollapse from '@/components/common/AnimatedCollapse'
import SortableList from '@/components/common/SortableList'
import { fieldsForCounts, orderedCountFields,
selectedCountChildren, selectedCountColumns, selectedCountFields } from '@/lib/counts'
import { useCountsDefinition } from '@/lib/countsDefinitions'
import { useClientMotion } from '@/lib/useClientMotion'
import { useUITranslation } from '@/lib/useUITranslation'
import type { CountField, CountsResource } from '@/lib/counts'
type BranchProps = {
field: CountField; path: string; enabled: boolean
selected: Set<string>; params: URLSearchParams
select: (path: string, checked: boolean) => void
update: (key: string, value: string) => void }
const FieldBranch = ({ field, path, enabled, selected, params, select, update }: BranchProps) => {
const { label, message } = useUITranslation ('counts')
const [open, setOpen] = useState (false)
const id = useId ()
const motion = useClientMotion ()
const checked = selected.has (path)
return <>
<div className="flex items-center gap-1">
<label className="inline-flex items-center gap-1">
<input type="checkbox" checked={checked} disabled={!enabled || (field.children != null
&& !field.children.some (child => selected.has (`${path}.${child.key}`)))}
onChange={event => select (path, event.target.checked)}/>
{label (field.label)}
</label>
{field.children != null && <button type="button" aria-expanded={open}
aria-controls={id} aria-label={message (field.label).text}
lang={message (field.label).language} onClick={() => setOpen (!open)}
className="rounded px-1 focus-visible:outline-none focus-visible:ring-2
focus-visible:ring-blue-500 dark:focus-visible:ring-blue-400">
<span aria-hidden className="inline-block" style={{
transform: open ? 'rotate(90deg)' : 'rotate(0deg)',
transition: `transform ${motion.duration ('collapse')}ms ${motion.easing}` }}>▸</span>
</button>}
</div>
{field.children != null && <AnimatedCollapse open={open} id={id}>
<div className="ml-2 border-l pl-1 dark:border-slate-700">
<FieldGroup fields={field.children} path={path} params={params}
enabled={enabled} selected={selected} select={select} update={update}/>
</div>
</AnimatedCollapse>}
</>
}
const FieldGroup = ({ fields, path, params, enabled, selected, select, update }: {
fields: CountField[]; path: string; params: URLSearchParams
enabled: boolean; selected: Set<string>; select: BranchProps['select']
update: BranchProps['update'] },
) => {
const { message } = useUITranslation ('counts')
const ordered = orderedCountFields (fields, params, path)
const prefix = path ? `${path}.` : ''
return <SortableList groups={{ items: ordered.map (field => field.key) }}
fixed={path === '' ? ['thumbnail'] : []}
name={key => message ('reorder', {
column: message (fields.find (field => field.key === key)!.label).text })}
renderGroup={(_, content) => content}
onChange={groups => {
const current = fields.map (field => `${prefix}${field.key}`)
const other = (params.get ('order') ?? '').split (' ')
.filter (key => key && !current.includes (key))
update ('order', [...other, ...groups.items.map (key => `${prefix}${key}`)].join (' '))
}} renderItem={key => {
const field = fields.find (field => field.key === key)!
return <FieldBranch field={field} path={`${prefix}${key}`} enabled={enabled}
selected={selected} params={params} select={select} update={update}/>
}}/>
}
const CountsFieldTree = (
{ resource, params, update, fields: suppliedFields,
selectionKey = 'columns', parent = '' }: {
resource: CountsResource
fields?: CountField[]
selectionKey?: string
parent?: string
params: URLSearchParams
update: (key: string, value: string) => void },
) => {
const definition = useCountsDefinition ()
const fields = suppliedFields ?? fieldsForCounts (resource, definition)
const roots = suppliedFields == null ? selectedCountColumns (resource, params, definition)
: selectedCountFields (fields, params, selectionKey, parent).map (field => field.key)
const selected = selectedCountChildren (params)
const rootPath = (key: string) => parent ? `${parent}.${key}` : key
for (const root of roots) selected.add (rootPath (root))
const select = (path: string, checked: boolean) => {
const root = fields.find (field => rootPath (field.key) === path)
if (root != null)
{
const next = new Set<string> (roots)
if (checked) next.add (root.key)
else next.delete (root.key)
update (selectionKey, fields.filter (field => next.has (field.key))
.map (field => field.key).join (' '))
return
}
if (checked)
{
const parts = path.split ('.')
for (let index = 1; index < parts.length; ++index)
selected.add (parts.slice (0, index + 1).join ('.'))
}
else selected.delete (path)
const nextParams = new URLSearchParams (params)
nextParams.set ('children', [...selected].filter (key => key.includes ('.')).join (' '))
const children = selectedCountChildren (nextParams)
update ('children', [...children].join (' '))
const nextRoots = new Set (roots)
const ancestor = fields.find (field => path.startsWith (`${rootPath (field.key)}.`))
if (checked && ancestor != null) nextRoots.add (ancestor.key)
for (const field of fields)
if (field.children != null && !field.children.some (child =>
children.has (`${rootPath (field.key)}.${child.key}`))) nextRoots.delete (field.key)
update (selectionKey, fields.filter (field => nextRoots.has (field.key))
.map (field => field.key).join (' '))
}
const visibleFields = parent === ''
? fields.filter (field => field.key !== 'thumbnail') : fields
return (
<>
<FieldGroup fields={visibleFields} path={parent} params={params} enabled selected={selected}
select={select} update={update}/>
</>)
}
export default CountsFieldTree
+114
ファイルの表示
@@ -0,0 +1,114 @@
import LocalisedText from '@/components/LocalisedText'
import PrefetchLink from '@/components/PrefetchLink'
import CountsSortHeader from '@/components/counts/CountsSortHeader'
import CountsSelectableTable from '@/components/counts/CountsSelectableTable'
import { changeCountSort, readCountSort, selectedCountChildren,
writeCountSort, orderedCountFields } from '@/lib/counts'
import { countsLogicalName, countsNestedClipboardTable, countsNestedExport,
countsNestedOrder, countsNestedText, countsNestedIsScalar, countsNestedScalarText }
from '@/lib/countsDisplay'
import { useLocale } from '@/lib/useLocale'
import { useUITranslation } from '@/lib/useUITranslation'
import type { CountField, CountRecord, CountsResource } from '@/lib/counts'
const CountsNestedTable = (
{ rows, fields, path, resource, title, params, update }: {
rows: CountRecord[]
fields: CountField[]
path: string
resource: CountsResource
title: { text: string; language: string }
params: URLSearchParams
update: (key: string, value: string) => void },
) => {
const { message } = useUITranslation ('counts')
const locale = useLocale ()
const selected = selectedCountChildren (params)
const columns = orderedCountFields (fields, params, path)
.filter (field => selected.has (`${path}.${field.key}`))
const allSorts = readCountSort (params.get ('child_sort'))
const sorts = allSorts.filter (sort => sort.key.startsWith (`${path}/`))
.map (sort => ({ ...sort, key: sort.key.slice (path.length + 1) }))
const ordered = countsNestedOrder (rows, path, params, locale)
const scalar = (row: CountRecord, field: string) => {
const value = row[field]
if (value == null) return null
if (field === 'tagId' || field === 'tagGroupId')
return <PrefetchLink to={field === 'tagId'
? `/tags/${encodeURIComponent (String (value))}` : '/wiki'}>
{String (value)}</PrefetchLink>
if (field === 'id')
{
if (row.kind === 'external') return String (value)
const target = path === 'parents' ? resource : 'tags'
return <PrefetchLink to={`/${target}/${encodeURIComponent (String (value))}`}>
{String (value)}</PrefetchLink>
}
const display = countsLogicalName (row, field, locale)
if (typeof value === 'boolean')
return <span aria-label={display.text} lang={display.language}>{value ? '✓' : '✗'}</span>
if (display.language != null)
return <LocalisedText language={display.language}>{display.text}</LocalisedText>
if (field === 'name' && typeof row.languageCode === 'string')
return <LocalisedText language={row.languageCode}>{String (value)}</LocalisedText>
return display.text
}
if (countsNestedIsScalar (fields, path, params))
return <span>{countsNestedScalarText (rows, fields, path, params, locale)}</span>
return (
<div className="counts-nested w-full min-w-0 max-w-full">
{params.get ('nested_json') === '1' ? (
<pre data-counts-scroll
aria-label={title.text} lang={title.language}
className="max-h-72 w-full overflow-auto whitespace-pre text-xs">
{JSON.stringify (countsNestedExport (rows, fields, path, params, locale))}
</pre>) : (columns.length > 0 && rows.length > 0 && (
<div data-counts-scroll
className="relative isolate max-h-72 w-full min-w-0 overflow-auto">
<CountsSelectableTable family={path} columns={columns.map (field => field.key)}
identity={`${path}:${params}`} name={title}
copyTable={() => countsNestedClipboardTable (rows, fields, path, params, locale)}
className="text-xs">
<thead className="sticky top-0 z-10 bg-background text-foreground
dark:bg-background dark:text-foreground"><tr>
{columns.map (field => (
<CountsSortHeader key={field.key} field={field.key}
label={message (field.label)}
sorts={sorts} structured={field.children != null}
change={(key, additive) => {
const next = changeCountSort (sorts, key, additive)
.map (sort => ({ ...sort, key: `${path}/${sort.key}` }))
update ('child_sort', writeCountSort ([
...allSorts.filter (sort => !sort.key.startsWith (`${path}/`)),
...next]))
}}/>))}
</tr></thead>
<tbody>{ordered.map ((row, index) => (
<tr key={index} className="border-t dark:border-slate-700">
{columns.map (field => (
<td key={field.key}
className="overflow-hidden break-words border-t px-3 py-2
align-top dark:border-slate-700"
data-counts-copy={
field.children
? countsNestedText (
row[field.key] as CountRecord[] ?? [], field.children,
`${path}.${field.key}`, params, locale)
: countsLogicalName (row, field.key, locale).text}>
{field.children
? <CountsNestedTable rows={row[field.key] as CountRecord[] ?? []}
fields={field.children} path={`${path}.${field.key}`}
resource={resource}
title={message (field.label)} params={params} update={update}/>
: scalar (row, field.key)}
</td>))}
</tr>))}</tbody>
</CountsSelectableTable>
</div>))}
</div>)
}
export default CountsNestedTable
+93
ファイルの表示
@@ -0,0 +1,93 @@
import { useContext, useEffect, useRef, useState } from 'react'
import { CountsTableScope, MAX_COUNT_WIDTH, MIN_COUNT_WIDTH,
useCountsTableLayout } from '@/lib/countsTableLayout'
import { cn } from '@/lib/utils'
import { useUITranslation } from '@/lib/useUITranslation'
const CountsResizeHandle = ({ field, label }: {
field: string; label: { text: string; language: string } }) => {
const family = useContext (CountsTableScope)
const layout = useCountsTableLayout ()
const { message } = useUITranslation ('counts')
const [active, setActive] = useState (false)
const press = useRef<{ x: number; y: number; width: number; armed: boolean
timer?: ReturnType<typeof setTimeout> } | null> (null)
const saved = useRef<{ cursor: string; userSelect: string } | null> (null)
const restore = () => {
if (saved.current == null) return
document.body.style.cursor = saved.current.cursor
document.body.style.userSelect = saved.current.userSelect
saved.current = null
}
const finish = () => {
if (press.current?.timer != null) clearTimeout (press.current.timer)
press.current = null
setActive (false)
restore ()
}
const arm = () => {
if (press.current == null) return
press.current.armed = true
saved.current = { cursor: document.body.style.cursor,
userSelect: document.body.style.userSelect }
document.body.style.cursor = 'col-resize'
document.body.style.userSelect = 'none'
setActive (true)
}
useEffect (() => () => {
if (press.current?.timer != null) clearTimeout (press.current.timer)
if (saved.current != null)
{
document.body.style.cursor = saved.current.cursor
document.body.style.userSelect = saved.current.userSelect
}
}, [])
const name = message ('resize.column', { column: label.text })
return (
<span role="separator" tabIndex={0} aria-orientation="vertical"
aria-label={name.text} lang={name.language}
aria-valuenow={layout.width (family, field)} aria-valuemin={MIN_COUNT_WIDTH}
aria-valuemax={MAX_COUNT_WIDTH} title={name.text}
data-counts-resize
className={cn ('absolute inset-y-0 right-0 z-30 w-3 cursor-col-resize touch-none',
'select-none outline-offset-[-2px] focus-visible:outline',
'before:absolute before:inset-y-1 before:left-1/2 before:border-l',
'before:border-slate-300 dark:before:border-slate-600',
'hover:before:border-blue-500 dark:hover:before:border-blue-400',
active && 'before:border-blue-500 dark:before:border-blue-400')}
onPointerDown={event => {
event.stopPropagation ()
if (!event.isPrimary || event.button !== 0) return
event.preventDefault ()
finish ()
event.currentTarget.setPointerCapture (event.pointerId)
press.current = { x: event.clientX, y: event.clientY,
width: layout.width (family, field), armed: false }
if (event.pointerType === 'mouse') arm ()
else press.current.timer = setTimeout (arm, 400)
}}
onPointerMove={event => {
event.stopPropagation ()
const state = press.current
if (state == null) return
if (state.armed)
layout.resize (family, field, state.width + event.clientX - state.x)
else if (Math.hypot (event.clientX - state.x, event.clientY - state.y) > 8)
finish ()
}}
onPointerUp={finish} onPointerCancel={finish} onLostPointerCapture={finish}
onClick={event => { event.preventDefault (); event.stopPropagation () }}
onContextMenu={event => event.preventDefault ()}
onKeyDown={event => {
event.stopPropagation ()
if (event.key !== 'ArrowLeft' && event.key !== 'ArrowRight') return
event.preventDefault ()
layout.resize (family, field, layout.width (family, field)
+ (event.key === 'ArrowLeft' ? -10 : 10))
}}/>)
}
export default CountsResizeHandle
+164
ファイルの表示
@@ -0,0 +1,164 @@
import SortableColumns from '@/components/common/SortableColumns'
import LocalisedText from '@/components/LocalisedText'
import PrefetchLink from '@/components/PrefetchLink'
import CountsRowNumber from '@/components/counts/CountsRowNumber'
import CountsNestedTable from '@/components/counts/CountsNestedTable'
import CountsSortHeader from '@/components/counts/CountsSortHeader'
import CountsResizeHandle from '@/components/counts/CountsResizeHandle'
import CountsSelectableTable from '@/components/counts/CountsSelectableTable'
import PostThumbnailPreview from '@/components/posts/PostThumbnailPreview'
import { changeCountSort, countColumns, countsRequestParams, fieldsForCounts,
readCountSort, selectedCountColumns, writeCountSort } from '@/lib/counts'
import { countsCellCopy, countsCellText, countsDetailClipboardTable,
countsImageURL, countsNestedRows } from '@/lib/countsDisplay'
import useDialogue from '@/lib/dialogues/useDialogue'
import { useLocale } from '@/lib/useLocale'
import { useCountsDefinition } from '@/lib/countsDefinitions'
import { useUITranslation } from '@/lib/useUITranslation'
import type { CountColumn, CountsResource,
CountsResult, CountsVersion } from '@/lib/counts'
const CountsResults = (
{ resource, data, params, pending, update }: {
resource: CountsResource
data: CountsResult
params: URLSearchParams
pending: boolean
update: (key: string, value: string) => void },
) => {
const definition = useCountsDefinition ()
const { label, message } = useUITranslation ('counts')
const locale = useLocale ()
const dialogue = useDialogue ()
const columns = selectedCountColumns (resource, params, definition)
const rowNumbers = params.get ('row_numbers') === '1'
const fields = fieldsForCounts (resource, definition)
const sorts = readCountSort (params.get ('sort') ?? 'created_at:desc')
const sort = (key: string, additive: boolean) =>
update ('sort', writeCountSort (changeCountSort (sorts, key, additive)))
const cell = (row: CountsVersion, column: CountColumn) => {
const field = fields.find (field => field.key === column)!
const value = countsCellText (row, column, locale, params)
if (field.children != null)
{
return <CountsNestedTable rows={countsNestedRows (row, column)}
fields={field.children} path={column} resource={resource}
title={message (field.label)} params={params} update={update}/>
}
switch (column)
{
case 'resource_id':
if (resource === 'nico_tags') return row.name ? (
<PrefetchLink to={`/tags/nico?${new URLSearchParams ({ name: row.name })}`}>
{row.resourceId}</PrefetchLink>) : row.resourceId
if (resource === 'tag_groups') return <PrefetchLink
to={`/wiki/${encodeURIComponent (`タググループ:${row.name ?? ''}`)}`}>
{row.resourceId}</PrefetchLink>
return <PrefetchLink to={`/${resource === 'tag_descriptions' ? 'tags' : resource}/${
encodeURIComponent (row.resourceId)}`}>
{row.resourceId}</PrefetchLink>
case 'name':
return resource === 'nico_tags' && row.name ? (
<PrefetchLink to={`/tags/nico?${new URLSearchParams ({ name: row.name })}`}>
{value}</PrefetchLink>) : value
case 'thumbnail':
{
const url = countsImageURL (row.thumbnailBase)
return url == null ? null : (
<button type="button" className="block" aria-label={message (
'shared:labels.thumbnail').text}
onClick={event => {
const trigger = event.currentTarget
void dialogue.form ({
title: label ('shared:labels.thumbnail'),
cancelText: label ('shared:dialogue.close'),
size: 'viewport',
body: () => <img src={url} alt={message ('shared:labels.thumbnail').text}
referrerPolicy="no-referrer" className="h-full w-full object-contain"/>
}).then (() => trigger.focus ({ preventScroll: true }))
}}>
<PostThumbnailPreview url={url} className="h-10 w-16"
alt={message ('shared:labels.thumbnail').text} referrerPolicy="no-referrer"/>
</button>)
}
case 'url':
case 'thumbnail_base':
return <a href={countsImageURL (value) ?? undefined} target="_blank" rel="noreferrer">
{value}</a>
case 'is_latest':
return <LocalisedText language={message ('latestValue').language}>
{value}</LocalisedText>
case 'user':
return row.createdByUser?.name ? value : (
<LocalisedText language={message (
row.createdByUser == null ? 'bot' : 'history:anonymous').language}>
{value}</LocalisedText>)
case 'event_type':
return <LocalisedText language={message (`events.${row.eventType}`).language}>
{value}</LocalisedText>
case 'category':
return <LocalisedText language={message (`common:categories.${row.category}`).language}>
{value}</LocalisedText>
default: return value
}
}
return <SortableColumns ids={columns} fixed={['thumbnail']}
name={key => message (fields.find (field => field.key === key)!.label)}
onChange={next => {
const all = fields.map (field => field.key)
const other = (params.get ('order') ?? '').split (' ').filter (key => !all.includes (key))
update ('order', [...next, ...all.filter (key => !next.includes (key)), ...other].join (' '))
}}>{columns => {
const tableColumns = rowNumbers ? ['row_number', ...columns] : columns
return (
<div className="min-h-0 flex-1 overflow-auto" data-counts-results aria-busy={pending}>
{tableColumns.length === 0 ? <p>{label ('nested.noColumns')}</p> : (
<CountsSelectableTable family={resource}
columns={tableColumns}
copyTable={() => countsDetailClipboardTable (data.versions ?? [],
columns.map (column => fields.find (field => field.key === column)!),
params, locale, rowNumbers ? (data.page - 1) * data.limit + 1 : null)}
identity={`${resource}:${countsRequestParams (params)}:${tableColumns.join (' ')}:${
(data.versions ?? []).map (row => row.id)}`}
className="text-left text-sm">
<thead className="sticky top-0 z-20 bg-background text-foreground
dark:bg-background dark:text-foreground"><tr>
{rowNumbers && <CountsRowNumber/>}
{columns.map (column => column === 'thumbnail'
? <th key={column} data-counts-heading={message (countColumns[column]).text}
className="relative px-2 py-1">
<span className="sr-only">{label (countColumns[column])}</span>
<CountsResizeHandle field={column} label={message (countColumns[column])}/>
</th>
: <CountsSortHeader key={column} field={column}
label={message (fields.find (field => field.key === column)!.label)}
dragId={column}
sorts={sorts} change={sort} structured={fields.find (
field => field.key === column)?.children != null}/>)}
</tr></thead>
<tbody>
{(data.versions ?? []).map ((row, index) => (
<tr key={row.id} className="border-b dark:border-slate-700">
{rowNumbers && <CountsRowNumber
value={(data.page - 1) * data.limit + index + 1}/>}
{columns.map (column => (
<td key={column}
className="overflow-hidden break-words border-b px-2 py-1
align-top dark:border-slate-700"
data-counts-copy={countsCellCopy (
row,
fields.find (field => field.key === column)!, params, locale)}>
{cell (row, column)}</td>))}
</tr>))}
</tbody>
</CountsSelectableTable>)}
{(data.versions?.length ?? 0) === 0
&& <p className="p-2">{label ('empty')}</p>}
</div>)
}}</SortableColumns>
}
export default CountsResults
+17
ファイルの表示
@@ -0,0 +1,17 @@
import CountsResizeHandle from '@/components/counts/CountsResizeHandle'
import { useUITranslation } from '@/lib/useUITranslation'
const CountsRowNumber = ({ value, rowSpan }: { value?: number; rowSpan?: number }) => {
const { label, message } = useUITranslation ('counts')
if (value != null) return <td data-counts-copy={String (value)}
className="border-b px-2 py-1 text-right align-top tabular-nums dark:border-slate-700">
{value}</td>
return <th rowSpan={rowSpan} data-counts-heading={message ('rowNumberHeading').text} className="relative px-2 py-1">
{label ('rowNumberHeading')}
<CountsResizeHandle field="row_number" label={message ('rowNumberHeading')}/>
</th>
}
export default CountsRowNumber
+354
ファイルの表示
@@ -0,0 +1,354 @@
import { useCallback, useEffect, useId, useLayoutEffect, useMemo, useRef, useState } from 'react'
import CountsCellMenu from '@/components/counts/CountsCellMenu'
import { countsClipboard, writeCountsClipboard } from '@/lib/countsClipboard'
import { CountsTableScope, useCountsTableLayout } from '@/lib/countsTableLayout'
import { cn } from '@/lib/utils'
import { useUITranslation } from '@/lib/useUITranslation'
import type { ClipboardEvent, ReactNode } from 'react'
import type { CountsClipboardTable } from '@/lib/countsClipboard'
import type { CountsMenuAction } from '@/components/counts/CountsCellMenu'
type Point = { row: number; column: number }
type Selection = { identity: string; anchor: Point; end: Point }
const interactive = 'a,button,input,select,textarea,summary,[contenteditable],[data-counts-resize]'
const cellGrid = (table: HTMLTableElement): HTMLTableCellElement[][] => {
const grid: HTMLTableCellElement[][] = []
for (const [row, tr] of Array.from (table.tBodies[0]?.rows ?? []).entries ())
{
grid[row] ??= []
let column = 0
for (const cell of Array.from (tr.cells))
{
while (grid[row][column] != null) ++column
for (let y = 0; y < cell.rowSpan; ++y)
{
grid[row + y] ??= []
for (let x = 0; x < cell.colSpan; ++x) grid[row + y][column + x] = cell
}
column += cell.colSpan
}
}
return grid
}
const cellAt = (table: HTMLTableElement, point: Point) =>
cellGrid (table)[point.row]?.[point.column]
const CountsSelectableTable = (
{ family, columns, identity, children, className, name, copyTable }: {
family: string
columns: string[]
identity: string
children: ReactNode
className?: string
name?: { text: string; language: string }
copyTable?: () => CountsClipboardTable },
) => {
const id = useId ()
const root = useRef<HTMLTableElement> (null)
const dragging = useRef (false)
const touchArmed = useRef (false)
const touchPress = useRef<{ x: number; y: number
timer: ReturnType<typeof setTimeout> } | null> (null)
const [menu, setMenu] = useState<{ x: number; y: number; point: Point
identity: string } | null> (null)
const suppressClick = useRef (false)
const [stored, setStored] = useState<Selection | null> (null)
const selection = stored?.identity === identity ? stored : null
const layout = useCountsTableLayout ()
const { label, message } = useUITranslation ('counts')
const bounds = useMemo (() => selection == null ? null : {
top: Math.min (selection.anchor.row, selection.end.row),
bottom: Math.max (selection.anchor.row, selection.end.row),
left: Math.min (selection.anchor.column, selection.end.column),
right: Math.max (selection.anchor.column, selection.end.column) }, [selection])
const finish = () => {
dragging.current = false
touchArmed.current = false
if (touchPress.current != null) clearTimeout (touchPress.current.timer)
touchPress.current = null
}
useEffect (() => {
const table = root.current
const preventPan = (event: TouchEvent) => {
if (touchArmed.current) event.preventDefault ()
}
table?.addEventListener ('touchmove', preventPan, { passive: false })
return () => {
table?.removeEventListener ('touchmove', preventPan)
if (touchPress.current != null) clearTimeout (touchPress.current.timer)
}
}, [])
useLayoutEffect (() => {
if (root.current == null) return
const grid = cellGrid (root.current)
const assigned = new Set<HTMLTableCellElement> ()
for (const [row, cells] of grid.entries ())
{
for (const [column, cell] of cells.entries ())
{
if (assigned.has (cell)) continue
assigned.add (cell)
const selected = bounds != null && row <= bounds.bottom
&& row + cell.rowSpan - 1 >= bounds.top && column <= bounds.right
&& column + cell.colSpan - 1 >= bounds.left
cell.id = `${id}-${row}-${column}`
if (cell.getAttribute ('role') !== 'rowheader') cell.setAttribute ('role', 'gridcell')
cell.setAttribute ('aria-selected', String (selected))
if (selected) cell.dataset.countsSelected = ''
else delete cell.dataset.countsSelected
}
}
if (bounds != null)
root.current.setAttribute ('aria-activedescendant', grid[bounds.bottom]?.[bounds.right]?.id ?? '')
else root.current.removeAttribute ('aria-activedescendant')
}, [bounds, children, id])
const pointFor = (element: Element | null, y?: number): Point | null => {
const table = root.current
if (table == null || element == null || !table.contains (element)) return null
let cell = element.closest ('td')
while (cell != null && cell.closest ('table') !== table)
cell = cell.closest ('table')?.parentElement?.closest ('td') ?? null
if (cell == null || cell.parentElement?.parentElement !== table.tBodies[0]) return null
const grid = cellGrid (table)
const hitRow = y == null ? -1 : Array.from (table.tBodies[0].rows).findIndex (row => {
const rect = row.getBoundingClientRect ()
return y >= rect.top && y < rect.bottom
})
for (const [row, cells] of grid.entries ())
{
if (hitRow >= 0 && row !== hitRow) continue
const column = cells.indexOf (cell as HTMLTableCellElement)
if (column >= 0) return { row, column }
}
return null
}
const closeMenu = useCallback (() => {
setMenu (null)
root.current?.focus ({ preventScroll: true })
}, [])
const openMenu = (point: Point, x: number, y: number) => {
const inside = bounds != null && point.row >= bounds.top && point.row <= bounds.bottom
&& point.column >= bounds.left && point.column <= bounds.right
if (inside) setStored (selection)
else setStored ({ identity, anchor: point, end: point })
setMenu ({ point, x, y, identity })
}
const clipboard = (headers = false) => {
if (bounds == null || root.current == null) return null
if (copyTable != null)
{
const table = copyTable ()
return countsClipboard ({
columns: table.columns.slice (bounds.left, bounds.right + 1),
headerRows: table.headerRows?.map (row => row.slice (bounds.left, bounds.right + 1)),
rows: table.rows.slice (bounds.top, bounds.bottom + 1)
.map (row => row.slice (bounds.left, bounds.right + 1)) }, headers)
}
const rows: string[][] = []
for (let row = bounds.top; row <= bounds.bottom; ++row)
{
const values: string[] = []
for (let column = bounds.left; column <= bounds.right; ++column)
{
const cell = cellAt (root.current, { row, column })
values.push (cell?.dataset.countsCopy ?? cell?.innerText ?? '')
}
rows.push (values)
}
if (headers)
{
const heading = root.current.tHead?.rows[0]
rows.unshift (Array.from ({ length: bounds.right - bounds.left + 1 }, (entry, index) => {
void entry
const cell = heading?.cells[bounds.left + index]
return cell?.dataset.countsHeading ?? cell?.textContent ?? ''
}))
}
return countsClipboard (rows, headers)
}
const menuAction = (action: CountsMenuAction) => {
if (menu == null || root.current == null) return
if (action === 'copy' || action === 'copyHeaders')
{
const payload = clipboard (action === 'copyHeaders')
if (payload != null) void writeCountsClipboard (payload).catch (error => {
console.error (error)
})
}
else
{
const lastRow = (root.current.tBodies[0]?.rows.length ?? 1) - 1
const lastColumn = columns.length - 1
const anchor = {
row: action === 'row' ? menu.point.row : 0,
column: action === 'column' ? menu.point.column : 0 }
const end = {
row: action === 'row' ? menu.point.row : lastRow,
column: action === 'column' ? menu.point.column : lastColumn }
setStored ({ identity, anchor, end })
}
closeMenu ()
}
const copy = (event: ClipboardEvent<HTMLTableElement>) => {
if (document.activeElement !== root.current) return
const payload = clipboard ()
if (payload == null) return
event.preventDefault ()
event.stopPropagation ()
event.clipboardData.setData ('text/plain', payload.text)
try
{
event.clipboardData.setData ('text/html', payload.html)
}
catch
{
// Plain TSV remains available when the browser disallows HTML clipboard data.
}
}
const totalWidth = columns.reduce ((sum, key) => sum + layout.width (family, key), 0)
return (
<CountsTableScope.Provider value={family}>
<table ref={root} role="grid" tabIndex={0} aria-multiselectable="true"
aria-label={name?.text ?? message ('selection.table').text}
aria-describedby={`${id}-hint`}
style={{ tableLayout: 'fixed', width: totalWidth }}
className={cn ('counts-grid select-none border-separate border-spacing-0 outline-none',
'focus-visible:outline focus-visible:outline-2 focus-visible:outline-blue-500',
'dark:focus-visible:outline-blue-400',
'[&>tbody>tr>td[data-counts-selected]]:bg-blue-100',
'[&>tbody>tr>td[data-counts-selected]]:text-slate-900',
'dark:[&>tbody>tr>td[data-counts-selected]]:bg-blue-900',
'dark:[&>tbody>tr>td[data-counts-selected]]:text-slate-100',
'[&>tbody>tr>td[data-counts-selected]]:outline',
'[&>tbody>tr>td[data-counts-selected]]:outline-blue-500',
'[&>tbody>tr>td[data-counts-selected]]:outline-offset-[-1px]', className)}
onCopy={copy}
onClickCapture={event => {
if ((event.target as Element).closest ('table') !== event.currentTarget
|| !suppressClick.current) return
event.preventDefault ()
event.stopPropagation ()
suppressClick.current = false
}}
onPointerDown={event => {
const element = event.target as Element
if (element.closest ('table') !== event.currentTarget) return
suppressClick.current = false
if (element.closest (interactive) != null || event.button !== 0) return
if (element instanceof HTMLElement && element.hasAttribute ('data-counts-scroll'))
{
const rect = element.getBoundingClientRect ()
if (event.clientX >= rect.left + element.clientLeft + element.clientWidth
|| event.clientY >= rect.top + element.clientTop + element.clientHeight) return
}
const point = pointFor (element, event.clientY)
if (point == null) return
event.stopPropagation ()
setMenu (null)
event.currentTarget.focus ({ preventScroll: true })
setStored ({ identity, anchor: event.shiftKey && selection != null
? selection.anchor : point, end: point })
finish ()
if (event.pointerType === 'mouse')
{
event.preventDefault ()
dragging.current = true
event.currentTarget.setPointerCapture (event.pointerId)
}
else
{
const table = event.currentTarget
const pointerId = event.pointerId
touchPress.current = { x: event.clientX, y: event.clientY,
timer: setTimeout (() => {
suppressClick.current = true
openMenu (point, event.clientX, event.clientY)
dragging.current = true
touchArmed.current = true
table.setPointerCapture (pointerId)
}, 450) }
}
}}
onPointerMove={event => {
if (touchPress.current != null && !touchArmed.current && Math.hypot (
event.clientX - touchPress.current.x, event.clientY - touchPress.current.y) > 8)
finish ()
if (!dragging.current || event.buttons === 0) return
const point = pointFor (document.elementFromPoint (event.clientX, event.clientY), event.clientY)
if (point == null) return
if (touchArmed.current && touchPress.current != null
&& Math.hypot (event.clientX - touchPress.current.x,
event.clientY - touchPress.current.y) <= 8) return
setMenu (null)
event.stopPropagation ()
setStored (previous => previous?.identity !== identity ? previous : {
...previous, end: point })
}}
onPointerUp={finish} onPointerCancel={finish}
onLostPointerCapture={event => {
if (event.target === event.currentTarget) finish ()
}}
onContextMenu={event => {
const element = event.target as Element
if (element.closest ('table') !== event.currentTarget) return
const point = pointFor (element, event.clientY)
if (point == null || element.closest ('[data-counts-resize]') != null) return
event.preventDefault ()
event.stopPropagation ()
openMenu (point, event.clientX, event.clientY)
}}
onKeyDown={event => {
if (event.target !== event.currentTarget) return
if (event.key === 'ContextMenu' || (event.shiftKey && event.key === 'F10'))
{
const point = selection?.end ?? { row: 0, column: 0 }
const rect = cellAt (event.currentTarget, point)?.getBoundingClientRect ()
if (rect != null)
{
event.preventDefault ()
openMenu (point, rect.left, rect.bottom)
}
return
}
if (event.key === 'Escape')
{
setStored (null)
return
}
if (!['ArrowUp', 'ArrowDown', 'ArrowLeft', 'ArrowRight'].includes (event.key)) return
const rows = event.currentTarget.tBodies[0]?.rows
if (rows == null || rows.length === 0) return
event.preventDefault ()
const previous = selection?.end ?? { row: 0, column: 0 }
const key = selection == null ? '' : event.key
const point = {
row: Math.max (0, Math.min (rows.length - 1, previous.row
+ (key === 'ArrowDown' ? 1 : (key === 'ArrowUp' ? -1 : 0)))),
column: Math.max (0, Math.min (columns.length - 1, previous.column
+ (key === 'ArrowRight' ? 1 : (key === 'ArrowLeft' ? -1 : 0)))) }
setStored ({ identity, anchor: event.shiftKey && selection != null
? selection.anchor : point, end: point })
cellAt (event.currentTarget, point)?.scrollIntoView ({
block: 'nearest', inline: 'nearest' })
}}>
<caption id={`${id}-hint`} className="sr-only">{label ('selection.hint')}</caption>
<colgroup>{columns.map (key => (
<col key={key} style={{ width: layout.width (family, key) }}/>))}</colgroup>
{children}
</table>
{menu?.identity === identity && <CountsCellMenu x={menu.x} y={menu.y}
act={menuAction} close={closeMenu}/>}
</CountsTableScope.Provider>)
}
export default CountsSelectableTable
+107
ファイルの表示
@@ -0,0 +1,107 @@
import { useEffect, useRef } from 'react'
import SortableItem from '@/components/common/SortableItem'
import CountsResizeHandle from '@/components/counts/CountsResizeHandle'
import { Button } from '@/components/ui/button'
import { useUITranslation } from '@/lib/useUITranslation'
import type { CountSort } from '@/lib/counts'
const CountsSortHeader = (
{ field, label, sorts, change, structured = false, rowSpan, colSpan, resizable = true, dragId, totalKind }: {
field: string
label: { text: string; language: string }
sorts: CountSort[]
change: (key: string, additive: boolean) => void
structured?: boolean
rowSpan?: number; colSpan?: number; resizable?: boolean
dragId?: string; totalKind?: string },
) => {
const { message } = useUITranslation ('counts')
const press = useRef<{ x: number; y: number
timer: ReturnType<typeof setTimeout> } | null> (null)
const suppressClick = useRef (false)
const cancel = () => {
if (press.current != null) clearTimeout (press.current.timer)
press.current = null
}
useEffect (() => () => {
if (press.current != null) clearTimeout (press.current.timer)
}, [])
const priority = sorts.findIndex (sort => sort.key === field)
const direction = sorts[priority]?.direction
const state = priority < 0 ? 'none' : (direction === 'asc' ? 'ascending' : 'descending')
const title = message (structured ? 'sort.structured' : 'sort.action')
const aria = message (priority < 0 ? 'sort.unsortedLabel'
: (sorts.length === 1 ? 'sort.singleLabel' : 'sort.label'), {
column: label.text, priority: priority + 1,
direction: message (`sort.${state}`).text })
const render = (shell?: Parameters<NonNullable<
Parameters<typeof SortableItem>[0]['renderShell']>>[0]) => (
<th ref={shell?.ref} style={shell?.style} data-counts-total={totalKind}
data-counts-heading={label.text} rowSpan={rowSpan} colSpan={colSpan}
className="relative overflow-hidden whitespace-nowrap px-1 py-1"
aria-sort={(sorts.length > 1 && priority >= 0) ? 'other' : state}>
<div className="flex min-w-0 items-center">
{shell?.handle}
<Button type="button" variant="ghost" size="sm"
className="h-auto w-full max-w-full justify-start select-none px-1 py-0 pr-3"
lang={aria.language} aria-label={aria.text} title={title.text}
onPointerDown={event => {
if (!event.isPrimary)
{
cancel ()
return
}
suppressClick.current = false
if (event.pointerType !== 'touch' && event.pointerType !== 'pen')
return
cancel ()
const x = event.clientX
const y = event.clientY
press.current = { x, y, timer: setTimeout (() => {
press.current = null
suppressClick.current = true
change (field, true)
}, 500) }
}}
onPointerMove={event => {
if (press.current != null && Math.hypot (
event.clientX - press.current.x, event.clientY - press.current.y) > 8)
cancel ()
}}
onPointerUp={cancel} onPointerCancel={cancel} onPointerLeave={cancel}
onContextMenu={event => {
if (press.current != null || suppressClick.current)
event.preventDefault ()
}}
onKeyDown={event => {
if (event.key !== 'Enter' && event.key !== ' ') return
event.preventDefault ()
suppressClick.current = false
change (field, event.ctrlKey || event.metaKey)
}}
onClick={event => {
if (suppressClick.current)
{
suppressClick.current = false
event.preventDefault ()
return
}
change (field, event.ctrlKey || event.metaKey)
}}>
<span className="truncate" lang={label.language}>{label.text}</span>
{priority >= 0 && <span aria-hidden>
{direction === 'asc' ? '▲' : '▼'}{sorts.length > 1 && priority + 1}</span>}
</Button>
</div>
{resizable && <CountsResizeHandle field={field} label={label}/>}
</th>)
return dragId == null ? render () : <SortableItem id={dragId} name={message ('reorder', { column: label.text })}
handleId={`column-${dragId}`} moveGroup={() => { }} renderShell={render}>{null}</SortableItem>
}
export default CountsSortHeader
+255
ファイルの表示
@@ -0,0 +1,255 @@
import SortableColumns from '@/components/common/SortableColumns'
import { countsSummaryValue } from '@/lib/countsDisplay'
import PrefetchLink from '@/components/PrefetchLink'
import CountsRowNumber from '@/components/counts/CountsRowNumber'
import CountsSelectableTable from '@/components/counts/CountsSelectableTable'
import CountsSortHeader from '@/components/counts/CountsSortHeader'
import { changeCountSort, countsRequestParams, readCountSort, writeCountSort } from '@/lib/counts'
import { countsAxisLabel, useCountsDefinition }
from '@/lib/countsDefinitions'
import { cn } from '@/lib/utils'
import { useLocale } from '@/lib/useLocale'
import { useUITranslation } from '@/lib/useUITranslation'
import { queryRecordParams, setQueryRecords, readCountState, encodeCountState, setCountState } from '@/lib/queryState'
import type { CountsPivotHeader, CountsResource, CountsResult } from '@/lib/counts'
import type { CountsAxis } from '@/lib/countsDefinitions'
const CountsSummaryTable = (
{ resource, data, params, pending, update, dragIds = {} }: {
dragIds?: Record<string, string>
resource: CountsResource; data: CountsResult; params: URLSearchParams
pending: boolean; update: (key: string, value: string, extra?: Record<string, string>) => void },
) => {
const definition = useCountsDefinition ()
const locale = useLocale ()
const { message } = useUITranslation ('counts')
const numbered = params.get ('row_numbers') === '1'
const rows = data.rows ?? []
const cols = data.cols ?? []
const cells = data.cells ?? []
const rowHeaders: CountsPivotHeader[] = data.rowHeaders ?? [...new Set (
cells.map (cell => JSON.stringify (cell.values.slice (0, rows.length))))]
.map (key => ({ values: JSON.parse (key), kind: 'data' }))
const columnHeaders: CountsPivotHeader[] = data.columnHeaders
?? (data.columnValues ?? [[]]).map (values => ({ values, kind: 'data' }))
const sorts = readCountSort (params.get ('sort'))
const change = (key: string, additive: boolean) => update (
'sort', writeCountSort (changeCountSort (sorts, key, additive)))
const display = (value: string | number | null, axis: CountsAxis, index: number) =>
countsSummaryValue (value, axis, index, data, locale)
const aggregateLabel = (header: CountsPivotHeader) => message (
header.kind === 'grand_total' ? (rows.length + cols.length === 1
? 'singleTotal' : 'grandTotal') : 'subtotal').text
const leading = Math.max (rows.length, cols.length > 0 ? 1 : 0)
const levels = Math.max (1, cols.length)
const names = (header: CountsPivotHeader, axes: CountsAxis[], offset: number) =>
header.values.map ((value, index) => display (value, axes[index], offset + index))
const columnKey = (header: CountsPivotHeader) => `${header.kind}:${JSON.stringify (header.values)}`
const measures = readCountState<Record<string, string | number | null>[]> (
params, 'measures', [])
const measureKey = (header: CountsPivotHeader) => {
if (header.values.length === 0) return 'count'
const ids = [...new Set (measures.map (entry => String (entry.key)))]
const existing = ids.find (id => JSON.stringify (measures.filter (entry => entry.key === id)
.sort ((a, b) => Number (a.axis) - Number (b.axis)).map (entry => entry.value))
=== JSON.stringify (header.values))
return existing ?? `m${Array.from ({ length: 100 }, (_, index) => index)
.find (index => !ids.includes (`m${index}`)) ?? 99}`
}
const changeMeasure = (header: CountsPivotHeader, additive: boolean) => {
const key = measureKey (header)
const next = new URLSearchParams (params)
const nextSort = changeCountSort (sorts, key, additive)
const active = new Set (nextSort.map (sort => sort.key))
const records = measures.filter (record => record.key !== key && active.has (String (record.key)))
setQueryRecords (next, 'measures', [...records, ...(active.has (key)
? header.values.map ((value, axis) => ({ key, axis, value })) : [])])
update ('sort', writeCountSort (nextSort),
queryRecordParams (next, 'measures'))
}
const keys = [...(numbered ? ['row_number'] : []),
...Array.from ({ length: leading }, (_, index) => `a${index}`),
...columnHeaders.map (columnKey)]
const full = new Map (cells.map (cell => [JSON.stringify (cell.values), cell.count]))
const totals = new Map ((data.pivotTotals ?? []).map (cell => [
JSON.stringify ([cell.row, cell.column]), cell.count]))
const count = (row: CountsPivotHeader, column: CountsPivotHeader) =>
(row.values.length === rows.length && column.values.length === cols.length)
? full.get (JSON.stringify ([...row.values, ...column.values])) ?? 0
: totals.get (JSON.stringify ([row.values, column.values])) ?? 0
const drill = (row: CountsPivotHeader, column: CountsPivotHeader) => {
const next = new URLSearchParams (params)
const existing = readCountState<Record<string, string | number | null>[]> (params, 'drill', [])
setCountState (next, 'drill', encodeCountState ('drill', [...existing,
...row.values.map ((value, index) => ({ ...rows[index], value })),
...column.values.map ((value, index) => ({ ...cols[index], value }))]))
next.set ('mode', 'details')
next.delete ('sort')
next.delete ('page')
return `/counts/${resource}?${next}`
}
let ordinal = (data.page - 1) * data.limit
const rowLabels = rowHeaders.map (header => {
const labels = names (header, rows, 0)
const number = header.kind === 'data' ? String (++ordinal) : ''
if (header.kind !== 'data') labels.push (aggregateLabel (header))
return [...(numbered ? [number] : []),
...Array.from ({ length: leading }, (_, index) => labels[index] ?? '')]
})
const spans = rowHeaders.map ((row, index) => rows.map ((_, axis) => {
if (row.kind !== 'data') return 1
const prefix = JSON.stringify (row.values.slice (0, axis + 1))
const previous = rowHeaders[index - 1]
if (previous?.kind === 'data'
&& JSON.stringify (previous.values.slice (0, axis + 1)) === prefix) return 0
let span = 1
while (index + span < rowHeaders.length)
{
const next = rowHeaders[index + span]
if (next.kind !== 'data'
|| JSON.stringify (next.values.slice (0, axis + 1)) !== prefix) break
++span
}
return span
}))
rowLabels.forEach ((labels, row) => spans[row].forEach ((span, column) => {
if (span === 0) labels[column + (numbered ? 1 : 0)] = ''
}))
// This unmerged matrix is also the single source for TSV and HTML clipboard headings.
const headingRows = Array.from ({ length: levels }, (_, level) => [
...(numbered ? [level === 0 ? message ('rowNumberHeading').text : ''] : []),
...Array.from ({ length: leading }, (_, index) =>
level === 0 && rows[index] != null
? countsAxisLabel (rows[index], definition, locale).text : ''),
...columnHeaders.map ((header, index) => {
if (cols.length === 0) return header.kind === 'data'
? message ('count').text : aggregateLabel (header)
if (level > header.values.length) return ''
if (level === header.values.length)
return header.kind === 'data' ? '' : aggregateLabel (header)
const prefix = header.values.slice (0, level + 1)
const previous = columnHeaders[index - 1]
if (level < levels - 1 && previous != null
&& JSON.stringify (previous.values.slice (0, level + 1)) === JSON.stringify (prefix))
return ''
return display (header.values[level], cols[level], rows.length + level)
})])
const matrix = rowHeaders.map ((header, index) => [...rowLabels[index],
...columnHeaders.map (column => String (count (header, column)))])
const headerCells = (level: number) => {
const output = []
for (let index = 0; index < columnHeaders.length; ++index)
{
const header = columnHeaders[index]
if (cols.length > 0 && level > header.values.length) continue
const aggregate = header.kind !== 'data' && level === header.values.length
const leaf = level === levels - 1 || aggregate
const text = aggregate ? aggregateLabel (header)
: (cols.length === 0 ? message ('count').text
: display (header.values[level], cols[level], rows.length + level))
let span = 1
if (!leaf)
{
const prefix = JSON.stringify (header.values.slice (0, level + 1))
while (index + span < columnHeaders.length)
{
const sibling = columnHeaders[index + span]
if (sibling.values.length <= level
|| JSON.stringify (sibling.values.slice (0, level + 1)) !== prefix) break
++span
}
}
const sortKey = leaf ? measureKey (header) : `a${rows.length + level}`
const field = leaf ? columnKey (header) : sortKey
output.push (<CountsSortHeader key={`${level}:${field}:${index}`} field={field}
colSpan={span} rowSpan={aggregate ? levels - level : 1} resizable={leaf}
totalKind={header.kind !== 'data' ? header.kind : undefined}
dragId={level === 0 && header.values.length > 0
? dragIds[JSON.stringify (header.values.slice (0, 1))] : undefined}
label={{ text, language: message ('count').language }}
sorts={sorts.map (sort => sort.key === sortKey ? { ...sort, key: field } : sort)}
change={(_, additive) => leaf ? changeMeasure (header, additive)
: change (sortKey, additive)}/>)
index += span - 1
}
return output
}
return <div className="min-h-0 flex-1 overflow-auto" data-counts-results aria-busy={pending}>
<CountsSelectableTable family={`pivot_${resource}`} columns={keys}
copyTable={() => ({ columns: keys.map ((_, index) => ({ label: headingRows[0][index] })),
headerRows: headingRows, rows: matrix })}
identity={`${countsRequestParams (params)}:${JSON.stringify (keys)}`}
className="text-left text-sm">
<thead className="sticky top-0 z-20">
{headingRows.map ((_, level) => <tr key={level}>
{level === 0 && <>
{numbered && <CountsRowNumber rowSpan={levels}/>}
{Array.from ({ length: leading }, (_, index) => rows[index] == null
? <th key={index} rowSpan={levels}/>
: <CountsSortHeader key={rows[index].field} field={`a${index}`}
rowSpan={levels} label={countsAxisLabel (rows[index], definition, locale)}
sorts={sorts} change={change}/>)}
</>}
{headerCells (level)}
</tr>)}
</thead>
<tbody>{rowHeaders.map ((row, index) => <tr key={columnKey (row)}>
{rowLabels[index].map ((value, offset) => {
const axis = offset - (numbered ? 1 : 0)
const span = axis < 0 ? 1 : spans[index][axis] ?? 1
return span === 0 ? null : <td key={offset} rowSpan={span}
role={axis >= 0 ? 'rowheader' : undefined} data-counts-copy={value}
data-counts-aggregate={row.kind !== 'data' ? '' : undefined}
data-counts-total={row.kind !== 'data' ? row.kind : undefined}
className={cn ('overflow-hidden break-words border px-2 py-1 dark:border-slate-700',
numbered && offset === 0 && 'text-right tabular-nums')}>{value}</td>
})}
{columnHeaders.map (column => <td key={columnKey (column)}
data-counts-copy={String (count (row, column))}
data-counts-total={(row.kind === 'grand_total' || column.kind === 'grand_total')
? 'grand_total' : ((row.kind === 'subtotal' || column.kind === 'subtotal')
? 'subtotal' : undefined)}
data-counts-aggregate={(row.kind !== 'data' || column.kind !== 'data')
? '' : undefined}
className={cn ('border px-2 py-1 dark:border-slate-700',
(row.kind !== 'data' || column.kind !== 'data')
&& 'bg-slate-200 text-slate-950 font-semibold',
(row.kind !== 'data' || column.kind !== 'data')
&& 'dark:bg-slate-700 dark:text-slate-50')}>
<PrefetchLink to={drill (row, column)}>{count (row, column)}</PrefetchLink>
</td>)}
</tr>)}</tbody>
</CountsSelectableTable>
</div>
}
const CountsSummaryResults = (props: Parameters<typeof CountsSummaryTable>[0]) => {
const locale = useLocale ()
const { message } = useUITranslation ('counts')
const headers = props.data.columnHeaders ?? []
const columns = props.data.cols ?? []
if (columns.length === 0) return <CountsSummaryTable {...props}/>
const groups = [...new Set (headers.filter (header => header.values.length > 0)
.map (header => JSON.stringify (header.values.slice (0, 1))))]
const ids = groups.map ((_, index) => `group${index}`)
const dragIds = Object.fromEntries (groups.map ((group, index) => [group, ids[index]]))
return <SortableColumns ids={ids} name={id => ({
text: countsSummaryValue (JSON.parse (groups[ids.indexOf (id)])[0], columns[0],
props.data.rows?.length ?? 0, props.data, locale), language: message ('count').language })}
onChange={order => props.update ('column_order', encodeCountState ('column_order',
order.map (id => ({ value: JSON.parse (groups[ids.indexOf (id)])[0] }))))}>
{order => {
const position = (header: CountsPivotHeader) => header.values.length === 0 ? order.length
: order.indexOf (dragIds[JSON.stringify (header.values.slice (0, 1))])
const sorted = [...headers].sort ((a, b) => position (a) - position (b))
return <CountsSummaryTable {...props} data={{ ...props.data, columnHeaders: sorted }}
dragIds={dragIds}/>
}}
</SortableColumns>
}
export default CountsSummaryResults
+24
ファイルの表示
@@ -0,0 +1,24 @@
import { useState } from 'react'
import { CountsLayoutContext, defaultCountWidths, MAX_COUNT_WIDTH, MIN_COUNT_WIDTH }
from '@/lib/countsTableLayout'
import type { ReactNode } from 'react'
const CountsTableLayout = ({ children }: { children: ReactNode }) => {
const [widths, setWidths] = useState<Record<string, number>> ({ })
const width = (family: string, key: string) =>
widths[`${family}/${key}`] ?? defaultCountWidths[key] ?? 160
const resize = (family: string, key: string, value: number) => {
const next = Math.round (Math.max (MIN_COUNT_WIDTH, Math.min (MAX_COUNT_WIDTH, value)))
setWidths (previous => ({ ...previous, [`${family}/${key}`]: next }))
}
return (
<CountsLayoutContext.Provider value={{ width, resize }}>
{children}
</CountsLayoutContext.Provider>)
}
export default CountsTableLayout
+79
ファイルの表示
@@ -0,0 +1,79 @@
.counts-grid > tbody > tr > td {
@apply h-6;
}
.counts-grid > tbody > tr:nth-child(odd) > td:not([data-counts-selected]) {
@apply bg-white text-slate-900;
}
.counts-grid > tbody > tr:nth-child(even) > td:not([data-counts-selected]) {
@apply bg-slate-50 text-slate-900;
}
.dark .counts-grid > tbody > tr:nth-child(odd) > td:not([data-counts-selected]) {
@apply bg-slate-950 text-slate-100;
}
.dark .counts-grid > tbody > tr:nth-child(even) > td:not([data-counts-selected]) {
@apply bg-slate-900 text-slate-100;
}
.counts-grid > tbody > tr > td[data-counts-aggregate]:not([data-counts-selected]) {
background-color: #e2e8f0;
color: #0f172a;
font-weight: 600;
}
.dark .counts-grid > tbody > tr > td[data-counts-aggregate]:not([data-counts-selected]) {
background-color: #334155;
color: #f8fafc;
}
.counts-grid > thead {
@apply bg-slate-200 text-slate-950;
}
.counts-grid > thead > tr > th {
@apply border-b border-r border-slate-300 bg-slate-200 text-slate-950;
}
.dark .counts-grid > thead,
.dark .counts-grid > thead > tr > th {
@apply border-slate-600 bg-slate-800 text-slate-50;
}
.counts-grid > tbody > tr > td {
@apply border-r border-slate-200 dark:border-slate-700;
-webkit-touch-callout: none;
}
.counts-grid > tbody > tr > td:has(> .counts-nested) {
padding: 0;
}
.counts-nested {
@apply border-l-2 border-slate-300 dark:border-slate-600;
}
.counts-grid > tbody > tr > td[data-counts-total='subtotal']:not([data-counts-selected]),
.counts-grid > thead > tr > th[data-counts-total='subtotal'] {
background-color: #e2e8f0;
color: #0f172a;
font-weight: 600;
}
.counts-grid > tbody > tr > td[data-counts-total='grand_total']:not([data-counts-selected]),
.counts-grid > thead > tr > th[data-counts-total='grand_total'] {
background-color: #cbd5e1;
color: #0f172a;
font-weight: 700;
}
.dark .counts-grid > tbody > tr > td[data-counts-total='subtotal']:not([data-counts-selected]),
.dark .counts-grid > thead > tr > th[data-counts-total='subtotal'] {
background-color: #334155;
color: #f1f5f9;
}
.dark .counts-grid > tbody > tr > td[data-counts-total='grand_total']:not([data-counts-selected]),
.dark .counts-grid > thead > tr > th[data-counts-total='grand_total'] {
background-color: #475569;
color: #fff;
}
+29 -10
ファイルの表示
@@ -8,6 +8,7 @@ import { Dialog,
DialogHeader, DialogHeader,
DialogTitle } from '@/components/ui/dialog' DialogTitle } from '@/components/ui/dialog'
import { DialogueContext } from '@/lib/dialogues/useDialogue' import { DialogueContext } from '@/lib/dialogues/useDialogue'
import { cn } from '@/lib/utils'
import { useUITranslation } from '@/lib/useUITranslation' import { useUITranslation } from '@/lib/useUITranslation'
import type { FC, ReactNode } from 'react' import type { FC, ReactNode } from 'react'
@@ -155,6 +156,22 @@ const DialogueProvider: FC<Props> = ({ children }) => {
}, },
[closeRequest, pendingIds]) [closeRequest, pendingIds])
const dismiss = async (request: DialogueRequest) => {
if (request.kind === 'form' && request.options.onCancel != null)
{
setPendingIds (current => [...current, request.id])
try
{
if (await request.options.onCancel () === false) return
}
finally
{
setPendingIds (current => current.filter (id => id !== request.id))
}
}
closeRequest (request.id, request.kind === 'confirm' ? false : null)
}
const active = queue[0] const active = queue[0]
const startActions = const startActions =
active?.kind === 'form' active?.kind === 'form'
@@ -178,18 +195,17 @@ const DialogueProvider: FC<Props> = ({ children }) => {
nestedConfirm?.parentId === active.id nestedConfirm?.parentId === active.id
|| pendingIds.includes (active.id) || pendingIds.includes (active.id)
if (!(open) && !(blocked)) if (!(open) && !(blocked))
closeRequest (active.id, active.kind !== 'confirm' && null) dismiss (active)
}}> }}>
<DialogContent <DialogContent
closeLabel={(active.kind === 'confirm' || active.kind === 'choice') closeLabel={(active.kind === 'confirm' || active.kind === 'choice')
? active.options.closeLabel : undefined} ? active.options.closeLabel : undefined}
className={ className={cn (active.kind === 'form'
active.kind === 'form' ? 'flex max-h-[calc(100dvh-1rem)] flex-col overflow-hidden gap-0'
? `flex max-h-[calc(100dvh-1rem)] ${ active.options.size === 'large' : 'px-6 pb-6 pt-7', active.kind === 'form' && (
? 'max-w-3xl' active.options.size === 'viewport'
: 'max-w-lg' } flex-col overflow-hidden gap-0` ? 'h-[88dvh] w-[88vw] max-w-none'
: 'px-6 pb-6 pt-7' : (active.options.size === 'large' ? 'max-w-3xl' : 'max-w-lg')))}
}
onEscapeKeyDown={event => { onEscapeKeyDown={event => {
if (nestedConfirm?.parentId === active.id || pendingIds.includes (active.id)) if (nestedConfirm?.parentId === active.id || pendingIds.includes (active.id))
event.preventDefault () event.preventDefault ()
@@ -237,11 +253,14 @@ const DialogueProvider: FC<Props> = ({ children }) => {
</Button>))} </Button>))}
</div> </div>
<div className="flex w-full flex-col gap-2 md:w-auto md:flex-row"> <div className={cn (
'flex w-full gap-2 md:w-auto md:flex-row',
active.options.footerLayout === 'inline'
? 'flex-row justify-end [&>button]:w-auto' : 'flex-col')}>
<Button <Button
className="w-full md:w-auto" className="w-full md:w-auto"
variant="outline" variant="outline"
onClick={() => closeRequest (active.id)} onClick={() => dismiss (active)}
disabled={pendingIds.includes (active.id) disabled={pendingIds.includes (active.id)
|| nestedConfirm?.parentId === active.id}> || nestedConfirm?.parentId === active.id}>
{active.options.cancelText ?? label ('actions.cancel')} {active.options.cancelText ?? label ('actions.cancel')}
+6 -2
ファイルの表示
@@ -7,6 +7,7 @@ import { useTranslation } from 'react-i18next'
import LocalisedText from '@/components/LocalisedText' import LocalisedText from '@/components/LocalisedText'
import { translateUIMessage } from '@/lib/i18n' import { translateUIMessage } from '@/lib/i18n'
import { useClientMotion } from '@/lib/useClientMotion'
import { cn } from '@/lib/utils' import { cn } from '@/lib/utils'
const Dialog = DialogPrimitive.Root const Dialog = DialogPrimitive.Root
@@ -37,14 +38,17 @@ const DialogContent = React.forwardRef<
React.ComponentPropsWithoutRef<typeof DialogPrimitive.Content> React.ComponentPropsWithoutRef<typeof DialogPrimitive.Content>
& { closeLabel?: React.ReactNode } & { closeLabel?: React.ReactNode }
>( >(
({ className, children, closeLabel, ...props }, ref) => { ({ className, children, closeLabel, style, ...props }, ref) => {
const { i18n } = useTranslation ('shared') const { i18n } = useTranslation ('shared')
const motion = useClientMotion ()
const animationDuration = `${motion.duration ('popover')}ms`
const close = translateUIMessage ('dialogue.close', 'shared', i18n.language) const close = translateUIMessage ('dialogue.close', 'shared', i18n.language)
return ( return (
<DialogPortal> <DialogPortal>
<DialogOverlay/> <DialogOverlay style={{ animationDuration }}/>
<DialogPrimitive.Content <DialogPrimitive.Content
ref={ref} ref={ref}
style={{ animationDuration, ...style }}
className={cn ( className={cn (
'fixed left-[50%] top-[50%] z-50 grid w-[calc(100%-2rem)] max-w-lg', 'fixed left-[50%] top-[50%] z-50 grid w-[calc(100%-2rem)] max-w-lg',
'translate-x-[-50%] translate-y-[-50%]', 'translate-x-[-50%] translate-y-[-50%]',
+3 -1
ファイルの表示
@@ -66,7 +66,9 @@ export const render = async (
authentication: Pick<RenderContext, 'cookie' | 'forwardedFor'> = { }, authentication: Pick<RenderContext, 'cookie' | 'forwardedFor'> = { },
): Promise<RenderedPage | RouteResponse> => { ): Promise<RenderedPage | RouteResponse> => {
const url = new URL (address, origin) const url = new URL (address, origin)
const legacy = legacyLocalePath ({ pathname: url.pathname, search: url.search, hash: '' }) const legacy = legacyLocalePath (
{ pathname: url.pathname, search: url.search, hash: url.hash },
pathname => resolvePageRoute (pathname) != null)
const pathname = legacy == null ? url.pathname : new URL (legacy, origin).pathname const pathname = legacy == null ? url.pathname : new URL (legacy, origin).pathname
const segment = pathname.split ('/')[1] const segment = pathname.split ('/')[1]
const known = resolvePageRoute (pathname) const known = resolvePageRoute (pathname)
+1
ファイルの表示
@@ -1,4 +1,5 @@
@import "@fontsource-variable/noto-sans-jp"; @import "@fontsource-variable/noto-sans-jp";
@import './components/counts/countsTable.css';
@tailwind base; @tailwind base;
@tailwind components; @tailwind components;
+37
ファイルの表示
@@ -199,3 +199,40 @@ export const apiDelete = async <T = void> (
export const isApiError = <T = unknown> (err: unknown): err is AxiosError<T> => export const isApiError = <T = unknown> (err: unknown): err is AxiosError<T> =>
axios.isAxiosError (err) axios.isAxiosError (err)
// The server acknowledges a complete file before native download starts.
export const apiDownload = async (
path: string,
body: Record<string, string>,
locale: LocaleCode,
onError?: () => void,
): Promise<void> => {
const { token } = await apiPost<{ token: string }> (`${path}/prepare`, body, { locale })
if (typeof token !== 'string' || token.length === 0)
throw new Error ('Missing download token')
startAPIDownload (path, token, onError)
}
export const startAPIDownload = (
path: string,
token: string,
onError?: () => void,
): void => {
const frame = document.createElement ('iframe')
frame.hidden = true
frame.addEventListener ('load', () => {
const document = frame.contentDocument
if (document != null && document.URL !== 'about:blank')
{
onError?.()
frame.remove ()
}
})
frame.addEventListener ('error', () => { onError?.(); frame.remove () })
frame.src = `${API_BASE_URL}${path}/files/${encodeURIComponent (token)}`
document.body.append (frame)
// Attachments do not reliably fire load; keep this isolated from page navigation.
window.setTimeout (() => frame.remove (), 10 * 60_000)
}
+6
ファイルの表示
@@ -32,3 +32,9 @@ export const clientScrollBehaviour = (
animationMode: ClientAnimationMode, animationMode: ClientAnimationMode,
): ScrollBehavior => ): ScrollBehavior =>
animationMode === 'off' ? 'auto' : 'smooth' animationMode === 'off' ? 'auto' : 'smooth'
export const clientMotionTokens = {
durationMS: { feedback: 120, popover: 160, collapse: 220, reorder: 200, drop: 180 },
easing: [.2, 0, 0, 1] as const,
cssEasing: 'cubic-bezier(.2, 0, 0, 1)',
distance: 4 }
+41
ファイルの表示
@@ -0,0 +1,41 @@
import { describe, expect, it } from 'vitest'
import { columnsForCounts, countColumns, countsCSV, normaliseCountsParams } from '@/lib/counts'
import { createUII18n, translateUIMessage } from '@/lib/i18n'
describe ('counts exports', () => {
it ('resolves every output and CSV header key in both locales', () => {
for (const locale of ['ja', 'ain-Kana'])
{
const instance = createUII18n (locale)
for (const key of Object.values (countColumns))
{
const translated = translateUIMessage (key, 'counts', locale, { }, instance)
expect (translated.text).not.toBe (key)
expect (translated.text).not.toContain ('counts:')
}
const fallback = translateUIMessage ('run', 'counts', locale, { }, instance)
expect (fallback).toEqual ({ text: '集計実行', language: 'ja' })
}
})
it ('makes the monthly default explicit for the existing API', () => {
expect (normaliseCountsParams ('posts', new URLSearchParams ()).get ('axis'))
.toBe ('month')
})
it ('escapes quotes, newlines and spreadsheet formulas in UTF-8 CSV', () => {
expect (countsCSV ([['日本語', 'a"b', 'a\nb', '=1+1']]))
.toBe ('\uFEFF"日本語","a""b","a\nb","\'=1+1"')
})
it ('keeps resource-specific columns out of the other resource', () => {
expect (columnsForCounts ('posts')).toContain ('added_tags')
expect (columnsForCounts ('posts')).not.toContain ('aliases')
expect (columnsForCounts ('tags')).toContain ('aliases')
expect (columnsForCounts ('tags')).not.toContain ('url')
expect (columnsForCounts ('posts')).not.toContain ('event_type')
expect (columnsForCounts ('tags')).not.toContain ('event_type')
})
})
+492
ファイルの表示
@@ -0,0 +1,492 @@
import { countsFieldLabel } from '@/lib/countsDefinitions'
import { apiDownload, apiGet } from '@/lib/api'
import { readCountState, encodeCountState } from '@/lib/queryState'
import type { CountsAxis, CountsDefinition } from '@/lib/countsDefinitions'
import type { LocaleCode } from '@/lib/locales'
import type { TagNameSnapshot } from '@/types'
export type CountsResource = 'posts' | 'tags' | 'nico_tags'
| 'tag_descriptions' | 'tag_groups' | 'materials'
export type CountsName = TagNameSnapshot & {
languageCode: string; languageName?: string | null; scriptName?: string | null }
export type CountsTag = {
id: number
kind: 'internal' | 'external'
tagId: number | null
externalTagId: number | null
versionNo: number | null
name: string | null
category: string | null
sections: { beginMs: number; endMs: number | null }[]
names: CountsName[] }
export type CountsVersion = {
[key: string]: unknown
eventType?: string
id: number
resourceId: number
versionNo: number
isLatest: boolean
createdAt: string
createdByUser: { id: number; name: string | null } | null
title?: string | null
url?: string
thumbnailBase?: string | null
tags?: CountsTag[]
parents: { id: number }[]
videoMs?: number | null
originalCreatedFrom?: string | null
originalCreatedBefore?: string | null
name?: string | null
names?: CountsName[]
category?: string
aliases?: { languageCode: string; name: string; scriptCode: string }[]
deprecated?: string | null }
export type CountsStateRow = {
kind: 'internal' | 'external'
tagId: number
name: string | null
deprecated?: string | null
period?: string
at: string
count: number | null }
export type CountsStateTotals = {
posts: number | null
tags: number | null
assignments: number | null
deprecatedTags: number | null
externalAssignments: number | null }
export type CountsPivotHeader = { values: (string | number | null)[]
kind: 'data' | 'subtotal' | 'grand_total' }
export type CountsResult = {
populationCount?: number
populationCountScope?: string
rowHeaders?: CountsPivotHeader[]
columnHeaders?: CountsPivotHeader[]
pivotTotals?: { row: (string | number | null)[]; column: (string | number | null)[]
count: number }[]
executionAt?: string
columnValues?: (string | number | null)[][]
cells?: { values: (string | number | null)[]; count: number }[]
axisLabels?: Record<string, Record<string, string | null>>
rows?: { field: string; grain?: string }[]
cols?: { field: string; grain?: string }[]
count: number
page: number
limit: number
groupCount?: number
groups?: { key: string | number | null; name: string | null; count: number }[]
versions?: CountsVersion[]
at?: string
from?: string
before?: string
grain?: string
totals?: CountsStateTotals
stateRows?: CountsStateRow[]
coverage?: { basis: string; exact: boolean
postHistoryFrom: string | null; tagHistoryFrom: string | null } }
export type CountSort = { key: string; direction: 'asc' | 'desc' }
export type CountField = { key: string; label: string; children?: CountField[] }
export type CountRecord = Record<string, unknown>
export const countColumns: Record<string, string> = {
event_type: 'counts:eventType',
thumbnail: 'counts:columns.thumbnailDisplay', resource_id: 'counts:columns.resourceId',
version_no: 'counts:columns.version', is_latest: 'counts:columns.latest',
title: 'counts:columns.postTitle', url: 'counts:columns.postURL',
thumbnail_base: 'counts:columns.thumbnailURL', tags: 'counts:columns.tags',
parents: 'shared:labels.parentPosts', video_ms: 'shared:labels.duration',
original_created_from: 'counts:columns.originalFrom',
original_created_before: 'counts:columns.originalBefore',
name: 'counts:columns.name', names: 'counts:columns.names',
category: 'common:categoryLabel', aliases: 'tags:aliases',
deprecated: 'counts:columns.deprecated', user: 'history:actor',
created_at: 'counts:columns.changedAt' }
export type CountColumn = string
const aliasFields: CountField[] = [
{ key: 'name', label: 'history:name' },
{ key: 'scriptCode', label: 'history:nameChanges.script' }]
const nameFields: CountField[] = [
{ key: 'languageCode', label: 'counts:columns.language' },
{ key: 'scriptCode', label: 'history:nameChanges.script' },
{ key: 'name', label: 'history:name' },
{ key: 'aliases', label: 'tags:aliases', children: aliasFields },
{ key: 'autoGenerated', label: 'history:nameChanges.automatic' }]
const placementFields = (depth: number): CountField[] => [
{ key: 'sectionName', label: 'counts:attributes.section_name' },
{ key: 'description', label: 'counts:attributes.description' },
{ key: 'tagId', label: 'counts:attributes.tag_id' },
{ key: 'tagGroupId', label: 'counts:resources.tag_groups' },
...(depth > 0 ? [{ key: 'children', label: 'counts:attributes.children',
children: placementFields (depth - 1) }] : [])]
const nested: Partial<Record<CountColumn, CountField[]>> = {
placements: placementFields (4),
export_paths_json: [
{ key: 'profile', label: 'counts:attributes.profile' },
{ key: 'path', label: 'counts:attributes.path' }],
members: [
{ key: 'tagId', label: 'counts:columns.resourceId' },
{ key: 'tagGroupId', label: 'counts:resources.tag_groups' },
{ key: 'versionNo', label: 'history:version' }],
tags: [
{ key: 'id', label: 'counts:columns.resourceId' },
{ key: 'versionNo', label: 'history:version' },
{ key: 'kind', label: 'counts:columns.kind' },
{ key: 'name', label: 'history:name' },
{ key: 'category', label: 'common:categoryLabel' },
{ key: 'sections', label: 'counts:columns.sections' },
{ key: 'names', label: 'counts:columns.names', children: nameFields }],
names: nameFields,
aliases: [{ key: 'languageCode', label: 'counts:columns.language' }, ...aliasFields],
parents: [{ key: 'id', label: 'counts:columns.resourceId' }] }
export const columnsForCounts = (
resource: CountsResource,
definition?: CountsDefinition,
): CountColumn[] => [
...(resource === 'posts' ? ['thumbnail'] : []),
'resource_id', 'version_no', 'is_latest',
...(resource === 'posts'
? ['title', 'url', 'thumbnail_base', 'tags', 'parents', 'video_ms',
'original_created_from', 'original_created_before']
: (resource === 'tags'
? ['name', 'names', 'category', 'aliases', 'parents', 'deprecated']
: (definition?.fields.filter (field => ![
'resource_id', 'version_no', 'event_type', 'user', 'created_at']
.includes (field.key)).map (field => field.key) ?? []))),
'event_type', 'user', 'created_at']
export const fieldsForCounts = (
resource: CountsResource,
definition?: CountsDefinition,
): CountField[] => columnsForCounts (resource, definition).map (key => ({
key,
label: (key === 'parents' && resource === 'tags') ? 'tags:parents'
: (countColumns[key] ?? countsFieldLabel (key)),
children: key === 'parents' && resource === 'tags'
? [...(nested.parents ?? []), { key: 'name', label: 'counts:columns.name' }]
: nested[key] }))
export const orderedCountFields = (
fields: CountField[],
params: URLSearchParams,
parent = '',
): CountField[] => {
const prefix = parent ? `${parent}.` : ''
const saved = (params.get ('order') ?? '').split (' ')
const keys = [...new Set ([...saved, ...fields.map (field => `${prefix}${field.key}`)])]
const ordered = keys.flatMap (key => {
const field = fields.find (field => `${prefix}${field.key}` === key)
return field == null ? [] : [field]
})
return parent ? ordered : [
...ordered.filter (field => field.key === 'thumbnail'),
...ordered.filter (field => field.key !== 'thumbnail')]
}
export const reorderCountFields = (
fields: CountField[],
params: URLSearchParams,
parent: string,
from: string,
to: string,
): string => {
const ordered = orderedCountFields (fields, params, parent)
const first = ordered.findIndex (field => field.key === from)
const last = ordered.findIndex (field => field.key === to)
if (first < 0 || last < 0) return params.get ('order') ?? ''
const prefix = parent ? `${parent}.` : ''
const current = ordered.map (field => `${prefix}${field.key}`)
const next = [...current]
next.splice (last, 0, ...next.splice (first, 1))
return [...(params.get ('order') ?? '').split (' ').filter (
key => key && !current.includes (key)), ...next].join (' ')
}
export const selectedCountColumns = (
resource: CountsResource,
params: URLSearchParams,
definition?: CountsDefinition,
): CountColumn[] => {
const resolved = normaliseCountsParams (resource, params, definition)
const common = ['resource_id', 'version_no', 'event_type', 'user', 'created_at']
const defaults: Record<CountsResource, string[]> = {
posts: ['title', 'url', 'tags', 'parents', 'video_ms'],
tags: ['name', 'category', 'aliases', 'parents'],
nico_tags: ['name', 'linked_tags'],
tag_descriptions: ['language_code', 'body', 'source_language_code'],
tag_groups: ['name', 'description', 'members'],
materials: ['url', 'source_kind', 'tag_id'] }
const selected = resolved.has ('columns') ? resolved.get ('columns')!.split (' ')
: [...common, ...defaults[resource]]
return orderedCountFields (fieldsForCounts (resource, definition), resolved)
.map (field => field.key as CountColumn).filter (key => selected.includes (key))
}
export const countChildPaths = (fields: CountField[], prefix = ''): string[] =>
fields.flatMap (field => {
const path = prefix ? `${prefix}.${field.key}` : field.key
return [path, ...countChildPaths (field.children ?? [], path)]
})
export const selectedCountChildren = (params: URLSearchParams): Set<string> => {
const selected = new Set (params.has ('children') ? params.get ('children')!.split (' ')
: ['tags.name', 'parents.id', 'aliases.name', 'members.tagId'])
const clean = (fields: CountField[], prefix: string) => {
for (const field of fields)
{
const path = `${prefix}.${field.key}`
if (field.children == null) continue
clean (field.children, path)
if (!field.children.some (child => selected.has (`${path}.${child.key}`)))
selected.delete (path)
}
}
for (const [path, fields] of Object.entries (nested)) clean (fields ?? [], path)
return selected
}
export const readCountSort = (value: string | null): CountSort[] =>
(value ?? '').split (' ').flatMap (entry => {
const [key, direction] = entry.split (':')
return (key && (direction === 'asc' || direction === 'desc'))
? [{ key, direction }] : []
})
export const writeCountSort = (sort: CountSort[]): string =>
sort.map (item => `${item.key}:${item.direction}`).join (' ')
export const changeCountSort = (
sort: CountSort[],
key: string,
additive: boolean,
): CountSort[] => {
const previous = sort.find (item => item.key === key)
if (!additive)
return [{ key, direction: previous?.direction === 'asc' ? 'desc' : 'asc' }]
if (previous == null)
return [...sort, { key, direction: 'asc' }]
return previous.direction === 'desc'
? sort.filter (item => item.key !== key)
: sort.map (item => item.key === key ? { key, direction: 'desc' } : item)
}
export const selectedCountFields = (
fields: CountField[],
params: URLSearchParams,
selectionKey: string,
parent = '',
): CountField[] => {
const selected = params.has (selectionKey)
? new Set (params.get (selectionKey)!.split (' ')) : null
return orderedCountFields (fields, params, parent)
.filter (field => selected == null || selected.has (field.key))
}
export const defaultCountsMode = (definition: string | null): 'summary' | 'details' =>
definition === 'history' ? 'details' : 'summary'
export const normaliseCountsParams = (
resource: CountsResource,
source: URLSearchParams,
definition?: CountsDefinition,
): URLSearchParams => {
const params = new URLSearchParams (source)
if (!params.has ('definition'))
params.set ('definition', params.get ('basis') === 'history' ? 'history' : 'latest')
if (!params.has ('current')) params.set ('current', params.get ('at') ? '0' : '1')
const legacy = params.get ('mode')
if (['state', 'trend', 'snapshot'].includes (legacy ?? ''))
params.set ('mode', legacy === 'snapshot' ? 'details' : 'summary')
if (!params.has ('mode')) params.set ('mode', defaultCountsMode (params.get ('definition')))
if (!params.has ('rows') && params.has ('axis'))
{
const field = params.get ('axis') === 'month' ? 'created_at' : params.get ('axis')!
params.set ('rows', encodeCountState ('rows', [{
field, ...(field === 'created_at' ? { grain: 'month' } : { }) }]))
}
if (params.has ('before') && !params.has ('through'))
params.set ('through', params.get ('before')!)
for (const key of ['basis', 'axis', 'latest', 'grain', 'before', 'summary_columns'])
params.delete (key)
if (params.has ('columns') && ['posts', 'tags'].includes (resource))
{
const selected = new Set (params.get ('columns')!.split (' '))
params.set ('columns', columnsForCounts (resource).filter (key => selected.has (key))
.join (' '))
}
if (params.has ('sort'))
params.set ('sort', params.get ('sort')!.split (' ')
.filter (entry => entry.split (':')[0] !== 'original_created_at').join (' '))
if (params.has ('order'))
params.set ('order', params.get ('order')!.split (' ')
.filter (key => key !== 'original_created_at').join (' '))
const defaults = definition?.defaults?.find (entry =>
entry.definition === params.get ('definition') && entry.mode === params.get ('mode'))
for (const { key, value } of defaults?.params ?? [])
{
if (params.has (key)) continue
if (key === 'sort')
{
const expected = defaults?.params.find (entry => entry.key === 'rows')?.value
const axes = readCountState<CountsAxis[]> (params, 'rows', [])
const first = expected == null ? null : readCountState<CountsAxis[]> (new URLSearchParams ({ rows: expected }), 'rows', [])[0]
if (!Array.isArray (axes) || axes[0]?.field !== first?.field
|| (axes[0]?.grain ?? 'month') !== (first?.grain ?? 'month')) continue
}
params.set (key, value)
}
const children = selectedCountChildren (params)
if (params.has ('children')) params.set ('children', [...children].filter (Boolean).join (' '))
if (params.has ('columns'))
{
const fields = fieldsForCounts (resource, definition)
params.set ('columns', params.get ('columns')!.split (' ').filter (key => {
const field = fields.find (field => field.key === key)
return field != null && (field.children == null
|| field.children.some (child => children.has (`${key}.${child.key}`)))
}).join (' '))
}
return params
}
export const isCountsState = (params: URLSearchParams): boolean =>
params.get ('definition') !== 'history'
export const countsUseLatest = (params: URLSearchParams): boolean =>
params.get ('current') !== '0'
export const countsRequestParams = (params: URLSearchParams): URLSearchParams => {
const result = new URLSearchParams (params)
for (const key of ['columns', 'summary_columns', 'children', 'child_sort',
'nested_json', 'order', 'basis', 'axis', 'latest', 'grain',
'current', 'row_numbers'])
result.delete (key)
if (isCountsState (params) && countsUseLatest (params)) result.delete ('at')
return result
}
export const fetchCounts = (
resource: CountsResource,
params: URLSearchParams,
locale: LocaleCode,
): Promise<CountsResult> => apiGet ('/counts/query', {
locale, params: { ...Object.fromEntries (countsRequestParams (params)), resource } })
export const exportCounts = (
resource: CountsResource,
params: URLSearchParams,
locale: LocaleCode,
format: 'json' | 'csv',
columns: string[],
headers: string[],
labels: Record<string, string>,
onError?: () => void,
): Promise<void> => apiDownload ('/counts/export', countsExportPayload (
resource, params, format, columns, headers, labels), locale, onError)
export const countsExportPayload = (
resource: CountsResource,
params: URLSearchParams,
format: 'json' | 'csv',
columns: string[],
headers: string[],
labels: Record<string, string>,
definition?: CountsDefinition,
): Record<string, string> => {
const payload = new URLSearchParams (params)
if (!payload.has ('children'))
{
const selected = selectedCountChildren (params)
payload.set ('children', countChildPaths (fieldsForCounts (resource, definition))
.filter (path => selected.has (path)).join (' '))
}
payload.delete ('current')
payload.delete ('row_numbers')
if (isCountsState (params) && countsUseLatest (params)) payload.delete ('at')
return { ...Object.fromEntries (payload), resource, format,
columns: columns.join (' '), headers: JSON.stringify (headers),
labels: JSON.stringify (labels) }
}
const stableValue = (value: unknown): unknown => {
if (Array.isArray (value)) return value.map (stableValue)
if (value != null && typeof value === 'object')
{
return Object.fromEntries (Object.entries (value)
.sort (([a], [b]) => a < b ? -1 : (a > b ? 1 : 0))
.map (([key, entry]) => [key, stableValue (entry)]))
}
return value
}
const compareValue = (first: unknown, second: unknown): number => {
if (Array.isArray (first) && Array.isArray (second) && first.length !== second.length)
return first.length - second.length
if (typeof first === 'number' && typeof second === 'number') return first - second
if (typeof first === 'boolean' && typeof second === 'boolean')
return Number (first) - Number (second)
const left = typeof first === 'object' ? JSON.stringify (stableValue (first)) : String (first)
const right = typeof second === 'object'
? JSON.stringify (stableValue (second)) : String (second)
return left < right ? -1 : (left > right ? 1 : 0)
}
export const sortedCountChildren = (
rows: CountRecord[],
path: string,
params: URLSearchParams,
valueFor: (row: CountRecord, key: string) => unknown = (row, key) => row[key],
): CountRecord[] => {
const sort = readCountSort (params.get ('child_sort'))
.filter (item => item.key.startsWith (`${path}/`))
return rows.map ((row, index) => ({ row, index })).sort ((first, second) => {
for (const item of sort)
{
const key = item.key.slice (path.length + 1)
const left = valueFor (first.row, key)
const right = valueFor (second.row, key)
if (left == null && right != null) return 1
if (left != null && right == null) return -1
if (left == null && right == null) continue
const compared = compareValue (left, right)
if (compared !== 0) return item.direction === 'asc' ? compared : -compared
}
return first.index - second.index
}).map (item => item.row)
}
export const countsCSV = (rows: string[][]): string =>
'\uFEFF' + rows.map (row => row.map (value => {
const safe = /^[=+@\-\t\r]/.test (value) ? `'${value}` : value
return `"${safe.replaceAll ('"', '""')}"`
}).join (',')).join ('\r\n')
+127
ファイルの表示
@@ -0,0 +1,127 @@
export type CountsClipboard = { text: string; html: string }
export type CountsClipboardColumn = { label: string; children?: CountsClipboardColumn[] }
export type CountsClipboardCell = string | CountsClipboardTable
export type CountsClipboardTable = {
columns: CountsClipboardColumn[]; rows: CountsClipboardCell[][]; headerRows?: string[][] }
const escapeHTML = (value: string): string => value.replace (/[&<>"']/g, character =>
({ '&': '&amp;', '<': '&lt;', '>': '&gt;', '"': '&quot;', "'": '&#39;' })[character]!)
const columnWidth = (column: CountsClipboardColumn): number =>
column.children?.reduce ((sum, child) => sum + columnWidth (child), 0) || 1
const headerDepth = (columns: CountsClipboardColumn[]): number =>
columns.length === 0 ? 0 : 1 + Math.max (0, ...columns.map (column =>
headerDepth (column.children ?? [])))
const headerRows = (columns: CountsClipboardColumn[]): string[][] => {
const width = columns.reduce ((sum, column) => sum + columnWidth (column), 0)
const rows = Array.from ({ length: headerDepth (columns) },
() => Array<string> (width).fill (''))
const place = (fields: CountsClipboardColumn[], row: number, offset: number) => {
for (const field of fields)
{
rows[row][offset] = field.label
if (field.children?.length) place (field.children, row + 1, offset)
offset += columnWidth (field)
}
}
if (columns.length) place (columns, 0, 0)
return rows
}
const dataRows = (
table: CountsClipboardTable,
columns = table.columns,
): string[][] => {
if (columns.length === 0) return []
return table.rows.flatMap (record => {
const blocks = columns.map ((column, index) => {
const value = record[index] ?? ''
return typeof value === 'string' ? [[value]]
: dataRows (value, column.children ?? value.columns)
})
const height = Math.max (1, ...blocks.map (block => block.length))
// Sibling arrays share the parent row's height; they never form a Cartesian product.
return Array.from ({ length: height }, (_, row) => columns.flatMap ((column, index) =>
Array.from ({ length: columnWidth (column) }, (_, offset) =>
blocks[index][row]?.[offset] ?? '')))
})
}
export const countsClipboard = (
source: string[][] | CountsClipboardTable,
headers = false,
): CountsClipboard => {
const headings = (!Array.isArray (source) && headers)
? source.headerRows ?? headerRows (source.columns) : []
const rows = Array.isArray (source) ? source : [...headings, ...dataRows (source)]
const headingCount = Array.isArray (source) ? (headers ? 1 : 0) : headings.length
// Excel accepts quoted TSV fields only when delimiters or quotes require them.
// Prefix formula-like text in both representations, including leading whitespace.
const safe = rows.map (row => row.map (value => /^\s*[=+@-]/.test (value)
? `'${value}` : value))
const text = safe.map (row => row.map (value => /[\t\r\n"]/.test (value)
? `"${value.replaceAll ('"', '""')}"` : value).join ('\t')).join ('\r\n')
const html = '<html><body><table>' + safe.map ((row, index) => '<tr>' + row.map (value => {
const tag = index < headingCount ? 'th' : 'td'
// Preserve time strings, leading zeroes, and user text instead of inferring dates/formulae.
const numeric = value.length <= 15 && /^(?:0|[1-9]\d*)(?:\.\d+)?$/.test (value)
const style = numeric ? 'white-space:pre-wrap' : 'white-space:pre-wrap;mso-number-format:"\\@"'
return `<${tag} style="${escapeHTML (style)}">${escapeHTML (value)
.replace (/\r\n|\r|\n/g, '<br>')}</${tag}>`
}).join ('') + '</tr>').join ('') + '</table></body></html>'
return { text, html }
}
export const writeCountsClipboard = async (payload: CountsClipboard): Promise<void> => {
if (navigator.clipboard?.write != null && typeof ClipboardItem !== 'undefined')
{
try
{
await navigator.clipboard.write ([new ClipboardItem ({
'text/plain': new Blob ([payload.text], { type: 'text/plain' }),
'text/html': new Blob ([payload.html], { type: 'text/html' }) })])
return
}
catch
{
// Browsers without HTML clipboard support may still permit plain TSV.
}
}
if (navigator.clipboard?.writeText != null)
{
try
{
await navigator.clipboard.writeText (payload.text)
return
}
catch
{
// Try the synchronous copy event when clipboard permission is unavailable.
}
}
// The synchronous copy event fallback is also used on non-secure local origins.
const listener = (event: ClipboardEvent) => {
event.preventDefault ()
event.stopImmediatePropagation ()
event.clipboardData?.setData ('text/plain', payload.text)
event.clipboardData?.setData ('text/html', payload.html)
}
document.addEventListener ('copy', listener, true)
try
{
if (!document.execCommand ('copy')) throw new Error ('Clipboard unavailable')
}
finally
{
document.removeEventListener ('copy', listener, true)
}
}
+77
ファイルの表示
@@ -0,0 +1,77 @@
import { createContext, useContext } from 'react'
import { translateUIMessage } from '@/lib/i18n'
import { apiGet } from '@/lib/api'
import type { LocaleCode } from '@/lib/locales'
import type { CountField, CountsResource } from '@/lib/counts'
export type CountsAxis = { field: string; grain?: string }
export type CountsCondition = {
field: string; operator: string; value?: string; end?: string; blank?: boolean }
export type CountsFieldType = 'string' | 'integer' | 'datetime' | 'enum' | 'json' | 'boolean'
export const countsResources: CountsResource[] = [
'posts', 'tags', 'nico_tags', 'tag_descriptions', 'tag_groups', 'materials']
export type CountsFieldMetadata = {
key: string; type: CountsFieldType; nullable: boolean; operators: string[] }
export type CountsDefinition = {
fields: CountsFieldMetadata[]
defaults?: { definition: string; mode: string; params: { key: string; value: string }[] }[]
axes: { latest: string[]; history: string[] }
grains: string[] }
export const CountsDefinitionContext = createContext<CountsDefinition | null> (null)
export const useCountsDefinition = (): CountsDefinition => {
const definition = useContext (CountsDefinitionContext)
if (definition == null) throw new Error ('Missing Counts definition')
return definition
}
export const fetchCountsDefinition = (resource: CountsResource): Promise<CountsDefinition> =>
apiGet (`/counts/definitions/${encodeURIComponent (resource)}`)
export const countsFilterFields = (
definition: CountsDefinition,
): Record<string, CountsFieldType> => Object.fromEntries (
definition.fields.map (field => [field.key, field.type]))
export const countsFieldLabel = (key: string): string => {
const labels: Record<string, string> = {
resource_id: 'columns.resourceId', version_no: 'columns.version',
user: 'history:actor', created_at: 'columns.changedAt', name: 'history:name',
title: 'columns.postTitle', url: 'columns.postURL', thumbnail_base: 'columns.thumbnailURL',
tags: 'columns.tags', tags_json: 'columns.tags', names: 'columns.names',
aliases: 'tags:aliases', category: 'common:categoryLabel',
language_code: 'columns.language', source_language_code: 'attributes.sourceLanguage',
changed_field: 'changedField' }
return labels[key] ?? `attributes.${key}`
}
export const countsEventTypes = ['create', 'update', 'discard', 'restore'] as const
export const countsAxisLabel = (
axis: CountsAxis,
definition: CountsDefinition,
locale: LocaleCode,
): { text: string; language: string } => {
const field = translateUIMessage (countsFieldLabel (axis.field), 'counts', locale)
if (definition.fields.find (item => item.key === axis.field)?.type !== 'datetime') return field
return translateUIMessage ('axisWithGrain', 'counts', locale, {
field: field.text,
grain: translateUIMessage (`grains.${axis.grain ?? 'month'}`, 'counts', locale).text })
}
export const countsAxisFields = (
definition: CountsDefinition,
history: boolean,
): CountField[] => definition.axes[history ? 'history' : 'latest']
.map (key => ({ key, label: countsFieldLabel (key) }))
+411
ファイルの表示
@@ -0,0 +1,411 @@
import { fieldsForCounts, orderedCountFields, selectedCountChildren,
sortedCountChildren } from '@/lib/counts'
import { countsEventTypes, countsFieldLabel } from '@/lib/countsDefinitions'
import { translateUIMessage } from '@/lib/i18n'
import { localisedDateString, msToTime } from '@/lib/utils'
import type { CountColumn, CountField, CountRecord, CountsResult,
CountsResource, CountsVersion } from '@/lib/counts'
import type { CountsClipboardCell, CountsClipboardColumn, CountsClipboardTable }
from '@/lib/countsClipboard'
import type { CountsAxis, CountsDefinition } from '@/lib/countsDefinitions'
import type { LocaleCode } from '@/lib/locales'
export const countsCellText = (
row: CountsVersion,
column: CountColumn,
locale: LocaleCode,
params = new URLSearchParams (),
): string => {
const timezone = params.get ('timezone') ?? 'UTC'
const date = (value: string) => localisedDateString (value, locale, null, timezone)
switch (column)
{
case 'resource_id': return String (row.resourceId)
case 'version_no': return String (row.versionNo)
case 'is_latest':
return row.isLatest ? translateUIMessage ('latestValue', 'counts', locale).text : ''
case 'title': return row.title ?? ''
case 'url': return row.url ?? ''
case 'thumbnail':
case 'thumbnail_base': return row.thumbnailBase ?? ''
case 'name': return row.name ?? ''
case 'category':
return translateUIMessage (`common:categories.${row.category}`, 'counts', locale).text
case 'video_ms': return row.videoMs == null ? '' : msToTime (row.videoMs)
case 'original_created_from':
case 'original_created_before':
{
const value = column === 'original_created_from'
? row.originalCreatedFrom : row.originalCreatedBefore
return value == null ? '' : date (value)
}
case 'deprecated':
return row.deprecated == null ? '' : date (row.deprecated)
case 'user':
if (row.createdByUser == null)
return translateUIMessage ('bot', 'counts', locale).text
return row.createdByUser.name
? translateUIMessage ('actorWithId', 'counts', locale, {
name: row.createdByUser.name, id: row.createdByUser.id }).text
: translateUIMessage ('history:anonymous', 'counts', locale, {
id: row.createdByUser.id }).text
case 'created_at': return date (row.createdAt)
default:
{
const key = column.replace (/_([a-z])/g, (_, letter: string) => letter.toUpperCase ())
const value = row[key]
if (['discarded_at', 'updated_at'].includes (column) && typeof value === 'string')
return date (value)
if (column === 'event_type')
return translateUIMessage (`events.${value}`, 'counts', locale).text
if (column === 'language_code' || column === 'source_language_code')
return countsLogicalName ({ languageCode: value }, 'languageCode', locale).text
return value == null ? '' : (typeof value === 'object'
? JSON.stringify (value, null, 2) : String (value))
}
}
}
export const countsNestedRows = (row: CountsVersion, column: string): CountRecord[] => {
const key = column.replace (/_([a-z])/g, (_, letter: string) => letter.toUpperCase ())
const value = row[key]
if (column === 'export_paths_json' && value != null && typeof value === 'object')
return Object.entries (value).map (([profile, path]) => ({ profile, path }))
return Array.isArray (value) ? value as CountRecord[] : []
}
export const countsCSVCell = (
row: CountsVersion,
column: CountColumn,
resource: CountsResource,
params: URLSearchParams,
locale: LocaleCode,
): string => {
const field = fieldsForCounts (resource).find (field => field.key === column)!
if (field.children != null)
{
const rows = countsNestedRows (row, column)
return JSON.stringify (countsNestedExport (rows, field.children, column, params, locale))
}
return countsCellText (row, column, locale, params)
}
const countsSectionRanges = (value: unknown, locale: LocaleCode): string[] | null => {
if (!Array.isArray (value)) return null
const time = (ms: number): string => {
const parts = msToTime (ms).split (':')
if (parts.length === 2) parts[0] = parts[0].padStart (2, '0')
return parts.join (':')
}
return value.map ((section: { beginMs: number; endMs: number | null }) =>
translateUIMessage ('sections.range', 'counts', locale, {
begin: time (section.beginMs),
end: section.endMs == null ? '' : time (section.endMs) }).text)
}
const displayNames = new Map<string, Intl.DisplayNames> ()
export const countsLogicalName = (
row: CountRecord,
field: string,
locale: LocaleCode,
): { text: string; language?: string } => {
const value = row[field]
if (value == null) return { text: '' }
if (field === 'sections')
return { text: (countsSectionRanges (value, locale) ?? []).join (
translateUIMessage ('sections.separator', 'counts', locale).text) }
const code = String (value)
if (field === 'languageCode' || field === 'scriptCode')
{
if (field === 'languageCode' && ['ja', 'ain'].includes (code))
return translateUIMessage (`history:languageNames.${code}`, 'counts', locale)
const master = row[field === 'languageCode' ? 'languageName' : 'scriptName']
if (typeof master === 'string' && master.trim () !== ''
&& master.toLowerCase () !== code.toLowerCase ())
return { text: master }
try
{
const type = field === 'languageCode' ? 'language' : 'script'
if (!displayNames.has (type))
displayNames.set (type, new Intl.DisplayNames (['ja'], { type, fallback: 'none' }))
const text = displayNames.get (type)!.of (code)
if (text != null && text !== code) return { text, language: 'ja' }
}
catch
{
// Invalid or unregistered codes have no logical label; never expose the code as a name.
}
return translateUIMessage ('notRegistered', 'counts', locale)
}
if (field === 'kind' || field === 'category')
return translateUIMessage (field === 'kind' ? `kinds.${code}`
: `common:categories.${code}`, 'counts', locale)
if (typeof value === 'boolean')
return translateUIMessage (`boolean.${value}`, 'counts', locale)
return { text: String (value) }
}
export const countsNestedOrder = (
rows: CountRecord[],
path: string,
params: URLSearchParams,
locale: LocaleCode,
): CountRecord[] => sortedCountChildren (rows, path, params, (row, key) =>
['languageCode', 'scriptCode'].includes (key)
? countsLogicalName (row, key, locale).text : row[key])
export const countsNestedExport = (
rows: CountRecord[],
fields: CountField[],
path: string,
params: URLSearchParams,
locale: LocaleCode,
): unknown => {
const selected = selectedCountChildren (params)
const columns = orderedCountFields (fields, params, path)
.filter (field => selected.has (`${path}.${field.key}`))
if (columns.length === 0) return rows.map (() => ({ }))
return countsNestedOrder (rows, path, params, locale).map (row =>
Object.fromEntries (columns.map (field => [
translateUIMessage (field.label, 'counts', locale).text,
field.children
? countsNestedExport (row[field.key] as CountRecord[] ?? [], field.children,
`${path}.${field.key}`, params, locale)
: (field.key === 'sections'
? countsSectionRanges (row[field.key], locale)
: countsLogicalName (row, field.key, locale).text)])))
}
// A chain with one selected scalar leaf is displayed and copied as one cell.
export const countsNestedIsScalar = (
fields: CountField[],
path: string,
params: URLSearchParams,
): boolean => {
const selected = selectedCountChildren (params)
const columns = fields.filter (field => selected.has (`${path}.${field.key}`))
if (columns.length !== 1) return false
const field = columns[0]
return field.children == null
|| countsNestedIsScalar (field.children, `${path}.${field.key}`, params)
}
export const countsNestedScalarText = (
rows: CountRecord[],
fields: CountField[],
path: string,
params: URLSearchParams,
locale: LocaleCode,
): string => {
const selected = selectedCountChildren (params)
const field = fields.find (field => selected.has (`${path}.${field.key}`))
if (field == null) return ''
return countsNestedOrder (rows, path, params, locale).map (row => field.children == null
? countsLogicalName (row, field.key, locale).text
: countsNestedScalarText (row[field.key] as CountRecord[] ?? [], field.children,
`${path}.${field.key}`, params, locale)).join (' ')
}
export const countsNestedText = (
rows: CountRecord[],
fields: CountField[],
path: string,
params: URLSearchParams,
locale: LocaleCode,
): string => {
if (countsNestedIsScalar (fields, path, params))
return countsNestedScalarText (rows, fields, path, params, locale)
const selected = selectedCountChildren (params)
const columns = orderedCountFields (fields, params, path)
.filter (field => selected.has (`${path}.${field.key}`))
if (columns.length === 0)
return `${translateUIMessage ('common:countLabel', 'counts', locale).text}: ${rows.length}`
return countsNestedOrder (rows, path, params, locale).map (row => columns.map (field => {
const value = field.children
? countsNestedText (row[field.key] as CountRecord[] ?? [], field.children,
`${path}.${field.key}`, params, locale)
: countsLogicalName (row, field.key, locale).text
return `${translateUIMessage (field.label, 'counts', locale).text}: ${value}`
}).join (' | ')).join ('\n')
}
export const countsCellCopy = (
row: CountsVersion,
field: CountField,
params: URLSearchParams,
locale: LocaleCode,
): string => {
if (field.children == null) return countsCellText (row, field.key as CountColumn, locale, params)
const rows = countsNestedRows (row, field.key)
if (countsNestedIsScalar (field.children, field.key, params))
return countsNestedScalarText (rows, field.children, field.key, params, locale)
return params.get ('nested_json') === '1'
? JSON.stringify (
countsNestedExport (rows, field.children, field.key, params, locale))
: countsNestedText (rows, field.children, field.key, params, locale)
}
// Historical URLs are displayed without fetching current attachments or metadata.
export const countsImageURL = (value: string | null | undefined): string | null => {
if (!value) return null
try
{
const url = new URL (value)
return (['https:', 'http:'].includes (url.protocol) && !url.username && !url.password)
? url.href : null
}
catch
{
return null
}
}
export const countsExportLabels = (
resource: CountsResource,
locale: LocaleCode,
definition?: CountsDefinition,
): Record<string, string> => {
const aliases: Record<string, string> = { versionNo: 'version_no',
languageCode: 'language', scriptCode: 'script', autoGenerated: 'auto_generated' }
const labels: Record<string, string> = { }
const visit = (fields: CountField[]) => {
for (const field of fields)
{
labels[aliases[field.key] ?? field.key] = translateUIMessage (
field.label, 'counts', locale).text
if (field.children) visit (field.children)
}
}
visit (fieldsForCounts (resource, definition))
for (const key of ['internal', 'external'])
labels[`kind:${key}`] = translateUIMessage (`kinds.${key}`, 'counts', locale).text
for (const key of ['deerjikist', 'meme', 'character', 'general', 'material', 'meta'])
labels[`category:${key}`] = translateUIMessage (
`common:categories.${key}`, 'counts', locale).text
for (const key of ['true', 'false'])
labels[`boolean:${key}`] = translateUIMessage (`boolean.${key}`, 'counts', locale).text
for (const key of ['ja', 'ain'])
labels[`language:${key}`] = translateUIMessage (
`history:languageNames.${key}`, 'counts', locale).text
labels.id = translateUIMessage ('columns.resourceId', 'counts', locale).text
labels.name = translateUIMessage ('history:name', 'counts', locale).text
for (const event of countsEventTypes)
labels[`event_type:${event}`] = translateUIMessage (`events.${event}`, 'counts', locale).text
labels.subtotal = translateUIMessage ('subtotal', 'counts', locale).text
labels.singleTotal = translateUIMessage ('singleTotal', 'counts', locale).text
labels.grandTotal = translateUIMessage ('grandTotal', 'counts', locale).text
labels.bot = translateUIMessage ('bot', 'counts', locale).text
labels.actorFormat = translateUIMessage ('actorWithId', 'counts', locale, {
name: '{{name}}', id: '{{id}}' }).text
labels.latest = translateUIMessage ('latestValue', 'counts', locale).text
labels.notRegistered = translateUIMessage ('notRegistered', 'counts', locale).text
return labels
}
const clipboardColumns = (
fields: CountField[],
path: string,
params: URLSearchParams,
locale: LocaleCode,
): CountsClipboardColumn[] => {
const selected = selectedCountChildren (params)
return orderedCountFields (fields, params, path)
.filter (field => selected.has (`${path}.${field.key}`))
.map (field => ({ label: translateUIMessage (field.label, 'counts', locale).text,
children: (field.children == null
|| countsNestedIsScalar (field.children, `${path}.${field.key}`, params))
? undefined : clipboardColumns (
field.children, `${path}.${field.key}`, params, locale) }))
}
export const countsNestedClipboardTable = (
rows: CountRecord[],
fields: CountField[],
path: string,
params: URLSearchParams,
locale: LocaleCode,
): CountsClipboardTable => {
const selected = selectedCountChildren (params)
const columns = orderedCountFields (fields, params, path)
.filter (field => selected.has (`${path}.${field.key}`))
return {
columns: clipboardColumns (fields, path, params, locale),
rows: countsNestedOrder (rows, path, params, locale).map (row => columns.map (field =>
field.children == null ? countsLogicalName (row, field.key, locale).text
: (countsNestedIsScalar (field.children, `${path}.${field.key}`, params)
? countsNestedScalarText (row[field.key] as CountRecord[] ?? [],
field.children, `${path}.${field.key}`, params, locale)
: countsNestedClipboardTable (row[field.key] as CountRecord[] ?? [],
field.children, `${path}.${field.key}`, params, locale)))) }
}
export const countsDetailClipboardTable = (
rows: CountsVersion[],
fields: CountField[],
params: URLSearchParams,
locale: LocaleCode,
rowStart: number | null = null,
): CountsClipboardTable => {
const columns = fields.map (field => ({
label: translateUIMessage (field.label, 'counts', locale).text,
children: (field.children == null
|| countsNestedIsScalar (field.children, field.key, params))
? undefined : clipboardColumns (
field.children, field.key, params, locale) }))
if (rowStart != null)
columns.unshift ({ label: translateUIMessage ('rowNumberHeading', 'counts', locale).text,
children: undefined })
return { columns, rows: rows.map ((row, index) => {
const cells: CountsClipboardCell[] = fields.map (field => field.children == null
? countsCellText (row, field.key, locale, params)
: (countsNestedIsScalar (field.children, field.key, params)
? countsNestedScalarText (countsNestedRows (row, field.key), field.children,
field.key, params, locale)
: countsNestedClipboardTable (countsNestedRows (row, field.key), field.children,
field.key, params, locale)))
return rowStart == null ? cells : [String (rowStart + index), ...cells]
}) }
}
export const countsSummaryValue = (
value: string | number | null,
axis: CountsAxis,
index: number,
data: CountsResult,
locale: LocaleCode,
): string => {
const message = (key: string, values?: Record<string, string | number>) =>
translateUIMessage (key, 'counts', locale, values).text
if (value == null) return axis.field === 'user' ? message ('bot') : ''
const logical = data.axisLabels?.[`a${index}`]?.[String (value)]
if (axis.field === 'user') return message ('actorWithId', {
name: logical || message ('notRegistered'), id: value })
if (logical != null) return logical
if (['tags', 'language_code', 'source_language_code'].includes (axis.field))
return message ('notRegistered')
if (axis.field === 'changed_field') return message (countsFieldLabel (String (value)))
if (axis.field === 'event_type') return message (`events.${value}`)
if (axis.field === 'category' || axis.field === 'tag_category')
return message (`common:categories.${value}`)
return String (value)
}
+94
ファイルの表示
@@ -0,0 +1,94 @@
import { isCountsState, selectedCountFields } from '@/lib/counts'
import { translateUIMessage } from '@/lib/i18n'
import type { CountField, CountsResource, CountsResult } from '@/lib/counts'
import type { LocaleCode } from '@/lib/locales'
export type CountsSummaryRow = {
key: string | number | null
name: string | null
count: number | null
kind?: 'internal' | 'external'
tagId?: number
deprecated?: string | null
at?: string }
export const countsSummaryFields = (
resource: CountsResource,
params: URLSearchParams,
): CountField[] => (isCountsState (params) && params.get ('axis') === 'tags')
? [{ key: 'kind', label: 'counts:columns.kind' },
{ key: 'tag_id', label: 'counts:columns.resourceId' },
{ key: 'name', label: 'history:tagName' },
{ key: 'deprecated', label: 'counts:columns.deprecated' },
{ key: 'count', label: 'counts:state.postCount' }]
: [{ key: 'group', label: `counts:axes.${params.get ('axis') ?? 'month'}` },
{ key: 'count', label: isCountsState (params)
? `counts:state.totals.${resource}` : 'counts:metric' }]
export const selectedCountsSummaryFields = (
resource: CountsResource,
params: URLSearchParams,
): CountField[] => selectedCountFields (
countsSummaryFields (resource, params), params, 'summary_columns', 'summary')
export const countsSummaryRows = (data: CountsResult): CountsSummaryRow[] =>
data.stateRows != null
? data.stateRows.map (row => ({ ...row, key: `${row.kind}:${row.tagId}` }))
: data.groups ?? []
export const countsSummaryCell = (
row: CountsSummaryRow,
key: string,
locale: LocaleCode,
): { text: string; language?: string } => {
switch (key)
{
case 'group': return row.key == null
? translateUIMessage ('bot', 'counts', locale)
: { text: row.name ?? String (row.key) }
case 'count': return { text: row.count == null ? '—' : String (row.count) }
case 'tag_id': return { text: String (row.tagId) }
case 'kind': return translateUIMessage (`kinds.${row.kind}`, 'counts', locale)
case 'name': return row.name == null
? translateUIMessage ('notRegistered', 'counts', locale) : { text: row.name }
case 'deprecated': return row.deprecated
? translateUIMessage ('common:states.deprecated', 'counts', locale) : { text: '' }
default: return { text: '' }
}
}
export const countsSummaryDrill = (
resource: CountsResource,
params: URLSearchParams,
row: CountsSummaryRow,
): string => {
const next = new URLSearchParams (params)
for (const key of ['sort', 'page']) next.delete (key)
next.set ('mode', 'details')
if (row.at) next.set ('at', row.at)
if (row.tagId != null)
{
next.set ('kind', row.kind!)
next.set ('tag', String (row.tagId))
}
else if (params.get ('axis') === 'user') next.set ('user_id', String (row.key ?? 'none'))
else
{
const first = `${row.key}-01T00:00:00Z`
const date = new Date (first)
date.setUTCMonth (date.getUTCMonth () + 1)
const last = date.toISOString ()
const from = next.get ('from')
const before = next.get ('before')
next.set ('from', (from != null && Date.parse (from) > Date.parse (first)) ? from : first)
next.set ('before',
(before != null && Date.parse (before) < Date.parse (last)) ? before : last)
}
return `/counts/${resource}?${next}`
}
+28
ファイルの表示
@@ -0,0 +1,28 @@
import { createContext, useContext } from 'react'
export const MIN_COUNT_WIDTH = 64
export const MAX_COUNT_WIDTH = 2000
export const defaultCountWidths: Record<string, number> = {
row_number: 72, thumbnail: 88, resource_id: 88, version_no: 88, is_latest: 104,
title: 300, url: 300, thumbnail_base: 300, tags: 560, parents: 180,
video_ms: 120, original_created_from: 240,
original_created_before: 240, name: 240, names: 480, category: 180,
aliases: 360, deprecated: 240, user: 200, created_at: 240, group: 240, count: 160,
period: 200, tag_id: 88, id: 88, versionNo: 88, kind: 140, sections: 260,
languageCode: 180, scriptCode: 200, autoGenerated: 120 }
type Layout = {
width: (family: string, key: string) => number
resize: (family: string, key: string, width: number) => void }
export const CountsLayoutContext = createContext<Layout | null> (null)
export const CountsTableScope = createContext ('')
export const useCountsTableLayout = (): Layout => {
const context = useContext (CountsLayoutContext)
if (context == null) throw new Error ('Counts table layout provider is missing')
return context
}
+3 -1
ファイルの表示
@@ -41,7 +41,9 @@ type DialogueFormOptions = { title: ReactNode
description?: ReactNode description?: ReactNode
body: (controls: DialogueFormControls) => ReactNode body: (controls: DialogueFormControls) => ReactNode
cancelText?: ReactNode cancelText?: ReactNode
size?: 'default' | 'large' } onCancel?: () => Promise<boolean | void> | boolean | void
footerLayout?: 'inline'
size?: 'default' | 'large' | 'viewport' }
type DialogueAPI = type DialogueAPI =
{ confirm: (options: ConfirmOptions) => Promise<boolean> { confirm: (options: ConfirmOptions) => Promise<boolean>
+8 -9
ファイルの表示
@@ -17,11 +17,6 @@ export const fetchLocales = (): Promise<{ locales: AvailableLocale[] }> =>
export const LOCALE_LABELS: Record<LocaleCode, string> = { export const LOCALE_LABELS: Record<LocaleCode, string> = {
ja: '日本語', 'ain-Kana': 'ヤウンモシルン アイヌ イタㇰ' } ja: '日本語', 'ain-Kana': 'ヤウンモシルン アイヌ イタㇰ' }
const LEGACY_ROOTS = [
'posts', 'tags', 'nico', 'theatres', 'materials', 'wiki', 'help',
'users', 'settings', 'tos', 'gekanator', 'more']
export const isLocaleCode = (value: unknown): value is LocaleCode => export const isLocaleCode = (value: unknown): value is LocaleCode =>
SUPPORTED_LOCALES.some (locale => locale === value) SUPPORTED_LOCALES.some (locale => locale === value)
@@ -73,11 +68,15 @@ export const switchLocalePath = (
`/${ locale }${ unlocalisedPath (location.pathname) }${ location.search }${ location.hash }` `/${ locale }${ unlocalisedPath (location.pathname) }${ location.search }${ location.hash }`
// Keep automatic selection independent of the page and API route trees.
export const automaticallySelectedLocale = (): LocaleCode => 'ja'
export const legacyLocalePath = ( export const legacyLocalePath = (
location: { pathname: string; search: string; hash: string }, location: { pathname: string; search: string; hash: string },
isPage: (pathname: string) => boolean = () => false,
): string | null => { ): string | null => {
const root = location.pathname.split ('/')[1] if (localeFromPath (location.pathname) != null || !isPage (location.pathname))
return (location.pathname === '/' || LEGACY_ROOTS.includes (root)) return null
? `/ja${ location.pathname }${ location.search }${ location.hash }` return switchLocalePath (location, automaticallySelectedLocale ())
: null
} }
+5 -4
ファイルの表示
@@ -1,4 +1,5 @@
import { apiDelete, apiGet, apiPost, apiPut } from '@/lib/api' import { apiDelete, apiGet, apiPost, apiPut } from '@/lib/api'
import { queryRecordParams, setQueryRecords } from '@/lib/queryState'
import { switchLocalePath } from '@/lib/locales' import { switchLocalePath } from '@/lib/locales'
import type { LocaleCode } from '@/lib/locales' import type { LocaleCode } from '@/lib/locales'
@@ -17,7 +18,7 @@ export const fetchPosts = async (
...(url && { url }), ...(url && { url }),
...(title && { title }), ...(title && { title }),
...(tags && { tags }), ...(tags && { tags }),
...(tagReferences && { tag_refs: tagReferences }), ...(tagReferences && queryRecordParams (new URLSearchParams (tagReferences), 'tag_refs')),
...(match && { match }), ...(match && { match }),
...(createdFrom && { created_from: createdFrom }), ...(createdFrom && { created_from: createdFrom }),
...(createdTo && { created_to: createdTo }), ...(createdTo && { created_to: createdTo }),
@@ -45,14 +46,14 @@ export const switchPostFilterLocale = async (
negative?: boolean negative?: boolean
missing?: boolean }[] }> ('/posts/tag_filters', { missing?: boolean }[] }> ('/posts/tag_filters', {
locale, params: { tags, target_locale: targetLocale, locale, params: { tags, target_locale: targetLocale,
tag_refs: query.get ('tag_refs') ?? undefined } }) ...queryRecordParams (query, 'tag_refs') } })
query.set ('tags', references.map (reference => reference.name).join (' ')) query.set ('tags', references.map (reference => reference.name).join (' '))
query.set ('tag_refs', JSON.stringify (references.map (reference => ({ setQueryRecords (query, 'tag_refs', references.map (reference => ({
name: reference.name, name: reference.name,
negative: reference.negative, negative: reference.negative,
...(reference.tagId != null && { tag_id: reference.tagId }), ...(reference.tagId != null && { tag_id: reference.tagId }),
...(reference.externalTagId != null && { external_tag_id: reference.externalTagId }), ...(reference.externalTagId != null && { external_tag_id: reference.externalTagId }),
...(reference.missing && { missing: true }) })))) ...(reference.missing && { missing: true }) })))
return switchLocalePath ({ ...location, search: `?${query}` }, targetLocale) return switchLocalePath ({ ...location, search: `?${query}` }, targetLocale)
} }
+3 -2
ファイルの表示
@@ -1,3 +1,4 @@
import { queryRecordParams } from '@/lib/queryState'
import { QueryClient } from '@tanstack/react-query' import { QueryClient } from '@tanstack/react-query'
import { match } from 'path-to-regexp' import { match } from 'path-to-regexp'
@@ -175,8 +176,8 @@ const prefetchPostsIndex: Prefetcher = async (qc, url, locale) => {
const keys = { const keys = {
tags, match: m, page, limit, url: qURL, title, tags, match: m, page, limit, url: qURL, title,
...(url.searchParams.get ('tag_refs') && { ...(url.searchParams.has ('tag_refs') && {
tagReferences: url.searchParams.get ('tag_refs')! }), tagReferences: new URLSearchParams (queryRecordParams (url.searchParams, 'tag_refs')).toString () }),
originalCreatedFrom, originalCreatedTo, createdFrom, createdTo, originalCreatedFrom, originalCreatedTo, createdFrom, createdTo,
updatedFrom, updatedTo, order } updatedFrom, updatedTo, order }
+5
ファイルの表示
@@ -161,3 +161,8 @@ export const theatresKeys = {
show: (id: string, locale: LocaleCode) => ['theatres', locale, id] as const, show: (id: string, locale: LocaleCode) => ['theatres', locale, id] as const,
programmes: (id: string, locale: LocaleCode) => programmes: (id: string, locale: LocaleCode) =>
['theatres', locale, id, 'programmes'] as const } ['theatres', locale, id, 'programmes'] as const }
export const countsKeys = {
definition: (resource: string) => ['counts', 'definition', resource] as const,
index: (resource: string, params: string, locale: LocaleCode) =>
['counts', resource, locale, params] as const }
+105
ファイルの表示
@@ -0,0 +1,105 @@
// Public URL grammar: space-delimited tokens and indexed scalar fields, never JSON.
export type QueryRecord = Record<string, string | number | boolean | null | undefined>
const schemas: Record<string, string[]> = {
conditions: ['field', 'operator', 'value', 'end', 'blank'],
drill: ['field', 'grain', 'value', 'value_type'],
tag_refs: ['name', 'tag_id', 'external_tag_id', 'negative', 'missing'],
column_order: ['value', 'value_type'],
measures: ['key', 'axis', 'value', 'value_type'] }
export const queryList = (value: string | null): string[] =>
value == null || value === '' ? [] : value.split (' ')
export const queryRecords = (params: URLSearchParams, key: string): QueryRecord[] => {
const indexed = new Map<number, QueryRecord> ()
for (const [name, value] of params)
{
if (!name.startsWith (`${key}.`)) continue
const parts = name.slice (key.length + 1).split ('.')
if (parts.length !== 2 || !/^\d{1,2}$/.test (parts[0])
|| !schemas[key]?.includes (parts[1])) continue
const index = Number (parts[0])
const record = indexed.get (index) ?? { }
const field = parts[1]
record[field] = ['blank', 'negative', 'missing'].includes (field)
? value === '1' : (['tag_id', 'external_tag_id', 'axis'].includes (field)
? Number (value) : value)
indexed.set (index, record)
}
return [...indexed].sort ((a, b) => a[0] - b[0]).map (([, record]) => {
if (record.value_type === 'null') record.value = null
if (record.value_type === 'number') record.value = Number (record.value)
if (record.value_type === 'boolean') record.value = record.value === 'true'
delete record.value_type
return record
})
}
export const setQueryRecords = (
params: URLSearchParams,
key: string,
records: QueryRecord[],
): void => {
for (const name of [...params.keys ()])
if (name === key || name.startsWith (`${key}.`)) params.delete (name)
params.set (key, '')
records.forEach ((record, index) => {
for (const [field, value] of Object.entries (record))
{
if (value == null && field !== 'value') continue
if (!schemas[key]?.includes (field)) continue
const name = `${key}.${index}.${field}`
if (field === 'value' && ['drill', 'measures', 'column_order'].includes (key))
params.set (`${key}.${index}.value_type`, value == null ? 'null' : typeof value)
if (value == null) continue
params.set (name, typeof value === 'boolean'
? (['blank', 'negative', 'missing'].includes (field) ? (value ? '1' : '0') : String (value))
: String (value))
}
})
}
export const readCountState = <T,> (
params: URLSearchParams,
key: string,
fallback: T,
): T => {
if (!params.has (key)) return fallback
if (key === 'rows' || key === 'cols')
return queryList (params.get (key)).map (token => {
const [field, grain] = token.split (':')
return grain == null ? { field } : { field, grain }
}) as T
return queryRecords (params, key) as T
}
export const encodeCountState = (key: string, records: QueryRecord[]): string => {
if (key === 'rows' || key === 'cols')
return records.map (record => record.grain == null
? String (record.field) : `${record.field}:${record.grain}`).join (' ')
const params = new URLSearchParams ()
setQueryRecords (params, key, records)
return params.toString ()
}
export const setCountState = (params: URLSearchParams, key: string, value: string): void => {
if (schemas[key] != null)
{
const fields = new URLSearchParams (value)
setQueryRecords (params, key, queryRecords (fields, key))
}
else params.set (key, value)
}
export const queryRecordParams = (
params: URLSearchParams,
key: string,
): Record<string, string> => Object.fromEntries ([...params].filter (([name]) =>
name === key || name.startsWith (`${key}.`)))
+22
ファイルの表示
@@ -0,0 +1,22 @@
import { useReducedMotion } from 'framer-motion'
import { useEffect, useState } from 'react'
import { clientAnimationTransition, clientMotionTokens } from '@/lib/clientAnimation'
import { useClientBehaviourSettings } from '@/lib/useClientBehaviourSettings'
export const useClientMotion = () => {
const settings = useClientBehaviourSettings ()
const osReduced = useReducedMotion ()
const [hydrated, setHydrated] = useState (false)
useEffect (() => setHydrated (true), [])
const mode = hydrated ? (osReduced ? 'reduced' : settings.animation ?? 'normal') : 'normal'
const reduced = mode !== 'normal'
const duration = (kind: keyof typeof clientMotionTokens.durationMS) =>
reduced ? 0 : clientMotionTokens.durationMS[kind]
const transition = (kind: keyof typeof clientMotionTokens.durationMS) =>
clientAnimationTransition (mode, {
normal: { duration: duration (kind) / 1000, ease: clientMotionTokens.easing },
reduced: { duration: 0 } })
return { reduced, duration, transition, easing: clientMotionTokens.cssEasing }
}
+25 -21
ファイルの表示
@@ -17,24 +17,26 @@ export const cn = (...inputs: ClassValue[]) => twMerge (clsx (...inputs))
type DatePrecision = 'month' | 'day' | 'hour' | 'minute' | 'second' | null type DatePrecision = 'month' | 'day' | 'hour' | 'minute' | 'second' | null
const dateFormatters = new Map<DatePrecision, Intl.DateTimeFormat> () const dateFormatters = new Map<string, Intl.DateTimeFormat> ()
let ainuDateFormatter: Intl.DateTimeFormat | null = null const ainuDateFormatters = new Map<string, Intl.DateTimeFormat> ()
const formatDateString = ( const formatDateString = (
date: Date, date: Date,
unknown: DatePrecision, unknown: DatePrecision,
locale: LocaleCode, locale: LocaleCode,
timeZone = 'Asia/Tokyo',
): string => { ): string => {
const key = `${timeZone}/${unknown}`
if (Number.isNaN (date.getTime ())) if (Number.isNaN (date.getTime ()))
return date.toString () return date.toString ()
if (locale === 'ja') if (locale === 'ja')
{ {
if (!dateFormatters.has (unknown)) if (!dateFormatters.has (key))
{ {
dateFormatters.set (unknown, new Intl.DateTimeFormat ('ja-JP-u-ca-japanese', { dateFormatters.set (key, new Intl.DateTimeFormat ('ja-JP-u-ca-japanese', {
era: 'long', year: 'numeric', era: 'long', year: 'numeric',
timeZone: 'Asia/Tokyo', timeZone,
month: unknown === 'month' ? undefined : 'long', month: unknown === 'month' ? undefined : 'long',
day: unknown != null && ['month', 'day'].includes (unknown) ? undefined : 'numeric', day: unknown != null && ['month', 'day'].includes (unknown) ? undefined : 'numeric',
weekday: unknown != null && ['month', 'day'].includes (unknown) ? undefined : 'short', weekday: unknown != null && ['month', 'day'].includes (unknown) ? undefined : 'short',
@@ -42,14 +44,15 @@ const formatDateString = (
minute: unknown == null || unknown === 'second' ? 'numeric' : undefined, minute: unknown == null || unknown === 'second' ? 'numeric' : undefined,
second: unknown == null ? 'numeric' : undefined })) second: unknown == null ? 'numeric' : undefined }))
} }
return dateFormatters.get (unknown)!.format (date) return dateFormatters.get (key)!.format (date)
} }
ainuDateFormatter ??= new Intl.DateTimeFormat ('en-CA', { if (!ainuDateFormatters.has (timeZone))
ainuDateFormatters.set (timeZone, new Intl.DateTimeFormat ('en-CA', {
year: 'numeric', month: '2-digit', day: '2-digit', year: 'numeric', month: '2-digit', day: '2-digit',
timeZone: 'Asia/Tokyo', timeZone,
hour: '2-digit', minute: '2-digit', second: '2-digit', hour: '2-digit', minute: '2-digit', second: '2-digit',
hourCycle: 'h23' }) hourCycle: 'h23' }))
const parts = ainuDateFormatter.formatToParts (date) const parts = ainuDateFormatters.get (timeZone)!.formatToParts (date)
const part = (type: string) => parts.find (entry => entry.type === type)?.value ?? '' const part = (type: string) => parts.find (entry => entry.type === type)?.value ?? ''
const omitDay = unknown != null && ['month', 'day'].includes (unknown) const omitDay = unknown != null && ['month', 'day'].includes (unknown)
const dateParts = [part ('year'), const dateParts = [part ('year'),
@@ -76,7 +79,8 @@ export const localisedDateString = (
value: string | Date, value: string | Date,
locale: LocaleCode, locale: LocaleCode,
unknown: DatePrecision = null, unknown: DatePrecision = null,
): string => dateString (value, unknown, locale) timeZone?: string,
): string => formatDateString (toDate (value), unknown, locale, timeZone)
export const originalCreatedAtString = ( export const originalCreatedAtString = (
@@ -91,36 +95,36 @@ export const originalCreatedAtString = (
const unknownLabel = (key: string) => i18n.t (`history:${key}`, { lng: locale }) const unknownLabel = (key: string) => i18n.t (`history:${key}`, { lng: locale })
const space = locale === 'ja' ? ' ' : '\u3000' const space = locale === 'ja' ? ' ' : '\u3000'
if (from && before) const fromParts = from == null ? null : new Date (from.getTime () + 9 * 3_600_000)
const beforeParts = before == null ? null : new Date (before.getTime () + 9 * 3_600_000)
if (from && before && fromParts && beforeParts)
{ {
const diff = before.getTime () - from.getTime () const diff = before.getTime () - from.getTime ()
if (diff <= 60_000 /* 1 分 */) if (diff <= 60_000 /* 1 分 */)
return format (from, 'second') return format (from, 'second')
if (fromParts.getUTCMinutes () === 0 && beforeParts.getUTCMinutes () === 0)
if (from.getMinutes () === 0 && before.getMinutes () === 0)
{ {
if (Math.abs (diff - 3_600_000 /* 1 時間 */) < 60_000) if (Math.abs (diff - 3_600_000 /* 1 時間 */) < 60_000)
return `${format (from, 'minute')}${space}(${unknownLabel ('unknownMinute')})` return `${format (from, 'minute')}${space}(${unknownLabel ('unknownMinute')})`
if (fromParts.getUTCHours () === 0 && beforeParts.getUTCHours () === 0)
if (from.getHours () === 0 && before.getHours () === 0)
{ {
if (Math.abs (diff - 86_400_000 /* 1 日 */) < 60_000) if (Math.abs (diff - 86_400_000 /* 1 日 */) < 60_000)
return `${format (from, 'hour')}${space}(${unknownLabel ('unknownTime')})` return `${format (from, 'hour')}${space}(${unknownLabel ('unknownTime')})`
if (fromParts.getUTCDate () === 1 && beforeParts.getUTCDate () === 1)
if (from.getDate () === 1 && before.getDate () === 1)
{ {
if (2_332_800_000 /* 27 日 */ < diff && diff < 2_764_800_000 /* 32 日 */) if (2_332_800_000 /* 27 日 */ < diff && diff < 2_764_800_000 /* 32 日 */)
return `${format (from, 'day')}${space}(${unknownLabel ('unknownDay')})` return `${format (from, 'day')}${space}(${unknownLabel ('unknownDay')})`
if (fromParts.getUTCMonth () === 0 && beforeParts.getUTCMonth () === 0
if (from.getMonth () === 0 && before.getMonth () === 0
&& (31_449_600_000 /* 364 日 */ <= diff && (31_449_600_000 /* 364 日 */ <= diff
&& diff < 31_708_800_000 /* 367 日 */)) && diff < 31_708_800_000 /* 367 日 */))
{
return `${format (from, 'month')}${space}(${unknownLabel ('unknownMonthDay')})` return `${format (from, 'month')}${space}(${unknownLabel ('unknownMonthDay')})`
} }
} }
} }
} }
}
const rtn = ([from ? format (from, 'second') : '', const rtn = ([from ? format (from, 'second') : '',
'〜', '〜',
+3
ファイルの表示
@@ -0,0 +1,3 @@
{
"_untranslated": ["*"]
}
+246
ファイルの表示
@@ -0,0 +1,246 @@
{
"titles": {
"posts": "広場の集計・明細",
"tags": "タグの集計・明細",
"nico_tags": "ニコニコタグの集計・明細",
"tag_descriptions": "タグ説明の集計・明細",
"tag_groups": "タググループの集計・明細",
"materials": "素材の集計・明細"
},
"target": "集計定義",
"run": "集計実行",
"conditions": "集計条件",
"output": "出力項目",
"filters": "抽出条件",
"mode": "表示",
"exportProgress": {
"title": "エクスポート",
"format": "形式:{{format}}",
"starting": "開始中",
"queued": "待機中",
"running": "生成中",
"cancelling": "取消処理中",
"cancelled": "取消済み",
"ready": "ファイル生成完了・ダウンロード開始(保存完了は未確認)",
"failed": "エラー",
"processed": "{{count}}件処理済み",
"measured": "{{processed}} / {{total}}件({{percent}}%)"
},
"displaySettings": "表示設定",
"summary": "集計",
"details": "明細",
"axis": "集計軸",
"axes": {
"month": "月別(UTC)",
"user": "$t(history:actor)",
"tags": "タグ別"
},
"metric": "履歴レコード件数",
"total": "履歴レコード件数:{{count}}",
"csv": "CSV",
"csvScope": "表示中のページ・選択列を CSV 出力",
"empty": "該当する履歴はありません。",
"columns": {
"changedAt": "操作日時",
"version": "版",
"name": "名称",
"resourceId": "Id.",
"postTitle": "タイトル",
"postURL": "URL",
"tags": "タグ",
"deprecated": "廃止状態",
"latest": "最新フラグ",
"thumbnailURL": "サムネール元 URL",
"originalFrom": "オリジナルの投稿日時(以降)",
"originalBefore": "オリジナルの投稿日時(より前)",
"language": "言語",
"kind": "内部・外部",
"sections": "記載区間",
"names": "言語別名称",
"thumbnailDisplay": "$t(shared:labels.thumbnail)"
},
"fields": {
"from": "開始(端末の時刻帯・以上)",
"before": "終了(端末の時刻帯・未満)",
"post": "投稿 ID",
"tag": "タグ名または ID",
"actor": "操作者 ID",
"limit": "表示件数"
},
"latestOnly": "最新のみ",
"boolean": {
"true": "はい",
"false": "いいえ"
},
"kinds": {
"internal": "内部タグ",
"external": "外部タグ"
},
"nested": {
"toggle": "表内表を展開・折り畳み",
"selection": "親を OFF にしても子項目の選択は保持します。",
"noColumns": "出力項目が選択されていません。",
"asJSON": "内表を JSON 表示"
},
"sort": {
"action": "クリックで単独ソート。Ctrl または長押しでキーを追加・方向変更・解除。NULL は末尾。",
"structured": "親表の複数値は要素数、保存スナップショットの文字列表現の順で比較します。Ctrl または長押しで複数キーソート。子表のソートは親表に影響しません。",
"label": "{{column}}:{{direction}}、優先順位 {{priority}}",
"none": "ソートなし",
"ascending": "昇順",
"descending": "降順",
"unsortedLabel": "{{column}}:ソートなし",
"singleLabel": "{{column}}:{{direction}}"
},
"notRegistered": "名称未登録",
"selection": {
"table": "集計・明細表",
"hint": "クリックでセル選択、ドラッグまたは Shift+クリックで範囲選択。タッチではセルを長押ししてドラッグ。Ctrl+C/Cmd+C でコピー。矢印キーで移動、Shift+矢印キーで範囲を拡張、Escape で解除。",
"copy": "コピー",
"copyHeaders": "見出し付きコピー",
"row": "行を選択",
"column": "列を選択",
"all": "すべて選択"
},
"resize": {
"column": "{{column}}の列幅を変更。境界をドラッグ、タッチでは長押ししてドラッグ。矢印キーでも調整できます。"
},
"latestValue": "最新",
"sections": {
"range": "{{begin}}~{{end}}",
"separator": "、"
},
"reorder": "{{column}}:出力項目の並べ替え",
"state": {
"list": "タグ別投稿件数一覧",
"trend": "タグ別投稿件数推移",
"details": "時点明細",
"history": "履歴集計",
"historyDetails": "履歴明細",
"at": "集計時点",
"boundary": "集計時点(UTC)",
"period": "時点(UTC)",
"grain": "粒度(UTC)",
"grains": {
"day": "日別",
"month": "月別",
"year": "年別"
},
"postCount": "投稿件数",
"totals": {
"posts": "投稿総数",
"tags": "タグ総数(内部タグ)",
"assignments": "タグ付け総件数"
}
},
"measure": "指標",
"export": "エクスポート",
"exportFormat": "エクスポート形式",
"exportFormats": {
"json": "JSON",
"csv": "CSV"
},
"resources": {
"posts": "広場",
"tags": "タグ",
"nico_tags": "ニコニコタグ",
"tag_descriptions": "タグ説明",
"tag_groups": "タググループ",
"materials": "素材"
},
"latest": "最新",
"history": "履歴",
"referenceAt": "基準日時",
"referenceHelp": "指定した日時における各データの最新版を集計・表示します。空欄の場合は現在の最新版を使用します。",
"from": "開始",
"through": "終了",
"axisItems": "項目一覧",
"rows": "行",
"cols": "列",
"axisPlace": "{{field}}を{{target}}に配置",
"field": "項目",
"operator": "演算子",
"add": "追加",
"remove": "削除",
"value": "値",
"blank": "空欄",
"additionalConditions": "追加条件",
"changedField": "変更項目",
"eventType": "異動区分",
"count": "件数",
"recordTotal": "閲覧可能件数:{{count}}",
"grain": "粒度",
"grains": {
"year": "年",
"quarter": "四半期",
"month": "月",
"week": "週",
"day": "日",
"hour": "時"
},
"operators": {
"eq": "等しい",
"ne": "等しくない",
"gt": "より大きい",
"lt": "未満",
"ge": "以上",
"le": "以下",
"like": "パターン",
"range": "範囲"
},
"events": {
"create": "起票",
"update": "異動",
"discard": "除票",
"restore": "恢復"
},
"attributes": {
"sourceLanguage": "翻訳元の言語",
"source_language_code": "翻訳元の言語",
"source_version_no": "翻訳元の版番号",
"event_type": "異動区分",
"body": "本文",
"linked_tags": "対応タグ",
"description": "説明",
"members": "構成要素",
"placements": "配置",
"parent_id": "親素材",
"tag_id": "タグ",
"tag_name": "タグ名称",
"tag_category": "タグカテゴリ",
"discarded_at": "破棄日時",
"updated_at": "履歴行更新日時",
"updated_by_user_id": "履歴行更新者",
"export_paths_json": "出力先",
"file_blob_id": "ファイル識別子",
"file_filename": "ファイル名",
"file_content_type": "ファイル形式",
"file_byte_size": "ファイル容量",
"file_checksum": "チェックサム",
"file_sha256": "SHA-256",
"source_kind": "取得元の種類",
"source_uri": "取得元 URI",
"source_path": "取得元のパス",
"source_file_id": "取得元ファイル識別子",
"normalized_source_key": "取得元識別キー",
"parent_post_ids": "親投稿",
"parent_tag_ids": "上位タグ",
"video_ms": "動画時間",
"original_created_from": "オリジナルの投稿日時(以降)",
"original_created_before": "オリジナルの投稿日時(より前)",
"deprecated_at": "廃止日時",
"section_name": "節の名称",
"children": "子項目",
"profile": "出力設定",
"path": "出力パス"
},
"rowNumberHeading": "No.",
"subtotal": "小計",
"singleTotal": "合計",
"grandTotal": "綜合計",
"rowNumbers": "行番号",
"periodSeparator": "〜",
"bot": "$t(history:bot)",
"actorWithId": "{{name}} (#{{id}})",
"axisWithGrain": "{{field}}({{grain}})"
}
+6
ファイルの表示
@@ -0,0 +1,6 @@
import CountsResourcePage from '@/pages/CountsResourcePage'
const CountsPage = () => <CountsResourcePage resource="posts"/>
export default CountsPage
+157
ファイルの表示
@@ -0,0 +1,157 @@
import { fireEvent, screen, waitFor } from '@testing-library/react'
import { beforeEach, describe, expect, it, vi } from 'vitest'
import CountsPage from '@/pages/CountsPage'
import CountsResourcePage from '@/pages/CountsResourcePage'
import { renderWithProviders } from '@/test/render'
import type { CountsResult } from '@/lib/counts'
const { fetchCounts, fetchTagByName } = vi.hoisted (() => ({
fetchCounts: vi.fn (), fetchTagByName: vi.fn () }))
vi.mock ('@/lib/counts', async importOriginal => ({
...await importOriginal<typeof import ('@/lib/counts')> (), fetchCounts }))
vi.mock ('@/lib/tags', async importOriginal => ({
...await importOriginal<typeof import ('@/lib/tags')> (), fetchTagByName }))
beforeEach (() => {
fetchCounts.mockReset ()
fetchTagByName.mockReset ()
fetchCounts.mockResolvedValue ({ count: 0, groupCount: 0, groups: [], page: 1, limit: 20 })
})
const conditions = () => screen.getByRole ('button', { name: '集計条件' })
const execute = () => fireEvent.submit (
screen.getByRole ('button', { name: '集計実行' }).closest ('form')!)
describe ('counts workspace', () => {
it ('opens the posts workspace at the root with a resource sidebar', async () => {
renderWithProviders (<CountsPage/>, { route: '/ja/counts' })
fireEvent.click (screen.getByRole ('button', { name: '集計対象' }))
expect (screen.getByRole ('link', { name: '投稿' }))
.toHaveAttribute ('href', '/ja/counts/posts')
expect (screen.getByRole ('link', { name: '投稿' }))
.toHaveAttribute ('aria-current', 'page')
expect (screen.getByRole ('link', { name: 'タグ' }))
.toHaveAttribute ('href', '/ja/counts/tags')
expect (conditions ()).toHaveAttribute ('aria-expanded', 'false')
await waitFor (() => expect (fetchCounts.mock.lastCall?.[1].get ('axis')).toBe ('month'))
})
it ('preserves draft filters on collapse without querying until execution', async () => {
renderWithProviders (<CountsResourcePage resource="posts"/>, {
route: '/ja/counts/posts?page=3&mode=details' })
await screen.findByText ('履歴レコード件数:0')
const calls = fetchCounts.mock.calls.length
fireEvent.click (conditions ())
const output = document.querySelector ('[data-counts-output]')!
const filters = document.querySelector ('[data-counts-filters]')!
expect (output.compareDocumentPosition (filters) & Node.DOCUMENT_POSITION_FOLLOWING)
.toBeTruthy ()
fireEvent.change (screen.getByRole ('textbox', { name: '投稿 ID' }), {
target: { value: '7' } })
fireEvent.change (screen.getByRole ('textbox', { name: '操作者 ID' }), {
target: { value: '9' } })
fireEvent.click (conditions ())
expect (screen.getByRole ('table')).toBeInTheDocument ()
fireEvent.click (conditions ())
expect (screen.getByRole ('textbox', { name: '投稿 ID' })).toHaveValue ('7')
expect (fetchCounts).toHaveBeenCalledTimes (calls)
execute ()
await waitFor (() => {
const params = fetchCounts.mock.lastCall?.[1] as URLSearchParams
expect (params.get ('resource_id')).toBe ('7')
expect (params.get ('user_id')).toBe ('9')
expect (params.has ('page')).toBe (false)
})
const refreshed = fetchCounts.mock.calls.length
execute ()
await waitFor (() => expect (fetchCounts.mock.calls.length).toBeGreaterThan (refreshed))
})
it ('applies selected columns only on execution and retains pagination state', async () => {
fetchCounts.mockResolvedValue ({ count: 21, page: 1, limit: 20,
versions: [{ postId: 7, versionNo: 2, url: { current: 'https://example.com' },
title: { current: 'Snapshot' } }] as CountsResult['versions'] })
renderWithProviders (<CountsResourcePage resource="posts"/>, {
route: '/ja/counts/posts?axis=month&mode=details&columns=title&user_id=3' })
expect (await screen.findByRole ('cell', { name: 'Snapshot' })).toBeInTheDocument ()
expect (screen.getByLabelText ('次のページ')).toHaveAttribute (
'href', '/ja/counts/posts?axis=month&mode=details&columns=title&user_id=3&page=2')
fireEvent.click (conditions ())
expect (screen.getByRole ('checkbox', { name: '投稿タイトル' })).toBeChecked ()
fireEvent.click (screen.getByRole ('checkbox', { name: '投稿 URL' }))
expect (fetchCounts.mock.lastCall?.[1].get ('columns')).toBe ('title')
execute ()
await waitFor (() => {
expect (fetchCounts.mock.lastCall?.[1].get ('columns')).toBe ('title,url')
})
})
it ('removes legacy event fields and axes from public query state', async () => {
renderWithProviders (<CountsResourcePage resource="tags"/>, {
route: '/ja/counts/tags?axis=event_type&event_type=update&columns=event_type,name' })
await waitFor (() => {
const params = fetchCounts.mock.lastCall?.[1] as URLSearchParams
expect (params.get ('axis')).toBe ('month')
expect (params.has ('event_type')).toBe (false)
expect (params.get ('columns')).toBe ('name')
})
fireEvent.click (conditions ())
expect (screen.queryByText ('操作種別')).not.toBeInTheDocument ()
expect (screen.getByRole ('combobox', { name: '集計軸' })).toHaveValue ('month')
})
it ('resolves a tag name with the existing lookup before running the query', async () => {
fetchTagByName.mockResolvedValue ({ id: 12 })
renderWithProviders (<CountsResourcePage resource="posts"/>, {
route: '/ja/counts/posts' })
fireEvent.click (conditions ())
fireEvent.change (screen.getByRole ('textbox', { name: 'タグ名または ID' }), {
target: { value: '日本語タグ' } })
execute ()
await waitFor (() => expect (fetchCounts.mock.lastCall?.[1].get ('tag')).toBe ('12'))
expect (fetchTagByName).toHaveBeenCalledWith ('日本語タグ', 'ja')
})
it ('uses Japanese fallback metadata in the Ainu workspace without exposing keys', async () => {
const { container } = renderWithProviders (<CountsResourcePage resource="tags"/>, {
route: '/ain-Kana/counts/tags' })
const title = screen.getByRole ('heading', { name: 'タグの集計・明細' })
expect (title.querySelector ('[lang="ja"]')).toHaveTextContent ('タグの集計・明細')
expect (conditions ().querySelector ('[lang="ja"]')).toBeInTheDocument ()
fireEvent.click (conditions ())
await screen.findByRole ('table')
expect (container.textContent).not.toMatch (/counts:|columns\.|fields\.|axes\./)
})
it ('drills through an actor group while preserving the existing filters', async () => {
fetchCounts.mockResolvedValue ({ count: 3, groupCount: 1, page: 1, limit: 20,
groups: [{ key: 9, name: 'Actor name', count: 3 }] })
renderWithProviders (<CountsResourcePage resource="posts"/>, {
route: '/ja/counts/posts?axis=user&resource_id=7' })
const link = await screen.findByRole ('link', { name: 'Actor name' })
const url = new URL (link.getAttribute ('href')!, 'http://localhost')
expect (url.searchParams.get ('resource_id')).toBe ('7')
expect (url.searchParams.get ('user_id')).toBe ('9')
expect (url.searchParams.get ('mode')).toBe ('details')
})
it ('keeps partial-month boundaries when drilling through to details', async () => {
fetchCounts.mockResolvedValue ({ count: 4, groupCount: 1, page: 1, limit: 20,
groups: [{ key: '2024-01', name: null, count: 4 }] })
renderWithProviders (<CountsResourcePage resource="tags"/>, {
route: '/ja/counts/tags?axis=month&from=2024-01-15T00:00Z' })
const link = await screen.findByRole ('link', { name: '2024-01' })
const url = new URL (link.getAttribute ('href')!, 'http://localhost')
expect (url.searchParams.get ('from')).toBe ('2024-01-15T00:00Z')
expect (url.searchParams.get ('before')).toBe ('2024-02-01T00:00Z')
expect (url.searchParams.get ('mode')).toBe ('details')
})
})
+275
ファイルの表示
@@ -0,0 +1,275 @@
import { useQuery } from '@tanstack/react-query'
import { useEffect, useRef, useState } from 'react'
import { useSearchParams } from 'react-router-dom'
import PrefetchLink from '@/components/PrefetchLink'
import CountsExportProgress from '@/components/counts/CountsExportProgress'
import CountsControls from '@/components/counts/CountsControls'
import CountsSummaryResults from '@/components/counts/CountsSummaryResults'
import CountsResults from '@/components/counts/CountsResults'
import CountsTableLayout from '@/components/counts/CountsTableLayout'
import AnimatedCollapse from '@/components/common/AnimatedCollapse'
import PageHead from '@/components/common/PageHead'
import PageTitle from '@/components/common/PageTitle'
import Pagination from '@/components/common/Pagination'
import MainArea from '@/components/layout/MainArea'
import SidebarComponent from '@/components/layout/SidebarComponent'
import { Button } from '@/components/ui/button'
import { countsRequestParams, defaultCountsMode, countsExportPayload, fieldsForCounts, fetchCounts,
isCountsState, normaliseCountsParams, selectedCountColumns } from '@/lib/counts'
import { countsExportLabels } from '@/lib/countsDisplay'
import { CountsDefinitionContext, countsAxisFields, countsAxisLabel,
countsResources, countsFieldLabel,
fetchCountsDefinition, useCountsDefinition }
from '@/lib/countsDefinitions'
import { countsKeys } from '@/lib/queryKeys'
import { cn } from '@/lib/utils'
import { useClientMotion } from '@/lib/useClientMotion'
import useDialogue from '@/lib/dialogues/useDialogue'
import { useLocale } from '@/lib/useLocale'
import { useUITranslation } from '@/lib/useUITranslation'
import { setCountState, readCountState } from '@/lib/queryState'
import type { CountsAxis } from '@/lib/countsDefinitions'
import type { CountsResource } from '@/lib/counts'
const CountsResourceContent = ({ resource }: { resource: CountsResource }) => {
const definition = useCountsDefinition ()
const [source, setParams] = useSearchParams ()
const params = normaliseCountsParams (resource, source, definition)
const canonical = params.toString ()
const controlParams = new URLSearchParams (params)
controlParams.delete ('nested_json')
controlParams.delete ('order')
controlParams.delete ('row_numbers')
controlParams.set ('columns', selectedCountColumns (resource, params, definition)
.filter (column => column !== 'thumbnail').sort ().join (' '))
const request = countsRequestParams (params)
const [open, setOpen] = useState (false)
const [sidebarOpen, setSidebarOpen] = useState (false)
const [expanded, setExpanded] = useState (() => new Set<string> ([resource]))
const motion = useClientMotion ()
useEffect (() => {
setExpanded (previous => new Set ([...previous, resource]))
}, [resource])
const locale = useLocale ()
const dialogue = useDialogue ()
const { label, message } = useUITranslation ('counts')
const mode = params.get ('mode') ?? 'summary'
const state = isCountsState (params)
const cacheParams = new URLSearchParams (request)
cacheParams.set ('definition', state ? 'latest' : 'history')
const [exporting, setExporting] = useState (false)
const exportInFlight = useRef (false)
const [exportError, setExportError] = useState (false)
const { data, isPending, isError, isPlaceholderData, refetch } = useQuery ({
enabled: params.has ('timezone'),
queryKey: countsKeys.index (resource, cacheParams.toString (), locale),
queryFn: () => fetchCounts (resource, params, locale), retry: false,
placeholderData: (previous, query) => {
const previousParams = new URLSearchParams (String (query?.queryKey[3] ?? ''))
return (query?.queryKey[1] === resource && query?.queryKey[2] === locale
&& previousParams.get ('mode') === mode
&& previousParams.get ('rows') === params.get ('rows')
&& previousParams.get ('cols') === params.get ('cols')
&& previousParams.get ('definition') === cacheParams.get ('definition'))
? previous : undefined
} })
useEffect (() => {
if (source.toString () !== canonical)
setParams (canonical, { replace: true })
}, [canonical, source, setParams])
useEffect (() => {
const timezone = Intl.DateTimeFormat ().resolvedOptions ().timeZone
if (params.get ('timezone') !== timezone)
{
const next = new URLSearchParams (canonical)
next.set ('timezone', timezone)
setParams (next, { replace: true })
}
else if (data?.executionAt != null && !isPlaceholderData && !params.has ('execution_at'))
{
const next = new URLSearchParams (canonical)
next.set ('execution_at', data.executionAt)
setParams (next, { replace: true })
}
}, [canonical, data, isPlaceholderData, params, setParams])
const update = (key: string, value: string, extra?: Record<string, string>) => {
const next = new URLSearchParams (params)
if (value || ['sort', 'columns', 'children', 'conditions', 'drill'].includes (key))
setCountState (next, key, value)
else next.delete (key)
if (extra != null)
{
for (const key of [...next.keys ()])
if (key === 'measures' || key.startsWith ('measures.')) next.delete (key)
for (const [key, value] of Object.entries (extra)) next.set (key, value)
}
if (key === 'sort') next.delete ('page')
setParams (next)
}
const download = async (format: 'csv' | 'json') => {
if (exportInFlight.current) return
exportInFlight.current = true
setExporting (true)
setExportError (false)
try
{
const axes = [...readCountState<CountsAxis[]> (params, 'rows', []),
...readCountState<CountsAxis[]> (params, 'cols', [])]
const fields = mode === 'summary' ? [
...axes.map (axis => ({
key: axis.field, label: countsFieldLabel (axis.field) })),
{ key: 'count', label: 'count' }]
: selectedCountColumns (resource, params, definition).map (column =>
fieldsForCounts (resource, definition).find (field => field.key === column)!)
const headers = mode === 'summary'
? [...axes.map (axis => countsAxisLabel (axis, definition, locale).text),
message ('count').text]
: fields.map (field => message (field.label).text)
const payload = countsExportPayload (resource, params, format,
fields.map (field => field.key), headers,
{ ...countsExportLabels (resource, locale, definition),
...Object.fromEntries (countsAxisFields (definition, !state).map (field => [
field.key, message (field.label).text])), count: message ('count').text },
definition)
const handle = { cancel: async () => false }
await dialogue.form ({ title: label ('exportProgress.title'),
footerLayout: 'inline', onCancel: () => handle.cancel (),
body: controls => <CountsExportProgress payload={payload} locale={locale}
handle={handle} controls={controls}/> })
}
catch
{
setExportError (true)
}
finally
{
exportInFlight.current = false
setExporting (false)
}
}
return (
<CountsTableLayout>
<div data-sidebar-container
className="flex min-h-0 flex-1 flex-col overflow-hidden md:flex-row">
<PageHead title={message (`titles.${resource}`)}/>
<Button type="button" variant="ghost" size="sm" className="shrink-0 md:hidden"
aria-expanded={sidebarOpen} aria-controls="counts-targets"
onClick={() => setSidebarOpen (!sidebarOpen)}>{label ('target')}</Button>
<SidebarComponent sidebarKey="counts" maxWidth={224}
className={cn ('md:block md:max-w-56', !sidebarOpen && 'hidden')}>
<nav id="counts-targets" aria-label={message ('target').text}
lang={message ('target').language}>
<h2 className="mb-2 font-semibold">{label ('target')}</h2>
<ul className="space-y-1">
{countsResources.map (target => <li key={target}>
<button type="button" className="flex w-full items-center gap-1 py-1 text-left
focus-visible:outline-none focus-visible:ring-2 focus-visible:ring-blue-500
dark:focus-visible:ring-blue-400"
aria-expanded={expanded.has (target)} aria-controls={`counts-definition-${target}`}
onClick={() => setExpanded (previous => {
const next = new Set (previous)
if (next.has (target)) next.delete (target)
else next.add (target)
return next
})}>
<span aria-hidden className="inline-block" style={{
transform: expanded.has (target) ? 'rotate(90deg)' : 'rotate(0deg)',
transition: `transform ${motion.duration ('collapse')}ms ${motion.easing}` }}>▸</span>
{label (`resources.${target}`)}
</button>
<AnimatedCollapse open={expanded.has (target)} id={`counts-definition-${target}`}>
<ul className="ml-3">
{['latest', 'history'].map (definition => {
const next = new URLSearchParams ({
definition, mode: defaultCountsMode (definition) })
for (const key of ['timezone', 'at', 'current', 'row_numbers'])
if (params.get (key)) next.set (key, params.get (key)!)
const active = target === resource
&& params.get ('definition') === definition
return <li key={definition}><PrefetchLink
to={`/counts/${target}?${next}`}
aria-current={active ? 'page' : undefined}
className={cn ('block py-1', active && 'font-bold underline')}>
{label (definition)}</PrefetchLink></li>
})}
</ul>
</AnimatedCollapse>
</li>)}
</ul>
</nav>
</SidebarComponent>
<MainArea className="flex min-h-0 min-w-0 flex-col p-2">
<PageTitle className="mb-1 text-base">{label (`titles.${resource}`)}</PageTitle>
<div className="shrink-0">
<CountsControls key={`${resource}:${controlParams}`}
resource={resource} applied={params}
open={open} setOpen={setOpen}
reorder={value => update ('order', value)}
displayJSON={enabled => update ('nested_json', enabled ? '1' : '0')}
displayRowNumbers={enabled => update ('row_numbers', enabled ? '1' : '0')}
displayThumbnail={enabled => {
const columns = selectedCountColumns (resource, params, definition)
.filter (column => column !== 'thumbnail')
update ('columns', (enabled ? ['thumbnail', ...columns] : columns).join (' '))
}}
execute={next => {
if (next.toString () === canonical) void refetch ()
else setParams (next)
}} actions={<div className="ml-auto flex shrink-0 items-center gap-2">
{(['csv', 'json'] as const).map (format => <Button key={format}
type="button" variant="outline" size="sm"
disabled={data == null || isError || isPlaceholderData
|| exporting || !params.get ('execution_at')}
onClick={() => void download (format)}>
{label (`exportFormats.${format}`)}</Button>)}
</div>}/>
{exportError && <p role="alert">{label ('common:fetchError')}</p>}
</div>
{isPending && <p>{label ('shared:messages.loadingEllipsis')}</p>}
{isError && <p role="alert">{label ('common:fetchError')}</p>}
{data != null && <>
<p className="my-1 flex flex-wrap gap-x-3 text-xs">
<span>{label ('recordTotal', { count: data.count })}</span>
{data.populationCount != null && <span>{label ('total', {
count: data.populationCount })}</span>}
</p>
{mode === 'summary' ? <CountsSummaryResults resource={resource}
data={data} params={params}
pending={isPlaceholderData} update={update}/> : (
<CountsResults resource={resource} data={data} params={params}
pending={isPlaceholderData} update={update}/>)}
<Pagination page={data.page} totalPages={Math.ceil (
(mode === 'summary' ? data.groupCount ?? 0 : data.count) / data.limit)}/>
</>}
</MainArea>
</div>
</CountsTableLayout>)
}
const CountsResourcePage = ({ resource }: { resource: CountsResource }) => {
const { label } = useUITranslation ('counts')
const { data, isError } = useQuery ({
queryKey: countsKeys.definition (resource),
queryFn: () => fetchCountsDefinition (resource), staleTime: Infinity })
if (data == null) return <p>{label (
isError ? 'common:fetchError' : 'shared:messages.loadingEllipsis')}</p>
return <CountsDefinitionContext.Provider value={data}>
<CountsResourceContent resource={resource}/>
</CountsDefinitionContext.Provider>
}
export default CountsResourcePage

変更されたファイルが多すぎるため、一部のファイルは表示されません さらに表示