コミットを比較

..
11 コミット
作成者 SHA1 メッセージ 日付
みてるぞ f91f9d0eb5 #434 2026-10-26 12:44:22 +09:00
みてるぞ 3e8f209542 #434 2026-10-26 12:19:43 +09:00
みてるぞ 8642225b94 #434 2026-10-25 23:08:48 +09:00
みてるぞ 96a5bb8601 #434 2026-10-25 22:16:20 +09:00
みてるぞ e71663bd22 #434 2026-10-25 16:47:05 +09:00
みてるぞ 52a8d62be5 #434 2026-10-25 16:25:04 +09:00
みてるぞ 3e05757820 #434 2026-10-25 16:00:37 +09:00
みてるぞ d411b3b40f #434 2026-10-25 14:53:35 +09:00
みてるぞ 304090f717 #434 2026-10-25 14:14:55 +09:00
みてるぞ 084a090cab アイヌ語追加 (#428) (#433)
Reviewed-on: #433
Co-authored-by: miteruzo <miteruzo@naver.com>
2026-10-20 20:44:14 +09:00
みてるぞ 75d7dac999 Wiki 依存の排除 (#431) (#432)
Reviewed-on: #432
Co-authored-by: miteruzo <miteruzo@naver.com>
2026-10-16 00:26:53 +09:00
348個のファイルの変更、19638行の追加、3546行の削除
+17 -4
ファイルの表示
@@ -573,10 +573,10 @@ and layout reuse, follow `frontend/AGENTS.md`.
- When new user-facing wording appears necessary, ask the user for the exact - When new user-facing wording appears necessary, ask the user for the exact
wording and placement before implementing it. wording and placement before implementing it.
- Do not invent replacement copy when removing unrequested wording. - Do not invent replacement copy when removing unrequested wording.
- Do not create, modify, or run tests unless the user explicitly asks for - 明示的な個別指示がない限り、テストの追加・修正・削除・実行を一切行わない。
test work. When the user asks for tests, keep working within the permitted fixture、mock、snapshot、テスト補助コードも対象とする。
test-file scope and rerun them until they pass or the remaining failure is 既存のテスト差分を勝手に修正・差戻ししない。
clearly blocked. この規約はディレクトリ固有の自動的なテスト実行推奨より優先する。
- Test-only work includes adding, updating, deleting, reorganising, or fixing - Test-only work includes adding, updating, deleting, reorganising, or fixing
SyntaxError in tests. During test-only work, do not modify production code. SyntaxError in tests. During test-only work, do not modify production code.
- During test-only work, do not change production constants, behaviour, API - During test-only work, do not change production constants, behaviour, API
@@ -631,6 +631,19 @@ and layout reuse, follow `frontend/AGENTS.md`.
## Frontend rules ## Frontend rules
- 原則として、ユーザ向け UI 文言を TSX / TypeScript に直接記述しない。
新しい文言は既存の i18n システムで管理し、適切な既存キーを先に確認する。
不必要な重複定義を避け、新規機能固有の文言は適切な namespace にまとめる。
- 対応 locale の辞書を整合させる。未翻訳項目には既存の日本語 fallback と
`_untranslated` 規約を使い、language metadata を維持する。
翻訳を捏造せず、未翻訳文字列を正式な翻訳として登録しない。
- UI 文言の追加・変更時はビルド、静的検査、実画面の確認等で、辞書と
SSR / hydration の整合及び翻訳キーの非露出を検証する。
i18n の検証を理由にテスト関連作業を開始しない。
- API の内部識別子、パラメータ、ユーザ入力データ、外部 URL は機械的に
翻訳しない。ボタン、項目名、CSV ヘッダ、エラー、空状態、ツールチップ、
アクセシビリティ用ラベルも UI 文言として扱う。
- Use `frontend/src/lib/api.ts` for API calls so headers and camelCase conversion stay consistent. - Use `frontend/src/lib/api.ts` for API calls so headers and camelCase conversion stay consistent.
- Add or reuse TanStack Query keys through `frontend/src/lib/queryKeys.ts`; - Add or reuse TanStack Query keys through `frontend/src/lib/queryKeys.ts`;
avoid ad hoc query key arrays. avoid ad hoc query key arrays.
+48 -2
ファイルの表示
@@ -1,22 +1,52 @@
class ApplicationController < ActionController::API class ApplicationController < ActionController::API
include ActionController::Cookies
include ActionController::RequestForgeryProtection
class UnsupportedLocale < StandardError
;
end
rescue_from ActiveRecord::RecordInvalid, with: :render_record_invalid rescue_from ActiveRecord::RecordInvalid, with: :render_record_invalid
rescue_from ActiveRecord::RecordNotUnique, with: :render_record_not_unique rescue_from ActiveRecord::RecordNotUnique, with: :render_record_not_unique
rescue_from(UnsupportedLocale) { head :not_found }
before_action :reject_banned_ip_address! before_action :reject_banned_ip_address!
before_action :authenticate_user before_action :authenticate_user
before_action :reject_banned_user! before_action :reject_banned_user!
before_action :validate_explicit_locale!
protect_from_forgery with: :exception
self.forgery_protection_origin_check = true
rescue_from ActionController::InvalidAuthenticityToken, with: :reject_invalid_csrf
after_action :prevent_authenticated_caching
def current_user = @current_user def current_user = @current_user
private private
def reject_invalid_csrf
response.headers['X-CSRF-Invalid'] = '1'
response.headers['Cache-Control'] = 'private, no-store'
head :forbidden
end
def authenticate_user def authenticate_user
code = request.headers['X-Transfer-Code'] || request.headers['HTTP_X_TRANSFER_CODE'] code = session[:inheritance_code]
return if code.blank? return if code.blank?
@current_user = User.find_by(inheritance_code: code) @current_user = User.find_by(inheritance_code: code)
end end
def prevent_authenticated_caching
response.headers['Cache-Control'] = 'private, no-store' if current_user
end
def establish_user_session user
reset_session
session[:inheritance_code] = user.inheritance_code
@current_user = user
response.headers['Cache-Control'] = 'private, no-store'
end
def bool? key, default: false def bool? key, default: false
return default if params[key].nil? return default if params[key].nil?
@@ -93,5 +123,21 @@ class ApplicationController < ActionController::API
value value
end end
def resolve_locale! = Locale.find_by(code: params[:locale]) || Locale.nipponese def resolve_locale!
return @resolved_locale if defined?(@resolved_locale)
code = params.fetch(:locale, 'ja')
unless code.is_a?(String) && Locale.codes.value?(code)
raise UnsupportedLocale
end
locale = Locale.find_by(code:)
raise UnsupportedLocale unless locale && locale.deprecated_at.nil?
@resolved_locale = locale
end
def validate_explicit_locale!
resolve_locale! if params.key?(:locale)
end
end end
+21
ファイルの表示
@@ -0,0 +1,21 @@
module CountsListing
extend ActiveSupport::Concern
def index
locale = resolve_locale!
resource = is_a?(CountsPostsController) ? 'posts' : 'tags'
query = CountsQuery.new(version_scope, resource, request.query_parameters.except('locale')
.merge('locale' => locale.code))
result = { count: query.count, page: query.page, limit: query.limit,
at: request.query_parameters['at'].presence || Time.current.utc.iso8601(6) }
if query.mode == 'summary'
result.merge!(query.summary)
else
result[:versions] = CountsVersionRepr.many(
query.details, resource:, language_code: locale.language_code)
end
render json: result
rescue ArgumentError, TypeError
head :bad_request
end
end
+45
ファイルの表示
@@ -0,0 +1,45 @@
class CountsExportsController < ApplicationController
wrap_parameters false
def prepare
locale = resolve_locale!
payload = request.request_parameters.merge(request.query_parameters)
exporter = CountsExporter.new(payload, language_code: locale.language_code)
session[:counts_export_owner] ||= SecureRandom.hex(32)
token = CountsExportDownload.prepare(exporter, session[:counts_export_owner])
response.headers['Cache-Control'] = 'private, no-store'
render json: { token: }
rescue ArgumentError, TypeError, KeyError, JSON::ParserError
head :bad_request
end
def download
file = CountsExportDownload.find(params[:token], session[:counts_export_owner])
return head :not_found unless file
response.headers['Cache-Control'] = 'private, no-store'
send_file file.fetch('path'),
filename: "counts-#{ file.fetch('resource') }.#{ file.fetch('format') }",
type: file['format'] == 'json' ? 'application/json' : 'text/csv',
disposition: 'attachment'
end
def index
locale = resolve_locale!
payload = request.request_parameters.except('authenticity_token')
.merge(request.query_parameters)
exporter = CountsExporter.new(payload,
language_code: locale.language_code)
response.headers['Content-Type'] = exporter.format == 'json' ?
'application/json; charset=utf-8' : 'text/csv; charset=utf-8'
response.headers['Content-Disposition'] =
"attachment; filename=\"counts-#{ exporter.resource }.#{ exporter.format }\""
response.headers['Cache-Control'] = 'private, no-store'
response.headers['X-Accel-Buffering'] = 'no'
# Rack::ETag otherwise materialises the response to calculate a digest.
response.headers['Last-Modified'] = Time.current.httpdate
self.response_body = exporter.body
rescue ArgumentError, TypeError, KeyError, JSON::ParserError
head :bad_request
end
end
+3
ファイルの表示
@@ -0,0 +1,3 @@
class CountsPostsController < PostVersionsController
include CountsListing
end
+10
ファイルの表示
@@ -0,0 +1,10 @@
class CountsStatesController < ApplicationController
def index
locale = resolve_locale!
result = CountsStateQuery.new(request.query_parameters,
language_code: locale.language_code).call
render json: result
rescue ArgumentError, TypeError
head :bad_request
end
end
+21
ファイルの表示
@@ -0,0 +1,21 @@
class CountsTablesController < ApplicationController
def definition
resolve_locale!
definition = CountsResourceDefinition.new(params[:resource])
render json: { fields: definition.metadata,
axes: { latest: definition.axes(false), history: definition.axes(true) },
grains: CountsAxes::GRAINS }
rescue ArgumentError
head :bad_request
end
def index
resolve_locale!
render json: CountsTableQuery.new(request.query_parameters).call
rescue CountsTableQuery::TooManyColumns
render json: { error: 'too_many_columns', max_columns: CountsTableQuery::MAX_COLUMNS },
status: :unprocessable_entity
rescue ArgumentError, TypeError, KeyError, JSON::ParserError
head :bad_request
end
end
+3
ファイルの表示
@@ -0,0 +1,3 @@
class CountsTagsController < TagVersionsController
include CountsListing
end
+10
ファイルの表示
@@ -0,0 +1,10 @@
class LocalesController < ApplicationController
def index
locales = Locale.where(deprecated_at: nil).order(:created_at).map do |locale|
{ code: locale.code_before_type_cast, name: locale.name }
end
response.headers['Cache-Control'] = 'no-store'
render json: { locales: }
end
end
+2 -1
ファイルの表示
@@ -255,7 +255,8 @@ class MaterialsController < ApplicationController
end end
def resolve_material_tag! locale, tag_name_raw def resolve_material_tag! locale, tag_name_raw
Tag.find_or_create_by_tag_name!(locale, tag_name_raw, category: :material) Tag.find_or_create_by_tag_name!(locale, tag_name_raw, category: :material,
created_by_user: current_user)
end end
def material_index_needs_tag_name? filters def material_index_needs_tag_name? filters
+22 -8
ファイルの表示
@@ -24,7 +24,7 @@ class NicoTagsController < ApplicationController
q = q =
ExternalTag ExternalTag
.joins("LEFT JOIN (#{ post_tag_max_sql }) post_tag_max " \ .joins("LEFT JOIN (#{ post_tag_max_sql }) post_tag_max " +
'ON post_tag_max.external_tag_id = external_tags.id') 'ON post_tag_max.external_tag_id = external_tags.id')
.includes(linked_tags: [:tag_names, :tag_descriptions]) .includes(linked_tags: [:tag_names, :tag_descriptions])
if name if name
@@ -37,7 +37,7 @@ class NicoTagsController < ApplicationController
linked_tag_ids = linked_tag_ids =
Tag Tag
.joins(:tag_names) .joins(:tag_names)
.where(tag_names: { language_code: 'ja', primary_flg: true }) .where(tag_names: { language_code: locale.language_code, primary_flg: true })
.where('tag_names.name LIKE ?', "%#{ linked_tag }%") .where('tag_names.name LIKE ?', "%#{ linked_tag }%")
.pluck(:id) .pluck(:id)
linked_nico_tag_ids = NicoTagRelation.where(tag_id: linked_tag_ids).pluck(:nico_tag_id) linked_nico_tag_ids = NicoTagRelation.where(tag_id: linked_tag_ids).pluck(:nico_tag_id)
@@ -45,7 +45,7 @@ class NicoTagsController < ApplicationController
end end
if link_status.in?(['linked', 'unlinked']) if link_status.in?(['linked', 'unlinked'])
exists_sql = exists_sql =
'EXISTS (SELECT 1 FROM nico_tag_relations ' \ 'EXISTS (SELECT 1 FROM nico_tag_relations ' +
'WHERE nico_tag_relations.nico_tag_id = external_tags.id)' 'WHERE nico_tag_relations.nico_tag_id = external_tags.id)'
q = link_status == 'linked' ? q.where(exists_sql) : q.where("NOT #{ exists_sql }") q = link_status == 'linked' ? q.where(exists_sql) : q.where("NOT #{ exists_sql }")
end end
@@ -98,7 +98,8 @@ class NicoTagsController < ApplicationController
ApplicationRecord.transaction do ApplicationRecord.transaction do
linked_tags = Tag.normalise_tags!(locale, linked_tag_names, linked_tags = Tag.normalise_tags!(locale, linked_tag_names,
with_tagme: false, with_tagme: false,
with_no_deerjikist: false) with_no_deerjikist: false,
created_by_user: current_user)
TagVersioning.record_tag_snapshots!( TagVersioning.record_tag_snapshots!(
linked_tags, linked_tags,
@@ -140,10 +141,23 @@ class NicoTagsController < ApplicationController
end end
def render_nico_tag_form_record_invalid record def render_nico_tag_form_record_invalid record
if record.is_a?(TagName) || record.is_a?(Tag) names =
render_validation_error fields: { tags: record.errors.full_messages.map { |message| case record
"タグ名 “#{ record.name }”: #{ message }" when TagName
} } [record]
when Tag
record.tag_names.to_a
else
[]
end
messages = names.flat_map do |tag_name|
tag_name.errors.full_messages.map do |message|
"タグ名 “#{ tag_name.name }”: #{ message }"
end
end
if messages.any?
render_validation_error fields: { tags: messages }
else else
render_validation_error record render_validation_error record
end end
+46 -27
ファイルの表示
@@ -1,5 +1,6 @@
class PostVersionsController < ApplicationController class PostVersionsController < ApplicationController
def index def index
locale = resolve_locale!
post_id = params[:post].presence post_id = params[:post].presence
tag_id = params[:tag].presence&.to_i tag_id = params[:tag].presence&.to_i
external_tag_id = params[:external_tag].presence&.to_i external_tag_id = params[:external_tag].presence&.to_i
@@ -13,21 +14,7 @@ class PostVersionsController < ApplicationController
offset = (page - 1) * limit offset = (page - 1) * limit
q = PostVersion.joins(<<~SQL.squish) q = version_scope
LEFT JOIN
post_versions prev
ON
prev.post_id = post_versions.post_id
AND prev.version_no = post_versions.version_no - 1
SQL
.select('post_versions.*',
'prev.title AS prev_title',
'prev.url AS prev_url',
'prev.thumbnail_base AS prev_thumbnail_base',
'prev.tags_json AS prev_tags_json',
'prev.video_ms AS prev_video_ms',
'prev.original_created_from AS prev_original_created_from',
'prev.original_created_before AS prev_original_created_before')
q = q.where('post_versions.post_id = ?', post_id) if post_id q = q.where('post_versions.post_id = ?', post_id) if post_id
if external_tag_id || (tag_id && !(Tag.exists?(id: tag_id))) if external_tag_id || (tag_id && !(Tag.exists?(id: tag_id)))
q = q.where('JSON_CONTAINS(post_versions.tags_json,' + q = q.where('JSON_CONTAINS(post_versions.tags_json,' +
@@ -45,12 +32,24 @@ class PostVersionsController < ApplicationController
.limit(limit) .limit(limit)
.offset(offset) .offset(offset)
render json: { versions: serialise_versions(versions), count: } render json: { versions: serialise_versions(versions, locale.language_code), count: }
end end
private private
def serialise_versions rows def version_scope
PostVersion.with_previous_version
.select('post_versions.*',
'prev.title AS prev_title',
'prev.url AS prev_url',
'prev.thumbnail_base AS prev_thumbnail_base',
'prev.tags_json AS prev_tags_json',
'prev.video_ms AS prev_video_ms',
'prev.original_created_from AS prev_original_created_from',
'prev.original_created_before AS prev_original_created_before')
end
def serialise_versions rows, language_code
rows = rows.to_a rows = rows.to_a
user_ids = rows.map(&:created_by_user_id).compact.uniq user_ids = rows.map(&:created_by_user_id).compact.uniq
users_by_id = User.where(id: user_ids).pluck(:id, :name).to_h users_by_id = User.where(id: user_ids).pluck(:id, :name).to_h
@@ -58,12 +57,16 @@ class PostVersionsController < ApplicationController
[normalise_json(row.tags_json), [normalise_json(row.tags_json),
normalise_json(row.attributes['prev_tags_json']) || []] normalise_json(row.attributes['prev_tags_json']) || []]
} }
tag_ids = snapshots.flatten.filter_map { _1['tag_id'] }.uniq
tag_names = TagName.where(tag_id: tag_ids, language_code:, primary_flg: true)
.pluck(:tag_id, :name).to_h
external_tag_names = external_tag_names_for(snapshots) external_tag_names = external_tag_names_for(snapshots)
rows.map do |row| rows.map do |row|
cur_tags = snapshot_tag_literals(normalise_json(row.tags_json), external_tag_names) cur_tags = snapshot_tag_literals(
normalise_json(row.tags_json), tag_names, external_tag_names)
prev_tags = snapshot_tag_literals( prev_tags = snapshot_tag_literals(
normalise_json(row.attributes['prev_tags_json']) || [], external_tag_names) normalise_json(row.attributes['prev_tags_json']) || [], tag_names, external_tag_names)
{ post_id: row.post_id, { post_id: row.post_id,
version_no: row.version_no, version_no: row.version_no,
@@ -75,6 +78,7 @@ class PostVersionsController < ApplicationController
prev: row.attributes['prev_thumbnail_base'] }, prev: row.attributes['prev_thumbnail_base'] },
video_ms: { current: row.video_ms, prev: row.attributes['prev_video_ms'] }, video_ms: { current: row.video_ms, prev: row.attributes['prev_video_ms'] },
tags: build_version_tags(cur_tags, prev_tags), tags: build_version_tags(cur_tags, prev_tags),
parent_post_ids: row.parent_post_ids,
original_created_from: { original_created_from: {
current: row.original_created_from&.iso8601, current: row.original_created_from&.iso8601,
prev: row.attributes['prev_original_created_from']&.iso8601 }, prev: row.attributes['prev_original_created_from']&.iso8601 },
@@ -106,17 +110,27 @@ class PostVersionsController < ApplicationController
names names
end end
def snapshot_tag_literals snapshots, external_tag_names def snapshot_tag_literals snapshots, tag_names, external_tag_names
snapshots snapshots
.filter_map { |snapshot| .filter_map { |snapshot|
if snapshot.key?('tag_id') if snapshot.key?('tag_id')
sections = snapshot.fetch('sections', []).map { |section|
[section.fetch('begin_ms'), section['end_ms']]
}.sort_by { |begin_ms, end_ms| [begin_ms, end_ms || Float::INFINITY] }
name = tag_names[snapshot['tag_id']] || snapshot.fetch('name')
[tag_category_order(snapshot['category']), [tag_category_order(snapshot['category']),
Post.tag_snapshot_literal(snapshot)] { identity: [:tag, snapshot['tag_id'], sections],
name: Post.tag_snapshot_literal(snapshot.merge('name' => name)),
restore_name: tag_names[snapshot['tag_id']] &&
Post.tag_snapshot_literal(snapshot.merge('name' => name)),
external: false }]
elsif external_tag_names[snapshot['external_tag_id']] elsif external_tag_names[snapshot['external_tag_id']]
[6, external_tag_names[snapshot['external_tag_id']]] [6, { identity: [:external_tag, snapshot['external_tag_id']],
name: external_tag_names[snapshot['external_tag_id']],
restore_name: nil, external: true }]
end end
} }
.sort_by { |order, name| [order, name.downcase] } .sort_by { |order, tag| [order, tag[:name].downcase] }
.map(&:second) .map(&:second)
end end
@@ -125,17 +139,22 @@ class PostVersionsController < ApplicationController
end end
def build_version_tags cur_tags, prev_tags def build_version_tags cur_tags, prev_tags
(cur_tags | prev_tags).map do |name| current = cur_tags.index_by { _1[:identity] }
previous = prev_tags.index_by { _1[:identity] }
(current.keys | previous.keys).map do |identity|
type = type =
if cur_tags.include?(name) && prev_tags.include?(name) if current.key?(identity) && previous.key?(identity)
'context' 'context'
elsif cur_tags.include?(name) elsif current.key?(identity)
'added' 'added'
else else
'removed' 'removed'
end end
{ name:, type: } entry = current[identity] || previous[identity]
{ name: entry[:name], restore_name: entry[:restore_name],
external: entry[:external], type: }
end end
end end
end end
+68 -23
ファイルの表示
@@ -400,8 +400,60 @@ class PostsController < ApplicationController
render_post_form_record_invalid e.record render_post_form_record_invalid e.record
end end
def tag_filters
source = resolve_locale!
code = params[:target_locale]
return head :not_found unless code.is_a?(String) && Locale.codes.value?(code)
target = Locale.find_by(code:, deprecated_at: nil)
return head :not_found unless target
literals = params[:tags].to_s.split
references = tag_filter_references(literals)
names = literals.each_with_index.map do |literal, index|
reference = references&.[](index)
(reference && reference['negative'] == false) ? literal : literal.sub(/\Anot:/i, '')
end
internal = references ? {} : TagName.where(
language_code: source.language_code, name: names).pluck(:name, :tag_id).to_h
canonical = references ? {} : TagName.where(
language_code: source.language_code, primary_flg: true,
tag_id: internal.values).pluck(:tag_id, :name).to_h
external_names = names.map { canonical[internal[_1]] || _1 }
external = references ? {} : ExternalTag.where(
"CONCAT(platform, ':', name) IN (?)", external_names).pluck(
Arel.sql("CONCAT(platform, ':', name)"), :id).to_h
ids = references ? references.filter_map { _1['tag_id'] } : internal.values
translated = TagName.where(language_code: target.language_code,
primary_flg: true, tag_id: ids).pluck(:tag_id, :name).to_h
filters = literals.each_with_index.map do |literal, index|
name = names[index]
reference = if references
references[index].except('name')
else
matches = {}
matches['tag_id'] = internal[name] if internal[name]
external_id = external[external_names[index]]
matches['external_tag_id'] = external_id if external_id
matches.empty? ? { 'missing' => true } : matches
end
name = translated[reference['tag_id']] || name
negative = reference.fetch('negative', literal.downcase.start_with?('not:'))
prefix = negative ? literal[/\Anot:/i] || 'not:' : ''
reference.merge('name' => "#{ prefix }#{ name }", 'negative' => negative)
end
render json: { references: filters }
end
private private
def tag_filter_references literals
PostTagSearch.parse_references(params[:tag_refs], literals)
rescue ArgumentError, JSON::ParserError, TypeError
raise ActionController::BadRequest
end
def filtered_posts locale def filtered_posts locale
tag_names = params[:tags].to_s.split tag_names = params[:tags].to_s.split
match_type = params[:match] match_type = params[:match]
@@ -413,20 +465,19 @@ class PostsController < ApplicationController
end end
def filter_posts_by_tags locale, tag_names, match_type def filter_posts_by_tags locale, tag_names, match_type
literals = tag_names.map do |raw_name| references = tag_filter_references(tag_names)
{ name: TagName.canonicalise(locale, raw_name.sub(/\Anot:/i, '')).first, literals = PostTagSearch.literals(locale, tag_names.join(' '), references:)
negative: raw_name.downcase.start_with?('not:') }
end
return Post.all if literals.empty? return Post.all if literals.empty?
if match_type == 'any' if match_type == 'any'
literals.reduce(Post.none) do |posts, literal| literals.reduce(Post.none) do |posts, literal|
posts.or(tag_literal_relation(locale, literal[:name], negative: literal[:negative])) posts.or(tag_literal_relation(locale, literal[:name], negative: literal[:negative],
reference: literal[:reference]))
end end
else else
literals.reduce(Post.all) do |posts, literal| literals.reduce(Post.all) do |posts, literal|
ids = tagged_post_ids_for(locale, literal[:name]) ids = tagged_post_ids_for(locale, literal[:name], reference: literal[:reference])
if literal[:negative] if literal[:negative]
posts.where.not(id: ids) posts.where.not(id: ids)
else else
@@ -436,8 +487,8 @@ class PostsController < ApplicationController
end end
end end
def tag_literal_relation locale, name, negative: def tag_literal_relation locale, name, negative:, reference: nil
ids = tagged_post_ids_for(locale, name) ids = tagged_post_ids_for(locale, name, reference:)
if negative if negative
Post.where.not(id: ids) Post.where.not(id: ids)
else else
@@ -445,19 +496,11 @@ class PostsController < ApplicationController
end end
end end
def tagged_post_ids_for locale, name def tagged_post_ids_for locale, name, reference: nil
posts_by_internal_tags = internal, external = PostTagSearch.references(locale, { name:, reference: })
Post Post.where(id: PostTag.where(tag_id: Array(internal)).select(:post_id))
.joins(tags: :tag_names) .or(Post.where(id: PostExternalTag.where(external_tag_id: Array(external))
.where(tag_names: { language_code: locale.language_code, name: }) .select(:post_id))).select(:id)
.select(:id)
posts_by_external_tags =
Post
.joins(:external_tags)
.where("CONCAT(external_tags.platform, ':', external_tags.name) = ?", name)
(posts_by_internal_tags + posts_by_external_tags).uniq(&:id)
end end
def sync_post_tags! post, desired_tags, sections def sync_post_tags! post, desired_tags, sections
@@ -728,7 +771,8 @@ class PostsController < ApplicationController
with_tagme: false, with_tagme: false,
with_no_deerjikist: false, with_no_deerjikist: false,
deny_deprecated: true, deny_deprecated: true,
with_sections: true) => { tags:, sections: } with_sections: true,
created_by_user: current_user) => { tags:, sections: }
tags = Tag.expand_parent_tags(tags).reject(&:deprecated?) tags = Tag.expand_parent_tags(tags).reject(&:deprecated?)
video_ms = normalise_video_ms(tags, video_ms_param:, duration_param:) video_ms = normalise_video_ms(tags, video_ms_param:, duration_param:)
@@ -874,7 +918,8 @@ class PostsController < ApplicationController
Tag.normalise_tags!(locale, snapshot[:tag_names], Tag.normalise_tags!(locale, snapshot[:tag_names],
with_tagme: false, with_tagme: false,
deny_deprecated: true, deny_deprecated: true,
with_sections: true) => { tags:, sections: } with_sections: true,
created_by_user: current_user) => { tags:, sections: }
TagVersioning.record_tag_snapshots!(tags, created_by_user: current_user, TagVersioning.record_tag_snapshots!(tags, created_by_user: current_user,
language_code: locale.language_code) language_code: locale.language_code)
+57
ファイルの表示
@@ -0,0 +1,57 @@
class SitemapEntriesController < ApplicationController
PAGE_SIZE = 1_000
RESOURCES = [
'posts', 'tags', 'deerjikists', 'materials',
'descriptions', 'tag_groups', 'theatres'].freeze
def index
locale = resolve_locale!
resources = RESOURCES.map { |resource| { resource:, count: entries(resource, locale).count } }
render json: { resources:, page_size: PAGE_SIZE }
end
def show
locale = resolve_locale!
resource = params[:resource]
return head :not_found unless RESOURCES.include?(resource)
page = Integer(params.fetch(:page, '1'), exception: false)
return head :bad_request unless page && page.positive?
scope = entries(resource, locale)
return head :not_found if page > [(scope.count.to_f / PAGE_SIZE).ceil, 1].max
ids = scope.order(:id).offset((page - 1) * PAGE_SIZE).limit(PAGE_SIZE).pluck(:id)
render json: { ids: }
end
private
def entries resource, locale
named_tags = TagName.where(language_code: locale.language_code, primary_flg: true)
.select(:tag_id)
case resource
when 'posts'
Post.all
when 'tags'
Tag.where(id: named_tags)
when 'deerjikists'
Tag.where(id: named_tags).where(id: Deerjikist.select(:tag_id))
when 'materials'
Material.all
when 'descriptions'
TagDescription.where(language_code: locale.language_code, tag_id: named_tags)
.where.not(body: [nil, ''])
.where(TagDescriptionVersion
.where('tag_description_versions.tag_id = tag_descriptions.tag_id')
.where('tag_description_versions.language_code = tag_descriptions.language_code')
.where('tag_description_versions.version_no = tag_descriptions.version_no')
.arel.exists)
when 'tag_groups'
TagGroup.joins(:versions)
.where('tag_group_versions.version_no = tag_groups.version_no')
when 'theatres'
Theatre.kept
end
end
end
+67 -21
ファイルの表示
@@ -10,17 +10,7 @@ class TagVersionsController < ApplicationController
offset = (page - 1) * limit offset = (page - 1) * limit
q = TagVersion.joins(<<~SQL.squish) q = version_scope
LEFT JOIN
tag_versions prev
ON
prev.tag_id = tag_versions.tag_id
AND prev.version_no = tag_versions.version_no - 1
SQL
.select('tag_versions.*', 'prev.name AS prev_name', 'prev.category AS prev_category',
'prev.deprecated_at AS prev_deprecated_at',
'prev.aliases AS prev_aliases', 'prev.names AS prev_names',
'prev.parent_tag_ids AS prev_parent_tag_ids')
q = q.where('tag_versions.tag_id = ?', tag_id) if tag_id q = q.where('tag_versions.tag_id = ?', tag_id) if tag_id
count = q.except(:select, :order, :limit, :offset).count count = q.except(:select, :order, :limit, :offset).count
@@ -34,9 +24,38 @@ class TagVersionsController < ApplicationController
private private
def version_scope
TagVersion.joins(<<~SQL.squish)
LEFT JOIN
tag_versions prev
ON
prev.tag_id = tag_versions.tag_id
AND prev.version_no = tag_versions.version_no - 1
SQL
.select('tag_versions.*', 'prev.name AS prev_name', 'prev.category AS prev_category',
'prev.deprecated_at AS prev_deprecated_at',
'prev.aliases AS prev_aliases', 'prev.names AS prev_names',
'prev.parent_tag_ids AS prev_parent_tag_ids')
end
def serialise_versions rows, language_code def serialise_versions rows, language_code
rows = rows.to_a
tag_ids = rows.map(&:tag_id).uniq
current_names_by_tag_id = TagName.where(tag_id: tag_ids, language_code:, primary_flg: true)
.pluck(:tag_id, :name, :auto_generated).to_h do |id, name, automatic|
[id, { name:, auto_generated: automatic }]
end
user_ids = rows.map(&:created_by_user_id).compact.uniq user_ids = rows.map(&:created_by_user_id).compact.uniq
users_by_id = User.where(id: user_ids).pluck(:id, :name).to_h users_by_id = User.where(id: user_ids).pluck(:id, :name).to_h
parent_ids = rows.flat_map do |row|
split_parent_tag_ids(row.parent_tag_ids) +
split_parent_tag_ids(row.attributes['prev_parent_tag_ids'])
end.uniq
parents_by_id = Tag.includes(
:tag_names, :materials, :deerjikists, :tag_descriptions,
parents: [:tag_names, :materials, :deerjikists, :tag_descriptions])
.where(id: parent_ids).index_by(&:id)
parent_json = parents_by_id.transform_values { TagRepr.base(_1, language_code:) }
rows.map do |row| rows.map do |row|
current_names = names_for_locale(row.names, language_code) current_names = names_for_locale(row.names, language_code)
@@ -47,18 +66,10 @@ class TagVersionsController < ApplicationController
cur_parent_tag_ids = split_parent_tag_ids(row.parent_tag_ids) cur_parent_tag_ids = split_parent_tag_ids(row.parent_tag_ids)
prev_parent_tag_ids = split_parent_tag_ids(row.attributes['prev_parent_tag_ids']) prev_parent_tag_ids = split_parent_tag_ids(row.attributes['prev_parent_tag_ids'])
all_parent_tag_ids = (cur_parent_tag_ids | prev_parent_tag_ids)
tags_by_id =
Tag
.includes(:tag_names, :materials, :tag_descriptions, parents: :tag_descriptions)
.where(id: all_parent_tag_ids)
.index_by(&:id)
parent_tags = parent_tags =
build_version_values(cur_parent_tag_ids, prev_parent_tag_ids, key: :tag_id) build_version_values(cur_parent_tag_ids, prev_parent_tag_ids, key: :tag_id)
.map do |h| .map do |h|
{ tag: TagRepr.base(tags_by_id[h[:tag_id]], language_code:), { tag: parent_json.fetch(h[:tag_id]),
type: h[:type] } type: h[:type] }
end end
@@ -66,6 +77,8 @@ class TagVersionsController < ApplicationController
version_no: row.version_no, version_no: row.version_no,
event_type: row.event_type, event_type: row.event_type,
names: row.names, names: row.names,
name_changes: name_changes(row.names, row.attributes['prev_names']),
current_name_reference: current_names_by_tag_id[row.tag_id],
name: { current: current_names['name'], prev: previous_names['name'] }, name: { current: current_names['name'], prev: previous_names['name'] },
category: { current: row.category, prev: row.attributes['prev_category'] }, category: { current: row.category, prev: row.attributes['prev_category'] },
deprecated_at: { current: row.deprecated_at&.iso8601, deprecated_at: { current: row.deprecated_at&.iso8601,
@@ -97,11 +110,44 @@ class TagVersionsController < ApplicationController
def split_values(values) = values.to_s.split(/\s+/).reject(&:blank?) def split_values(values) = values.to_s.split(/\s+/).reject(&:blank?)
def names_for_locale names, language_code def names_for_locale names, language_code
values = snapshot_names(names)[language_code]
values.is_a?(Hash) ? values : {}
end
def snapshot_names names
values = names.is_a?(String) ? JSON.parse(names) : names values = names.is_a?(String) ? JSON.parse(names) : names
values&.fetch(language_code, {}) || {} values.is_a?(Hash) ? values : {}
rescue JSON::ParserError rescue JSON::ParserError
{} {}
end end
def name_changes current_names, previous_names
current = snapshot_names(current_names)
previous = snapshot_names(previous_names)
(current.keys | previous.keys).sort.filter_map do |language|
current_name = language_snapshot(current[language])
previous_name = language_snapshot(previous[language])
next if current_name == previous_name
{ language_code: language, current: current_name, prev: previous_name }
end
end
def language_snapshot entry
return nil unless entry.is_a?(Hash)
aliases = Array(entry['aliases']).filter_map do |value|
value = { 'name' => value } if value.is_a?(String)
next unless value.is_a?(Hash) && value['name'].is_a?(String)
{ name: value['name'], script_code: value['script_code'] }
end
aliases = aliases.uniq.sort_by { [_1[:name], _1[:script_code].to_s] }
{ name: entry['name'],
script_code: entry['script_code'],
auto_generated: entry['auto_generated'],
aliases: }
end
def split_parent_tag_ids(values) = split_values(values).map(&:to_i) def split_parent_tag_ids(values) = split_values(values).map(&:to_i)
end end
+20 -10
ファイルの表示
@@ -175,7 +175,8 @@ class TagsController < ApplicationController
tags_by_id = tags_by_id =
Tag Tag
.joins(:tag_names) .joins(:tag_names)
.includes(:tag_names, :materials, :tag_descriptions, parents: :tag_descriptions) .preload(:tag_names, :materials, :deerjikists, :tag_descriptions,
parents: [:tag_names, :materials, :deerjikists, :tag_descriptions])
.where(tag_names: { language_code: locale.language_code, primary_flg: true }, .where(tag_names: { language_code: locale.language_code, primary_flg: true },
id: tag_ids) id: tag_ids)
.index_by(&:id) .index_by(&:id)
@@ -238,7 +239,8 @@ class TagsController < ApplicationController
base = base =
Tag Tag
.joins(:tag_names) .joins(:tag_names)
.includes(:tag_names, :materials, :tag_descriptions, parents: :tag_descriptions) .preload(:tag_names, :materials, :deerjikists, :tag_descriptions,
parents: [:tag_names, :materials, :deerjikists, :tag_descriptions])
.where(tag_names: { language_code: locale.language_code, primary_flg: true }, .where(tag_names: { language_code: locale.language_code, primary_flg: true },
deprecated_at: nil) deprecated_at: nil)
base = base.where('tags.post_count > 0') if present_only base = base.where('tags.post_count > 0') if present_only
@@ -282,7 +284,8 @@ class TagsController < ApplicationController
tag = tag =
Tag Tag
.joins(:tag_names) .joins(:tag_names)
.includes(:tag_names, :materials, :tag_descriptions, parents: :tag_descriptions) .preload(:tag_names, :materials, :deerjikists, :tag_descriptions,
parents: [:tag_names, :materials, :deerjikists, :tag_descriptions])
.find_by(id: params[:id], .find_by(id: params[:id],
tag_names: { language_code: locale.language_code, primary_flg: true }) tag_names: { language_code: locale.language_code, primary_flg: true })
return render json: TagRepr.base(tag, language_code: locale.language_code) if tag return render json: TagRepr.base(tag, language_code: locale.language_code) if tag
@@ -302,8 +305,9 @@ class TagsController < ApplicationController
tag = tag =
Tag Tag
.joins(:tag_names) .joins(:tag_names)
.includes(:tag_names, :materials, :tag_descriptions, parents: :tag_descriptions) .preload(:tag_names, :materials, :deerjikists, :tag_descriptions,
.find_by(tag_names: { name:, language_code: locale.language_code, primary_flg: true }) parents: [:tag_names, :materials, :deerjikists, :tag_descriptions])
.find_by(tag_names: { name:, language_code: locale.language_code })
return render json: TagRepr.base(tag, language_code: locale.language_code) if tag return render json: TagRepr.base(tag, language_code: locale.language_code) if tag
platform, external_name = name.split(':', 2) platform, external_name = name.split(':', 2)
@@ -320,7 +324,8 @@ class TagsController < ApplicationController
tag = tag =
Tag.joins(:tag_names) Tag.joins(:tag_names)
.includes(:tag_names, :tag_descriptions, parents: :tag_descriptions) .preload(:tag_names, :materials, :deerjikists, :tag_descriptions,
parents: [:tag_names, :materials, :deerjikists, :tag_descriptions])
.find_by(id: params[:id], .find_by(id: params[:id],
tag_names: { language_code: locale.language_code, primary_flg: true }) tag_names: { language_code: locale.language_code, primary_flg: true })
return head :not_found unless tag return head :not_found unless tag
@@ -340,7 +345,8 @@ class TagsController < ApplicationController
tag = tag =
Tag Tag
.joins(:tag_names) .joins(:tag_names)
.includes(:tag_names, :tag_descriptions, parents: :tag_descriptions) .preload(:tag_names, :materials, :deerjikists, :tag_descriptions,
parents: [:tag_names, :materials, :deerjikists, :tag_descriptions])
.find_by(tag_names: { name:, language_code: locale.language_code, primary_flg: true }) .find_by(tag_names: { name:, language_code: locale.language_code, primary_flg: true })
return head :not_found unless tag return head :not_found unless tag
@@ -359,7 +365,8 @@ class TagsController < ApplicationController
tag = tag =
Tag Tag
.joins(:tag_names) .joins(:tag_names)
.includes(:tag_names, :tag_descriptions, parents: :tag_descriptions) .preload(:tag_names, :materials, :deerjikists, :tag_descriptions,
parents: [:tag_names, :materials, :deerjikists, :tag_descriptions])
.find_by(id: params[:id], .find_by(id: params[:id],
tag_names: { language_code: locale.language_code, primary_flg: true }) tag_names: { language_code: locale.language_code, primary_flg: true })
return head :not_found unless tag return head :not_found unless tag
@@ -368,6 +375,7 @@ class TagsController < ApplicationController
return if performed? return if performed?
ApplicationRecord.transaction do ApplicationRecord.transaction do
SharedResourceIdAllocator.lock_space!(Tag, 'tags')
tag.lock! tag.lock!
requested_keys = rows.map { |row| [row[:platform], row[:code]] }.uniq requested_keys = rows.map { |row| [row[:platform], row[:code]] }.uniq
@@ -424,7 +432,8 @@ class TagsController < ApplicationController
tag = tag =
Tag Tag
.joins(:tag_names) .joins(:tag_names)
.includes(:tag_names, :materials, :tag_descriptions, parents: :tag_descriptions) .preload(:tag_names, :materials, :deerjikists, :tag_descriptions,
parents: [:tag_names, :materials, :deerjikists, :tag_descriptions])
.find_by(tag_names: { name:, language_code: locale.language_code, primary_flg: true }) .find_by(tag_names: { name:, language_code: locale.language_code, primary_flg: true })
return head :not_found unless tag return head :not_found unless tag
@@ -835,7 +844,8 @@ class TagsController < ApplicationController
def update_parent_tags! locale, tag, parent_names def update_parent_tags! locale, tag, parent_names
parent_tags = Tag.normalise_tags!(locale, parent_names, parent_tags = Tag.normalise_tags!(locale, parent_names,
with_tagme: false, with_tagme: false,
with_no_deerjikist: false) with_no_deerjikist: false,
created_by_user: current_user)
old_parent_tags = tag.parents.to_a old_parent_tags = tag.parents.to_a
+33 -3
ファイルの表示
@@ -1,39 +1,69 @@
class UsersController < ApplicationController class UsersController < ApplicationController
def create def create
if current_user
return render json: { code: current_user.inheritance_code,
user: current_user.slice(:id, :name, :inheritance_code, :role) }
end
user = nil user = nil
User.transaction do User.transaction do
user = User.create!(inheritance_code: SecureRandom.uuid, role: :guest) user = User.create!(inheritance_code: SecureRandom.uuid, role: :guest)
attach_ip_address!(user) attach_ip_address!(user)
end end
establish_user_session(user)
render json: { code: user.inheritance_code, render json: { code: user.inheritance_code,
user: user.slice(:id, :name, :inheritance_code, :role) }, user: user.slice(:id, :name, :inheritance_code, :role) },
status: :created status: :created
end end
def verify def verify
user = User.find_by(inheritance_code: params[:code]) user = if bool?(:restore) && current_user
current_user
else
User.find_by(inheritance_code: params[:code])
end
return render json: { valid: false } unless user return render json: { valid: false } unless user
return head :forbidden if user.banned? return head :forbidden if user.banned?
attach_ip_address!(user) attach_ip_address!(user)
establish_user_session(user)
render json: { valid: true, user: user.slice(:id, :name, :inheritance_code, :role) } render json: { valid: true, user: user.slice(:id, :name, :inheritance_code, :role) }
end end
def show_session
return head :unauthorized unless current_user
render json: current_user.slice(:id, :name, :role)
end
def csrf
response.headers['Cache-Control'] = 'private, no-store'
render json: { token: form_authenticity_token }
end
def destroy_session
reset_session
response.headers['Cache-Control'] = 'private, no-store'
head :no_content
end
def renew def renew
return head :unauthorized unless current_user return head :unauthorized unless current_user
user = current_user user = current_user
user.inheritance_code = SecureRandom.uuid user.inheritance_code = SecureRandom.uuid
user.save! user.save!
establish_user_session(user)
render json: { code: user.inheritance_code } render json: { code: user.inheritance_code }
end end
def me def me
user = User.find_by(inheritance_code: params[:code]) user = current_user
return head :not_found unless user return head :unauthorized unless user
attach_ip_address!(user)
render json: user.slice(:id, :name, :inheritance_code, :role) render json: user.slice(:id, :name, :inheritance_code, :role)
end end
+35 -1
ファイルの表示
@@ -34,11 +34,33 @@ class WikiPagesController < ApplicationController
render json: render json:
end end
def autolink_candidates
locale = resolve_locale!
language = params.fetch(:language, locale.language_code)
language = 'ja' if language == 'und'
return head :not_found unless language.is_a?(String)
return head :not_found unless Language.exists?(code: language)
names = TagDescription.joins(tag: :tag_names)
.where(language_code: locale.language_code, tag_names: { language_code: language })
.pluck('tag_descriptions.tag_id', 'tag_descriptions.id', 'tag_names.name')
candidates = names.map do |tag_id, description_id, name|
WikiPageRepr.autolink_tag(locale, tag_id:, description_id:, name:, language:)
end
candidates.concat(TagGroup.order(:id).pluck(:id, :name).map do |id, name|
WikiPageRepr.autolink_group(locale, id:, name:, language:)
end)
render json: candidates
end
def show def show
locale = resolve_locale! locale = resolve_locale!
tag_description = TagDescription.find_by(id: params[:id], language_code: locale.language_code) reference = TagDescription.find_by(id: params[:id])
tag_description = reference && TagDescription.find_by(
tag_id: reference.tag_id, language_code: locale.language_code)
return render_wiki_page_by_tag_description locale, tag_description if tag_description return render_wiki_page_by_tag_description locale, tag_description if tag_description
return head :not_found if reference
tag_group = TagGroup.find_by(id: params[:id]) tag_group = TagGroup.find_by(id: params[:id])
return render_tag_group locale, tag_group if tag_group return render_tag_group locale, tag_group if tag_group
@@ -63,6 +85,17 @@ class WikiPagesController < ApplicationController
head :not_found head :not_found
end end
def tag_reference
locale = resolve_locale!
description = TagDescription.find_by(id: params[:id])
return head :not_found unless description
tag = Tag.preload(:tag_names, :materials, :deerjikists, :tag_descriptions,
parents: [:tag_names, :materials, :deerjikists, :tag_descriptions])
.find(description.tag_id)
render json: TagRepr.base(tag, language_code: locale.language_code)
end
def exists def exists
locale = resolve_locale! locale = resolve_locale!
if TagDescription.exists?(id: params[:id], language_code: locale.language_code) if TagDescription.exists?(id: params[:id], language_code: locale.language_code)
@@ -214,6 +247,7 @@ class WikiPagesController < ApplicationController
base_revision_id = params[:base_revision_id].presence base_revision_id = params[:base_revision_id].presence
ApplicationRecord.transaction do ApplicationRecord.transaction do
SharedResourceIdAllocator.lock_space!(Tag, 'tags')
tag.lock! tag.lock!
old_title = tag_name.name old_title = tag_name.name
if title != old_title if title != old_title
+1 -1
ファイルの表示
@@ -1,5 +1,5 @@
class Language < ApplicationRecord class Language < ApplicationRecord
has_many :locales, foreign_key: :language_code, primary_key: :code has_many :locales, foreign_key: :language_code, primary_key: :code
enum :code, ja: 'ja' enum :code, ja: 'ja', ain: 'ain'
end end
+25 -13
ファイルの表示
@@ -1,26 +1,38 @@
class Locale < ApplicationRecord class Locale < ApplicationRecord
after_create_commit :generate_tag_names! attr_accessor :certified_tag_names_path
around_save :save_with_tag_names!, if: :will_generate_tag_names?
belongs_to :language, foreign_key: :language_code, primary_key: :code belongs_to :language, foreign_key: :language_code, primary_key: :code
belongs_to :script, foreign_key: :script_code, primary_key: :code belongs_to :script, foreign_key: :script_code, primary_key: :code
enum :code, ja: 'ja', en: 'en', fr: 'fr' # Enum deserialisation uses the first key for each stored value.
# Keep the legacy key for existing predicates, scopes and query arguments.
enum :code, { ja: 'ja', 'ain-Kana' => 'ain-Kana', ain_kana: 'ain-Kana' }
def code= value
super(self.class.codes.fetch(value.to_s, value))
end
def self.nipponese = Locale.find('ja') def self.nipponese = Locale.find('ja')
private private
def generate_tag_names! def will_generate_tag_names?
ApplicationRecord.transaction do deprecated_at.nil? && (will_save_change_to_code? || will_save_change_to_deprecated_at?)
tag_ids = TagName.where(language_code:, primary_flg: true).pluck(:tag_id) end
Tag.where.not(id: tag_ids).includes(:tag_names).find_each do
TagName.create!(tag_id: _1.id, def save_with_tag_names!
language_code:, completed = self.class.transaction(requires_new: true) do
name: TagName.generate_name(self, _1, _1.tag_name('ja')), SharedResourceIdAllocator.lock_space!(Tag, 'tags')
script_code:, saved = yield
primary_flg: true, raise ActiveRecord::RecordNotSaved, 'Locale save was halted' unless saved
auto_generated: true)
end TagNameCompletion.for_locale!(self)
true
end
unless completed
raise ActiveRecord::RecordNotSaved, 'Locale tag completion was rolled back'
end end
end end
end end
+5
ファイルの表示
@@ -1,6 +1,11 @@
class PostVersion < ApplicationRecord class PostVersion < ApplicationRecord
include VersionRecord include VersionRecord
scope :with_previous_version, -> {
joins('LEFT JOIN post_versions prev ON prev.post_id = post_versions.post_id ' +
'AND prev.version_no = post_versions.version_no - 1')
}
belongs_to :post belongs_to :post
belongs_to :parent, class_name: 'Post', optional: true belongs_to :parent, class_name: 'Post', optional: true
+7 -1
ファイルの表示
@@ -1,5 +1,11 @@
class Script < ApplicationRecord class Script < ApplicationRecord
has_many :locales, foreign_key: :script_code, primary_key: :code has_many :locales, foreign_key: :script_code, primary_key: :code
enum :code, Jpan: 'Jpan' # Canonical keys come first so reads and plucks retain ISO 15924 spelling.
# Legacy keys preserve the existing enum predicates, scopes and queries.
enum :code, { 'Jpan' => 'Jpan', 'Kana' => 'Kana', jpan: 'Jpan', kana: 'Kana' }
def code= value
super(self.class.codes.fetch(value.to_s, value))
end
end end
+21 -33
ファイルの表示
@@ -110,7 +110,12 @@ class Tag < ApplicationRecord
def has_deerjikists = deerjikists.loaded? ? deerjikists.any? : deerjikists.exists? def has_deerjikists = deerjikists.loaded? ? deerjikists.any? : deerjikists.exists?
def snapshot_aliases(language_code) def snapshot_aliases language_code
if tag_names.loaded?
return tag_names.select { _1.language_code == language_code && !_1.primary? }
.sort_by(&:name).map(&:name)
end
tag_names.where(language_code:, primary_flg: false).order(:name).pluck(:name) tag_names.where(language_code:, primary_flg: false).order(:name).pluck(:name)
end end
@@ -137,7 +142,8 @@ class Tag < ApplicationRecord
with_tagme: true, with_tagme: true,
with_no_deerjikist: true, with_no_deerjikist: true,
deny_deprecated: false, deny_deprecated: false,
with_sections: false with_sections: false,
created_by_user: nil
if tag_names.any? { |n| n.downcase.start_with?('nico:') } if tag_names.any? { |n| n.downcase.start_with?('nico:') }
raise NicoTagNormalisationError raise NicoTagNormalisationError
end end
@@ -166,7 +172,8 @@ class Tag < ApplicationRecord
name = TagName.canonicalise(locale, name).first name = TagName.canonicalise(locale, name).first
find_or_create_by_tag_name!(locale, name, category: (cat || :general)).tap do |tag| find_or_create_by_tag_name!(locale, name, category: (cat || :general),
created_by_user:).tap do |tag|
if deny_deprecated && tag.deprecated? if deny_deprecated && tag.deprecated?
raise DeprecatedTagNormalisationError, [tag.name(locale.language_code)] raise DeprecatedTagNormalisationError, [tag.name(locale.language_code)]
end end
@@ -254,25 +261,28 @@ class Tag < ApplicationRecord
[left_end_ms, right_end_ms].max [left_end_ms, right_end_ms].max
end end
def find_or_create_by_tag_name! locale, name, category: def find_or_create_by_tag_name! locale, name, category:, created_by_user: nil
language_code = locale.language_code language_code = locale.language_code
name = name.to_s.strip name = name.to_s.strip
ApplicationRecord.transaction do ApplicationRecord.transaction do
tn = TagName.find_by(language_code:, name:) SharedResourceIdAllocator.lock_space!(Tag, 'tags')
tn = TagName.lock.find_by(language_code:, name:)
next tn.tag if tn next tn.tag if tn
tag = Tag.create!(category:) tag = Tag.new(category:)
tn = TagName.create!(tag:, language_code:, name:, tag.tag_names.build(language_code:, name:, script_code: locale.script_code,
script_code: locale.script_code, primary_flg: true, primary_flg: true, auto_generated: false)
auto_generated: false) TagNameCompletion.for_tag!(tag, created_by_user:)
generate_tag_names!(locale, tag, tn)
tag.association(:tag_names).reset tag.association(:tag_names).reset
tag tag
end end
rescue ActiveRecord::RecordNotUnique rescue ActiveRecord::RecordNotUnique
retry existing = TagName.find_by(language_code:, name:)
raise unless existing
existing.tag
end end
def merge_tags! target_tag, source_tags, created_by_user: nil def merge_tags! target_tag, source_tags, created_by_user: nil
@@ -352,28 +362,6 @@ class Tag < ApplicationRecord
total_s * 1_000 + match[:ms].to_s.ljust(3, '0')[0, 3].to_i total_s * 1_000 + match[:ms].to_s.ljust(3, '0')[0, 3].to_i
end end
private
def generate_tag_names! locale, tag, base_tag_name
ApplicationRecord.transaction do
locales =
Locale
.where(deprecated_at: nil)
.where.not(language_code: locale.language_code)
.order(:language_code, :created_at, :script_code)
.uniq { _1.language_code }
locales.each do |locale|
language_code = locale.language_code
script_code = locale.script_code
TagName.create!(tag_id: tag.id,
language_code:,
name: TagName.generate_name(locale, tag, base_tag_name),
script_code:,
primary_flg: true,
auto_generated: true)
end
end
end
end end
private private
+18 -3
ファイルの表示
@@ -34,9 +34,22 @@ class TagName < ApplicationRecord
end end
def self.generate_name locale, tag, base_tag_name def self.generate_name locale, tag, base_tag_name
# TODO: 言語ごとの自動命名ロジック完成したら書く. fallback = "Tag_##{ tag.id }"
name =
"Tag_##{ tag.id }" case [locale.language_code, base_tag_name&.language_code]
when ['ain', 'ja']
TagNameTranscription.ainu(base_tag_name.name)
when ['ja', 'ain']
base_tag_name.name
else
return fallback
end
candidate = new(tag:, name:, language_code: locale.language_code,
script_code: locale.script_code,
primary_flg: true, auto_generated: true)
candidate.valid? ? name : fallback
rescue TagNameTranscription::Unconvertible
fallback
end end
private private
@@ -54,6 +67,8 @@ class TagName < ApplicationRecord
end end
def name_must_be_sanitised def name_must_be_sanitised
return if auto_generated && name == "Tag_##{ tag_id }"
if name? && name != TagNameSanitisationRule.sanitise(name) if name? && name != TagNameSanitisationRule.sanitise(name)
errors.add :name, '名前に使用できない文字が含まれてゐます.' errors.add :name, '名前に使用できない文字が含まれてゐます.'
end end
+103
ファイルの表示
@@ -0,0 +1,103 @@
# Counts uses recorded values only; history/diff endpoints retain their own contract.
class CountsVersionRepr
def self.many(rows, resource:, language_code:, at: nil, inclusive: true)
new(rows.to_a, resource:, language_code:, at:, inclusive:).serialise
end
def initialize(rows, resource:, language_code:, at:, inclusive:)
@rows = rows
@at = at
@inclusive = inclusive
@resource = resource
@language = language_code
@users = User.where(id: rows.map(&:created_by_user_id).compact).pluck(:id, :name).to_h
@languages = master_names(Language)
@scripts = master_names(Script)
@tag_versions = {}
load_tag_versions if resource == 'posts'
end
def serialise
@rows.map do |row|
resource_id = row.public_send(@resource == 'posts' ? :post_id : :tag_id)
common = { id: row.id, resource_id:,
version_no: row.version_no, is_latest: row.attributes['counts_latest'].to_i == 1,
created_at: row.created_at.iso8601,
created_by_user: row.created_by_user_id &&
{ id: row.created_by_user_id, name: @users[row.created_by_user_id] } }
common.merge(@resource == 'posts' ? post(row) : tag(row))
end
end
private
def master_names(model)
model.select('code AS counts_code', :name).to_h do |record|
[record.attributes['counts_code'], record.name]
end
end
def json(value)
value.is_a?(String) ? JSON.parse(value) : value
end
def load_tag_versions
pairs = @rows.flat_map { Array(json(_1.tags_json)) }
.filter_map { |entry| [entry['tag_id'], entry['version_no']] if entry['tag_id'] }
.uniq
pairs.each_slice(500) do |batch|
placeholders = batch.map { '(?, ?)' }.join(', ')
scope = TagVersion.all
if @at
scope = scope.where("created_at #{ @inclusive ? '<=' : '<' } ?", @at)
end
scope.where(["(tag_id, version_no) IN (#{ placeholders })", *batch.flatten])
.select(:tag_id, :version_no, :name, :category, :names).each do |version|
@tag_versions[[version.tag_id, version.version_no]] = version
end
end
end
def names(value)
(json(value) || {}).sort.map do |language, entry|
aliases = entry.fetch('aliases', []).map do |alias_entry|
alias_entry.merge('script_name' => @scripts[alias_entry['script_code']])
end
{ language_code: language, language_name: @languages[language],
script_code: entry['script_code'], script_name: @scripts[entry['script_code']],
name: entry['name'], auto_generated: entry['auto_generated'], aliases: }
end
end
def post(row)
snapshots = Array(json(row.tags_json)).map do |entry|
version = @tag_versions[[entry['tag_id'], entry['version_no']]]
{ id: entry['tag_id'] || entry['external_tag_id'],
kind: entry['tag_id'] ? 'internal' : 'external',
tag_id: entry['tag_id'], external_tag_id: entry['external_tag_id'],
version_no: entry['version_no'], name: entry['name'] || version&.name,
category: entry['category'] || version&.category,
sections: entry.fetch('sections', []), names: version ? names(version.names) : [] }
end
{ title: row.title, url: row.url, thumbnail_base: row.thumbnail_base,
tags: snapshots, parents: row.parent_post_ids.to_s.split.map { { id: _1.to_i } },
video_ms: row.video_ms,
original_created_from: row.original_created_from&.iso8601,
original_created_before: row.original_created_before&.iso8601 }
end
def tag(row)
recorded_names = names(row.names)
selected = recorded_names.find { _1[:language_code] == @language }
aliases = recorded_names.flat_map do |entry|
entry[:aliases].map {
_1.merge('language_code' => entry[:language_code],
'language_name' => entry[:language_name])
}
end
{ name: selected&.fetch(:name) || row.name, names: recorded_names,
category: row.category, aliases:,
parents: row.parent_tag_ids.to_s.split.map { { id: _1.to_i } },
deprecated: row.deprecated_at&.iso8601 }
end
end
+22
ファイルの表示
@@ -9,6 +9,8 @@ module WikiPageRepr
{ 'id' => description.id, { 'id' => description.id,
'resource_type' => 'tag_description', 'resource_type' => 'tag_description',
'tag_id' => tag.id,
'language_code' => description.language_code,
'tag_name_id' => tag.tag_name(locale.language_code)&.id, 'tag_name_id' => tag.tag_name(locale.language_code)&.id,
'body' => description.body, 'body' => description.body,
'created_user_id' => description.created_by_user_id, 'created_user_id' => description.created_by_user_id,
@@ -24,4 +26,24 @@ module WikiPageRepr
def tag_group tag_group def tag_group tag_group
TagGroupRepr.tag_group(tag_group).merge('resource_type' => 'tag_group') TagGroupRepr.tag_group(tag_group).merge('resource_type' => 'tag_group')
end end
def autolink_tag locale, tag_id:, description_id:, name:, language:
{ resource_type: 'tag_description',
resource_id: description_id,
tag_id:,
tag_description_id: description_id,
name:,
language_code: language,
locale_code: locale.code_before_type_cast }
end
def autolink_group locale, id:, name:, language:
{ resource_type: 'tag_group',
resource_id: id,
tag_id: nil,
tag_description_id: nil,
name: "#{ TagGroupRepr::PREFIX }#{ name }",
language_code: language,
locale_code: locale.code_before_type_cast }
end
end end
+105
ファイルの表示
@@ -0,0 +1,105 @@
require 'json'
require 'pathname'
class CertifiedTagNameTranslations
FORMAT = 'btrc-hub.certified-tag-name-translations.v1'.freeze
class Error < ArgumentError
;
end
def self.load! path, locale:
new(path, locale:).load!
end
def initialize path, locale:
@path = path
@locale = locale
end
def load!
document = read_document
unless document.is_a?(Hash) && document['format'] == FORMAT
fail!('invalid format or version')
end
source_code = document['source_locale']
target_code = document['target_locale']
unless target_code.is_a?(String) && target_code == Locale.codes.fetch(@locale.code)
fail!("target_locale #{ target_code.inspect } does not match the new Locale")
end
unless source_code.is_a?(String) && Locale.codes.value?(source_code)
fail!("invalid source_locale #{ source_code.inspect }")
end
source_locale = Locale.find_by(code: source_code)
fail!("source Locale #{ source_code.inspect } does not exist") unless source_locale
fail!('source_locale and target_locale must differ') if source_code == target_code
entries = validate_entries(document['entries'], source_code, target_code)
resolve_tags(entries, source_locale.language_code)
end
private
def read_document
unless (@path.is_a?(String) && !@path.include?("\0") &&
Pathname.new(@path).absolute?)
fail!('certified_tag_names_path must be an absolute file path')
end
fail!('JSON file does not exist or is not a regular file') unless File.file?(@path)
text = File.read(@path, encoding: Encoding::UTF_8)
fail!('JSON file is not valid UTF-8') unless text.valid_encoding?
JSON.parse(text)
rescue JSON::ParserError, SystemCallError, IOError, EncodingError => error
fail!("cannot read JSON: #{ error.message }")
end
def validate_entries entries, source_code, target_code
fail!('entries must be an array') unless entries.is_a?(Array)
sources = {}
targets = {}
entries.each_with_index.map do |entry, index|
fail!("entry #{ index + 1 } must be an object") unless entry.is_a?(Hash)
source = entry[source_code]
target = entry[target_code]
unless source.is_a?(String) && source.present? && target.is_a?(String) && target.present?
fail!("entry #{ index + 1 }: missing or empty #{ source_code }/#{ target_code } name")
end
fail!("duplicate source name #{ source.inspect }") if sources.key?(source)
fail!("duplicate target name #{ target.inspect }") if targets.key?(target)
sources[source] = true
targets[target] = true
[source, target]
end
end
def resolve_tags entries, source_language
source_names = entries.map(&:first)
# SQL may use a case/accent-insensitive collation; the Ruby keys are exact.
matches = TagName.where(language_code: source_language, primary_flg: true,
name: source_names).pluck(:name, :tag_id).group_by(&:first)
translations = {}
entries.each do |source, target|
rows = matches.fetch(source, [])
next if rows.empty?
unless rows.length == 1
fail!("source primary name #{ source.inspect } must match exactly one tag")
end
tag_id = rows.first.last
if translations.key?(tag_id)
fail!("multiple source primary names resolve to Tag##{ tag_id }: #{ source.inspect }")
end
translations[tag_id] = target
end
translations
end
def fail! message
raise Error, "Certified tag names #{@path.inspect}: #{ message }"
end
end
+98
ファイルの表示
@@ -0,0 +1,98 @@
class CountsAxes
GRAINS = ['year', 'quarter', 'month', 'week', 'day', 'hour'].freeze
def self.time_expression(expression, grain, timezone)
raise ArgumentError unless GRAINS.include?(grain)
connection = ApplicationRecord.connection
# Use the installed TZInfo database, including DST, rather than MySQL timezone tables.
first = Time.utc(1900)
last = Time.utc(2101)
offset = timezone.period_for_utc(first).utc_total_offset
clauses = timezone.transitions_up_to(last, first).filter_map do |transition|
next if transition.offset.utc_total_offset == offset
previous = offset
offset = transition.offset.utc_total_offset
"WHEN #{ expression } < #{ connection.quote(transition.at.to_time) } THEN #{ previous }"
end
seconds = clauses.empty? ? offset.to_s : "CASE #{ clauses.join(' ') } ELSE #{ offset } END"
local = "DATE_ADD(#{ expression }, INTERVAL (#{ seconds }) SECOND)"
case grain
when 'year' then "DATE_FORMAT(#{ local }, '%Y')"
when 'quarter' then "CONCAT(YEAR(#{ local }), '-Q', QUARTER(#{ local }))"
when 'month' then "DATE_FORMAT(#{ local }, '%Y-%m')"
when 'week' then "DATE_FORMAT(#{ local }, '%x-W%v')"
when 'day' then "DATE_FORMAT(#{ local }, '%Y-%m-%d')"
else "DATE_FORMAT(#{ local }, '%Y-%m-%d %H:00')"
end
end
attr_reader :rows, :columns, :all
def initialize(population)
@population = population
@definition = population.definition
@rows = parse(population.params.fetch('rows', '[]'))
@columns = parse(population.params.fetch('cols', '[]'))
@all = rows + columns
raise ArgumentError if all.length > 8 || all.map { _1['field'] }.uniq.length != all.length
end
def expressions
all.map do |axis|
field = axis.fetch('field')
case field
when 'tags'
"CASE WHEN tag_entry.tag_id IS NOT NULL " +
"THEN CONCAT('internal:', tag_entry.tag_id) " +
"ELSE CONCAT('external:', tag_entry.external_tag_id) END"
when 'changed_field' then 'changed.field_key'
else
expression = @definition.expression(field)
@definition.fields[field] == :datetime ? self.class.time_expression(
expression, axis.fetch('grain', 'month'), @population.timezone) : expression
end
end
end
def joins
result = ''
if all.any? { _1['field'] == 'tags' }
result += <<~SQL.squish
JOIN JSON_TABLE(CAST(v.tags_json AS JSON), '$[*]' COLUMNS (
tag_id BIGINT PATH '$.tag_id' NULL ON EMPTY,
external_tag_id BIGINT PATH '$.external_tag_id' NULL ON EMPTY)) tag_entry
SQL
end
if all.any? { _1['field'] == 'changed_field' }
fields = @definition.changes.map do |field|
"SELECT #{ ApplicationRecord.connection.quote(field) } AS field_key"
end.join(' UNION ALL ')
condition = @definition.changes.map do |field|
"(changed.field_key = '#{ field }' AND #{ @population.changed_sql(field) })"
end.join(' OR ')
result += " JOIN (#{ fields }) changed ON (#{ condition })"
end
result
end
private
def parse(value)
axes = JSON.parse(value)
raise ArgumentError unless axes.is_a?(Array)
axes.each do |axis|
raise ArgumentError unless axis.is_a?(Hash) && (axis.keys - ['field', 'grain']).empty?
raise ArgumentError unless @definition.axes(@population.history).include?(axis['field'])
if @definition.fields[axis['field']] == :datetime
raise ArgumentError unless GRAINS.include?(axis.fetch('grain', 'month'))
elsif axis.key?('grain')
raise ArgumentError
end
end
axes
end
end
+51
ファイルの表示
@@ -0,0 +1,51 @@
require 'tempfile'
require 'fileutils'
# Finish generation before reporting readiness; downloads never hold the full file in RAM.
class CountsExportDownload
EXPIRY = 10.minutes
def self.directory = Rails.root.join('tmp', 'counts_exports')
def self.verifier = Rails.application.message_verifier('counts_export_download')
def self.prepare(exporter, owner)
FileUtils.mkdir_p(directory, mode: 0o700)
Dir.glob(directory.join('counts-*.export')).each do |path|
File.unlink(path) if File.mtime(path) < Time.current - EXPIRY
rescue Errno::ENOENT
# Another request may have removed an expired download.
end
file = Tempfile.create(['counts-', '.part'], directory)
complete = false
path = file.path
begin
file.binmode
exporter.body.each { file.write(_1) }
file.close
finished_path = path.sub(/\.part\z/, '.export')
File.rename(path, finished_path)
path = finished_path
token = verifier.generate({ 'file' => File.basename(path),
'resource' => exporter.resource, 'format' => exporter.format,
'owner' => owner }, expires_in: EXPIRY, purpose: 'counts_export')
complete = true
token
ensure
file.close unless file.closed?
File.unlink(path) if !complete && File.exist?(path)
end
end
def self.find(token, owner)
metadata = verifier.verified(token, purpose: 'counts_export')
return unless metadata.is_a?(Hash) && metadata['owner'] == owner
return unless metadata['file']&.match?(/\Acounts-[a-zA-Z0-9_-]+\.export\z/)
return unless CountsResourceDefinition::DEFINITIONS.key?(metadata['resource'])
return unless ['json', 'csv'].include?(metadata['format'])
path = directory.join(metadata['file'])
return unless path.file?
metadata.merge('path' => path.to_s)
end
end
+229
ファイルの表示
@@ -0,0 +1,229 @@
require 'set'
# Export identifiers are stable; UI translations are supplied separately as column labels.
class CountsExportProjection
ROOT_KEYS = { 'event_type' => 'event_type', 'resource_id' => 'resource_id', 'version_no' => 'version_no',
'is_latest' => 'is_latest', 'thumbnail' => 'thumbnail_url',
'thumbnail_base' => 'thumbnail_base', 'title' => 'title', 'url' => 'url',
'tags' => 'tags', 'parents' => 'parents', 'video_ms' => 'video_ms',
'original_created_at' => 'original_created_at',
'original_created_from' => 'original_created_from',
'original_created_before' => 'original_created_before',
'user' => 'created_by_user', 'created_at' => 'created_at',
'name' => 'name', 'names' => 'names', 'category' => 'category',
'aliases' => 'aliases', 'deprecated' => 'deprecated' }.freeze
CHILD_KEYS = { 'id' => 'id', 'versionNo' => 'version_no', 'kind' => 'kind',
'name' => 'name', 'category' => 'category', 'sections' => 'sections',
'names' => 'names', 'aliases' => 'aliases',
'languageCode' => 'language', 'scriptCode' => 'script',
'autoGenerated' => 'auto_generated' }.freeze
NAME_FIELDS = ['languageCode', 'scriptCode', 'name', 'aliases', 'autoGenerated'].freeze
CHILDREN = {
'tags' => ['id', 'versionNo', 'kind', 'name', 'category', 'sections', 'names'],
'names' => NAME_FIELDS,
'aliases' => ['languageCode', 'name', 'scriptCode'],
'parents' => ['id'] }.freeze
attr_reader :columns, :headers
def initialize(params, resource:, summary:, tags:)
@params = params
@labels = JSON.parse(params.fetch('labels', '{}'))
raise ArgumentError unless @labels.is_a?(Hash) && @labels.length <= 100
raise ArgumentError unless @labels.all? { |key, value| valid_label?(key, value) }
@summary = summary
allowed = if summary
tags ? ['kind', 'tag_id', 'name', 'deprecated', 'count'] : ['group', 'count']
else
CountsQuery::COLUMNS.fetch(resource)
end
@columns = params.fetch('columns', allowed.join(',')).split(',')
raise ArgumentError if (@columns - allowed).any? || @columns.uniq != @columns
@headers = JSON.parse(params.fetch('headers', JSON.generate(@columns)))
unless (@headers.is_a?(Array) && @headers.length == @columns.length &&
@headers.all? { _1.is_a?(String) && _1.length <= 200 })
raise ArgumentError
end
@order = params.fetch('order', '').split(',')
@sorts = params.fetch('child_sort', '').split(',').map do |entry|
parts = entry.split(':', -1)
raise ArgumentError unless parts.length == 2
path, direction = parts
raise ArgumentError unless ['asc', 'desc'].include?(direction)
[path, direction]
end
@selected = params.key?('children') ? params['children'].split(',').to_set : nil
valid = CHILDREN.flat_map do |key, fields|
paths(fields, key)
end
raise ArgumentError if @selected && (@selected.to_a - valid).any?
allowed_sorts = valid.map do |path|
parts = path.split('.')
field = parts.pop
"#{ parts.join('.') }/#{ field }"
end
if ((@sorts.map(&:first) - allowed_sorts).any? ||
@sorts.map(&:first).uniq.length != @sorts.length)
raise ArgumentError
end
end
def metadata
@columns.each_with_index.map do |key, index|
{ key: @summary ? key : ROOT_KEYS.fetch(key), label: @headers[index] }
end
end
def project(row)
@columns.to_h do |key|
if @summary
value = key == 'group' ? { key: row[:key], name: row[:name] } : row[key.to_sym]
[key, value]
else
value = case key
when 'thumbnail' then row[:thumbnail_base]
when 'original_created_at'
{ from: row[:original_created_from], before: row[:original_created_before] }
else row[ROOT_KEYS.fetch(key).to_sym]
end
value = nested(value || [], CHILDREN.fetch(key), key) if CHILDREN.key?(key)
[ROOT_KEYS.fetch(key), value]
end
end
end
def csv_record(record, row)
record.map do |key, value|
key == 'group' ? row[:name] || row[:key] : csv_display(value, key)
end
end
private
def valid_label?(key, value)
key.is_a?(String) && value.is_a?(String) && key.length <= 100 && value.length <= 300
end
def csv_display(value, key)
return value ? @labels.fetch('latest', 'true') : '' if key == 'is_latest'
return @labels.fetch("kind:#{ value }", value) if key == 'kind'
return @labels.fetch("category:#{ value }", value) if key == 'category'
if ['language', 'script'].include?(key) && value.is_a?(Hash)
return @labels.fetch("#{ key }:#{ value[:code] }",
value[:name] || @labels.fetch('notRegistered', ''))
end
return @labels.fetch("boolean:#{ value }", value) if value == true || value == false
return value.map { csv_display(_1, key) } if value.is_a?(Array)
if value.is_a?(Hash)
return value.to_h do |field, item|
[@labels.fetch(field.to_s, field), csv_display(item, field.to_s)]
end
end
value
end
def fields_for(key)
return NAME_FIELDS if key == 'names'
return ['name', 'scriptCode'] if key == 'aliases'
[]
end
def paths(fields, prefix)
fields.flat_map do |key|
path = "#{ prefix }.#{ key }"
[path, *paths(fields_for(key), path)]
end
end
def fetch(row, key) = row.key?(key) ? row[key] : row[key.to_sym]
def named_code(row, kind)
code = fetch(row, "#{ kind }_code")
name = fetch(row, "#{ kind }_name")
{ code:, name: name.present? && name.to_s.downcase != code.to_s.downcase ? name : nil }
end
def stable(value)
case value
when Hash
value.stringify_keys.sort.to_h.transform_values { stable(_1) }
when Array
value.map { stable(_1) }
else
value
end
end
def sort_value(row, key)
if key == 'languageCode'
language = named_code(row, 'language')
return @labels.fetch("language:#{ language[:code] }", language[:name])
end
return named_code(row, 'script')[:name] if key == 'scriptCode'
fetch(row, CHILD_KEYS.fetch(key))
end
def comparison(left, right)
if left.is_a?(Array) && right.is_a?(Array) && left.length != right.length
return left.length <=> right.length
end
return left <=> right if left.is_a?(Numeric) && right.is_a?(Numeric)
if [true, false].include?(left) && [true, false].include?(right)
return (left ? 1 : 0) <=> (right ? 1 : 0)
end
a = left.is_a?(Hash) || left.is_a?(Array) ? JSON.generate(stable(left)) : left.to_s
b = right.is_a?(Hash) || right.is_a?(Array) ? JSON.generate(stable(right)) : right.to_s
a <=> b
end
def sorted(rows, prefix, fields)
sorts = @sorts.filter_map do |path, direction|
parent, key = path.split('/', 2)
next unless parent == prefix
raise ArgumentError unless fields.include?(key)
[key, direction]
end
rows.each_with_index.sort do |(left, a), (right, b)|
result = 0
sorts.each do |key, direction|
first = sort_value(left, key)
last = sort_value(right, key)
result = if first.nil? || last.nil?
first.nil? == last.nil? ? 0 : (first.nil? ? 1 : -1)
else
comparison(first, last) * (direction == 'asc' ? 1 : -1)
end
break unless result.zero?
end
result.zero? ? a <=> b : result
end.map(&:first)
end
def nested(rows, fields, prefix)
keys = fields.select { !@selected || @selected.include?("#{ prefix }.#{ _1 }") }
ordered = (@order + keys.map { "#{ prefix }.#{ _1 }" }).uniq
.filter_map { |path| keys.find { "#{ prefix }.#{ _1 }" == path } }
sorted(rows, prefix, fields).map do |row|
ordered.to_h do |key|
stable = CHILD_KEYS.fetch(key)
value = case key
when 'languageCode' then named_code(row, 'language')
when 'scriptCode' then named_code(row, 'script')
else fetch(row, stable)
end
if fields_for(key).any?
value = nested(value || [], fields_for(key), "#{ prefix }.#{ key }")
end
[stable, value]
end
end
end
end
+146
ファイルの表示
@@ -0,0 +1,146 @@
require 'csv'
# One read view and a fixed point for the entire download; only one page is held in memory.
class CountsExporter
EXTRA_KEYS = ['basis', 'format', 'resource', 'headers', 'children', 'order',
'summary_columns', 'nested_json', 'child_sort', 'labels'].freeze
attr_reader :format, :resource
def initialize(params, language_code:)
if params.key?('definition')
@table_export = CountsTableExport.new(params)
@format = @table_export.format
@resource = @table_export.resource
return
end
allowed = (CountsQuery::KEYS + CountsStateQuery::KEYS + EXTRA_KEYS).uniq
raise ArgumentError if (params.keys - allowed).any?
raise ArgumentError unless params.values.all? { _1.is_a?(String) }
@params = params
@language = language_code
@format = params.fetch('format', 'json')
@resource = params.fetch('resource', 'posts')
@mode = params.fetch('mode', 'summary')
@history = params.fetch('basis', 'state') == 'history'
raise ArgumentError unless ['json', 'csv'].include?(format)
raise ArgumentError unless ['posts', 'tags'].include?(resource)
raise ArgumentError unless ['summary', 'details'].include?(@mode)
raise ArgumentError unless ['state', 'history'].include?(params.fetch('basis', 'state'))
@query_params = params.except(*EXTRA_KEYS, 'columns', 'page', 'limit')
.merge('mode' => @mode, 'page' => '1', 'limit' => '100')
point = CountsStateQuery.new(@query_params.slice('at').merge('resource' => resource),
language_code: @language).at
@query_params['at'] = point.iso8601(6)
@query_params['axis'] ||= !@history && resource == 'posts' ? 'tags' : 'month'
@projection = CountsExportProjection.new(params, resource:,
summary: @mode == 'summary',
tags: !@history && @query_params['axis'] == 'tags')
if @history
@query_params.delete('kind')
history_query
else
@query_params.delete('latest')
@state_query = CountsStateQuery.new(@query_params.merge('resource' => resource),
language_code: @language).validate_query!
end
end
def body
return @table_export.body if @table_export
Enumerator.new do |output|
ApplicationRecord.transaction(isolation: :repeatable_read) do
first_row = true
started = false
pages do |page|
unless started
meta = { resource:, mode: @mode, basis: @history ? 'history' : 'state',
at: @query_params['at'], boundary: 'inclusive',
filters: @query_params.except('sort', 'page', 'limit', 'mode'),
sort: export_sort,
child_sort: sort_metadata(@params.fetch('child_sort', '')),
columns: @projection.metadata,
children: @params['children']&.split(','),
field_order: @params.fetch('order', '').split(','), timezone: 'UTC',
count: @mode == 'summary' ? page[:group_count] : page[:count],
totals: page[:totals], coverage: page[:coverage] }
output << (format == 'json' ? "{\"metadata\":#{ meta.to_json },\"data\":[" : "\uFEFF")
if format == 'csv'
output << CSV.generate_line(
@projection.headers.map { csv_value(_1) }, row_sep: "\r\n")
end
started = true
end
(page[:versions] || page[:state_rows] || page[:groups] || []).each do |row|
record = @projection.project(row)
if format == 'json'
output << ',' unless first_row
output << record.to_json
else
values = @projection.csv_record(record, row).map { csv_value(_1) }
output << CSV.generate_line(values, row_sep: "\r\n")
end
first_row = false
end
end
output << ']}' if format == 'json'
end
end
end
private
def export_sort
sort_metadata(@query_params.fetch('sort', default_sort))
end
def sort_metadata(value)
value.split(',').map do |entry|
key, direction = entry.split(':', 2)
{ key:, direction: }
end
end
def default_sort
return 'created_at:desc' if @mode == 'details'
!@history && @query_params['axis'] == 'tags' ? 'count:desc' : 'group:asc'
end
def history_query
scope = resource == 'posts' ? PostVersion.with_previous_version : TagVersion.all
CountsQuery.new(scope, resource, @query_params)
end
def pages
unless @history
@state_query.each_page { yield _1 }
return
end
query = history_query
total = @mode == 'summary' ? nil : query.count
loop do
result = if @mode == 'summary'
query.summary(include_count: @query_params['page'] == '1')
else
{ count: total,
versions: CountsVersionRepr.many(query.details, resource:,
language_code: @language,
at: Time.iso8601(@query_params['at'])) }
end
total ||= result[:group_count]
yield result
break if @query_params['page'].to_i * 100 >= total
@query_params['page'] = (@query_params['page'].to_i + 1).to_s
query = history_query
end
end
def csv_value(value)
text = value.is_a?(Hash) || value.is_a?(Array) ? JSON.generate(value) : value.to_s
text.match?(/\A\s*[=+@-]/) ? "'#{ text }" : text
end
end
+208
ファイルの表示
@@ -0,0 +1,208 @@
# Population first, then filters. Previous versions are joined before period filtering.
class CountsPopulation
attr_reader :definition, :params, :at, :history, :scope, :timezone
def initialize(params)
@params = params
@definition = CountsResourceDefinition.new(params.fetch('resource', 'posts'))
@history = params.fetch('definition', 'latest') == 'history'
raise ArgumentError unless ['latest', 'history'].include?(params.fetch('definition', 'latest'))
@timezone = TZInfo::Timezone.get(params.fetch('timezone', 'UTC'))
@at = timestamp(params['execution_at']) || Time.current.utc
raise ArgumentError if at > Time.current + 1.second
@scope = definition.model.all
make_population
apply_filters
rescue TZInfo::InvalidTimezoneIdentifier
raise ArgumentError
end
def timestamp(value, end_date: false)
return nil if value.nil? || value == ''
raise ArgumentError unless value.is_a?(String)
if value.match?(/\A\d{4}-\d{2}-\d{2}\z/)
date = Date.iso8601(value)
date += 1 if end_date
timezone.local_to_utc(Time.utc(date.year, date.month, date.day))
else
raise ArgumentError unless value.match?(/(?:Z|[+-]\d{2}:\d{2})\z/)
time = Time.iso8601(value).utc
raise ArgumentError unless time.year.between?(1900, 2100)
time
end
rescue TZInfo::AmbiguousTime, TZInfo::PeriodNotFound, Date::Error
raise ArgumentError
end
def latest_sql(row = 'v', cutoff = nil)
cutoff ||= history ? at : timestamp(params['at']) || at
keys = definition.keys.map { "newer.#{ _1 } = #{ row }.#{ _1 }" }.join(' AND ')
bound = definition.model.connection.quote(cutoff)
<<~SQL.squish
NOT EXISTS (SELECT 1 FROM #{ definition.table } newer
WHERE #{ keys } AND newer.version_no > #{ row }.version_no
AND newer.#{ definition.time_column } <= #{ bound })
SQL
end
def changed_sql(key)
column = definition.column(key)
"(previous.id IS NOT NULL AND NOT (v.#{ column } <=> previous.#{ column }))"
end
private
def make_population
table = definition.table
@scope = @scope.from("#{ table } v").select('v.*')
keys = definition.keys.map { "previous.#{ _1 } = v.#{ _1 }" }.join(' AND ')
@scope = @scope.joins("LEFT JOIN #{ table } previous ON #{ keys } " +
'AND previous.version_no = v.version_no - 1')
@scope = @scope.where("v.#{ definition.time_column } <= ?", at)
unless history
point = timestamp(params['at']) || at
raise ArgumentError if point > at
@scope = @scope.where("v.#{ definition.time_column } <= ?", point)
.where(Arel.sql(latest_sql('v', point))).where.not('v.event_type' => 'discard')
return
end
first = timestamp(params['from'])
last = timestamp(params['through'], end_date: true)
raise ArgumentError if first && last && first > last
@scope = @scope.where('v.created_at >= ?', first) if first
if last
operator = params['through'].length == 10 ? '<' : '<='
@scope = @scope.where("v.created_at #{ operator } ?", last)
end
end
def apply_filters
['resource_id', 'user'].each do |key|
value = params[key == 'user' ? 'user_id' : key]
next if value.nil? || value == ''
raise ArgumentError unless value.match?(/\A[0-9]+\z/)
@scope = @scope.where("#{ definition.expression(key) } = ?", value.to_i)
end
apply_tags if params['tags'].present? || params['tag'].present?
conditions = JSON.parse(params.fetch('conditions', '[]'))
raise ArgumentError unless conditions.is_a?(Array) && conditions.length <= 100
conditions.each { apply_condition(_1) }
drills = JSON.parse(params.fetch('drill', '[]'))
raise ArgumentError unless drills.is_a?(Array) && drills.length <= 100
drills.each do |entry|
key = entry.fetch('field')
if key == 'tags'
@scope = @scope.where(snapshot_membership(entry.fetch('value')))
elsif key == 'changed_field'
raise ArgumentError unless history && definition.changes.include?(entry['value'])
@scope = @scope.where(Arel.sql(changed_sql(entry['value'])))
elsif definition.fields[key] == :datetime && entry['grain']
@scope = @scope.where(definition.expression(key) => nil) if entry['value'].nil?
if entry['value']
expression = CountsAxes.time_expression(definition.expression(key),
entry['grain'], timezone)
@scope = @scope.where("#{ expression } = ?", entry['value'])
end
else
raise ArgumentError unless definition.axes(history).include?(key)
@scope = @scope.where(definition.expression(key) => entry['value'])
end
end
end
def apply_condition(entry)
raise ArgumentError unless entry.is_a?(Hash)
raise ArgumentError if (entry.keys - ['field', 'operator', 'value', 'end', 'blank']).any?
key = entry.fetch('field')
type = definition.fields.fetch(key) { raise ArgumentError }
operator = entry.fetch('operator')
raise ArgumentError unless CountsResourceDefinition::OPERATORS.fetch(type, []).include?(operator)
raise ArgumentError unless [true, false, nil].include?(entry['blank'])
expression = definition.expression(key)
blank = type == :string || type == :enum ?
"(#{ expression } IS NULL OR #{ expression } = '')" : "#{ expression } IS NULL"
if entry['blank']
@scope = @scope.where(Arel.sql(blank))
return
end
values = [entry['value'], entry['end']]
raise ArgumentError unless values.all? { _1.nil? || _1.is_a?(String) }
return if values.all? { _1.nil? || _1 == '' }
parsed = values.map do |value|
next nil if value.nil? || value == ''
case type
when :integer
raise ArgumentError unless value.match?(/\A-?\d+\z/)
value.to_i
when :datetime then timestamp(value)
else value
end
end
if operator == 'range'
@scope = @scope.where("#{ expression } >= ?", parsed[0]) if parsed[0]
@scope = @scope.where("#{ expression } <= ?", parsed[1]) if parsed[1]
return
end
return unless parsed[0]
sql_operator = { 'eq' => '=', 'ne' => '<>', 'gt' => '>', 'lt' => '<',
'ge' => '>=', 'le' => '<=', 'like' => 'LIKE' }.fetch(operator)
sql = "#{ expression } #{ sql_operator } ?"
sql = "(#{ sql } OR #{ blank })" if operator == 'ne'
@scope = @scope.where(sql, parsed[0])
end
def snapshot_membership(value)
kind, identifier = value.to_s.split(':', 2)
raise ArgumentError unless ['internal', 'external'].include?(kind)
raise ArgumentError unless identifier&.match?(/\A[1-9]\d*\z/)
key = kind == 'internal' ? 'tag_id' : 'external_tag_id'
definition.model.sanitize_sql_array([
"JSON_CONTAINS(v.tags_json, JSON_OBJECT('#{ key }', CAST(? AS UNSIGNED)))",
identifier.to_i])
end
def apply_tags
raise ArgumentError unless definition.resource == 'posts'
if params['tag'].present?
@scope = @scope.where(snapshot_membership(
"#{ params.fetch('kind', 'internal') }:#{ params['tag'] }"))
end
return unless params['tags'].present?
locale = Locale.find_by!(code: params.fetch('locale', 'ja'))
refs = PostTagSearch.parse_references(params['tag_refs'], params['tags'].split)
raw = params['tags']
literals = PostTagSearch.literals(locale, raw, references: refs)
predicates = literals.map do |literal|
internal, external = PostTagSearch.references(locale, literal)
parts = Array(internal).map { snapshot_membership("internal:#{ _1 }") } +
Array(external).map { snapshot_membership("external:#{ _1 }") }
expression = parts.empty? ? 'FALSE' : "(#{ parts.join(' OR ') })"
literal[:negative] ? "NOT #{ expression }" : expression
end
return if predicates.empty?
any = params['match'] == 'any' || raw.split.any? { _1.downcase == 'type:or' }
@scope = @scope.where(Arel.sql("(#{ predicates.join(any ? ' OR ' : ' AND ') })"))
end
end
+208
ファイルの表示
@@ -0,0 +1,208 @@
# Read-only history queries. SQL identifiers and expressions are server-owned.
class CountsQuery
AXES = ['month', 'user'].freeze
COMMON_COLUMNS = ['resource_id', 'version_no', 'is_latest', 'event_type', 'user', 'created_at'].freeze
COLUMNS = {
'posts' => COMMON_COLUMNS + [
'thumbnail', 'thumbnail_base', 'title', 'url', 'tags', 'parents', 'video_ms',
'original_created_at', 'original_created_from', 'original_created_before'],
'tags' => COMMON_COLUMNS + [
'name', 'names', 'category', 'aliases', 'parents', 'deprecated'] }.freeze
KEYS = [
'locale', 'mode', 'axis', 'page', 'limit', 'from', 'before', 'latest',
'user_id', 'resource_id', 'tag', 'columns', 'sort', 'at'].freeze
attr_reader :page, :limit, :mode, :axis
def initialize(scope, resource, params, latest_expression: nil)
@table = scope.klass.table_name
@resource = resource
@id_column = resource == 'posts' ? 'post_id' : 'tag_id'
@params = params
@latest_expression = latest_expression
@scope = scope.except(:select, :order).joins(
"LEFT JOIN users counts_actor ON counts_actor.id = #{ @table }.created_by_user_id")
raise ArgumentError unless (params.keys - KEYS).empty?
raise ArgumentError unless params.values.all? { _1.is_a?(String) }
@mode = params.fetch('mode', 'summary')
@axis = params.fetch('axis', 'month')
raise ArgumentError unless ['summary', 'details'].include?(mode) && AXES.include?(axis)
@page = integer('page', 1)
@limit = integer('limit', 20)
raise ArgumentError if limit > 100 || page > 1_000_000
columns = params.fetch('columns', COLUMNS.fetch(resource).join(',')).split(',')
if columns.empty? || columns.uniq != columns || (columns - COLUMNS.fetch(resource)).any?
raise ArgumentError
end
@sort = parse_sort
apply_filters
end
def relation = @scope
def count = @scope.count
def latest_expression
return @latest_expression if @latest_expression
at = timestamp('at')
cutoff = at ? "AND newer.created_at <= #{ @scope.klass.connection.quote(at) }" : ''
<<~SQL.squish
NOT EXISTS (SELECT 1 FROM #{ @table } newer
WHERE newer.#{ @id_column } = #{ @table }.#{ @id_column }
AND newer.version_no > #{ @table }.version_no #{ cutoff })
SQL
end
def details
@scope.select("#{ @table }.*", Arel.sql("(#{ latest_expression }) AS counts_latest"))
.order(Arel.sql(order_sql(detail_expressions, ["#{ @table }.id DESC"])))
.limit(limit).offset((page - 1) * limit)
end
def summary(include_count: true)
expression = axis == 'month' ? "DATE_FORMAT(#{ @table }.created_at, '%Y-%m')" :
"#{ @table }.created_by_user_id"
group_order = axis == 'user' ? 'MAX(counts_actor.name)' : expression
sql = @scope.group(Arel.sql(expression)).select(Arel.sql(
"#{ expression } AS group_key, COUNT(*) AS record_count, " +
"#{ group_order } AS group_name")).to_sql
connection = @scope.klass.connection
source = "FROM (#{ sql }) counts_groups"
groups = include_count ? connection.select_value("SELECT COUNT(*) #{ source }") : nil
expressions = { 'group' => ['group_name'], 'count' => ['record_count'] }
order = order_sql(expressions, ['group_key IS NULL ASC', 'group_key ASC'])
rows = connection.select_all(
"SELECT group_key, group_name, record_count #{ source } " +
"ORDER BY #{ order } LIMIT #{ limit } OFFSET #{ (page - 1) * limit }")
{ group_count: groups&.to_i,
groups: rows.map do |row|
{ key: row['group_key'],
name: axis == 'user' ? row['group_name'] : nil,
count: row['record_count'].to_i }
end }
end
private
def parse_sort
value = @params.fetch('sort', mode == 'details' ? 'created_at:desc' : 'group:asc')
return [] if value.empty?
values = value.split(',', -1).map { _1.split(':', -1) }
allowed = mode == 'details' ? COLUMNS.fetch(@resource) - ['thumbnail'] : ['group', 'count']
invalid = values.any? do |key, direction, *extra|
extra.any? || !allowed.include?(key) || !['asc', 'desc'].include?(direction)
end
if (values.empty? || values.length > allowed.length || invalid ||
values.map(&:first).uniq.length != values.length)
raise ArgumentError
end
values
end
# NULL always sorts last, including descending keys. The final key is unique.
def order_sql(expressions, final)
(@sort.flat_map do |key, direction|
expressions.fetch(key).flat_map do |expression|
["(#{ expression }) IS NULL ASC", "#{ expression } #{ direction.upcase }"]
end
end + final).join(', ')
end
def detail_expressions
common = { 'resource_id' => ["#{ @table }.#{ @id_column }"],
'version_no' => ["#{ @table }.version_no"],
'is_latest' => ["(#{ latest_expression })"],
'user' => ['counts_actor.name', "#{ @table }.created_by_user_id"],
'created_at' => ["#{ @table }.created_at"] }
if @resource == 'posts'
fields = [
'title', 'url', 'thumbnail_base', 'video_ms',
'original_created_from', 'original_created_before']
common.merge(fields.to_h { [_1, ["#{ @table }.#{ _1 }"]] }).merge(
'original_created_at' => [
'post_versions.original_created_from', 'post_versions.original_created_before'],
'tags' => json_order('post_versions.tags_json'),
'parents' => list_order('post_versions.parent_post_ids'))
else
name = "JSON_UNQUOTE(JSON_EXTRACT(tag_versions.names, #{ locale_name_path }))"
common.merge(
'name' => ["COALESCE(#{ name }, tag_versions.name)"],
'names' => json_order('tag_versions.names'),
'category' => ['tag_versions.category'],
'aliases' => json_order(
"JSON_EXTRACT(tag_versions.names, '$.*.aliases[*]')"),
'parents' => list_order('tag_versions.parent_tag_ids'),
'deprecated' => ['tag_versions.deprecated_at'])
end
end
def locale_name_path
language = Locale.find_by(code: @params.fetch('locale', 'ja'))&.language_code || 'ja'
path = "$.#{ JSON.generate(language) }.name"
@scope.klass.connection.quote(path)
end
# Multi-valued parent cells sort by cardinality, then binary snapshot text.
def json_order(column)
[
"COALESCE(JSON_LENGTH(#{ column }), 0)",
"COALESCE(CAST(#{ column } AS CHAR), '[]') COLLATE utf8mb4_bin"]
end
def list_order(column)
[
"CASE WHEN #{ column } = '' THEN 0 ELSE " +
"1 + LENGTH(#{ column }) - LENGTH(REPLACE(#{ column }, ' ', '')) END",
"#{ column } COLLATE utf8mb4_bin"]
end
def integer(key, default = nil)
value = @params[key].presence
return default if value.nil? && !default.nil?
raise ArgumentError unless value&.match?(/\A[1-9][0-9]{0,9}\z/)
value.to_i
end
def timestamp(key)
value = @params[key].presence
return nil unless value
pattern = /\A\d{4}-\d{2}-\d{2}T\d{2}:\d{2}(?::\d{2}(?:\.\d{1,6})?)?Z\z/
raise ArgumentError unless value.match?(pattern)
Date.iso8601(value.first(10))
Time.iso8601(value.length == 17 ? value.sub(/Z\z/, ':00Z') : value)
end
def apply_filters
at = timestamp('at')
first = timestamp('from')
last = timestamp('before')
raise ArgumentError if first && last && first >= last
raise ArgumentError unless ['0', '1'].include?(@params.fetch('latest', '1'))
@scope = @scope.where("#{ @table }.created_at <= ?", at) if at
@scope = @scope.where(Arel.sql(latest_expression)) unless @params['latest'] == '0'
@scope = @scope.where("#{ @table }.created_at >= ?", first) if first
@scope = @scope.where("#{ @table }.created_at < ?", last) if last
if @params['resource_id'].present?
@scope = @scope.where(@table => { @id_column => integer('resource_id') })
end
if @params['user_id'].present?
user = @params['user_id'] == 'none' ? nil : integer('user_id')
@scope = @scope.where(@table => { created_by_user_id: user })
end
return unless @params['tag'].present?
raise ArgumentError unless @resource == 'posts'
id = integer('tag')
@scope = @scope.where(
"JSON_CONTAINS(post_versions.tags_json, JSON_OBJECT('tag_id', CAST(? AS UNSIGNED))) OR " +
"JSON_CONTAINS(prev.tags_json, JSON_OBJECT('tag_id', CAST(? AS UNSIGNED)))", id, id)
end
end
+65
ファイルの表示
@@ -0,0 +1,65 @@
# Server-owned version identities, attributes and field types. No client SQL is accepted.
class CountsResourceDefinition
COMMON = { 'resource_id' => :integer, 'version_no' => :integer,
'event_type' => :enum, 'user' => :integer, 'created_at' => :datetime }.freeze
DEFINITIONS = {
'posts' => [PostVersion, ['post_id'], {
'title' => :string, 'url' => :string, 'thumbnail_base' => :string,
'tags_json' => :json, 'parent_post_ids' => :string, 'video_ms' => :integer,
'original_created_from' => :datetime, 'original_created_before' => :datetime }],
'tags' => [TagVersion, ['tag_id'], {
'name' => :string, 'names' => :json, 'category' => :enum, 'aliases' => :string,
'parent_tag_ids' => :string, 'deprecated_at' => :datetime }],
'nico_tags' => [NicoTagVersion, ['tag_id'], {
'name' => :string, 'linked_tags' => :string }],
'tag_descriptions' => [TagDescriptionVersion, ['tag_id', 'language_code'], {
'language_code' => :enum, 'body' => :string,
'source_language_code' => :enum, 'source_version_no' => :integer }],
'tag_groups' => [TagGroupVersion, ['tag_group_id'], {
'name' => :string, 'description' => :string,
'members' => :json, 'placements' => :json }],
'materials' => [MaterialVersion, ['material_id'], {
'url' => :string, 'parent_id' => :integer, 'tag_id' => :integer,
'tag_name' => :string, 'tag_category' => :enum,
'discarded_at' => :datetime, 'updated_at' => :datetime,
'updated_by_user_id' => :integer, 'export_paths_json' => :json,
'file_blob_id' => :integer, 'file_filename' => :string,
'file_content_type' => :string, 'file_byte_size' => :integer,
'file_checksum' => :string, 'file_sha256' => :string,
'source_kind' => :enum, 'source_uri' => :string, 'source_path' => :string,
'source_file_id' => :string, 'normalized_source_key' => :string }] }.freeze
OPERATORS = {
string: ['eq', 'ne', 'like'], enum: ['eq', 'ne'], boolean: ['eq', 'ne'],
integer: ['eq', 'ne', 'gt', 'lt', 'ge', 'le', 'range'],
datetime: ['eq', 'ne', 'gt', 'lt', 'ge', 'le', 'range'] }.freeze
attr_reader :model, :keys, :fields, :resource
def initialize(resource)
@resource = resource
@model, @keys, attributes = DEFINITIONS.fetch(resource) { raise ArgumentError }
@fields = COMMON.merge(attributes)
end
def table = model.table_name
def time_column = 'created_at'
def column(key)
raise ArgumentError unless fields.key?(key)
{ 'resource_id' => keys.first, 'user' => 'created_by_user_id' }.fetch(key, key)
end
def expression(key, row = 'v') = "#{ row }.#{ column(key) }"
def changes = fields.keys - COMMON.keys - ['updated_at', 'updated_by_user_id']
def axes(history)
fields.reject { |_key, type| type == :json }.keys +
(resource == 'posts' ? ['tags'] : []) + (history ? ['changed_field'] : [])
end
def metadata
fields.map do |key, type|
{ key:, type:, nullable: model.columns_hash.fetch(column(key)).null,
operators: OPERATORS.fetch(type, []) }
end
end
end
+366
ファイルの表示
@@ -0,0 +1,366 @@
# Restores one recorded version per resource before applying record filters.
class CountsStateQuery
KEYS = ['locale', 'resource', 'mode', 'at', 'from', 'before', 'grain',
'kind', 'tag', 'page', 'limit', 'sort', 'axis', 'user_id', 'resource_id'].freeze
MAX_PERIODS = 366
attr_reader :at
def initialize(params, language_code:)
raise ArgumentError unless (params.keys - KEYS).empty?
raise ArgumentError unless params.values.all? { _1.is_a?(String) }
@params = params
@connection = ApplicationRecord.connection
@language = language_code
@resource = params.fetch('resource', 'posts')
@mode = params.fetch('mode', 'summary')
@inclusive = ['summary', 'details'].include?(@mode)
@axis = params.fetch('axis', @resource == 'posts' ? 'tags' : 'month')
@kind = params.fetch('kind', 'internal')
@grain = params.fetch('grain', 'month')
raise ArgumentError unless ['posts', 'tags'].include?(@resource)
raise ArgumentError unless ['summary', 'details', 'state', 'trend', 'snapshot'].include?(@mode)
raise ArgumentError unless ['internal', 'external', 'all'].include?(@kind)
raise ArgumentError unless ['day', 'month', 'year'].include?(@grain)
raise ArgumentError unless ['tags', 'month', 'user'].include?(@axis)
if @mode == 'summary' && @axis == 'tags' && @resource != 'posts'
raise ArgumentError
end
@page = integer('page', 1)
@limit = integer('limit', 20)
@tag = integer('tag') if params['tag'].present?
raise ArgumentError if @limit > 100 || @page > 1_000_000
raise ArgumentError if @tag && @kind == 'all'
raise ArgumentError if @mode == 'trend' && !@tag
@now = Time.current.utc
@at = timestamp('at') || @now
raise ArgumentError if @at > @now
end
def call(metadata: true)
@metadata = metadata
# Keep totals, rows, and detail pages on one repeatable-read database snapshot.
ApplicationRecord.transaction do
unless @history_loaded
@post_start = PostVersion.minimum(:created_at)
@tag_start = TagVersion.minimum(:created_at)
@history_loaded = true
end
result = { page: @page, limit: @limit, at: @at.iso8601(6),
coverage: { basis: 'recorded_snapshots', exact: false,
post_history_from: @post_start&.iso8601(6),
tag_history_from: @tag_start&.iso8601(6) } }
case @mode
when 'trend'
result.merge(trend)
when 'summary'
result.merge(metadata ? { totals: totals } : {}).merge(summary)
when 'details', 'snapshot'
result.merge(metadata ? { totals: totals } : {}).merge(details)
else
result.merge(metadata ? { totals: totals } : {}).merge(listing)
end
end
end
def each_page
return enum_for(:each_page) unless block_given?
@page = 1
first = call
yield first
total = ['summary', 'state', 'trend'].include?(@mode) ? first[:group_count] : first[:count]
while @page * @limit < total
@page += 1
yield call(metadata: false)
end
end
def validate_query!
filtered_scope(@resource)
case @mode
when 'summary'
if @axis == 'tags'
order_sql({ 'name' => 'name', 'count' => 'post_count', 'tag_id' => 'tag_id',
'kind' => 'kind', 'deprecated' => 'deprecated_at' }, 'count:desc', [])
else
record_query('summary')
end
when 'details', 'snapshot'
record_query('details')
end
self
end
private
def quote(value) = @connection.quote(value)
def integer(key, default = nil)
value = @params[key]
return default if value.blank? && default
raise ArgumentError unless value&.match?(/\A[1-9][0-9]{0,9}\z/)
value.to_i
end
def timestamp(key)
value = @params[key].presence
return nil unless value
pattern = /\A\d{4}-\d{2}-\d{2}T\d{2}:\d{2}(?::\d{2}(?:\.\d{1,6})?)?Z\z/
raise ArgumentError unless value.match?(pattern)
Date.iso8601(value.first(10))
Time.iso8601(value.length == 17 ? value.sub(/Z\z/, ':00Z') : value).utc
end
# Version numbers order equal-timestamp writes. Do not filter tags before this selection.
def latest_condition(table, row, boundary)
key = table == 'post_versions' ? 'post_id' : 'tag_id'
<<~SQL.squish
#{ row }.created_at #{ @inclusive ? '<=' : '<' } #{ boundary } AND NOT EXISTS (
SELECT 1 FROM #{ table } newer
WHERE newer.#{ key } = #{ row }.#{ key }
AND newer.created_at #{ @inclusive ? '<=' : '<' } #{ boundary }
AND newer.version_no > #{ row }.version_no)
SQL
end
def state_sql(table, boundary, live: false)
sql = "SELECT v.* FROM #{ table } v WHERE #{ latest_condition(table, 'v', boundary) }"
live ? "#{ sql } AND v.event_type <> 'discard'" : sql
end
def membership(row)
key = @kind == 'external' ? 'external_tag_id' : 'tag_id'
"JSON_CONTAINS(#{ row }.tags_json, JSON_OBJECT('#{ key }', #{ @tag }))"
end
# Deprecation keeps a tag in existence. Only an explicit discard removes it.
def tag_visible(boundary)
return '1 = 1' if @kind == 'external'
<<~SQL.squish
NOT EXISTS (SELECT 1 FROM tag_versions t
WHERE t.tag_id = #{ @tag } AND t.event_type = 'discard'
AND #{ latest_condition('tag_versions', 't', boundary) })
SQL
end
def ctes
boundary = quote(@at)
<<~SQL
WITH post_at AS (#{ filtered_scope('posts').to_sql }),
tag_at AS (#{ state_sql('tag_versions', boundary) }),
nico_at AS (#{ state_sql('nico_tag_versions', boundary) }),
pairs AS (
SELECT DISTINCT p.post_id,
CASE WHEN entry.tag_id IS NULL THEN 'external' ELSE 'internal' END AS kind,
COALESCE(entry.tag_id, entry.external_tag_id) AS tag_id
FROM post_at p
JOIN JSON_TABLE(CAST(p.tags_json AS JSON), '$[*]' COLUMNS (
tag_id BIGINT PATH '$.tag_id' NULL ON EMPTY ERROR ON ERROR,
external_tag_id BIGINT PATH '$.external_tag_id' NULL ON EMPTY ERROR ON ERROR
)) entry
LEFT JOIN tag_at t ON t.tag_id = entry.tag_id
WHERE COALESCE(entry.tag_id, entry.external_tag_id) IS NOT NULL
AND (t.event_type IS NULL OR t.event_type <> 'discard')
)
SQL
end
def totals
row = @connection.select_one(<<~SQL)
#{ ctes }
SELECT (SELECT COUNT(*) FROM post_at) AS posts,
(SELECT COUNT(*) FROM (#{ filtered_scope('tags').to_sql }) counted_tags) AS tags,
(SELECT COUNT(*) FROM tag_at WHERE event_type <> 'discard'
AND deprecated_at IS NOT NULL) AS deprecated_tags,
(SELECT COUNT(*) FROM pairs) AS assignments,
(SELECT COUNT(*) FROM pairs WHERE kind = 'external') AS external_assignments
SQL
row.transform_values!(&:to_i)
row.merge('posts' => available?(@post_start, @at) ? row['posts'] : nil,
'assignments' => available?(@post_start, @at) ? row['assignments'] : nil,
'external_assignments' => available?(@post_start, @at) ?
row['external_assignments'] : nil,
'tags' => available?(@tag_start, @at) ? row['tags'] : nil,
'deprecated_tags' => available?(@tag_start, @at) ? row['deprecated_tags'] : nil)
end
def available?(first, boundary)
first && (@inclusive ? first <= boundary : first < boundary)
end
def listing
path = quote("$.#{ JSON.generate(@language) }.name")
catalog = <<~SQL
, tag_counts AS (
SELECT membership.kind, membership.tag_id, COUNT(*) AS post_count
FROM pairs membership GROUP BY membership.kind, membership.tag_id
), catalog AS (
SELECT 'internal' AS kind, tag_id,
COALESCE(JSON_UNQUOTE(JSON_EXTRACT(names, #{ path })), name) AS name,
deprecated_at FROM tag_at WHERE event_type <> 'discard'
UNION ALL
SELECT tc.kind, tc.tag_id, external_version.name, NULL AS deprecated_at
FROM tag_counts tc
LEFT JOIN tag_at internal_version
ON tc.kind = 'internal' AND internal_version.tag_id = tc.tag_id
LEFT JOIN nico_at external_version
ON tc.kind = 'external' AND external_version.tag_id = tc.tag_id
WHERE tc.kind = 'external' OR internal_version.tag_id IS NULL
)
SQL
filters = []
filters << "c.kind = #{ quote(@kind) }" unless @kind == 'all'
filters << "c.tag_id = #{ @tag }" if @tag
where = filters.any? ? "WHERE #{ filters.join(' AND ') }" : ''
source = 'FROM catalog c LEFT JOIN tag_counts n ON n.kind = c.kind AND n.tag_id = c.tag_id'
prefix = "#{ ctes } #{ catalog }"
total = if @metadata
@connection.select_value("#{ prefix } SELECT COUNT(*) #{ source } #{ where }").to_i
end
order = order_sql({ 'name' => 'c.name', 'count' => 'COALESCE(n.post_count, 0)',
'tag_id' => 'c.tag_id', 'kind' => 'c.kind',
'deprecated' => 'c.deprecated_at' }, 'count:desc',
['c.kind ASC', 'c.tag_id ASC'])
rows = @connection.select_all(<<~SQL)
#{ prefix }
SELECT c.kind, c.tag_id, c.name, c.deprecated_at,
COALESCE(n.post_count, 0) AS post_count #{ source } #{ where }
ORDER BY #{ order } LIMIT #{ @limit } OFFSET #{ (@page - 1) * @limit }
SQL
{ count: total, group_count: total, state_rows: rows.map do |row|
{ kind: row['kind'], tag_id: row['tag_id'], name: row['name'],
deprecated: row['deprecated_at']&.iso8601,
count: available?(@post_start, @at) ? row['post_count'].to_i : nil,
at: @at.iso8601(6) }
end }
end
# All record filters apply after selecting the latest recorded version at the point.
def restored_scope(resource)
model = resource == 'posts' ? PostVersion : TagVersion
table = model.table_name
scope = model.where(Arel.sql(latest_condition(table, table, quote(@at))))
.where.not(event_type: 'discard')
if @tag
scope = if resource == 'posts'
scope.where("#{ membership(table) } AND #{ tag_visible(quote(@at)) }")
elsif @kind == 'internal'
scope.where(tag_id: @tag)
else
scope.none
end
end
scope
end
def filtered_scope(resource)
params = @params.slice('from', 'before', 'user_id', 'resource_id')
.merge('mode' => 'details', 'latest' => '0', 'sort' => '')
CountsQuery.new(restored_scope(resource), resource, params,
latest_expression: '1').relation
end
def record_query(mode)
params = @params.slice('locale', 'sort', 'page', 'limit')
.merge('mode' => mode, 'latest' => '0', 'axis' => @axis == 'tags' ? 'month' : @axis,
'page' => @page.to_s, 'limit' => @limit.to_s)
CountsQuery.new(filtered_scope(@resource), @resource, params, latest_expression: '1')
end
def summary
return listing if @axis == 'tags'
query = record_query('summary')
{ count: @metadata ? query.count : nil }.merge(query.summary(include_count: @metadata))
end
def details
query = record_query('details')
{ count: @metadata ? query.count : nil,
versions: CountsVersionRepr.many(query.details, resource: @resource,
language_code: @language, at: @at,
inclusive: @inclusive) }
end
def next_boundary(time)
case @grain
when 'day' then time.beginning_of_day + 1.day
when 'month' then time.beginning_of_month.next_month
else time.beginning_of_year.next_year
end
end
def periods
@before = timestamp('before') || @now.beginning_of_month
@from = timestamp('from') || @before.prev_year
raise ArgumentError if @from >= @before || @before > @now
rows = []
boundary = next_boundary(@from)
while boundary <= @before
raise ArgumentError if rows.length >= MAX_PERIODS
format = { 'day' => '%Y-%m-%d', 'month' => '%Y-%m', 'year' => '%Y' }.fetch(@grain)
rows << [boundary, (boundary - 1.second).strftime(format)]
boundary = next_boundary(boundary)
end
rows
end
def trend
points = periods
metadata = { count: points.length, group_count: points.length, state_rows: [],
from: @from.iso8601, before: @before.iso8601, grain: @grain }
return metadata if points.empty?
values = points.map do |boundary, period|
"SELECT CAST(#{ quote(boundary) } AS DATETIME(6)) AS boundary, " +
"#{ quote(period) } AS period"
end.join(' UNION ALL ')
order = order_sql({ 'period' => 'boundary', 'count' => 'post_count' },
'period:asc', ['boundary ASC'])
known = @post_start ? "periods.boundary > #{ quote(@post_start) }" : 'FALSE'
rows = @connection.select_all(<<~SQL)
WITH periods AS (#{ values }), series AS (
SELECT periods.boundary, periods.period,
CASE WHEN #{ known } THEN COUNT(DISTINCT p.post_id) ELSE NULL END AS post_count
FROM periods
LEFT JOIN post_versions p ON p.event_type <> 'discard'
AND #{ membership('p') }
AND #{ latest_condition('post_versions', 'p', 'periods.boundary') }
AND #{ tag_visible('periods.boundary') }
GROUP BY periods.boundary, periods.period
)
SELECT * FROM series ORDER BY #{ order }
LIMIT #{ @limit } OFFSET #{ (@page - 1) * @limit }
SQL
metadata.merge(state_rows: rows.map do |row|
{ kind: @kind, tag_id: @tag, name: nil, period: row['period'],
at: row['boundary'].utc.iso8601(6), count: row['post_count']&.to_i }
end)
end
def order_sql(expressions, default, final)
raw = @params.fetch('sort', default)
sorts = raw.empty? ? [] : raw.split(',', -1).map { _1.split(':', -1) }
raise ArgumentError if sorts.map(&:first).uniq.length != sorts.length
raise ArgumentError if sorts.length > expressions.length
order = sorts.flat_map do |key, direction, *extra|
if extra.any? || !expressions.key?(key) || !['asc', 'desc'].include?(direction)
raise ArgumentError
end
expression = expressions.fetch(key)
["(#{ expression }) IS NULL ASC", "#{ expression } #{ direction.upcase }"]
end
(order + final).join(', ')
end
end
+172
ファイルの表示
@@ -0,0 +1,172 @@
require 'csv'
# Export uses the exact same population and grouped cells as the interactive table.
class CountsTableExport
EXTRA = ['format', 'columns', 'headers', 'labels', 'children', 'child_sort',
'order', 'nested_json', 'summary_columns', 'basis', 'axis', 'latest'].freeze
attr_reader :format, :resource
def initialize(params)
raise ArgumentError if (params.keys - CountsTableQuery::KEYS - EXTRA).any?
raise ArgumentError unless params.values.all? { _1.is_a?(String) }
@params = params
@labels = JSON.parse(params.fetch('labels', '{}'))
raise ArgumentError unless @labels.is_a?(Hash)
@selected = params.key?('children') ? params['children'].split(',') : nil
@order = params.fetch('order', '').split(',')
@child_sort = params.fetch('child_sort', '').split(',').map { _1.split(':', -1) }
@child_sort.each do |path, direction, *extra|
raise ArgumentError if extra.any? || !['asc', 'desc'].include?(direction)
raise ArgumentError unless path&.match?(/\A[a-zA-Z_]+(?:\.[a-zA-Z]+)*\/[a-zA-Z]+\z/)
end
@format = params.fetch('format', 'json')
@resource = params.fetch('resource', 'posts')
raise ArgumentError unless ['json', 'csv'].include?(format)
@query = CountsTableQuery.new(params.slice(*CountsTableQuery::KEYS)
.merge('page' => '1', 'limit' => '100'))
@summary = @query.mode == 'summary'
@columns = params.fetch('columns', '').split(',')
@headers = JSON.parse(params.fetch('headers', '[]'))
raise ArgumentError unless @headers.is_a?(Array) && @headers.all? { _1.is_a?(String) }
unless @summary
if ['posts', 'tags'].include?(resource)
@projection = CountsExportProjection.new(params, resource:, summary: false, tags: false)
else
allowed = @query.population.definition.fields.keys + ['is_latest']
raise ArgumentError if (@columns - allowed).any?
raise ArgumentError unless @columns.length == @headers.length && @columns.uniq == @columns
end
end
end
def body
Enumerator.new do |output|
ApplicationRecord.transaction(isolation: :repeatable_read) do
@language_names = Language.pluck(:code, :name).to_h
started = false
first = true
@query.each_page do |page|
unless started
metadata = page.except(:versions, :cells, :axis_labels, :column_values).merge(
resource:, filters: @params.slice(*CountsTableQuery::KEYS),
fields: @query.population.definition.metadata,
columns: @columns, column_labels: @headers,
order: @params['order'], children: @params['children'],
sort: @params.fetch('sort', ''), child_sort: @params['child_sort'])
output << (format == 'json' ? "{\"metadata\":#{ metadata.to_json },\"data\":[" : "\uFEFF")
if format == 'csv'
headers = @summary ? summary_headers : @headers
output << CSV.generate_line(headers.map { safe(_1) }, row_sep: "\r\n")
end
started = true
end
(page[:cells] || page[:versions]).each do |row|
record = project(row)
if format == 'json'
output << ',' unless first
output << record.to_json
else
values = if @summary
row[:values].each_with_index.map do |value, index|
page[:axis_labels].fetch("a#{ index }", {}).fetch(value.to_s, value)
end + [row[:count]]
elsif @projection
@projection.csv_record(record, row)
else
record.map { |key, value| csv_logical(key, value) }
end
output << CSV.generate_line(values.map { safe(_1) }, row_sep: "\r\n")
end
first = false
end
end
output << ']}' if format == 'json'
end
end
end
private
def summary_headers
labels = @labels
@query.axes.all.map { labels.fetch(_1['field'], _1['field']) } +
[labels.fetch('count', 'count')]
end
def project(row)
return { row: row[:values].first(@query.axes.rows.length),
column: row[:values].drop(@query.axes.rows.length), count: row[:count] } if @summary
return @projection.project(row) if @projection
@columns.to_h do |column|
key = column == 'user' ? 'created_by_user' : column
value = row[key]
value = project_nested(value, key) if ['members', 'placements', 'export_paths_json'].include?(key)
[key, value]
end
end
def camel(key)
key.gsub(/_([a-z])/) { Regexp.last_match(1).upcase }
end
def project_nested(value, path)
rows = path == 'export_paths_json' ? (value || {}).map { |profile, target|
{ 'profile' => profile, 'path' => target }
} : Array(value)
sorts = @child_sort.filter_map do |key, direction|
parent, field = key.split('/', 2)
[field, direction] if parent == path
end
rows = rows.each_with_index.sort do |(first, a), (last, b)|
compared = 0
sorts.each do |field, direction|
left = first.find { |key, _| camel(key) == field }&.last
right = last.find { |key, _| camel(key) == field }&.last
compared = if left.nil? || right.nil?
left.nil? == right.nil? ? 0 : (left.nil? ? 1 : -1)
else
(left.to_s <=> right.to_s) * (direction == 'asc' ? 1 : -1)
end
break unless compared.zero?
end
compared.zero? ? a <=> b : compared
end.map(&:first)
rows.map do |row|
fields = row.keys.select { !@selected || @selected.include?("#{ path }.#{ camel(_1) }") }
ordered = (@order + fields.map { "#{ path }.#{ camel(_1) }" }).uniq
.filter_map { |item| fields.find { "#{ path }.#{ camel(_1) }" == item } }
ordered.to_h do |key|
item = row[key]
item = project_nested(item, "#{ path }.#{ camel(key) }") if item.is_a?(Array)
[key, item]
end
end
end
def csv_logical(key, value)
return value ? @labels.fetch('latest', '') : '' if key == 'is_latest'
if value.is_a?(Time) || value.is_a?(ActiveSupport::TimeWithZone)
return @query.population.timezone.utc_to_local(value.utc).iso8601
end
if key == 'language_code' || key == 'source_language_code'
return @language_names[value]
end
if value.is_a?(Hash)
return value.to_h { |field, item| [@labels.fetch(camel(field.to_s), ''), csv_logical(field, item)] }
end
return value.map { csv_logical(key, _1) } if value.is_a?(Array)
@labels.fetch("#{ key }:#{ value }", value)
end
def safe(value)
return value if value.is_a?(Numeric)
text = value.is_a?(Hash) || value.is_a?(Array) ? JSON.generate(value) : value.to_s
text.match?(/\A\s*[=+@-]/) ? "'#{ text }" : text
end
end
+252
ファイルの表示
@@ -0,0 +1,252 @@
# Common entrypoint for six version resources; legacy endpoints keep their contracts.
class CountsTableQuery
MAX_COLUMNS = 100
class TooManyColumns < ArgumentError; end
KEYS = ['resource', 'definition', 'mode', 'at', 'execution_at', 'from', 'through',
'timezone', 'rows', 'cols', 'conditions', 'drill', 'tags', 'tag_refs', 'match',
'tag', 'kind', 'resource_id', 'user_id', 'page', 'limit', 'sort', 'locale'].freeze
attr_reader :population, :axes, :page, :limit, :mode
def initialize(params)
raise ArgumentError if (params.keys - KEYS).any?
raise ArgumentError unless params.values.all? { _1.is_a?(String) }
@params = params
@population = CountsPopulation.new(params)
@definition = population.definition
@connection = @definition.model.connection
@axes = CountsAxes.new(population)
@page = Integer(params.fetch('page', '1'), 10)
@limit = Integer(params.fetch('limit', '20'), 10)
@mode = params.fetch('mode', 'summary')
raise ArgumentError unless page.between?(1, 1_000_000) && limit.between?(1, 100)
raise ArgumentError unless ['summary', 'details'].include?(mode)
@sort = params.fetch('sort', '').split(',').map { _1.split(':', -1) }
allowed = mode == 'summary' ? axes.all.each_index.map { "a#{ _1 }" } + ['count'] :
@definition.fields.keys + ['is_latest'] +
(@definition.resource == 'posts' ? ['tags', 'parents', 'original_created_at'] : []) +
(@definition.resource == 'tags' ? ['parents', 'deprecated'] : [])
raise ArgumentError if @sort.map(&:first).uniq.length != @sort.length
@sort.each do |key, direction, *extra|
measure = mode == 'summary' && key&.match?(/\Ameasure_[0-9a-f]+\z/)
measure_values(key) if measure
if extra.any? || (!allowed.include?(key) && !measure) || !['asc', 'desc'].include?(direction)
raise ArgumentError
end
end
end
def call
ApplicationRecord.transaction do
metadata.merge(mode == 'summary' ? summary : details)
end
end
def metadata
{ page:, limit:, execution_at: population.at.iso8601(6), at: @params['at'].presence,
definition: population.history ? 'history' : 'latest',
timezone: population.timezone.identifier, rows: axes.rows, cols: axes.columns,
fields: @definition.metadata,
count: population.scope.except(:select).optimizer_hints('MAX_EXECUTION_TIME(10000)').count }
end
def summary
expressions = axes.expressions
selected = expressions.each_with_index.map do |expression, index|
"#{ expression } AS a#{ index }"
end
source = population.scope
source = source.joins(axes.joins) unless axes.joins.empty?
sql = if expressions.empty?
source.except(:select).select('COUNT(DISTINCT v.id) AS count').to_sql
else
source.except(:select).group(*expressions.map { Arel.sql(_1) })
.select(Arel.sql((selected + ['COUNT(DISTINCT v.id) AS count']).join(', ')))
.to_sql
end
prefix = "WITH cells AS (#{ sql })"
column_values = limited_columns(prefix)
row_keys = axes.rows.each_index.map { "a#{ _1 }" }
# Page whole row tuples so a pivot row is never split across display pages.
row_selection = row_keys.empty? ? '1 AS singleton' : row_keys.join(', ')
row_group = row_keys.empty? ? 'GROUP BY singleton' : "GROUP BY #{ row_keys.join(', ') }"
row_sort = @sort.each_with_index.filter_map do |(key, direction), index|
if key == 'count' || key.start_with?('measure_')
condition = if key == 'count'
'TRUE'
else
measure_values(key).each_with_index.map do |value, offset|
"a#{ axes.rows.length + offset } <=> #{ @connection.quote(value) }"
end.join(' AND ').presence || 'TRUE'
end
["MAX(CASE WHEN #{ condition } THEN count END) AS sort#{ index }",
"sort#{ index } IS NULL ASC, sort#{ index } #{ direction.upcase }"]
elsif row_keys.include?(key)
[nil, "#{ key } IS NULL ASC, #{ key } #{ direction.upcase }"]
end
end
extras = row_sort.filter_map(&:first)
order_rows = row_sort.map(&:last) + row_keys.map { "#{ _1 } IS NULL, #{ _1 }" }
order_rows = ['singleton ASC'] if order_rows.empty?
group_sql = "SELECT #{ ([row_selection] + extras).join(', ') } FROM cells #{ row_group }"
total = @connection.select_value(<<~SQL).to_i
#{ prefix }
SELECT /*+ MAX_EXECUTION_TIME(10000) */ COUNT(*) FROM (#{ group_sql }) row_groups
SQL
join = row_keys.map { "cells.#{ _1 } <=> page_rows.#{ _1 }" }.join(' AND ')
join = 'TRUE' if row_keys.empty?
order_cols = @sort.filter_map do |key, direction|
next unless key.match?(/\Aa[0-9]+\z/) && !row_keys.include?(key)
"cells.#{ key } #{ direction.upcase }"
end
order = ['page_rows.position'] + order_cols +
axes.all.each_index.map { "cells.a#{ _1 }" }
records = @connection.select_all(<<~SQL)
#{ prefix }, row_groups AS (#{ group_sql }),
numbered AS (SELECT row_groups.*,
ROW_NUMBER() OVER (ORDER BY #{ order_rows.join(', ') }) AS position FROM row_groups),
page_rows AS (SELECT * FROM numbered
WHERE position > #{ (page - 1) * limit } AND position <= #{ page * limit })
SELECT /*+ MAX_EXECUTION_TIME(10000) */ cells.* FROM cells JOIN page_rows ON #{ join }
ORDER BY #{ order.join(', ') }
SQL
{ group_count: total, column_values:, cells: records.map do |row|
{ values: axes.all.each_index.map { row["a#{ _1 }"] }, count: row['count'].to_i }
end, axis_labels: axis_labels(records, column_values) }
end
def details
query = population.scope.select(Arel.sql(
"(#{ population.latest_sql }) AS counts_latest"))
order = sort_sql { detail_expression(_1) }
query = query.order(Arel.sql((order + ['v.id DESC']).join(', ')))
.limit(limit).offset((page - 1) * limit)
rows = query.to_a
if ['posts', 'tags'].include?(@definition.resource)
point = population.history ? population.at :
population.timestamp(@params['at']) || population.at
values = CountsVersionRepr.many(rows, resource: @definition.resource,
language_code: locale.language_code,
at: point)
values.each_with_index { |value, index| value[:event_type] = rows[index].event_type }
else
users = User.where(id: rows.map(&:created_by_user_id)).pluck(:id, :name).to_h
values = rows.map do |row|
attributes = @definition.fields.keys.to_h do |key|
[key, row.public_send(@definition.column(key))]
end
attributes.merge('id' => row.id, 'resource_id' => row[@definition.keys.first],
'is_latest' => row.attributes['counts_latest'].to_i == 1,
'created_by_user' => row.created_by_user_id &&
{ id: row.created_by_user_id, name: users[row.created_by_user_id] })
end
end
{ versions: values }
end
def each_page
total = nil
loop do
result = call
total ||= mode == 'summary' ? result[:group_count] : result[:count]
yield result
break if page * limit >= total
@page += 1
end
end
private
def locale = @locale ||= Locale.find_by!(code: @params.fetch('locale', 'ja'))
def limited_columns(prefix)
keys = axes.columns.each_index.map { "a#{ axes.rows.length + _1 }" }
return [[]] if keys.empty?
order = keys.map { "#{ _1 } IS NULL, #{ _1 }" }.join(', ')
values = @connection.select_rows(<<~SQL)
#{ prefix }
SELECT /*+ MAX_EXECUTION_TIME(10000) */ DISTINCT #{ keys.join(', ') } FROM cells
ORDER BY #{ order } LIMIT #{ MAX_COLUMNS + 1 }
SQL
raise TooManyColumns if values.length > MAX_COLUMNS
values
end
def measure_values(key)
value = JSON.parse([key.delete_prefix('measure_')].pack('H*'))
unless (value.is_a?(Array) && value.length == axes.columns.length &&
value.all? { _1.nil? || _1.is_a?(String) || _1.is_a?(Numeric) })
raise ArgumentError
end
value
end
def detail_expression(key)
return "(#{ population.latest_sql })" if key == 'is_latest'
aliases = { 'tags' => 'tags_json', 'parents' => @definition.resource == 'posts' ?
'parent_post_ids' : 'parent_tag_ids', 'deprecated' => 'deprecated_at',
'original_created_at' => 'original_created_from' }
field = aliases.fetch(key, key)
expression = @definition.expression(field)
return expression unless @definition.fields[field] == :json
"CAST(#{ expression } AS CHAR) COLLATE utf8mb4_bin"
end
def sort_sql
@sort.flat_map do |key, direction|
expression = yield key
["(#{ expression }) IS NULL ASC", "#{ expression } #{ direction.upcase }"]
end
end
def axis_labels(records, column_values)
result = {}
axes.all.each_with_index do |axis, index|
values = records.map { _1["a#{ index }"] }
if index >= axes.rows.length
values += column_values.map { _1[index - axes.rows.length] }
end
values = values.compact.uniq
labels = case axis['field']
when 'user'
User.where(id: values).pluck(:id, :name).to_h.transform_keys(&:to_s)
when 'language_code', 'source_language_code'
Language.where(code: values).pluck(:code, :name).to_h
when 'tags'
tag_labels(values)
else {}
end
result["a#{ index }"] = labels
end
result
end
def tag_labels(values)
internal = values.grep(/\Ainternal:/).map { _1.split(':').last.to_i }
external = values.grep(/\Aexternal:/).map { _1.split(':').last.to_i }
point = population.timestamp(@params['at']) || population.at
labels = {}
sources = [[TagVersion, internal, 'internal'], [NicoTagVersion, external, 'external']]
sources.each do |model, ids, kind|
next if ids.empty?
table = model.table_name
model.where(tag_id: ids).where('created_at <= ?', point)
.where("NOT EXISTS (SELECT 1 FROM #{ table } newer " +
"WHERE newer.tag_id = #{ table }.tag_id AND newer.created_at <= ? " +
"AND newer.version_no > #{ table }.version_no)", point).each do |row|
names = row.respond_to?(:names) ? row.names : nil
name = names&.dig(locale.language_code, 'name') || row.name
labels["#{ kind }:#{ row.tag_id }"] = name
end
end
labels
end
end
+1 -1
ファイルの表示
@@ -105,7 +105,7 @@ class PostCreator
category = spec[:category] || spec['category'] category = spec[:category] || spec['category']
next if name.blank? || category.blank? next if name.blank? || category.blank?
tag = Tag.find_or_create_by_tag_name!(locale, name, category:) tag = Tag.find_or_create_by_tag_name!(locale, name, category:, created_by_user: @actor)
tag.update!(category:) if tag.category.to_sym != category.to_sym tag.update!(category:) if tag.category.to_sym != category.to_sym
tags[name] ||= tag tags[name] ||= tag
end.values end.values
+42
ファイルの表示
@@ -0,0 +1,42 @@
# Shared literal parsing and reference resolution for current and snapshot post search.
class PostTagSearch
def self.parse_references(raw, literals)
return nil if raw.blank?
references = JSON.parse(raw)
valid = references.is_a?(Array) && references.all? do |reference|
next false unless reference.is_a?(Hash) && reference['name'].is_a?(String)
ids = [reference['tag_id'], reference['external_tag_id']].compact
((ids.any? || reference['missing'] == true) &&
ids.all? { _1.is_a?(Integer) && _1.positive? } &&
(!reference.key?('missing') || (reference['missing'] == true && ids.empty?)) &&
(!reference.key?('negative') || [true, false].include?(reference['negative'])))
end
raise ArgumentError unless valid
references.map { _1['name'] } == literals ? references : nil
end
def self.literals(locale, raw, references: nil)
raw.split.each_with_index.filter_map do |token, index|
next if token.downcase == 'type:or'
reference = references&.[](index)
name = token.sub(/\Anot:/i, '')
name = TagName.canonicalise(locale, name).first unless reference
negative = reference&.key?('negative') ? reference['negative'] :
token.downcase.start_with?('not:')
{ name:, negative:, reference: }
end
end
def self.references(locale, literal)
reference = literal[:reference]
return [reference['tag_id'], reference['external_tag_id']] if reference
[TagName.where(language_code: locale.language_code, name: literal[:name])
.distinct.pluck(:tag_id),
ExternalTag.where("CONCAT(platform, ':', name) = ?", literal[:name]).pluck(:id)]
end
end
+76 -30
ファイルの表示
@@ -13,42 +13,26 @@ class SharedResourceIdAllocator
['tag_description_versions', 'tag_id']], ['tag_description_versions', 'tag_id']],
'wiki' => [['tag_group_versions', 'tag_group_id']] }.freeze 'wiki' => [['tag_group_versions', 'tag_group_id']] }.freeze
LOCK_TIMEOUT = 10 LOCK_TIMEOUT = 10
CONNECTION_LOCKS = ObjectSpace::WeakMap.new
class LockCleanup
def initialize &finish
@finish = finish
end
def trigger_transactional_callbacks? = false
def before_committed! = nil
def committed!(**) = @finish.call(self)
def rolledback!(**) = @finish.call(self)
end
private_constant :LockCleanup
def self.assign! record def self.assign! record
space, tables = SPACES.find { |_name, names| names.include?(record.class.table_name) } space, tables = SPACES.find { |_name, names| names.include?(record.class.table_name) }
raise ArgumentError, 'Unknown shared ID resource' unless tables raise ArgumentError, 'Unknown shared ID resource' unless tables
connection = record.class.connection connection = record.class.connection
transaction = record.class.current_transaction lock_space!(record.class, space)
unless transaction.open?
raise ActiveRecord::ActiveRecordError, 'Shared ID requires a transaction'
end
database = connection.pool.db_config.database
name = "btrc:ids:#{ Digest::SHA256.hexdigest(database)[0, 16] }:#{ space }"
quoted_name = connection.quote(name)
acquired = record.class.uncached do
connection.select_value("SELECT GET_LOCK(#{ quoted_name }, #{ LOCK_TIMEOUT })")
end
unless acquired.to_i == 1
raise ActiveRecord::LockWaitTimeout, "Shared ID lock unavailable: #{ space }"
end
release = lambda do
result = record.class.uncached do
connection.select_value("SELECT RELEASE_LOCK(#{ quoted_name })")
end
unless result.to_i == 1
raise ActiveRecord::ActiveRecordError, "Shared ID lock release failed: #{ space }"
end
end
begin
transaction.after_commit(&release)
transaction.after_rollback(&release)
rescue StandardError
release.call
raise
end
# 明示-tic な Id. 指定は許可する. # 明示-tic な Id. 指定は許可する.
return if record.id.present? return if record.id.present?
@@ -66,6 +50,68 @@ class SharedResourceIdAllocator
record.id = next_id record.id = next_id
end end
def self.lock_space! record_class, space
raise ArgumentError, 'Unknown shared ID resource' unless SPACES.key?(space)
connection = record_class.connection
transaction = connection.current_transaction
unless transaction.open?
raise ActiveRecord::ActiveRecordError, 'Shared ID requires a transaction'
end
database = connection.pool.db_config.database
name = "btrc:ids:#{ Digest::SHA256.hexdigest(database)[0, 16] }:#{ space }"
held_locks = CONNECTION_LOCKS[connection] ||= {}
return if held_locks.key?(name)
quoted_name = connection.quote(name)
acquired = record_class.uncached do
connection.select_value("SELECT GET_LOCK(#{ quoted_name }, #{ LOCK_TIMEOUT })")
end
unless acquired.to_i == 1
raise ActiveRecord::LockWaitTimeout, "Shared ID lock unavailable: #{ space }"
end
held_locks[name] = true
finish = lambda do |cleanup|
next unless held_locks.key?(name)
# Savepoint callbacks may run before the outer transaction ends.
# Keep one session lock and transfer its cleanup to the remaining parent.
parent = connection.current_transaction
if parent.open?
connection.add_transaction_record(cleanup)
next
end
result = record_class.uncached do
connection.select_value("SELECT RELEASE_LOCK(#{ quoted_name })")
end
unless result.to_i == 1
raise ActiveRecord::ActiveRecordError, "Shared ID lock release failed: #{ space }"
end
held_locks.delete(name)
rescue StandardError
# Never return a session with an untracked advisory lock to the pool.
begin
connection.disconnect!
ensure
held_locks.clear
end
raise
end
begin
# Rails finalises remaining records in ensure if another callback raises.
connection.add_transaction_record(LockCleanup.new(&finish))
rescue StandardError
begin
connection.disconnect!
ensure
held_locks.clear
end
raise
end
end
def self.next_ids connection, tables def self.next_ids connection, tables
expiry = connection.select_value('SELECT @@SESSION.information_schema_stats_expiry').to_i expiry = connection.select_value('SELECT @@SESSION.information_schema_stats_expiry').to_i
begin begin
+2
ファイルの表示
@@ -5,6 +5,7 @@ class TagDescriptionOperation
def self.create! locale, tag:, body:, created_by_user:, message: nil def self.create! locale, tag:, body:, created_by_user:, message: nil
ApplicationRecord.transaction do ApplicationRecord.transaction do
SharedResourceIdAllocator.lock_space!(Tag, 'tags')
tag.lock! tag.lock!
raise ActiveRecord::RecordNotUnique if TagDescription.exists?( raise ActiveRecord::RecordNotUnique if TagDescription.exists?(
tag_id: tag.id, language_code: locale.language_code) tag_id: tag.id, language_code: locale.language_code)
@@ -19,6 +20,7 @@ class TagDescriptionOperation
def self.update! locale, tag:, body:, updated_by_user:, message: nil, base_revision_id: nil def self.update! locale, tag:, body:, updated_by_user:, message: nil, base_revision_id: nil
ApplicationRecord.transaction do ApplicationRecord.transaction do
SharedResourceIdAllocator.lock_space!(Tag, 'tags')
tag.lock! tag.lock!
description = TagDescription.lock.find_by!( description = TagDescription.lock.find_by!(
tag_id: tag.id, language_code: locale.language_code) tag_id: tag.id, language_code: locale.language_code)
+122
ファイルの表示
@@ -0,0 +1,122 @@
class TagNameCompletion
class << self
def for_locale! locale
completed = Tag.transaction do
SharedResourceIdAllocator.lock_space!(Tag, 'tags')
certified_names =
if locale.certified_tag_names_path.nil?
{ }
else
CertifiedTagNameTranslations.load!(locale.certified_tag_names_path, locale:)
end
Tag.where(id: certified_names.keys).lock.find_each(order: :asc) do |tag|
for_tag!(tag, locales: [locale],
certified_names: { locale.language_code => certified_names.fetch(tag.id) },
created_by_user: nil)
end
named_tags =
TagName.where(language_code: locale.language_code, primary_flg: true)
.select(:tag_id)
Tag.where.not(id: named_tags).lock.find_each(order: :asc) do |tag|
for_tag!(tag, locales: [locale], created_by_user: nil)
end
true
end
unless completed
raise ActiveRecord::RecordNotSaved, 'Locale tag completion was rolled back'
end
end
def for_tag! tag, locales: nil, certified_names: { }, created_by_user: nil
locales ||=
Locale.where(deprecated_at: nil)
.order(:language_code, :created_at, :script_code).lock
Tag.transaction do
SharedResourceIdAllocator.lock_space!(Tag, 'tags')
# Creation includes every language in its first snapshot.
creating = tag.new_record?
tag.save! if creating
completed = tag.with_lock do
primary_names = tag.tag_names.where(primary_flg: true).lock.index_by(&:language_code)
certified_names.each do |language, name|
primary = primary_names[language]
next unless primary
next if primary.name == name && !primary.auto_generated
raise CertifiedTagNameTranslations::Error,
"Tag##{ tag.id } already has a primary #{ language } name; " +
"cannot register certified name #{ name.inspect }"
end
missing_locales = locales.to_a.uniq(&:language_code).reject { |locale|
primary_names.key?(locale.language_code)
}
next true if missing_locales.empty? && !creating
TagVersioning.ensure_snapshot!(tag, created_by_user:) unless creating
missing_locales.each do |locale|
primary_names[locale.language_code] =
if certified_names.key?(locale.language_code)
create_certified_name!(locale, tag, certified_names.fetch(locale.language_code))
else
source_language = locale.language_code == 'ja' ? 'ain' : 'ja'
name = TagName.generate_name(locale, tag, primary_names[source_language])
create_name!(locale, tag, name)
end
end
tag.association(:tag_names).reset
event_type = creating ? :create : :update
version = TagVersioning.record!(tag, event_type:, created_by_user:)
unless version&.persisted?
raise ActiveRecord::RecordNotSaved, 'Tag version was not persisted'
end
true
end
unless completed
raise ActiveRecord::RecordNotSaved, 'Tag completion was rolled back'
end
end
end
private
def create_certified_name! locale, tag, name
persist_name!(locale, tag, name, auto_generated: false)
rescue ActiveRecord::RecordInvalid => error
raise unless error.record.is_a?(TagName)
raise CertifiedTagNameTranslations::Error,
"Tag##{ tag.id } certified name #{ name.inspect }: " +
error.record.errors.full_messages.join(', ')
rescue ActiveRecord::RecordNotUnique => error
raise CertifiedTagNameTranslations::Error,
"Tag##{ tag.id } certified name #{ name.inspect } conflicts: #{ error.message }"
end
def create_name! locale, tag, name
persist_name!(locale, tag, name, auto_generated: true)
rescue ActiveRecord::RecordNotUnique, ActiveRecord::RecordInvalid => error
fallback = "Tag_##{ tag.id }"
invalid_name =
if error.is_a?(ActiveRecord::RecordNotUnique)
TagName.where(language_code: locale.language_code, name:).lock.exists?
else
error.record.errors[:name].present? &&
error.record.errors.attribute_names.all? { _1 == :name }
end
raise if name == fallback || !(invalid_name)
persist_name!(locale, tag, fallback, auto_generated: true)
end
def persist_name! locale, tag, name, auto_generated:
tag_name = TagName.transaction(requires_new: true) do
TagName.create!(tag:, name:, language_code: locale.language_code,
script_code: locale.script_code, primary_flg: true, auto_generated:)
end
unless tag_name&.persisted?
raise ActiveRecord::RecordNotSaved, 'Generated tag name was not persisted'
end
tag_name
end
end
end
+56
ファイルの表示
@@ -0,0 +1,56 @@
module TagNameTranscription
class Unconvertible < StandardError
;
end
def self.ainu name
previous_script = nil
parts = []
NipponeseSegments.split(name).each do |script, text|
output = transcribe_segment(script, text)
if (parts.last && output.first && previous_script && script != previous_script &&
NipponeseSegments.nipponese?(previous_script) && NipponeseSegments.nipponese?(script))
parts << '_'
end
parts.concat(output)
previous_script = script
end
replace_unconvertible(parts)
end
def self.transcribe_segment script, text
return [text] if script == :literal
return [nil] if script == :unconvertible
return transcribe_reading(text) unless script == :kanzi
text.each_char.flat_map do |character|
transcribe_reading(JisReadings.read(character))
rescue Unconvertible
[nil]
end
end
def self.transcribe_reading reading
Phonology.adapt(reading).map do |sound|
sound && Katakana.render([sound])
rescue Unconvertible
nil
end
end
def self.replace_unconvertible parts
grouped = parts.chunk { _1.nil? }.flat_map { |unconvertible, group|
unconvertible ? [nil] : group
}
first = grouped.index { _1&.match?(/[^_]/) }
last = grouped.rindex { _1&.match?(/[^_]/) }
grouped.each_with_index.map { |part, index|
part || (first && first < index && index < last ? '_' : '')
}.join
end
private_class_method :transcribe_segment, :transcribe_reading, :replace_unconvertible
end
+13
ファイルの表示
@@ -0,0 +1,13 @@
module TagNameTranscription
module JisReadings
SOURCE = Rails.root.join('config/tag_name_readings/jis_level_one.tsv')
READINGS = File.foreach(SOURCE, chomp: true).reject { _1.start_with?('#') }.to_h { |line|
character, reading, = line.split("\t")
[character, reading]
}.freeze
def self.read text
text.each_char.map { READINGS.fetch(_1) { raise Unconvertible } }.join
end
end
end
+26
ファイルの表示
@@ -0,0 +1,26 @@
module TagNameTranscription
module Katakana
ROWS = {
'' => 'アイウエオ',
'k' => 'カキクケコ',
's' => 'サシスセソ',
'c' => ['チャ', 'チ', 'チュ', 'チェ', 'チョ'],
't' => ['タ', 'ティ', 'トゥ', 'テ', 'ト'],
'n' => 'ナニヌネノ',
'h' => 'ハヒフヘホ',
'p' => 'パピプペポ',
'm' => 'マミムメモ',
'r' => 'ラリルレロ' }.freeze
SYLLABLES = ROWS.flat_map { |onset, letters|
kana = letters.is_a?(String) ? letters.chars : letters
['a', 'i', 'u', 'e', 'o'].zip(kana).map { |vowel, letter| [onset + vowel, letter] }
}.to_h.merge(
'ya' => 'ヤ', 'yu' => 'ユ', 'yo' => 'ヨ', 'wa' => 'ワ',
'k' => 'ㇰ', 's' => 'ㇱ', 't' => 'ッ', 'n' => 'ン', 'm' => 'ㇺ', 'p' => 'ㇷ゚',
'r:a' => 'ㇻ', 'r:i' => 'ㇼ', 'r:u' => 'ㇽ', 'r:e' => 'ㇾ', 'r:o' => 'ㇿ').freeze
def self.render sounds
sounds.map { SYLLABLES.fetch(_1) { raise Unconvertible } }.join
end
end
end
+30
ファイルの表示
@@ -0,0 +1,30 @@
module TagNameTranscription
module NipponeseSegments
NIPPONESE_SCRIPTS = [:kanzi, :xiragana, :katakana].freeze
def self.nipponese?(script) = NIPPONESE_SCRIPTS.include?(script)
def self.split name
segments = []
name.to_s.scan(/\X/).each do |character|
script = classify(character, segments.last&.first)
if segments.last&.first == script
segments.last[1] << character
else
segments << [script, character.dup]
end
end
segments
end
def self.classify character, previous_script
return previous_script if character == 'ー' && nipponese?(previous_script)
return :kanzi if character.match?(/\A\p{Han}\z/)
return :xiragana if character.match?(/\A\p{Hiragana}\p{M}*\z/)
return :katakana if character.match?(/\A\p{Katakana}\p{M}*\z/)
return :literal if character.match?(/\A(?:\p{Latin}\p{M}*|[!-~])\z/)
:unconvertible
end
end
end
+59
ファイルの表示
@@ -0,0 +1,59 @@
module TagNameTranscription
module Phonology
ROWS = {
'アイウエオ' => ['a', 'i', 'u', 'e', 'o'],
'カキクケコ' => ['ka', 'ki', 'ku', 'ke', 'ko'],
'ガギグゲゴ' => ['ka', 'ki', 'ku', 'ke', 'ko'],
'サシスセソ' => ['sa', 'si', 'su', 'se', 'so'],
'ザジズゼゾ' => ['ca', 'ci', 'cu', 'ce', 'co'],
'タチツテト' => ['ta', 'ci', 'cu', 'te', 'to'],
'ダヂヅデド' => ['ta', 'ci', 'cu', 'te', 'to'],
'ナニヌネノ' => ['na', 'ni', 'nu', 'ne', 'no'],
'ハヒフヘホ' => ['ha', 'hi', 'hu', 'he', 'ho'],
'バビブベボ' => ['pa', 'pi', 'pu', 'pe', 'po'],
'パピプペポ' => ['pa', 'pi', 'pu', 'pe', 'po'],
'マミムメモ' => ['ma', 'mi', 'mu', 'me', 'mo'],
'ヤユヨ' => ['ya', 'yu', 'yo'],
'ラリルレロ' => ['ra', 'ri', 'ru', 're', 'ro'],
'ワ' => ['wa'],
'ン' => ['n'],
'ッ' => ['t'],
'ㇰ' => ['k'],
'ㇱㇲ' => ['s', 's'],
'ㇳ' => ['t'],
'ㇴ' => ['n'],
'ㇺ' => ['m'],
'ㇻㇼㇽㇾㇿ' => ['r:a', 'r:i', 'r:u', 'r:e', 'r:o'] }.freeze
SOUNDS = ROWS.flat_map { |characters, sounds| characters.chars.zip(sounds) }.to_h.freeze
DIGRAPHS = {
'シャ' => 'sa', 'シュ' => 'su', 'シェ' => 'se', 'ショ' => 'so',
'チャ' => 'ca', 'チュ' => 'cu', 'チェ' => 'ce', 'チョ' => 'co',
'ジャ' => 'ca', 'ジュ' => 'cu', 'ジェ' => 'ce', 'ジョ' => 'co',
'ヂャ' => 'ca', 'ヂュ' => 'cu', 'ヂェ' => 'ce', 'ヂョ' => 'co' }.freeze
def self.adapt reading
kana = reading.unicode_normalize(:nfc).tr('ぁ-ゖ', 'ァ-ヶ').delete('ー')
characters = kana.scan(/\X/)
sounds = []
index = 0
while index < characters.length
pair = characters[index, 2].join
if DIGRAPHS.key?(pair)
sounds << DIGRAPHS.fetch(pair)
index += 2
next
end
character = characters[index].tr('ャュョ', 'ヤユヨ')
sounds << (character == 'ㇷ゚' ? 'p' : SOUNDS[character])
index += 1
end
sounds
end
end
end
+22 -6
ファイルの表示
@@ -1,6 +1,9 @@
class TagVersionRecorder < VersionRecorder class TagVersionRecorder < VersionRecorder
def self.record! tag:, event_type:, created_by_user:, **_options def self.record! tag:, event_type:, created_by_user:, **_options
new(tag:, event_type:, created_by_user:).record! Tag.transaction do
SharedResourceIdAllocator.lock_space!(Tag, 'tags')
new(tag:, event_type:, created_by_user:).record!
end
end end
def initialize tag:, event_type:, created_by_user: def initialize tag:, event_type:, created_by_user:
@@ -15,26 +18,39 @@ class TagVersionRecorder < VersionRecorder
def snapshot_attributes def snapshot_attributes
names = snapshot_names names = snapshot_names
compatibility_names = names.fetch('ja') compatibility_names = names['ja'] || names['ain'] || names.values.first
unless compatibility_names
@record.errors.add(:tag_names, :invalid)
raise ActiveRecord::RecordInvalid, @record
end
parent_ids =
TagImplication.where(tag_id: @record.id).order(:parent_tag_id)
.lock.pluck(:parent_tag_id)
{ name: compatibility_names.fetch('name'), { name: compatibility_names.fetch('name'),
category: @record.category, category: @record.category,
deprecated_at: @record.deprecated_at, deprecated_at: @record.deprecated_at,
aliases: compatibility_names.fetch('aliases').map { _1.fetch('name') }.join(' '), aliases: compatibility_names.fetch('aliases').map { _1.fetch('name') }.join(' '),
names:, names:,
parent_tag_ids: @record.snapshot_parent_tag_ids.join(' ') } parent_tag_ids: parent_ids.join(' ') }
end end
def snapshot_names def snapshot_names
@record.tag_names.order(:language_code, :name).group_by(&:language_code) @record.tag_names.order(:language_code, :name).lock.group_by(&:language_code)
.transform_values do |tag_names| .transform_values do |tag_names|
primary = tag_names.find(&:primary?) primary = tag_names.find(&:primary?)
unless primary
@record.errors.add(:tag_names, :invalid,
language_code: tag_names.first.language_code)
raise ActiveRecord::RecordInvalid, @record
end
aliases = tag_names.reject(&:primary?).sort_by(&:name).map { |tag_name| aliases = tag_names.reject(&:primary?).sort_by(&:name).map { |tag_name|
{ 'name' => tag_name.name, 'script_code' => tag_name.script_code } { 'name' => tag_name.name,
'script_code' => Script.codes.fetch(tag_name.script_code, tag_name.script_code) }
} }
{ 'name' => primary.name, { 'name' => primary.name,
'script_code' => primary.script_code, 'script_code' => Script.codes.fetch(primary.script_code, primary.script_code),
'auto_generated' => primary.auto_generated, 'auto_generated' => primary.auto_generated,
'aliases' => aliases } 'aliases' => aliases }
end end
+13 -4
ファイルの表示
@@ -4,14 +4,23 @@ class TagVersioning
end end
def self.ensure_snapshot! tag, created_by_user:, language_code: 'ja' def self.ensure_snapshot! tag, created_by_user:, language_code: 'ja'
return if tag.tag_versions.exists? Tag.transaction do
SharedResourceIdAllocator.lock_space!(Tag, 'tags')
locked_tag = Tag.unscoped.lock.find(tag.id)
next if locked_tag.tag_versions.lock.exists?
TagVersionRecorder.record!(tag:, event_type: :create, created_by_user:, language_code:) TagVersionRecorder.record!(
tag: locked_tag, event_type: :create, created_by_user:, language_code:)
end
end end
def self.record_tag_snapshot! tag, created_by_user:, language_code: 'ja' def self.record_tag_snapshot! tag, created_by_user:, language_code: 'ja'
event_type = tag.tag_versions.exists? ? :update : :create Tag.transaction do
record!(tag, event_type:, created_by_user:, language_code:) SharedResourceIdAllocator.lock_space!(Tag, 'tags')
locked_tag = Tag.unscoped.lock.find(tag.id)
event_type = locked_tag.tag_versions.lock.exists? ? :update : :create
record!(locked_tag, event_type:, created_by_user:, language_code:)
end
end end
def self.record_tag_snapshots! tags, created_by_user:, language_code: 'ja' def self.record_tag_snapshots! tags, created_by_user:, language_code: 'ja'
+1 -1
ファイルの表示
@@ -35,7 +35,7 @@ class VersionRecorder
private private
def latest_version = versions.order(version_no: :desc).first def latest_version = versions.order(version_no: :desc).lock.first
def versions = @record.public_send(version_association) def versions = @record.public_send(version_association)
+9
ファイルの表示
@@ -31,5 +31,14 @@ module Backend
# Middleware like session, flash, cookies can be added back manually. # Middleware like session, flash, cookies can be added back manually.
# Skip views, helpers and assets when generating a new resource. # Skip views, helpers and assets when generating a new resource.
config.api_only = true config.api_only = true
config.middleware.use ActionDispatch::Cookies
config.session_store :cookie_store,
key: Rails.env.production? ? '__Host-btrc-session' : 'btrc-session',
path: '/',
secure: Rails.env.production?,
httponly: true,
same_site: :lax,
expire_after: 20.years
config.middleware.use ActionDispatch::Session::CookieStore, config.session_options
end end
end end
+2 -2
ファイルの表示
@@ -4,5 +4,5 @@
# Use this to limit dissemination of sensitive information. # Use this to limit dissemination of sensitive information.
# See the ActiveSupport::ParameterFilter documentation for supported notations and behaviors. # See the ActiveSupport::ParameterFilter documentation for supported notations and behaviors.
Rails.application.config.filter_parameters += [ Rails.application.config.filter_parameters += [
:passw, :email, :secret, :token, :_key, :crypt, :salt, :certificate, :otp, :ssn, :cvv, :cvc :passw, :email, :secret, :token, :_key, :crypt, :salt, :certificate,
] :otp, :ssn, :cvv, :cvc, :code, :inheritance_code]
+19
ファイルの表示
@@ -1,4 +1,17 @@
Rails.application.routes.draw do Rails.application.routes.draw do
get 'counts/definitions/:resource', to: 'counts_tables#definition'
get 'counts/query', to: 'counts_tables#index'
post 'counts/export/prepare', to: 'counts_exports#prepare'
get 'counts/export/files/:token', to: 'counts_exports#download'
get 'counts/export', to: 'counts_exports#index'
post 'counts/export', to: 'counts_exports#index'
get 'counts/state', to: 'counts_states#index'
get 'counts/posts', to: 'counts_posts#index'
get 'counts/tags', to: 'counts_tags#index'
resources :locales, only: [:index]
resources :sitemap_entries, only: [:index, :show], param: :resource
resources :tag_groups, only: [:index, :show, :create, :update] resources :tag_groups, only: [:index, :show, :create, :update]
resources :nico_tags, path: 'tags/nico', only: [:index, :show, :update] resources :nico_tags, path: 'tags/nico', only: [:index, :show, :update]
@@ -43,6 +56,7 @@ Rails.application.routes.draw do
collection do collection do
get :search, action: :index get :search, action: :index
get :changes get :changes
get :autolink_candidates
scope :title do scope :title do
get ':title/exists', action: :exists_by_title get ':title/exists', action: :exists_by_title
@@ -53,12 +67,14 @@ Rails.application.routes.draw do
member do member do
get :exists get :exists
get :diff get :diff
get :tag, action: :tag_reference
end end
end end
resources :posts, only: [:index, :show, :create, :update] do resources :posts, only: [:index, :show, :create, :update] do
collection do collection do
get :metadata get :metadata
get :tag_filters
post :bulk post :bulk
get :random get :random
get :versions, to: 'post_versions#index' get :versions, to: 'post_versions#index'
@@ -97,6 +113,9 @@ Rails.application.routes.draw do
collection do collection do
post :verify post :verify
get :me get :me
get :session, action: :show_session
get :csrf
delete :session, action: :destroy_session
post 'code/renew', action: :renew post 'code/renew', action: :renew
end end
end end
ファイル差分が大きすぎるため省略します 差分を読み込み
+5
ファイルの表示
@@ -8,6 +8,11 @@
# MovieGenre.find_or_create_by!(name: genre_name) # MovieGenre.find_or_create_by!(name: genre_name)
# end # end
ApplicationRecord.transaction do
Language.find_or_create_by!(code: 'ain') { _1.name = 'ain' }
Script.find_or_create_by!(code: 'Kana') { _1.name = 'Kana' }
end
post_url_sanitisation_rules = [ post_url_sanitisation_rules = [
{ priority: 10, { priority: 10,
source_pattern: '\Ahttps?://youtu\.be/([^/?#]+)(?:[?#].*)?\z', source_pattern: '\Ahttps?://youtu\.be/([^/?#]+)(?:[?#].*)?\z',
+1 -11
ファイルの表示
@@ -24,17 +24,7 @@ FactoryBot.define do
evaluator.primary_tag_name.update!(tag:, primary_flg: true) evaluator.primary_tag_name.update!(tag:, primary_flg: true)
Locale.where(deprecated_at: nil).order(:language_code, :created_at, :script_code).to_a TagNameCompletion.for_tag!(tag)
.uniq(&:language_code).each do |locale|
next if tag.tag_names.exists?(language_code: locale.language_code,
primary_flg: true)
create(:tag_name,
tag:, language_code: locale.language_code,
script_code: locale.script_code,
name: TagName.generate_name(locale, tag, evaluator.primary_tag_name),
primary_flg: true, auto_generated: true)
end
end end
end end
end end
+2 -3
ファイルの表示
@@ -6,9 +6,8 @@ RSpec.describe Language, type: :model do
expect(described_class.where(deprecated_at: nil).pluck(:code)).to eq(['ja']) expect(described_class.where(deprecated_at: nil).pluck(:code)).to eq(['ja'])
end end
it 'accepts English when multilingual support is enabled' do it 'accepts Ainu when multilingual support is enabled' do
pending '日本語以外の Language 有効化は別課題で実装する' expect { described_class.create!(code: 'ain', name: 'ain') }
expect { described_class.create!(code: 'en', name: 'English') }
.to change(described_class, :count).by(1) .to change(described_class, :count).by(1)
end end
+82
ファイルの表示
@@ -0,0 +1,82 @@
require 'rails_helper'
RSpec.describe 'Locale activation', type: :model do
let!(:language) { Language.create!(code: 'ain', name: 'ain') }
let!(:script) { Script.create!(code: 'Kana', name: 'Kana') }
let!(:first) { create(:tag, primary_name: 'activation_first') }
let!(:second) { create(:tag, primary_name: 'activation_second') }
def build_locale deprecated_at: nil
Locale.new(code: 'ain-Kana', language_code: language.code,
script_code: script.code, region_code: 392, variants: '',
name: 'ain-Kana', deprecated_at:)
end
def generated_names
TagName.where(language_code: 'ain', primary_flg: true)
end
def fail_second_generation!
allow(TagName).to receive(:generate_name).and_call_original
allow(TagName).to receive(:generate_name)
.with(kind_of(Locale), second, second.tag_name('ja'))
.and_raise('backfill failed')
end
it 'rolls back a newly created locale and the preceding generated names' do
locale = build_locale
fail_second_generation!
expect { locale.save! }.to raise_error('backfill failed')
expect(Locale.exists?(code: 'ain-Kana')).to be(false)
expect(generated_names).to be_empty
expect(first.reload.name('ja')).to eq('activation_first')
expect(second.reload.name('ja')).to eq('activation_second')
end
it 'does not backfill while the locale is inactive' do
build_locale(deprecated_at: Time.current).save!
expect(generated_names).to be_empty
end
it 'backfills missing primary names when the locale becomes active' do
locale = build_locale(deprecated_at: Time.current)
locale.save!
locale.update!(deprecated_at: nil)
expect(generated_names.pluck(:tag_id)).to contain_exactly(first.id, second.id)
expect(generated_names).to all(have_attributes(
script_code: 'Kana', primary_flg: true, auto_generated: true))
end
it 'rolls back activation and all generated names when backfill fails' do
locale = build_locale(deprecated_at: Time.current)
locale.save!
previous_deprecated_at = locale.reload.deprecated_at
fail_second_generation!
expect { locale.update!(deprecated_at: nil) }.to raise_error('backfill failed')
expect(locale.reload.deprecated_at).to eq(previous_deprecated_at)
expect(generated_names).to be_empty
end
it 'preserves a human primary through deactivation and repeated activation' do
locale = build_locale
locale.save!
primary = generated_names.find_by!(tag: first)
primary.update!(name: 'human_confirmed', auto_generated: false)
snapshot = generated_names.order(:id).map(&:attributes)
2.times do
locale.update!(deprecated_at: Time.current)
locale.update!(deprecated_at: nil)
end
expect(generated_names.order(:id).map(&:attributes)).to eq(snapshot)
expect(primary.reload).to have_attributes(name: 'human_confirmed', auto_generated: false)
end
end
+111 -55
ファイルの表示
@@ -1,13 +1,13 @@
require 'rails_helper' require 'rails_helper'
RSpec.describe Locale, type: :model do RSpec.describe Locale, type: :model do
include_context 'English locale' include_context 'Ainu locale'
def create_english_locale! def create_ainu_locale!
prepare_english_reference! prepare_ainu_reference!
described_class.create!(code: 'en', language_code: 'en', described_class.create!(code: 'ain-Kana', language_code: 'ain',
script_code: 'Latn', region_code: 826, script_code: 'Kana', region_code: 392,
variants: '', name: 'English') variants: '', name: 'Ainu')
end end
it 'keeps the active Nipponese locale consistent with its identity tuple' do it 'keeps the active Nipponese locale consistent with its identity tuple' do
@@ -27,7 +27,6 @@ RSpec.describe Locale, type: :model do
end end
it 'generates a primary name for every existing tag using the name generator' do it 'generates a primary name for every existing tag using the name generator' do
pending '日本語以外の Locale 有効化は別課題で実装する'
first = create(:tag, primary_name: 'first_existing_tag') first = create(:tag, primary_name: 'first_existing_tag')
second = create(:tag, primary_name: 'second_existing_tag') second = create(:tag, primary_name: 'second_existing_tag')
allow(TagName).to receive(:generate_name).and_call_original allow(TagName).to receive(:generate_name).and_call_original
@@ -35,95 +34,152 @@ RSpec.describe Locale, type: :model do
.with(kind_of(described_class), first, primary_tag_name_for(first, 'ja')) .with(kind_of(described_class), first, primary_tag_name_for(first, 'ja'))
.and_return('name_from_generator') .and_return('name_from_generator')
locale = create_english_locale! locale = create_ainu_locale!
expect(TagName).to have_received(:generate_name) expect(TagName).to have_received(:generate_name)
.with(locale, first, primary_tag_name_for(first, 'ja')) .with(locale, first, primary_tag_name_for(first, 'ja'))
expect(TagName).to have_received(:generate_name) expect(TagName).to have_received(:generate_name)
.with(locale, second, primary_tag_name_for(second, 'ja')) .with(locale, second, primary_tag_name_for(second, 'ja'))
expect(first.tag_names.find_by!(language_code: 'en', primary_flg: true)) expect(first.tag_names.find_by!(language_code: 'ain', primary_flg: true))
.to have_attributes(name: 'name_from_generator', tag_id: first.id, .to have_attributes(name: 'name_from_generator', tag_id: first.id,
language_code: 'en', script_code: 'Latn', language_code: 'ain', script_code: 'Kana',
primary_flg: true, auto_generated: true) primary_flg: true, auto_generated: true)
expect(second.tag_names.find_by!(language_code: 'en', primary_flg: true)) expect(second.tag_names.find_by!(language_code: 'ain', primary_flg: true))
.to have_attributes(tag_id: second.id, language_code: 'en', .to have_attributes(tag_id: second.id, language_code: 'ain',
script_code: 'Latn', primary_flg: true, script_code: 'Kana', primary_flg: true,
auto_generated: true) auto_generated: true)
expect(TagName.where(language_code: 'en', primary_flg: true, expect(TagName.where(language_code: 'ain', primary_flg: true,
tag_id: [first.id, second.id]).count).to eq(2) tag_id: [first.id, second.id]).count).to eq(2)
end end
it 'does not duplicate an existing primary name in the new language' do it 'does not duplicate an existing primary name in the new language' do
pending '日本語以外の Locale 有効化は別課題で実装する' prepare_ainu_reference!
prepare_english_reference!
tag = create(:tag, primary_name: 'already_named') tag = create(:tag, primary_name: 'already_named')
existing = create(:tag_name, name: 'existing_english', tag:, existing = create(:tag_name, name: 'existing_ainu', tag:,
language_code: 'en', script_code: 'Latn') language_code: 'ain', script_code: 'Kana')
versions_before = tag.tag_versions.order(:version_no).map(&:attributes)
create_english_locale! create_ainu_locale!
expect(TagName.where(tag_id: tag.id, language_code: 'en', primary_flg: true)) expect(TagName.where(tag_id: tag.id, language_code: 'ain', primary_flg: true))
.to contain_exactly(existing) .to contain_exactly(existing)
expect(existing.reload).to have_attributes( expect(existing.reload).to have_attributes(
tag_id: tag.id, language_code: 'en', script_code: 'Latn', tag_id: tag.id, language_code: 'ain', script_code: 'Kana',
primary_flg: true, auto_generated: false) primary_flg: true, auto_generated: false)
expect(tag.tag_versions.order(:version_no).map(&:attributes)).to eq(versions_before)
end end
it 'creates a primary name when the language has only an alias' do it 'adds the new language primary while preserving existing names and aliases' do
pending '日本語以外の Locale 有効化は別課題で実装する' tag = create(:tag, primary_name: 'named_tag')
prepare_english_reference! create(:tag_name, :alias, name: 'existing_alias', tag:,
tag = create(:tag, primary_name: 'alias_only_tag') language_code: 'ja', script_code: 'Jpan')
alias_name = create(:tag_name, :alias, name: 'alias_en', original_names = tag.tag_names.order(:id).map(&:attributes)
tag:,
language_code: 'en', script_code: 'Latn')
create_english_locale! create_ainu_locale!
expect(alias_name.reload).to have_attributes( expect(tag.tag_names.where(language_code: 'ja').order(:id).map(&:attributes))
tag_id: tag.id, .to eq(original_names)
language_code: 'en', script_code: 'Latn', primary_flg: false) generated = TagName.find_by!(tag_id: tag.id, language_code: 'ain', primary_flg: true)
generated = TagName.find_by!(tag_id: tag.id, language_code: 'en', primary_flg: true) expect(generated).to have_attributes(name: 'named_tag', tag_id: tag.id, language_code: 'ain',
expect(generated).to have_attributes(tag_id: tag.id, language_code: 'en', script_code: 'Kana', primary_flg: true,
script_code: 'Latn', primary_flg: true,
auto_generated: true) auto_generated: true)
expect(TagName.where(tag_id: tag.id, language_code: 'en', primary_flg: true).count).to eq(1) expect(TagName.where(tag_id: tag.id, language_code: 'ain', primary_flg: true).count).to eq(1)
end end
it 'rolls back the locale and all generated names when backfill fails' do it 'rolls back the locale and all generated names when backfill fails' do
pending '日本語以外の Locale 有効化は別課題で実装する' prepare_ainu_reference!
prepare_english_reference!
first = create(:tag, primary_name: 'atomic_first') first = create(:tag, primary_name: 'atomic_first')
second = create(:tag, primary_name: 'atomic_second') second = create(:tag, primary_name: 'atomic_second')
versions_before = TagVersion.order(:id).map(&:attributes)
allow(TagName).to receive(:generate_name).and_call_original allow(TagName).to receive(:generate_name).and_call_original
allow(TagName).to receive(:generate_name) allow(TagName).to receive(:generate_name)
.with(kind_of(described_class), second, primary_tag_name_for(second, 'ja')) .with(kind_of(described_class), second, primary_tag_name_for(second, 'ja'))
.and_raise('backfill failed') .and_raise('backfill failed')
expect { create_english_locale! }.to raise_error('backfill failed') expect { create_ainu_locale! }.to raise_error('backfill failed')
expect(described_class.exists?(code: 'en')).to be(false) expect(described_class.exists?(code: 'ain-Kana')).to be(false)
expect(TagName.where(language_code: 'en', tag_id: [first.id, second.id])).to be_empty expect(TagName.where(language_code: 'ain', tag_id: [first.id, second.id])).to be_empty
expect(TagVersion.order(:id).map(&:attributes)).to eq(versions_before)
end end
it 'accepts a stable locale code outside the historical enum values' do it 'records a separate automatic update for each tag without changing its authored create' do
pending '日本語以外の Locale 有効化は別課題で実装する' author = create(:user, :admin)
prepare_english_reference! tags = ['locale_history_first', 'locale_history_second'].map do |name|
Tag.find_or_create_by_tag_name!(Locale.nipponese, name, category: :general,
created_by_user: author)
end
originals = tags.map { |tag| tag.tag_versions.sole.attributes }
locale = described_class.create!( expect { create_ainu_locale! }.to change(TagVersion, :count).by(2)
code: 'en-x-rspec', language_code: 'en',
script_code: 'Latn', region_code: 826,
variants: 'x-rspec', name: 'English')
expect(locale.reload).to have_attributes( tags.zip(originals).each do |tag, original|
code: 'en-x-rspec', language_code: 'en', script_code: 'Latn', versions = tag.tag_versions.order(:version_no).to_a
region_code: 826, variants: 'x-rspec') expect(versions.map(&:event_type)).to eq(['create', 'update'])
expect(versions.first.attributes).to eq(original)
expect(versions.first.created_by_user_id).to eq(author.id)
expect(versions.last).to have_attributes(version_no: 2, created_by_user_id: nil)
expect(versions.last.names.fetch('ain')).to eq(
'name' => tag.name('ja'), 'script_code' => 'Kana',
'auto_generated' => true, 'aliases' => [])
expect(versions.first.names.keys).to eq(['ja'])
expect(tag.reload.version_no).to eq(2)
end
expect { TagNameCompletion.for_locale!(Locale.find('ain-Kana')) }
.not_to change(TagVersion, :count)
end end
it 'registers the future British English locale tuple' do it 'records automatic before and after snapshots for a tag without any history' do
pending '日本語以外の Locale 有効化は別課題で実装する' tag = create(:tag, primary_name: 'legacy_before_locale')
expect(tag.tag_versions).to be_empty
expect(locale).to have_attributes(code: 'en', language_code: 'en', create_ainu_locale!
script_code: 'Latn', region_code: 826,
versions = tag.tag_versions.order(:version_no).to_a
expect(versions.map(&:event_type)).to eq(['create', 'update'])
expect(versions.map(&:created_by_user_id)).to eq([nil, nil])
expect(versions.first.names.keys).to eq(['ja'])
expect(versions.last.names.keys).to match_array(['ja', 'ain'])
expect(versions.last.names.fetch('ja')).to eq(versions.first.names.fetch('ja'))
end
it 'rolls back earlier tag names and histories when a later history fails' do
first = create(:tag, primary_name: 'history_atomic_first')
second = create(:tag, primary_name: 'history_atomic_second')
names_before = TagName.order(:id).map(&:attributes)
versions_before = TagVersion.order(:id).map(&:attributes)
allow(TagVersioning).to receive(:record!).and_wrap_original do |original, tag, **options|
if tag.id == second.id && options[:event_type] == :update
raise ActiveRecord::RecordNotSaved, 'history failed'
end
original.call(tag, **options)
end
expect { create_ainu_locale! }
.to raise_error(ActiveRecord::RecordNotSaved, 'history failed')
expect(described_class.exists?(code: 'ain-Kana')).to be(false)
expect(TagName.order(:id).map(&:attributes)).to eq(names_before)
expect(TagVersion.order(:id).map(&:attributes)).to eq(versions_before)
expect([first.reload.version_no, second.reload.version_no]).to eq([1, 1])
end
it 'rejects a private locale code outside the supported locale set' do
prepare_ainu_reference!
expect {
described_class.create!(
code: 'ain-Kana-x-rspec', language_code: 'ain',
script_code: 'Kana', region_code: 392,
variants: 'x-rspec', name: 'Ainu')
}.to raise_error(ArgumentError)
end
it 'registers the supported Ainu locale tuple' do
expect(locale).to have_attributes(code: 'ain-Kana', language_code: 'ain',
script_code: 'Kana', region_code: 392,
variants: '') variants: '')
end end
end end
+3 -4
ファイルの表示
@@ -1,7 +1,7 @@
require 'rails_helper' require 'rails_helper'
RSpec.describe Script, type: :model do RSpec.describe Script, type: :model do
include_context 'English locale' include_context 'Ainu locale'
it 'selects locales by script code, not language code' do it 'selects locales by script code, not language code' do
expect(described_class.find('Jpan').locales).to contain_exactly(Locale.nipponese) expect(described_class.find('Jpan').locales).to contain_exactly(Locale.nipponese)
@@ -25,11 +25,10 @@ RSpec.describe Script, type: :model do
} }
end end
it 'selects Latin locales when multilingual support is enabled' do it 'selects Kana locales when multilingual support is enabled' do
pending '日本語以外の Locale 有効化は別課題で実装する'
locale locale
expect(described_class.find('Latn').locales).to contain_exactly(locale) expect(described_class.find('Kana').locales).to contain_exactly(locale)
expect(described_class.find('Jpan').locales).not_to include(locale) expect(described_class.find('Jpan').locales).not_to include(locale)
end end
end end
+14 -14
ファイルの表示
@@ -95,38 +95,38 @@ RSpec.describe TagNameSanitisationRule, type: :model do
end end
context 'when only another language has the sanitised name' do context 'when only another language has the sanitised name' do
include_context 'English locale' include_context 'Ainu locale'
it 'keeps both names when sanitisation produces a cross-language match' do it 'keeps both names when sanitisation produces a cross-language match' do
pending '日本語以外の Locale 有効化は別課題で実装する'
locale locale
nipponese = create(:tag_name, name: 'foobar') nipponese = create(:tag_name, name: 'foobar')
english = create(:tag_name, name: 'temporary', ainu = create(:tag_name, name: 'temporary',
language_code: 'en', script_code: 'Latn') language_code: 'ain', script_code: 'Kana')
english.update_columns(name: 'foo_bar') ainu.update_columns(name: 'foo_bar')
expect { described_class.apply! }.not_to change(TagName, :count) expect { described_class.apply! }.not_to change(TagName, :count)
expect(english.reload).to have_attributes(name: 'foobar', language_code: 'en') expect(ainu.reload).to have_attributes(name: 'foobar', language_code: 'ain')
expect(nipponese.reload).to have_attributes(name: 'foobar', language_code: 'ja') expect(nipponese.reload).to have_attributes(name: 'foobar', language_code: 'ja')
end end
it 'renames without merging tags or deleting either language identity' do it 'renames without merging tags or deleting either language identity' do
pending '日本語以外の Locale 有効化は別課題で実装する'
locale locale
nipponese = create(:tag, primary_name: 'foobar') nipponese = create(:tag, primary_name: 'foobar')
english_name = create(:tag_name, name: 'temporary', primary_tag_name_for(nipponese, 'ain').update!(
language_code: 'en', script_code: 'Latn') name: 'different_ainu_name', auto_generated: false)
english = create(:tag, primary_tag_name: english_name) ainu_name = create(:tag_name, name: 'temporary',
english_name.update_columns(name: 'foo_bar') language_code: 'ain', script_code: 'Kana')
ainu = create(:tag, primary_tag_name: ainu_name)
ainu_name.update_columns(name: 'foo_bar')
described_class.apply! described_class.apply!
expect(english_name.reload).to have_attributes( expect(ainu_name.reload).to have_attributes(
name: 'foobar', language_code: 'en', tag_id: english.id) name: 'foobar', language_code: 'ain', tag_id: ainu.id)
expect(primary_tag_name_for(nipponese.reload, 'ja')).to have_attributes( expect(primary_tag_name_for(nipponese.reload, 'ja')).to have_attributes(
name: 'foobar', language_code: 'ja', tag_id: nipponese.id) name: 'foobar', language_code: 'ja', tag_id: nipponese.id)
expect(primary_tag_name_for(english.reload, 'en')).to eq(english_name) expect(primary_tag_name_for(ainu.reload, 'ain')).to eq(ainu_name)
end end
end end
+25 -29
ファイルの表示
@@ -1,17 +1,16 @@
require 'rails_helper' require 'rails_helper'
RSpec.describe TagName, type: :model do RSpec.describe TagName, type: :model do
include_context 'English locale' include_context 'Ainu locale'
it 'allows the same name in different languages' do it 'allows the same name in different languages' do
pending '日本語以外の Locale 有効化は別課題で実装する'
locale locale
create(:tag_name, name: 'shared_name') create(:tag_name, name: 'shared_name')
english = build(:tag_name, name: 'shared_name', ainu = build(:tag_name, name: 'shared_name',
language_code: 'en', script_code: 'Latn') language_code: 'ain', script_code: 'Kana')
expect(english).to be_valid expect(ainu).to be_valid
expect { english.save! }.to change(described_class, :count).by(1) expect { ainu.save! }.to change(described_class, :count).by(1)
end end
it 'rejects a duplicate name within the same language' do it 'rejects a duplicate name within the same language' do
@@ -67,11 +66,10 @@ RSpec.describe TagName, type: :model do
end end
it 'completes an ordinary tag factory under future active languages' do it 'completes an ordinary tag factory under future active languages' do
pending '日本語以外の Locale 有効化は別課題で実装する'
locale locale
tag = create(:tag, primary_name: 'future_factory') tag = create(:tag, primary_name: 'future_factory')
expect(TagName.where(tag:, language_code: 'en', primary_flg: true)) expect(TagName.where(tag:, language_code: 'ain', primary_flg: true))
.to contain_exactly(have_attributes(auto_generated: true)) .to contain_exactly(have_attributes(auto_generated: true))
end end
@@ -114,46 +112,44 @@ RSpec.describe TagName, type: :model do
end end
it 'resolves the primary name of the alias language on the same tag' do it 'resolves the primary name of the alias language on the same tag' do
pending '日本語以外の Locale 有効化は別課題で実装する'
locale locale
tag = create(:tag, primary_name: '日本語正本') tag = create(:tag, primary_name: '日本語正本')
english_primary = primary_tag_name_for(tag, 'en') ainu_primary = primary_tag_name_for(tag, 'ain')
english_primary.update!(name: 'english_primary', auto_generated: false) ainu_primary.update!(name: 'ainu_primary', auto_generated: false)
english_alias = create(:tag_name, :alias, tag:, name: 'english_alias', ainu_alias = create(:tag_name, :alias, tag:, name: 'ainu_alias',
language_code: 'en', script_code: 'Latn') language_code: 'ain', script_code: 'Kana')
representative_id = tag.tag_name('ja')&.id representative_id = tag.tag_name('ja')&.id
primary = described_class.find_by!(tag_id: english_alias.tag_id, primary = described_class.find_by!(tag_id: ainu_alias.tag_id,
language_code: english_alias.language_code, language_code: ainu_alias.language_code,
primary_flg: true) primary_flg: true)
expect(primary).to eq(english_primary) expect(primary).to eq(ainu_primary)
expect(described_class.canonicalise(locale, [english_alias.name])) expect(described_class.canonicalise(locale, [ainu_alias.name]))
.to eq([english_primary.name]) .to eq([ainu_primary.name])
expect(tag.reload.tag_name('ja')&.id).to eq(representative_id) expect(tag.reload.tag_name('ja')&.id).to eq(representative_id)
end end
it 'resolves only aliases in the requested language' do it 'resolves only aliases in the requested language' do
pending '日本語以外の Locale 有効化は別課題で実装する'
locale locale
nipponese = create(:tag, primary_name: 'nipponese_canonical') nipponese = create(:tag, primary_name: 'nipponese_canonical')
english_name = create(:tag_name, name: 'english_canonical', ainu_name = create(:tag_name, name: 'ainu_canonical',
language_code: 'en', script_code: 'Latn') language_code: 'ain', script_code: 'Kana')
english = create(:tag, primary_tag_name: english_name) ainu = create(:tag, primary_tag_name: ainu_name)
nipponese_alias = create(:tag_name, :alias, name: 'shared_alias', nipponese_alias = create(:tag_name, :alias, name: 'shared_alias',
tag: nipponese) tag: nipponese)
english_alias = create(:tag_name, :alias, name: 'shared_alias', ainu_alias = create(:tag_name, :alias, name: 'shared_alias',
tag: english, tag: ainu,
language_code: 'en', script_code: 'Latn') language_code: 'ain', script_code: 'Kana')
expect(nipponese_alias.reload).to have_attributes( expect(nipponese_alias.reload).to have_attributes(
tag_id: nipponese.id, tag_id: nipponese.id,
primary_flg: false, language_code: 'ja') primary_flg: false, language_code: 'ja')
expect(english_alias.reload).to have_attributes( expect(ainu_alias.reload).to have_attributes(
tag_id: english.id, tag_id: ainu.id,
primary_flg: false, language_code: 'en') primary_flg: false, language_code: 'ain')
expect(described_class.canonicalise(locale, ['shared_alias'])) expect(described_class.canonicalise(locale, ['shared_alias']))
.to eq(['english_canonical']) .to eq(['ainu_canonical'])
expect(described_class.canonicalise(Locale.nipponese, ['shared_alias'])) expect(described_class.canonicalise(Locale.nipponese, ['shared_alias']))
.to eq(['nipponese_canonical']) .to eq(['nipponese_canonical'])
end end
+63 -74
ファイルの表示
@@ -8,7 +8,7 @@ RSpec.describe Tag, type: :model do
it 'checks the requested language and treats an empty description as present' do it 'checks the requested language and treats an empty description as present' do
TagDescription.create!(tag:, language_code: 'ja', body: '', version_no: 1) TagDescription.create!(tag:, language_code: 'ja', body: '', version_no: 1)
expect(tag.has_description?(language_code: 'ja')).to be(true) expect(tag.has_description?(language_code: 'ja')).to be(true)
expect(tag.has_description?(language_code: 'en')).to be(false) expect(tag.has_description?(language_code: 'ain')).to be(false)
end end
it 'returns false without a description' do it 'returns false without a description' do
@@ -23,16 +23,16 @@ RSpec.describe Tag, type: :model do
subscriber = ->(*args) { queries << args.last.fetch(:sql) } subscriber = ->(*args) { queries << args.last.fetch(:sql) }
ActiveSupport::Notifications.subscribed(subscriber, 'sql.active_record') do ActiveSupport::Notifications.subscribed(subscriber, 'sql.active_record') do
expect(loaded.has_description?(language_code: 'ja')).to be(true) expect(loaded.has_description?(language_code: 'ja')).to be(true)
expect(loaded.has_description?(language_code: 'en')).to be(false) expect(loaded.has_description?(language_code: 'ain')).to be(false)
end end
expect(queries).to be_empty expect(queries).to be_empty
end end
it 'keeps language selection generic without enabling another Language record' do it 'keeps language selection generic without enabling another Language record' do
tag.association(:tag_descriptions).target = [ tag.association(:tag_descriptions).target = [
TagDescription.new(tag:, language_code: 'en', body: '', version_no: 1)] TagDescription.new(tag:, language_code: 'ain', body: '', version_no: 1)]
tag.association(:tag_descriptions).loaded! tag.association(:tag_descriptions).loaded!
expect(tag.has_description?(language_code: 'en')).to be(true) expect(tag.has_description?(language_code: 'ain')).to be(true)
expect(tag.has_description?(language_code: 'ja')).to be(false) expect(tag.has_description?(language_code: 'ja')).to be(false)
end end
end end
@@ -61,33 +61,32 @@ RSpec.describe Tag, type: :model do
end end
describe '.normalise_tags!' do describe '.normalise_tags!' do
include_context 'English locale' include_context 'Ainu locale'
it 'canonicalises aliases in the supplied language only' do it 'canonicalises aliases in the supplied language only' do
pending '日本語以外の Locale 有効化は別課題で実装する' prepare_ainu_reference!
prepare_english_reference!
nipponese = create(:tag, primary_name: 'nipponese_canonical') nipponese = create(:tag, primary_name: 'nipponese_canonical')
english_name = create(:tag_name, name: 'english_canonical', ainu_name = build(:tag_name, tag: nil, name: 'ainu_canonical',
language_code: 'en', script_code: 'Latn') language_code: 'ain', script_code: 'Kana')
english = create(:tag, primary_tag_name: english_name) ainu = create(:tag, primary_tag_name: ainu_name)
primary_tag_name_for(english, 'ja').update!( primary_tag_name_for(ainu, 'ja').update!(
name: 'english_in_nipponese', auto_generated: false) name: 'ainu_in_nipponese', auto_generated: false)
english_locale = locale ainu_locale = locale
nipponese_alias = create(:tag_name, :alias, name: 'shared_alias', nipponese_alias = create(:tag_name, :alias, name: 'shared_alias',
tag: nipponese) tag: nipponese)
english_alias = create(:tag_name, :alias, name: 'shared_alias', ainu_alias = create(:tag_name, :alias, name: 'shared_alias',
tag: english, tag: ainu,
language_code: 'en', script_code: 'Latn') language_code: 'ain', script_code: 'Kana')
expect(nipponese_alias.reload).to have_attributes( expect(nipponese_alias.reload).to have_attributes(
tag_id: nipponese.id, tag_id: nipponese.id,
primary_flg: false, language_code: 'ja') primary_flg: false, language_code: 'ja')
expect(english_alias.reload).to have_attributes( expect(ainu_alias.reload).to have_attributes(
tag_id: english.id, tag_id: ainu.id,
primary_flg: false, language_code: 'en') primary_flg: false, language_code: 'ain')
expect(described_class.normalise_tags!( expect(described_class.normalise_tags!(
english_locale, ['shared_alias'], ainu_locale, ['shared_alias'],
with_tagme: false, with_no_deerjikist: false)).to eq([english]) with_tagme: false, with_no_deerjikist: false)).to eq([ainu])
expect(described_class.normalise_tags!( expect(described_class.normalise_tags!(
Locale.nipponese, ['shared_alias'], Locale.nipponese, ['shared_alias'],
with_tagme: false, with_no_deerjikist: false)).to eq([nipponese]) with_tagme: false, with_no_deerjikist: false)).to eq([nipponese])
@@ -253,129 +252,120 @@ RSpec.describe Tag, type: :model do
describe '.find_or_create_by_tag_name!' do describe '.find_or_create_by_tag_name!' do
context 'with an explicit locale' do context 'with an explicit locale' do
include_context 'English locale' include_context 'Ainu locale'
it 'creates the representative name with locale attributes and ownership' do it 'creates the representative name with locale attributes and ownership' do
pending '日本語以外の Locale 有効化は別課題で実装する'
locale locale
tag = described_class.find_or_create_by_tag_name!( tag = described_class.find_or_create_by_tag_name!(
locale, 'english_name', category: :character) locale, 'ainu_name', category: :character)
expect(primary_tag_name_for(tag, 'en')).to have_attributes( expect(primary_tag_name_for(tag, 'ain')).to have_attributes(
language_code: 'en', script_code: 'Latn', language_code: 'ain', script_code: 'Kana',
primary_flg: true, tag_id: tag.id) primary_flg: true, tag_id: tag.id)
expect(tag.category).to eq('character') expect(tag.category).to eq('character')
end end
it 'uses the English primary as the source for a generated Nipponese name' do it 'copies an Ainu-origin name to Japanese without reverse transcription' do
pending '日本語以外の Locale 有効化は別課題で実装する'
locale locale
allow(TagName).to receive(:generate_name).and_return('generated_nipponese') allow(TagName).to receive(:generate_name).and_call_original
tag = described_class.find_or_create_by_tag_name!( tag = described_class.find_or_create_by_tag_name!(
locale, 'english_origin', category: :general) locale, 'ainu_origin', category: :general)
expect(TagName.where(tag:, language_code: 'en', primary_flg: true)) expect(TagName.where(tag:, language_code: 'ain', primary_flg: true))
.to contain_exactly(have_attributes( .to contain_exactly(have_attributes(
name: 'english_origin', auto_generated: false)) name: 'ainu_origin', auto_generated: false))
expect(TagName.where(tag:, language_code: 'ja', primary_flg: true)) expect(TagName.where(tag:, language_code: 'ja', primary_flg: true))
.to contain_exactly(have_attributes( .to contain_exactly(have_attributes(
name: 'generated_nipponese', auto_generated: true)) name: 'ainu_origin', auto_generated: true))
source_tag_name = TagName.find_by!(tag:, language_code: 'en', primary_flg: true)
expect(TagName).to have_received(:generate_name) expect(TagName).to have_received(:generate_name)
.with(Locale.nipponese, tag, source_tag_name) .with(Locale.nipponese, tag, tag.tag_name('ain'))
end end
it 'gives a new tag one primary name for every active language' do it 'gives a new tag one primary name for every active language' do
pending '日本語以外の Locale 有効化は別課題で実装する'
locale locale
tag = described_class.find_or_create_by_tag_name!( tag = described_class.find_or_create_by_tag_name!(
Locale.nipponese, 'new_multilingual_tag', category: :general) Locale.nipponese, 'new_multilingual_tag', category: :general)
expect(TagName.where(tag:, language_code: 'ja', primary_flg: true).count).to eq(1) expect(TagName.where(tag:, language_code: 'ja', primary_flg: true).count).to eq(1)
expect(TagName.where(tag:, language_code: 'en', primary_flg: true).count).to eq(1) expect(TagName.where(tag:, language_code: 'ain', primary_flg: true).count).to eq(1)
end end
it 'creates one primary per language when multiple locales use English' do it 'creates one primary per language when multiple locales use Ainu' do
pending '日本語以外の Locale 有効化は別課題で実装する'
locale locale
described_class.find_or_create_by_tag_name!( described_class.find_or_create_by_tag_name!(
locale, 'before_second_english_locale', category: :general) locale, 'before_second_ainu_locale', category: :general)
Locale.create!(code: 'en-x-rspec', language_code: 'en',
script_code: 'Latn', region_code: 826,
variants: 'x-rspec', name: 'English')
tag = described_class.find_or_create_by_tag_name!( tag = described_class.find_or_create_by_tag_name!(
locale, 'one_english_primary', category: :general) locale, 'one_ainu_primary', category: :general)
TagNameCompletion.for_tag!(tag, locales: [locale, locale.dup, Locale.nipponese])
expect(TagName.where(tag:, language_code: 'en', primary_flg: true).count).to eq(1) expect(TagName.where(tag:, language_code: 'ain', primary_flg: true).count).to eq(1)
expect(TagName.where(tag:, language_code: 'ja', primary_flg: true).count).to eq(1) expect(TagName.where(tag:, language_code: 'ja', primary_flg: true).count).to eq(1)
end end
it 'rolls back a new tag and requested name if name generation fails' do it 'rolls back a new tag and requested name if name generation fails' do
pending '日本語以外の Locale 有効化は別課題で実装する'
locale locale
original_tag_count = described_class.count original_tag_count = described_class.count
allow(TagName).to receive(:generate_name).and_raise('generation failed') allow(TagName).to receive(:generate_name).and_raise('generation failed')
expect { expect {
described_class.find_or_create_by_tag_name!( described_class.find_or_create_by_tag_name!(
locale, 'atomic_english_name', category: :general) locale, 'atomic_ainu_name', category: :general)
}.to raise_error('generation failed') }.to raise_error('generation failed')
expect(described_class.count).to eq(original_tag_count) expect(described_class.count).to eq(original_tag_count)
expect(TagName.where(language_code: 'en', name: 'atomic_english_name')).to be_empty expect(TagName.where(language_code: 'ain', name: 'atomic_ainu_name')).to be_empty
expect(described_class.joins(:tag_names) expect(described_class.joins(:tag_names)
.where(tag_names: { name: 'atomic_english_name' })).to be_empty .where(tag_names: { name: 'atomic_ainu_name' })).to be_empty
end end
it 'creates separate tag identities for the same name in different languages' do it 'creates separate tag identities for the same name in different languages' do
pending '日本語以外の Locale 有効化は別課題で実装する'
locale locale
nipponese = described_class.find_or_create_by_tag_name!( nipponese = described_class.find_or_create_by_tag_name!(
Locale.nipponese, 'same_name', category: :general) Locale.nipponese, 'same_name', category: :general)
english = described_class.find_or_create_by_tag_name!( primary_tag_name_for(nipponese, 'ain').update!(
name: 'other_ainu_name', auto_generated: false)
ainu = described_class.find_or_create_by_tag_name!(
locale, 'same_name', category: :character) locale, 'same_name', category: :character)
expect(english).not_to eq(nipponese) expect(ainu).not_to eq(nipponese)
expect(primary_tag_name_for(english, 'en')).to have_attributes( expect(primary_tag_name_for(ainu, 'ain')).to have_attributes(
name: 'same_name', tag_id: english.id, language_code: 'en', primary_flg: true) name: 'same_name', tag_id: ainu.id, language_code: 'ain', primary_flg: true)
expect(primary_tag_name_for(nipponese, 'ja')).to have_attributes( expect(primary_tag_name_for(nipponese, 'ja')).to have_attributes(
name: 'same_name', tag_id: nipponese.id, language_code: 'ja', primary_flg: true) name: 'same_name', tag_id: nipponese.id, language_code: 'ja', primary_flg: true)
expect(described_class.find_or_create_by_tag_name!( expect(described_class.find_or_create_by_tag_name!(
locale, 'same_name', category: :general)).to eq(english) locale, 'same_name', category: :general)).to eq(ainu)
expect(described_class.find_or_create_by_tag_name!( expect(described_class.find_or_create_by_tag_name!(
Locale.nipponese, 'same_name', category: :general)).to eq(nipponese) Locale.nipponese, 'same_name', category: :general)).to eq(nipponese)
end end
it 'preserves the V1 representative when looking up another primary language' do it 'preserves the V1 representative when looking up another primary language' do
pending '日本語以外の Locale 有効化は別課題で実装する'
locale locale
tag = create(:tag, primary_name: '日本語代表名') tag = create(:tag, primary_name: '日本語代表名')
representative_id = tag.tag_name('ja')&.id representative_id = tag.tag_name('ja')&.id
english = primary_tag_name_for(tag, 'en') ainu = primary_tag_name_for(tag, 'ain')
english.update!(name: 'english_primary', auto_generated: false) ainu.update!(name: 'ainu_primary', auto_generated: false)
found = described_class.find_or_create_by_tag_name!( found = described_class.find_or_create_by_tag_name!(
locale, english.name, category: :general) locale, ainu.name, category: :general)
expect(found).to eq(tag) expect(found).to eq(tag)
expect(tag.reload.tag_name('ja')&.id).to eq(representative_id) expect(tag.reload.tag_name('ja')&.id).to eq(representative_id)
expect(english.reload.tag_id).to eq(tag.id) expect(ainu.reload.tag_id).to eq(tag.id)
end end
it 'normalises names using the supplied locale' do it 'normalises names using the supplied locale' do
pending '日本語以外の Locale 有効化は別課題で実装する'
locale locale
tags = described_class.normalise_tags!( tags = described_class.normalise_tags!(
locale, ['character:normalised_english'], locale, ['character:normalised_ainu'],
with_tagme: false, with_no_deerjikist: false) with_tagme: false, with_no_deerjikist: false)
expect(tags.length).to eq(1) expect(tags.length).to eq(1)
expect(primary_tag_name_for(tags.first, 'en')).to have_attributes( expect(primary_tag_name_for(tags.first, 'ain')).to have_attributes(
name: 'normalised_english', language_code: 'en', script_code: 'Latn', name: 'normalised_ainu', language_code: 'ain', script_code: 'Kana',
primary_flg: true, tag_id: tags.first.id) primary_flg: true, tag_id: tags.first.id)
end end
end end
@@ -444,26 +434,25 @@ RSpec.describe Tag, type: :model do
end end
context 'with names in multiple languages' do context 'with names in multiple languages' do
include_context 'English locale' include_context 'Ainu locale'
it 'keeps the target primary and moves source names as aliases' do it 'keeps the target primary and moves source names as aliases' do
pending '日本語以外の Locale 有効化は別課題で実装する'
locale locale
english_primary = primary_tag_name_for(target_tag, 'en') ainu_primary = primary_tag_name_for(target_tag, 'ain')
source_primary = primary_tag_name_for(source_tag, 'en') source_primary = primary_tag_name_for(source_tag, 'ain')
source_alias = create(:tag_name, :alias, tag: source_tag, source_alias = create(:tag_name, :alias, tag: source_tag,
name: 'source_english_alias', language_code: 'en', script_code: 'Latn') name: 'source_ainu_alias', language_code: 'ain', script_code: 'Kana')
expect([english_primary, source_primary]).to all(have_attributes( expect([ainu_primary, source_primary]).to all(have_attributes(
primary_flg: true, auto_generated: true)) primary_flg: true, auto_generated: true))
described_class.merge_tags!(target_tag, [source_tag]) described_class.merge_tags!(target_tag, [source_tag])
expect(primary_tag_name_for(target_tag.reload, 'en')).to eq(english_primary) expect(primary_tag_name_for(target_tag.reload, 'ain')).to eq(ainu_primary)
expect(TagName.where(tag: target_tag, language_code: 'en', primary_flg: true)) expect(TagName.where(tag: target_tag, language_code: 'ain', primary_flg: true))
.to contain_exactly(english_primary) .to contain_exactly(ainu_primary)
expect([source_primary.reload, source_alias.reload]).to all(have_attributes( expect([source_primary.reload, source_alias.reload]).to all(have_attributes(
tag_id: target_tag.id, language_code: 'en', primary_flg: false)) tag_id: target_tag.id, language_code: 'ain', primary_flg: false))
expect(TagName.where(tag: target_tag, language_code: 'ja', primary_flg: true)) expect(TagName.where(tag: target_tag, language_code: 'ja', primary_flg: true))
.to contain_exactly(primary_tag_name_for(target_tag, 'ja')) .to contain_exactly(primary_tag_name_for(target_tag, 'ja'))
expect(source_tag_name.reload).to have_attributes( expect(source_tag_name.reload).to have_attributes(
+8
ファイルの表示
@@ -53,6 +53,14 @@ RSpec.configure do |config|
config.include AuthHelper, type: :request config.include AuthHelper, type: :request
config.include JsonHelper, type: :request config.include JsonHelper, type: :request
config.around(:each, type: :request) do |example|
previous = ApplicationController.allow_forgery_protection
ApplicationController.allow_forgery_protection = example.metadata.fetch(:csrf, false)
example.run
ensure
ApplicationController.allow_forgery_protection = previous
end
# Remove this line if you're not using ActiveRecord or ActiveRecord fixtures # Remove this line if you're not using ActiveRecord or ActiveRecord fixtures
config.fixture_paths = [ config.fixture_paths = [
Rails.root.join('spec/fixtures') Rails.root.join('spec/fixtures')
+96
ファイルの表示
@@ -0,0 +1,96 @@
require 'rails_helper'
RSpec.describe 'Counts API', type: :request do
let!(:actor) { create(:user, :member) }
let!(:post) { create(:post) }
let!(:tag) { create(:tag, primary_name: 'counts tag', category: :general) }
before do
snapshot = { tag_id: tag.id, version_no: 1, name: 'counts tag',
category: 'general', sections: [] }
[1, 2, 3].each do |number|
time = Time.utc(2024, number == 3 ? 2 : 1, number, 0, 0)
PostVersion.create!(post:, version_no: number,
event_type: number == 1 ? 'create' : 'update',
tags: '', url: post.url, title: "Snapshot #{ number }", parent_post_ids: '',
tags_json: number == 1 ? [snapshot] : [],
created_by_user: number == 3 ? nil : actor, created_at: time)
TagVersion.create!(tag:, version_no: number,
event_type: number == 1 ? 'create' : 'update',
name: 'counts tag', category: 'general', aliases: '', parent_tag_ids: '',
names: { ja: { name: 'counts tag', script_code: 'Jpan',
auto_generated: false, aliases: [] } },
created_by_user: number == 3 ? nil : actor, created_at: time)
end
end
['posts', 'tags'].each do |resource|
context resource do
let(:path) { "/counts/#{ resource }" }
let(:id) { resource == 'posts' ? post.id : tag.id }
let(:filters) { { resource_id: id, user_id: actor.id,
from: '2024-01-01T00:00Z', before: '2024-02-01T00:00Z' } }
it 'uses the same AND filters for SQL totals, groups and details' do
get path, params: filters.merge(axis: 'event_type')
expect(response).to have_http_status(:ok)
summary = response.parsed_body
expect(summary['count']).to eq(2)
expect(summary['groups'].sum { _1['count'] }).to eq(2)
expect(summary['groups'].map { _1['key'] }).to eq(['create', 'update'])
get path, params: filters.merge(mode: 'details')
expect(response.parsed_body['count']).to eq(2)
expect(response.parsed_body['versions'].map { _1['version_no'] }).to eq([2, 1])
end
it 'paginates details and groups independently from totals' do
get path, params: { resource_id: id, mode: 'details', limit: 1, page: 2 }
expect(response.parsed_body['count']).to eq(3)
expect(response.parsed_body['versions'].sole['version_no']).to eq(2)
get path, params: { resource_id: id, axis: 'month', limit: 1, page: 2 }
expect(response.parsed_body['group_count']).to eq(2)
expect(response.parsed_body['groups'].sole).to include('key' => '2024-02', 'count' => 1)
end
it 'groups null actors and supports their drill-down filter' do
get path, params: { resource_id: id, axis: 'user' }
expect(response.parsed_body['groups']).to include(
{ 'key' => nil, 'name' => nil, 'count' => 1 })
get path, params: { resource_id: id, mode: 'details', user_id: 'none' }
expect(response.parsed_body['count']).to eq(1)
end
it 'uses an inclusive start and exclusive end' do
get path, params: { resource_id: id,
from: '2024-01-01T00:00Z', before: '2024-01-02T00:00Z' }
expect(response.parsed_body['count']).to eq(1)
end
it 'rejects unsupported or malformed query values' do
[
{ axis: 'SQL' }, { limit: 101 }, { page: '1x' }, { user_id: '-1' },
{ event_type: 'invalid' }, { columns: 'password' }, { order: 'id' },
{ from: '2024-02-30T00:00Z' }, { from: '2024-01-01' },
{ user_id: ['1'] }, { mode: 'invalid' }].each do |invalid|
get path, params: invalid
expect(response).to have_http_status(:bad_request)
end
end
it 'retains the existing banned-user boundary' do
actor.update!(banned_at: Time.current)
sign_in_as(actor)
get path
expect(response).to have_http_status(:forbidden)
end
end
end
it 'matches removed tags from the previous snapshot, not current post_tags' do
get '/counts/posts', params: { tag: tag.id, mode: 'details' }
expect(response.parsed_body['count']).to eq(2)
expect(response.parsed_body['versions'].map { _1['version_no'] }).to eq([2, 1])
expect(response.parsed_body['versions'].first['tags'].sole['type']).to eq('removed')
end
end
+54
ファイルの表示
@@ -0,0 +1,54 @@
require 'rails_helper'
RSpec.describe 'API locale resolution', type: :request do
let!(:tag) { create(:tag, primary_name: '日本語正本') }
it 'defaults an omitted locale to Japanese' do
get "/tags/#{ tag.id }"
expect(response).to have_http_status(:ok)
expect(json.fetch('name')).to eq('日本語正本')
end
it 'accepts an explicitly requested active Japanese locale' do
get "/tags/#{ tag.id }", params: { locale: 'ja' }
expect(response).to have_http_status(:ok)
expect(json.fetch('name')).to eq('日本語正本')
end
['', 'en', 'ain', 'Jpan', 'JA', 'unknown', ['ja']].each do |code|
it "rejects invalid locale #{ code.inspect } instead of defaulting to Japanese" do
get "/tags/#{ tag.id }", params: { locale: code }
expect(response).to have_http_status(:not_found)
expect(response.body).not_to include('日本語正本')
end
end
it 'rejects a supported locale that has not been registered' do
get "/tags/#{ tag.id }", params: { locale: 'ain-Kana' }
expect(response).to have_http_status(:not_found)
end
it 'rejects an inactive locale' do
Locale.nipponese.update!(deprecated_at: Time.current)
get "/tags/#{ tag.id }", params: { locale: 'ja' }
expect(response).to have_http_status(:not_found)
end
it 'does not write posts or tag names under an invalid locale' do
sign_in_as(create_member_user!)
expect do
post '/posts', params: {
locale: 'en', title: 'invalid locale', url: 'https://example.com/invalid-locale',
tags: 'unwritten_tag' }
end.to change(Post, :count).by(0).and change(TagName, :count).by(0)
expect(response).to have_http_status(:not_found)
end
end
+81 -81
ファイルの表示
@@ -1,7 +1,7 @@
require 'rails_helper' require 'rails_helper'
RSpec.describe 'Locale propagation on write paths', type: :request do RSpec.describe 'Locale propagation on write paths', type: :request do
include_context 'English locale' include_context 'Ainu locale'
let(:member) { create(:user, :member) } let(:member) { create(:user, :member) }
@@ -14,7 +14,19 @@ RSpec.describe 'Locale propagation on write paths', type: :request do
expect(tag_name).to have_attributes( expect(tag_name).to have_attributes(
script_code: expected_locale.script_code, primary_flg: true) script_code: expected_locale.script_code, primary_flg: true)
expect(tag_name.tag_id).to be_present expect(tag_name.tag_id).to be_present
tag_name.tag tag = tag_name.tag
version = tag.tag_versions.sole
expect(version).to have_attributes(
event_type: 'create', version_no: 1, created_by_user_id: member.id)
expect(tag.reload.version_no).to eq(1)
expect(version.names.keys).to match_array(
Locale.where(deprecated_at: nil).distinct.pluck(:language_code))
tag.tag_names.where(primary_flg: true).each do |primary|
expect(version.names.fetch(primary.language_code)).to include(
'name' => primary.name, 'script_code' => primary.script_code,
'auto_generated' => primary.auto_generated, 'aliases' => [])
end
tag
end end
it 'confirms a generated Nipponese primary when a human renames it' do it 'confirms a generated Nipponese primary when a human renames it' do
@@ -54,13 +66,12 @@ RSpec.describe 'Locale propagation on write paths', type: :request do
end end
end end
['en', nil].each do |requested_locale| ['ain-Kana', nil].each do |requested_locale|
context "with locale #{ requested_locale.inspect }" do context "with locale #{ requested_locale.inspect }" do
let(:locale_params) { requested_locale ? { locale: requested_locale } : { } } let(:locale_params) { requested_locale ? { locale: requested_locale } : { } }
let(:expected_locale) { requested_locale ? locale : Locale.nipponese } let(:expected_locale) { requested_locale ? locale : Locale.nipponese }
it 'creates post tags in the requested language or Nipponese fallback' do it 'creates post tags in the requested language or Nipponese fallback' do
pending '日本語以外の Locale 有効化は別課題で実装する' if requested_locale
locale if requested_locale locale if requested_locale
post '/posts', params: locale_params.merge( post '/posts', params: locale_params.merge(
title: 'Locale post', url: 'https://example.com/locale-post', title: 'Locale post', url: 'https://example.com/locale-post',
@@ -72,7 +83,6 @@ RSpec.describe 'Locale propagation on write paths', type: :request do
end end
it 'updates post tags in the requested language or Nipponese fallback' do it 'updates post tags in the requested language or Nipponese fallback' do
pending '日本語以外の Locale 有効化は別課題で実装する' if requested_locale
locale if requested_locale locale if requested_locale
record = create(:post) record = create(:post)
PostVersionRecorder.record!(post: record, event_type: :create, created_by_user: member) PostVersionRecorder.record!(post: record, event_type: :create, created_by_user: member)
@@ -87,7 +97,6 @@ RSpec.describe 'Locale propagation on write paths', type: :request do
end end
it 'passes the resolved locale to bulk creation' do it 'passes the resolved locale to bulk creation' do
pending '日本語以外の Locale 有効化は別課題で実装する' if requested_locale
locale if requested_locale locale if requested_locale
# Worker propagation is exercised in post_bulk_creator_spec. # Worker propagation is exercised in post_bulk_creator_spec.
expect_any_instance_of(PostBulkCreator).to receive(:run) expect_any_instance_of(PostBulkCreator).to receive(:run)
@@ -102,7 +111,6 @@ RSpec.describe 'Locale propagation on write paths', type: :request do
end end
it 'creates materials with tags in the resolved locale' do it 'creates materials with tags in the resolved locale' do
pending '日本語以外の Locale 有効化は別課題で実装する' if requested_locale
locale if requested_locale locale if requested_locale
post '/materials', params: locale_params.merge( post '/materials', params: locale_params.merge(
tag: 'locale_material', url: 'https://example.com/material') tag: 'locale_material', url: 'https://example.com/material')
@@ -113,7 +121,6 @@ RSpec.describe 'Locale propagation on write paths', type: :request do
end end
it 'updates materials with tags in the resolved locale' do it 'updates materials with tags in the resolved locale' do
pending '日本語以外の Locale 有効化は別課題で実装する' if requested_locale
locale if requested_locale locale if requested_locale
tag = create(:tag, category: :material) tag = create(:tag, category: :material)
material = Material.create!(tag:, url: 'https://example.com/material') material = Material.create!(tag:, url: 'https://example.com/material')
@@ -127,7 +134,6 @@ RSpec.describe 'Locale propagation on write paths', type: :request do
end end
it 'links Nico external tags to internal tags in the resolved locale' do it 'links Nico external tags to internal tags in the resolved locale' do
pending '日本語以外の Locale 有効化は別課題で実装する' if requested_locale
locale if requested_locale locale if requested_locale
external = create(:external_tag) external = create(:external_tag)
NicoTagVersionRecorder.record!( NicoTagVersionRecorder.record!(
@@ -141,7 +147,6 @@ RSpec.describe 'Locale propagation on write paths', type: :request do
end end
it 'creates parent tags in the resolved locale' do it 'creates parent tags in the resolved locale' do
pending '日本語以外の Locale 有効化は別課題で実装する' if requested_locale
locale if requested_locale locale if requested_locale
tag = create(:tag) tag = create(:tag)
@@ -157,184 +162,179 @@ RSpec.describe 'Locale propagation on write paths', type: :request do
end end
context 'with existing names in both languages' do context 'with existing names in both languages' do
let(:english_name) do let(:ainu_name) do
create(:tag_name, name: 'shared_name', language_code: 'en', script_code: 'Latn') create(:tag_name, name: 'shared_name', language_code: 'ain', script_code: 'Kana')
end
let(:ainu_tag) do
create(:tag, primary_tag_name: ainu_name, category: :material).tap do |tag|
primary_tag_name_for(tag, 'ja').update!(name: 'ainu_in_japanese', auto_generated: false)
end
end end
let(:english_tag) { create(:tag, primary_tag_name: english_name, category: :material) }
let(:nipponese_tag) do let(:nipponese_tag) do
create(:tag, primary_name: 'temporary_nipponese', category: :general).tap do |tag| create(:tag, primary_name: 'temporary_nipponese', category: :general).tap do |tag|
# Isolate lookup from the separately tested uniqueness validation. primary_tag_name_for(tag, 'ja').update!(name: 'shared_name')
primary_tag_name_for(tag, 'ja').update_columns(name: 'shared_name')
end end
end end
it 'plans and creates the post using the English identity and category' do it 'plans and creates the post using the Ainu identity and category' do
pending '日本語以外の Locale 有効化は別課題で実装する'
locale locale
english_tag ainu_tag
nipponese_tag nipponese_tag
post '/posts', params: { post '/posts', params: {
locale: 'en', title: 'English identity', locale: 'ain-Kana', title: 'Ainu identity',
url: 'https://example.com/english-identity', tags: 'shared_name', parent_post_ids: '' } url: 'https://example.com/ainu-identity', tags: 'shared_name', parent_post_ids: '' }
expect(response).to have_http_status(:created), response.body expect(response).to have_http_status(:created), response.body
record = Post.find(json.fetch('id')) record = Post.find(json.fetch('id'))
expect(record.tags).to include(english_tag) expect(record.tags).to include(ainu_tag)
expect(record.tags).not_to include(nipponese_tag) expect(record.tags).not_to include(nipponese_tag)
expect(english_tag.reload.category).to eq('material') expect(ainu_tag.reload.category).to eq('material')
expect(nipponese_tag.reload.category).to eq('general') expect(nipponese_tag.reload.category).to eq('general')
end end
it 'updates the post using the English identity' do it 'updates the post using the Ainu identity' do
pending '日本語以外の Locale 有効化は別課題で実装する'
locale locale
english_tag ainu_tag
nipponese_tag nipponese_tag
record = create(:post) record = create(:post)
PostVersionRecorder.record!(post: record, event_type: :create, created_by_user: member) PostVersionRecorder.record!(post: record, event_type: :create, created_by_user: member)
put "/posts/#{ record.id }", params: { put "/posts/#{ record.id }", params: {
locale: 'en', title: record.title, tags: 'shared_name', parent_post_ids: '', locale: 'ain-Kana', title: record.title, tags: 'shared_name', parent_post_ids: '',
base_version_no: record.reload.version_no } base_version_no: record.reload.version_no }
expect(response).to have_http_status(:ok), response.body expect(response).to have_http_status(:ok), response.body
expect(record.reload.tags).to include(english_tag) expect(record.reload.tags).to include(ainu_tag)
expect(record.tags).not_to include(nipponese_tag) expect(record.tags).not_to include(nipponese_tag)
end end
it 'resolves the English material tag' do it 'resolves the Ainu material tag' do
pending '日本語以外の Locale 有効化は別課題で実装する'
locale locale
english_tag ainu_tag
nipponese_tag nipponese_tag
post '/materials', params: { post '/materials', params: {
locale: 'en', tag: 'shared_name', url: 'https://example.com/english-material' } locale: 'ain-Kana', tag: 'shared_name', url: 'https://example.com/ainu-material' }
expect(response).to have_http_status(:created), response.body expect(response).to have_http_status(:created), response.body
expect(Material.find(json.fetch('id')).tag).to eq(english_tag) expect(Material.find(json.fetch('id')).tag).to eq(ainu_tag)
expect(Material.where(tag: nipponese_tag)).to be_empty expect(Material.where(tag: nipponese_tag)).to be_empty
end end
it 'resolves the English parent tag' do it 'resolves the Ainu parent tag' do
pending '日本語以外の Locale 有効化は別課題で実装する'
locale locale
english_tag ainu_tag
nipponese_tag nipponese_tag
child = create(:tag) child = create(:tag)
put "/tags/#{ child.id }", params: { put "/tags/#{ child.id }", params: {
locale: 'en', name: primary_tag_name_for(child, 'ja').name, locale: 'ain-Kana', name: primary_tag_name_for(child, 'ja').name,
category: child.category, deprecated: false, category: child.category, deprecated: false,
aliases: '', parent_tags: 'shared_name' } aliases: '', parent_tags: 'shared_name' }
expect(response).to have_http_status(:ok), response.body expect(response).to have_http_status(:ok), response.body
expect(TagImplication.where(tag: child).pluck(:parent_tag_id)).to eq([english_tag.id]) expect(TagImplication.where(tag: child).pluck(:parent_tag_id)).to eq([ainu_tag.id])
end end
it 'resolves the English internal tag for a Nico link' do it 'resolves the Ainu internal tag for a Nico link' do
pending '日本語以外の Locale 有効化は別課題で実装する'
locale locale
english_tag ainu_tag
nipponese_tag nipponese_tag
external = create(:external_tag) external = create(:external_tag)
NicoTagVersionRecorder.record!( NicoTagVersionRecorder.record!(
external_tag: external, event_type: :create, created_by_user: member) external_tag: external, event_type: :create, created_by_user: member)
put "/tags/nico/#{ external.id }", params: { locale: 'en', tags: 'shared_name' } put "/tags/nico/#{ external.id }", params: { locale: 'ain-Kana', tags: 'shared_name' }
expect(response).to have_http_status(:ok), response.body expect(response).to have_http_status(:ok), response.body
expect(external.reload.linked_tags).to contain_exactly(english_tag) expect(external.reload.linked_tags).to contain_exactly(ainu_tag)
end end
end end
context 'when renaming or updating aliases in English' do context 'when renaming or updating aliases in Ainu' do
let(:english_name) do let(:ainu_name) do
create(:tag_name, name: 'english_original', language_code: 'en', script_code: 'Latn') create(:tag_name, name: 'ainu_original', language_code: 'ain', script_code: 'Kana')
end end
let(:tag) { create(:tag, primary_tag_name: english_name) } let(:tag) { create(:tag, primary_tag_name: ainu_name) }
it 'marks a human-renamed generated primary as no longer auto-generated' do it 'marks a human-renamed generated primary as no longer auto-generated' do
pending '日本語以外の Locale 有効化は別課題で実装する'
locale locale
generated_tag = create(:tag, primary_name: 'rename_generated_source') generated_tag = create(:tag, primary_name: 'rename_generated_source')
generated_name = primary_tag_name_for(generated_tag, 'en') generated_name = primary_tag_name_for(generated_tag, 'ain')
expect(generated_name.auto_generated).to be(true) expect(generated_name.auto_generated).to be(true)
patch "/tags/#{ generated_tag.id }", params: { patch "/tags/#{ generated_tag.id }", params: {
locale: 'en', name: 'human_english_name' } locale: 'ain-Kana', name: 'human_ainu_name' }
expect(response).to have_http_status(:ok), response.body expect(response).to have_http_status(:ok), response.body
expect(generated_name.reload).to have_attributes( expect(generated_name.reload).to have_attributes(
name: 'human_english_name', auto_generated: false) name: 'human_ainu_name', auto_generated: false)
end end
it 'does not steal another tag primary for an alias' do it 'does not steal another tag primary for an alias' do
pending '日本語以外の Locale 有効化は別課題で実装する'
locale locale
other_tag = create(:tag, primary_name: 'other_nipponese') other_tag = create(:tag, primary_name: 'other_nipponese')
other_primary = primary_tag_name_for(other_tag, 'en') other_primary = primary_tag_name_for(other_tag, 'ain')
other_primary.update!(name: 'other_english_primary', auto_generated: false) other_primary.update!(name: 'other_ainu_primary', auto_generated: false)
create(:tag_name, :alias, tag:, name: 'existing_english_alias', create(:tag_name, :alias, tag:, name: 'existing_ainu_alias',
language_code: 'en', script_code: 'Latn') language_code: 'ain', script_code: 'Kana')
primary_name_before = primary_tag_name_for(tag, 'en').name primary_name_before = primary_tag_name_for(tag, 'ain').name
aliases_before = aliases_before =
tag.tag_names.where(language_code: 'en', primary_flg: false) tag.tag_names.where(language_code: 'ain', primary_flg: false)
.order(:name).pluck(:name) .order(:name).pluck(:name)
category_before = tag.category category_before = tag.category
put "/tags/#{ tag.id }", params: { put "/tags/#{ tag.id }", params: {
locale: 'en', name: 'renamed_before_alias_collision', locale: 'ain-Kana', name: 'renamed_before_alias_collision',
category: 'character', deprecated: false, category: 'character', deprecated: false,
aliases: 'other_english_primary', parent_tags: '' } aliases: 'other_ainu_primary', parent_tags: '' }
aggregate_failures do aggregate_failures do
expect(response).to have_http_status(:unprocessable_entity) expect(response).to have_http_status(:unprocessable_entity)
expect(other_primary.reload).to have_attributes( expect(other_primary.reload).to have_attributes(
tag_id: other_tag.id, language_code: 'en', primary_flg: true) tag_id: other_tag.id, language_code: 'ain', primary_flg: true)
expect(TagName.where(tag: other_tag, language_code: 'en', primary_flg: true)) expect(TagName.where(tag: other_tag, language_code: 'ain', primary_flg: true))
.to contain_exactly(other_primary) .to contain_exactly(other_primary)
expect(primary_tag_name_for(tag.reload, 'en').name).to eq(primary_name_before) expect(primary_tag_name_for(tag.reload, 'ain').name).to eq(primary_name_before)
expect(tag.category).to eq(category_before) expect(tag.category).to eq(category_before)
aliases_after = aliases_after =
tag.tag_names.where(language_code: 'en', primary_flg: false) tag.tag_names.where(language_code: 'ain', primary_flg: false)
.order(:name).pluck(:name) .order(:name).pluck(:name)
expect(aliases_after).to eq(aliases_before) expect(aliases_after).to eq(aliases_before)
end end
end end
it 'allows a rename to a name already used in Nipponese' do it 'allows a rename to a name already used in Nipponese' do
pending '日本語以外の Locale 有効化は別課題で実装する'
locale locale
nipponese = create(:tag, primary_name: 'rename_target') nipponese = create(:tag, primary_name: 'rename_target')
primary_tag_name_for(nipponese, 'ain').update!(
name: 'different_ainu_target', auto_generated: false)
patch "/tags/#{ tag.id }", params: { locale: 'en', name: 'rename_target' } patch "/tags/#{ tag.id }", params: { locale: 'ain-Kana', name: 'rename_target' }
expect(response).to have_http_status(:ok), response.body expect(response).to have_http_status(:ok), response.body
expect(primary_tag_name_for(tag.reload, 'en')).to have_attributes( expect(primary_tag_name_for(tag.reload, 'ain')).to have_attributes(
name: 'rename_target', language_code: 'en', tag_id: tag.id) name: 'rename_target', language_code: 'ain', tag_id: tag.id)
expect(primary_tag_name_for(nipponese.reload, 'ja')).to have_attributes( expect(primary_tag_name_for(nipponese.reload, 'ja')).to have_attributes(
name: 'rename_target', language_code: 'ja', tag_id: nipponese.id) name: 'rename_target', language_code: 'ja', tag_id: nipponese.id)
end end
it 'creates aliases in the requested language' do it 'creates aliases in the requested language' do
pending '日本語以外の Locale 有効化は別課題で実装する'
locale locale
put "/tags/#{ tag.id }", params: { put "/tags/#{ tag.id }", params: {
locale: 'en', name: english_name.name, locale: 'ain-Kana', name: ainu_name.name,
category: tag.category, deprecated: false, category: tag.category, deprecated: false,
aliases: 'english_alias', parent_tags: '' } aliases: 'ainu_alias', parent_tags: '' }
expect(response).to have_http_status(:ok), response.body expect(response).to have_http_status(:ok), response.body
expect(TagName.find_by!(language_code: 'en', name: 'english_alias')) expect(TagName.find_by!(language_code: 'ain', name: 'ainu_alias'))
.to have_attributes( .to have_attributes(
script_code: 'Latn', script_code: 'Kana',
tag_id: tag.id, primary_flg: false, language_code: 'en') tag_id: tag.id, primary_flg: false, language_code: 'ain')
primary = TagName.find_by!(tag_id: tag.id, language_code: 'en', primary_flg: true) primary = TagName.find_by!(tag_id: tag.id, language_code: 'ain', primary_flg: true)
expect(primary).to have_attributes(name: 'english_original', script_code: 'Latn') expect(primary).to have_attributes(name: 'ainu_original', script_code: 'Kana')
end end
it 'does not take an alias from another language' do it 'does not take an alias from another language' do
pending '日本語以外の Locale 有効化は別課題で実装する'
locale locale
nipponese = create(:tag, primary_name: 'nipponese_owner') nipponese = create(:tag, primary_name: 'nipponese_owner')
nipponese_alias = create(:tag_name, :alias, name: 'shared_alias', nipponese_alias = create(:tag_name, :alias, name: 'shared_alias',
@@ -344,7 +344,7 @@ RSpec.describe 'Locale propagation on write paths', type: :request do
primary_flg: false, language_code: 'ja') primary_flg: false, language_code: 'ja')
put "/tags/#{ tag.id }", params: { put "/tags/#{ tag.id }", params: {
locale: 'en', name: english_name.name, locale: 'ain-Kana', name: ainu_name.name,
category: tag.category, deprecated: false, category: tag.category, deprecated: false,
aliases: 'shared_alias', parent_tags: '' } aliases: 'shared_alias', parent_tags: '' }
@@ -352,12 +352,12 @@ RSpec.describe 'Locale propagation on write paths', type: :request do
expect(nipponese_alias.reload).to have_attributes( expect(nipponese_alias.reload).to have_attributes(
tag_id: nipponese.id, tag_id: nipponese.id,
primary_flg: false, language_code: 'ja') primary_flg: false, language_code: 'ja')
expect(TagName.find_by!(language_code: 'en', name: 'shared_alias')) expect(TagName.find_by!(language_code: 'ain', name: 'shared_alias'))
.to have_attributes( .to have_attributes(
tag_id: tag.id, script_code: 'Latn', tag_id: tag.id, script_code: 'Kana',
primary_flg: false, language_code: 'en') primary_flg: false, language_code: 'ain')
expect(TagName.where(name: 'shared_alias').pluck(:language_code, :tag_id)) expect(TagName.where(name: 'shared_alias').pluck(:language_code, :tag_id))
.to contain_exactly(['ja', nipponese.id], ['en', tag.id]) .to contain_exactly(['ja', nipponese.id], ['ain', tag.id])
end end
end end
end end
+3
ファイルの表示
@@ -231,11 +231,14 @@ RSpec.describe 'NicoTags', type: :request do
replacement: 'valid' replacement: 'valid'
) )
nico_tag nico_tag
before_counts = [Tag.count, TagName.count, TagVersion.count]
expect { expect {
patch "/tags/nico/#{nico_tag.id}", params: { tags: 'created_first invalid' } patch "/tags/nico/#{nico_tag.id}", params: { tags: 'created_first invalid' }
}.not_to change(TagName, :count) }.not_to change(TagName, :count)
expect([Tag.count, TagName.count, TagVersion.count]).to eq(before_counts)
expect(response).to have_http_status(:unprocessable_entity) expect(response).to have_http_status(:unprocessable_entity)
expect(json.fetch('errors').fetch('tags')).to include( expect(json.fetch('errors').fetch('tags')).to include(
a_string_including('タグ名 “invalid”:', '名前に使用できない文字が含まれてゐます.')) a_string_including('タグ名 “invalid”:', '名前に使用できない文字が含まれてゐます.'))
+44
ファイルの表示
@@ -0,0 +1,44 @@
require 'rails_helper'
RSpec.describe 'Post filter identity across locales', type: :request do
include_context 'Ainu locale'
let!(:activated_locale) { locale }
let!(:tag) { create(:tag, primary_name: 'legacy_union_source') }
let!(:external) { create(:external_tag, name: 'shared') }
let!(:internal_post) { create(:post) }
let!(:external_post) { create(:post) }
let!(:other_post) { create(:post) }
before do
# Legacy data can predate the current prohibition on internal nico: names.
primary_tag_name_for(tag, 'ja').update_columns(name: 'nico:shared')
primary_tag_name_for(tag, 'ain').update!(name: 'シロシ', auto_generated: false)
create(:tag_name, :alias, tag:, name: 'union_alias', language_code: 'ja')
PostTag.create!(post: internal_post, tag:)
PostExternalTag.create!(post: external_post, external_tag: external)
end
['nico:shared', 'union_alias', 'not:nico:shared'].each do |literal|
it "preserves the complete match set for #{ literal }" do
get '/posts', params: { locale: 'ja', tags: literal }
expect(response).to have_http_status(:ok)
original_ids = json.fetch('posts').map { _1.fetch('id') }
expected = literal.start_with?('not:') ? [other_post.id] :
[internal_post.id, external_post.id]
expect(original_ids).to match_array(expected)
get '/posts/tag_filters', params: {
locale: 'ja', target_locale: 'ain-Kana', tags: literal }
expect(response).to have_http_status(:ok)
references = json.fetch('references')
expect(references.first).to include('tag_id' => tag.id, 'external_tag_id' => external.id)
get '/posts', params: {
locale: 'ain-Kana', tags: references.map { _1.fetch('name') }.join(' '),
tag_refs: references.to_json }
expect(response).to have_http_status(:ok)
expect(json.fetch('posts').map { _1.fetch('id') }).to match_array(original_ids)
end
end
end
+16 -8
ファイルの表示
@@ -900,7 +900,7 @@ RSpec.describe 'Posts API', type: :request do
query_count = query_count =
count_sql_queries do count_sql_queries do
get "/posts/#{ post_record.id }", get "/posts/#{ post_record.id }",
headers: { 'X-Transfer-Code' => user.inheritance_code } headers: cookie_headers(user)
end end
expect(response).to have_http_status(:ok) expect(response).to have_http_status(:ok)
@@ -2323,8 +2323,10 @@ RSpec.describe 'Posts API', type: :request do
'prev' => nil 'prev' => nil
) )
expect(latest.fetch('tags')).to include( expect(latest.fetch('tags')).to include(
{ 'name' => 'spec_tag_2', 'type' => 'added' }, { 'name' => 'spec_tag_2', 'restore_name' => 'spec_tag_2',
{ 'name' => 'spec_tag', 'type' => 'removed' } 'external' => false, 'type' => 'added' },
{ 'name' => 'spec_tag', 'restore_name' => 'spec_tag',
'external' => false, 'type' => 'removed' }
) )
expect(latest.fetch('original_created_from')).to eq( expect(latest.fetch('original_created_from')).to eq(
'current' => oc_from.iso8601, 'current' => oc_from.iso8601,
@@ -2351,7 +2353,8 @@ RSpec.describe 'Posts API', type: :request do
'prev' => nil 'prev' => nil
) )
expect(first.fetch('tags')).to include( expect(first.fetch('tags')).to include(
{ 'name' => 'spec_tag', 'type' => 'added' } { 'name' => 'spec_tag', 'restore_name' => 'spec_tag',
'external' => false, 'type' => 'added' }
) )
expect(first.fetch('created_at')).to eq(t_v1.iso8601) expect(first.fetch('created_at')).to eq(t_v1.iso8601)
end end
@@ -2435,7 +2438,9 @@ RSpec.describe 'Posts API', type: :request do
expect(response).to have_http_status(:ok) expect(response).to have_http_status(:ok)
expect(json.fetch('count')).to eq(3) expect(json.fetch('count')).to eq(3)
expect(json.fetch('versions').first.fetch('tags')).to include( expect(json.fetch('versions').first.fetch('tags')).to include(
'name' => primary_tag_name_for(tag2, 'ja').name, 'type' => 'context') 'name' => primary_tag_name_for(tag2, 'ja').name,
'restore_name' => primary_tag_name_for(tag2, 'ja').name,
'external' => false, 'type' => 'context')
end end
it 'filters versions by tag when the current snapshot includes the tag' do it 'filters versions by tag when the current snapshot includes the tag' do
@@ -2449,7 +2454,8 @@ RSpec.describe 'Posts API', type: :request do
expect(versions[0]['post_id']).to eq(post_record.id) expect(versions[0]['post_id']).to eq(post_record.id)
expect(versions[0]['version_no']).to eq(2) expect(versions[0]['version_no']).to eq(2)
expect(versions[0]['tags']).to include( expect(versions[0]['tags']).to include(
{ 'name' => 'spec_tag_2', 'type' => 'added' } { 'name' => 'spec_tag_2', 'restore_name' => 'spec_tag_2',
'external' => false, 'type' => 'added' }
) )
end end
@@ -2467,10 +2473,12 @@ RSpec.describe 'Posts API', type: :request do
first = versions[1] first = versions[1]
expect(latest['tags']).to include( expect(latest['tags']).to include(
{ 'name' => 'spec_tag', 'type' => 'removed' } { 'name' => 'spec_tag', 'restore_name' => 'spec_tag',
'external' => false, 'type' => 'removed' }
) )
expect(first['tags']).to include( expect(first['tags']).to include(
{ 'name' => 'spec_tag', 'type' => 'added' } { 'name' => 'spec_tag', 'restore_name' => 'spec_tag',
'external' => false, 'type' => 'added' }
) )
end end
+155
ファイルの表示
@@ -0,0 +1,155 @@
require 'rails_helper'
RSpec.describe 'Sitemap entries API', type: :request do
include_context 'Ainu locale'
let(:user) { create(:user, :member) }
it 'serves public resource counts with the default locale and page size' do
create(:post)
create(:tag, primary_name: 'sitemap_count_tag')
get '/sitemap_entries'
expect(response).to have_http_status(:ok)
expect(json.fetch('page_size')).to eq(1_000)
counts = json.fetch('resources').to_h { |entry|
[entry.fetch('resource'), entry.fetch('count')]
}
expect(counts).to eq(
'posts' => Post.count, 'tags' => Tag.count, 'deerjikists' => 0,
'materials' => 0, 'descriptions' => 0, 'tag_groups' => 0, 'theatres' => 0)
end
it 'paginates public post IDs in ascending order without exposing post contents' do
stub_const('SitemapEntriesController::PAGE_SIZE', 2)
posts = create_list(:post, 3)
get '/sitemap_entries/posts'
expect(response).to have_http_status(:ok)
expect(json).to eq('ids' => posts.first(2).map(&:id))
get '/sitemap_entries/posts', params: { page: 2 }
expect(response).to have_http_status(:ok)
expect(json).to eq('ids' => [posts.last.id])
get '/sitemap_entries/posts', params: { page: 3 }
expect(response).to have_http_status(:not_found)
end
['0', '-1', 'invalid', '1.5'].each do |page|
it "rejects invalid sitemap page #{ page.inspect }" do
get '/sitemap_entries/posts', params: { page: }
expect(response).to have_http_status(:bad_request)
end
end
it 'rejects unsupported resource names' do
get '/sitemap_entries/users'
expect(response).to have_http_status(:not_found)
end
it 'returns an empty first page for a resource without entries' do
get '/sitemap_entries/posts'
expect(response).to have_http_status(:ok)
expect(json).to eq('ids' => [])
end
it 'rejects unregistered, deprecated and unsupported locales' do
get '/sitemap_entries', params: { locale: 'ain-Kana' }
expect(response).to have_http_status(:not_found)
locale.update!(deprecated_at: Time.current)
['/sitemap_entries', '/sitemap_entries/posts'].each do |path|
get path, params: { locale: 'ain-Kana' }
expect(response).to have_http_status(:not_found)
get path, params: { locale: 'en' }
expect(response).to have_http_status(:not_found)
end
end
it 'only lists live materials and theatres' do
material = Material.create!(url: 'https://example.com/sitemap-material')
discarded_material = Material.create!(url: 'https://example.com/sitemap-discarded')
discarded_material.discard!
theatre = create(:theatre)
create(:theatre).discard!
get '/sitemap_entries/materials'
expect(response).to have_http_status(:ok)
expect(json).to eq('ids' => [material.id])
get '/sitemap_entries/theatres'
expect(response).to have_http_status(:ok)
expect(json).to eq('ids' => [theatre.id])
end
context 'with an active Ainu locale' do
before { locale }
it 'uses tag IDs and only includes deerjikists with linked accounts' do
tagged = create(:tag, primary_name: 'sitemap_deer', category: :deerjikist)
unlinked = create(:tag, primary_name: 'sitemap_unlinked', category: :deerjikist)
Deerjikist.create!(tag: tagged, platform: 'youtube', code: 'sitemap-channel')
Deerjikist.create!(tag: tagged, platform: 'nico', code: 'sitemap-nico')
get '/sitemap_entries/tags', params: { locale: 'ain-Kana' }
expect(response).to have_http_status(:ok)
expect(json.fetch('ids')).to contain_exactly(tagged.id, unlinked.id)
get '/sitemap_entries/deerjikists', params: { locale: 'ain-Kana' }
expect(response).to have_http_status(:ok)
expect(json).to eq('ids' => [tagged.id])
end
it 'lists description IDs for the requested language with a nonempty current version' do
tag = create(:tag, primary_name: 'sitemap_description')
japanese = TagDescriptionOperation.create!(
Locale.nipponese, tag:, body: 'Japanese sitemap body', created_by_user: user)
ainu = TagDescriptionOperation.create!(
locale, tag:, body: 'Ainu sitemap body', created_by_user: user)
unversioned_tag = create(:tag, primary_name: 'sitemap_unversioned')
TagDescription.create!(tag: unversioned_tag, language_code: 'ain',
body: 'No revision', version_no: 1)
empty_tag = create(:tag, primary_name: 'sitemap_empty')
# Legacy empty bodies can exist even though the write service rejects them.
empty = TagDescription.create!(tag: empty_tag, language_code: 'ain',
body: '', version_no: 1)
TagDescriptionVersion.create!(tag_id: empty_tag.id, language_code: 'ain',
body: '', version_no: 1, event_type: 'create',
created_by_user_id: user.id, created_at: Time.current)
get '/sitemap_entries/descriptions', params: { locale: 'ain-Kana' }
expect(response).to have_http_status(:ok)
expect(json).to eq('ids' => [ainu.id])
expect(json.fetch('ids')).not_to include(japanese.id, empty.id)
get '/sitemap_entries/descriptions', params: { locale: 'ja' }
expect(response).to have_http_status(:ok)
expect(json).to eq('ids' => [japanese.id])
get '/sitemap_entries', params: { locale: 'ain-Kana' }
expect(response).to have_http_status(:ok)
descriptions = json.fetch('resources').find { |entry|
entry.fetch('resource') == 'descriptions'
}
expect(descriptions.fetch('count')).to eq(1)
end
it 'excludes groups without a revision matching the current version' do
group = TagGroupOperation.create!(
Locale.nipponese, name: 'Sitemap group', description: 'Group description',
members: [], placements: [], actor: user)
TagGroup.create!(name: 'Unversioned sitemap group', version_no: 1)
get '/sitemap_entries/tag_groups', params: { locale: 'ain-Kana' }
expect(response).to have_http_status(:ok)
expect(json).to eq('ids' => [group.id])
end
end
end
+1 -1
ファイルの表示
@@ -4,7 +4,7 @@ RSpec.describe 'Tag description Wiki facade writes', type: :request do
include WikiRecords include WikiRecords
let!(:user) { create_member_user! } let!(:user) { create_member_user! }
let!(:tag) { create(:tag, primary_name: 'description_write') } let!(:tag) { create(:tag, primary_name: 'description_write') }
let(:headers) { { 'X-Transfer-Code' => user.inheritance_code } } let(:headers) { cookie_headers(user) }
def stored_state def stored_state
[Tag, TagName, TagVersion, TagDescription, TagDescriptionVersion].map do |model| [Tag, TagName, TagVersion, TagDescription, TagDescriptionVersion].map do |model|
+3 -3
ファイルの表示
@@ -7,7 +7,7 @@ RSpec.describe 'Tag description native API', type: :request do
let(:editor) { create(:user, :member, name: 'description editor') } let(:editor) { create(:user, :member, name: 'description editor') }
let(:tag_name) { create(:tag_name, name: 'native_description') } let(:tag_name) { create(:tag_name, name: 'native_description') }
let(:tag) { create(:tag, primary_tag_name: tag_name) } let(:tag) { create(:tag, primary_tag_name: tag_name) }
let(:headers) { { 'X-Transfer-Code' => editor.inheritance_code } } let(:headers) { cookie_headers(editor) }
let(:endpoint) { "/tags/#{ tag.id }/description" } let(:endpoint) { "/tags/#{ tag.id }/description" }
def stored_state def stored_state
@@ -285,7 +285,7 @@ RSpec.describe 'Tag description native API', type: :request do
guest = create(:user, :guest) guest = create(:user, :guest)
before_state = stored_state before_state = stored_state
native_write(body: 'forbidden', native_write(body: 'forbidden',
request_headers: { 'X-Transfer-Code' => guest.inheritance_code }) request_headers: cookie_headers(guest))
expect(response).to have_http_status(:forbidden) expect(response).to have_http_status(:forbidden)
expect(stored_state).to eq(before_state) expect(stored_state).to eq(before_state)
end end
@@ -294,7 +294,7 @@ RSpec.describe 'Tag description native API', type: :request do
banned = create(:user, :member, :banned) banned = create(:user, :member, :banned)
before_state = stored_state before_state = stored_state
native_write(body: 'forbidden', native_write(body: 'forbidden',
request_headers: { 'X-Transfer-Code' => banned.inheritance_code }) request_headers: cookie_headers(banned))
expect(response).to have_http_status(:forbidden) expect(response).to have_http_status(:forbidden)
expect(stored_state).to eq(before_state) expect(stored_state).to eq(before_state)
end end
+17 -16
ファイルの表示
@@ -7,7 +7,7 @@ RSpec.describe 'Tag group locale contracts', type: :request do
let!(:user) { create_member_user! } let!(:user) { create_member_user! }
let!(:tag) { create(:tag, primary_name: 'japanese_member') } let!(:tag) { create(:tag, primary_name: 'japanese_member') }
let!(:ainu_name) { add_ainu_name(tag, 'ainu_member') } let!(:ainu_name) { add_ainu_name(tag, 'ainu_member') }
let(:headers) { { 'X-Transfer-Code' => user.inheritance_code } } let(:headers) { cookie_headers(user) }
let(:members) { [{ 'tag_id' => tag.id }] } let(:members) { [{ 'tag_id' => tag.id }] }
let(:placements) do let(:placements) do
[{ 'section_name' => 'section', 'description' => nil, 'children' => [ [{ 'section_name' => 'section', 'description' => nil, 'children' => [
@@ -22,7 +22,7 @@ RSpec.describe 'Tag group locale contracts', type: :request do
it 'uses selected-language names in native collection and nested placements' do it 'uses selected-language names in native collection and nested placements' do
["/tag_groups/#{ group.id }", '/tag_groups'].each do |path| ["/tag_groups/#{ group.id }", '/tag_groups'].each do |path|
get path, params: { locale: 'ain' } get path, params: { locale: 'ain-Kana' }
expect(response).to have_http_status(:ok) expect(response).to have_http_status(:ok)
representation = json.is_a?(Array) ? json.sole : json representation = json.is_a?(Array) ? json.sole : json
expect(representation.fetch('members').sole.fetch('tag').fetch('name')) expect(representation.fetch('members').sole.fetch('tag').fetch('name'))
@@ -37,14 +37,14 @@ RSpec.describe 'Tag group locale contracts', type: :request do
it 'uses snapshot names recursively for history and current names for the latest view' do it 'uses snapshot names recursively for history and current names for the latest view' do
version = group.versions.sole version = group.versions.sole
ainu_name.update!(name: 'renamed_ainu') ainu_name.update!(name: 'renamed_ainu')
get "/wiki/#{ group.id }", params: { locale: 'ain' } get "/wiki/#{ group.id }", params: { locale: 'ain-Kana' }
expect(response).to have_http_status(:ok) expect(response).to have_http_status(:ok)
expect(json.fetch('body')).to include('renamed_ainu') expect(json.fetch('body')).to include('renamed_ainu')
get "/wiki/#{ group.id }", params: { locale: 'ain', version: version.id } get "/wiki/#{ group.id }", params: { locale: 'ain-Kana', version: version.id }
expect(response).to have_http_status(:ok) expect(response).to have_http_status(:ok)
expect(json.fetch('body').scan('ainu_member').length).to eq(2) expect(json.fetch('body').scan('ainu_member').length).to eq(2)
expect(json.fetch('body')).not_to include('renamed_ainu', 'japanese_member') expect(json.fetch('body')).not_to include('renamed_ainu', 'japanese_member')
get "/wiki/#{ group.id }/diff", params: { locale: 'ain', from: version.id } get "/wiki/#{ group.id }/diff", params: { locale: 'ain-Kana', from: version.id }
expect(response).to have_http_status(:ok) expect(response).to have_http_status(:ok)
expect(json.fetch('diff').map { _1.fetch('type') }.uniq).to eq(['context']) expect(json.fetch('diff').map { _1.fetch('type') }.uniq).to eq(['context'])
end end
@@ -52,23 +52,24 @@ RSpec.describe 'Tag group locale contracts', type: :request do
['タググループ:', 'tag_group:', 'TAG_GROUP:'].each do |prefix| ['タググループ:', 'tag_group:', 'TAG_GROUP:'].each do |prefix|
it "round trips #{ prefix } through create, lookup, search and update" do it "round trips #{ prefix } through create, lookup, search and update" do
post '/wiki', params: { post '/wiki', params: {
locale: 'ain', title: "#{ prefix }parent", body: '- tag_group:locale group' }, locale: 'ain-Kana', title: "#{ prefix }parent", body: '- tag_group:locale group' },
headers: headers headers: headers
expect(response).to have_http_status(:created) expect(response).to have_http_status(:created)
id = json.fetch('id') id = json.fetch('id')
expect(json.fetch('body')).to include('タググループ:locale group') expect(json.fetch('body')).to include('タググループ:locale group')
path = "/wiki/title/#{ CGI.escapeURIComponent("#{ prefix }parent") }" path = "/wiki/title/#{ CGI.escapeURIComponent("#{ prefix }parent") }"
get path, params: { locale: 'ain' } get path, params: { locale: 'ain-Kana' }
expect(response).to have_http_status(:ok) expect(response).to have_http_status(:ok)
expect(json.fetch('id')).to eq(id) expect(json.fetch('id')).to eq(id)
get "#{ path }/exists", params: { locale: 'ain' } get "#{ path }/exists", params: { locale: 'ain-Kana' }
expect(response).to have_http_status(:no_content) expect(response).to have_http_status(:no_content)
get '/wiki/search', params: { locale: 'ain', title: "#{ prefix }parent" } get '/wiki/search', params: { locale: 'ain-Kana', title: "#{ prefix }parent" }
expect(json.map { _1.fetch('id') }).to eq([id]) expect(json.map { _1.fetch('id') }).to eq([id])
put "/wiki/#{ id }", params: { put "/wiki/#{ id }", params: {
locale: 'ain', title: "#{ prefix }renamed", body: '- ainu_member' }, headers: headers locale: 'ain-Kana', title: "#{ prefix }renamed", body: '- ainu_member' },
headers: headers
expect(response).to have_http_status(:ok) expect(response).to have_http_status(:ok)
get "/tag_groups/#{ id }", params: { locale: 'ain' } get "/tag_groups/#{ id }", params: { locale: 'ain-Kana' }
expect(json.fetch('members').sole.fetch('tag').fetch('name')).to eq('ainu_member') expect(json.fetch('members').sole.fetch('tag').fetch('name')).to eq('ainu_member')
end end
end end
@@ -81,7 +82,7 @@ RSpec.describe 'Tag group locale contracts', type: :request do
end.to raise_error(TagGroupMarkdownParser::InvalidDocument) end.to raise_error(TagGroupMarkdownParser::InvalidDocument)
before_state = group_state before_state = group_state
put "/wiki/#{ group.id }", params: { put "/wiki/#{ group.id }", params: {
locale: 'ain', title: 'タググループ:partial rename', body: '- japanese_member' }, locale: 'ain-Kana', title: 'タググループ:partial rename', body: '- japanese_member' },
headers: headers headers: headers
expect(response).to have_http_status(:unprocessable_entity) expect(response).to have_http_status(:unprocessable_entity)
expect(group_state).to eq(before_state) expect(group_state).to eq(before_state)
@@ -89,15 +90,15 @@ RSpec.describe 'Tag group locale contracts', type: :request do
it 'passes locale into native validation and facade compatibility checks' do it 'passes locale into native validation and facade compatibility checks' do
post '/tag_groups', params: { post '/tag_groups', params: {
locale: 'ain', name: 'native', description: nil, members:, placements: }, locale: 'ain-Kana', name: 'native', description: nil, members:, placements: },
headers:, as: :json headers:, as: :json
expect(response).to have_http_status(:created) expect(response).to have_http_status(:created)
id = json.fetch('id') id = json.fetch('id')
expect(json.fetch('members').sole.fetch('tag').fetch('name')).to eq('ainu_member') expect(json.fetch('members').sole.fetch('tag').fetch('name')).to eq('ainu_member')
get "/wiki/#{ id }", params: { locale: 'ain' } get "/wiki/#{ id }", params: { locale: 'ain-Kana' }
expect(response).to have_http_status(:ok) expect(response).to have_http_status(:ok)
body = json.fetch('body') body = json.fetch('body')
put "/wiki/#{ id }", params: { locale: 'ain', title: 'tag_group:native', body: }, put "/wiki/#{ id }", params: { locale: 'ain-Kana', title: 'tag_group:native', body: },
headers: headers headers: headers
expect(response).to have_http_status(:ok) expect(response).to have_http_status(:ok)
expect(TagGroup.find(id).version_no).to eq(1) expect(TagGroup.find(id).version_no).to eq(1)
@@ -115,7 +116,7 @@ RSpec.describe 'Tag group locale contracts', type: :request do
representation = TagGroupRepr.historical(ainu_locale, version) representation = TagGroupRepr.historical(ainu_locale, version)
expect(representation.fetch('members').sole.fetch('tag').fetch('name')).to be_nil expect(representation.fetch('members').sole.fetch('tag').fetch('name')).to be_nil
expect(representation.fetch('placements').sole.fetch('tag').fetch('name')).to be_nil expect(representation.fetch('placements').sole.fetch('tag').fetch('name')).to be_nil
get "/wiki/#{ old_group.id }", params: { locale: 'ain', version: version.id } get "/wiki/#{ old_group.id }", params: { locale: 'ain-Kana', version: version.id }
expect(response).to have_http_status(:unprocessable_entity) expect(response).to have_http_status(:unprocessable_entity)
expect(response.body).not_to include('historical_japanese', 'current_ainu') expect(response.body).not_to include('historical_japanese', 'current_ainu')
end end
+1 -1
ファイルの表示
@@ -37,7 +37,7 @@ RSpec.describe 'Tag group Markdown round trips through the Wiki facade', type: :
markdown.lines.map(&:chomp).reject(&:empty?)) markdown.lines.map(&:chomp).reject(&:empty?))
post '/wiki', params: { title: 'タググループ:再解析', body: }, post '/wiki', params: { title: 'タググループ:再解析', body: },
headers: { 'X-Transfer-Code' => user.inheritance_code } headers: cookie_headers(user)
expect(response).to have_http_status(:created) expect(response).to have_http_status(:created)
parsed = group_rows('tag_groups', name: '再解析').sole parsed = group_rows('tag_groups', name: '再解析').sole
expect_group_snapshot(parsed.fetch('id'), number: 1, placements:, members:, description:) expect_group_snapshot(parsed.fetch('id'), number: 1, placements:, members:, description:)
+4 -4
ファイルの表示
@@ -7,7 +7,7 @@ RSpec.describe 'Tag group Wiki facade writes', type: :request do
let!(:user) { create_member_user! } let!(:user) { create_member_user! }
let!(:tag) { group_tag('洗澡鹿') } let!(:tag) { group_tag('洗澡鹿') }
let!(:other_tag) { group_tag('伊地知ニジカ') } let!(:other_tag) { group_tag('伊地知ニジカ') }
let(:headers) { { 'X-Transfer-Code' => user.inheritance_code } } let(:headers) { cookie_headers(user) }
let(:members) { [{ 'tag_id' => tag.id, 'version_no' => tag.version_no }] } let(:members) { [{ 'tag_id' => tag.id, 'version_no' => tag.version_no }] }
let(:placements) { [{ 'tag_id' => tag.id, 'children' => [] }] } let(:placements) { [{ 'tag_id' => tag.id, 'children' => [] }] }
@@ -148,7 +148,7 @@ RSpec.describe 'Tag group Wiki facade writes', type: :request do
body = group_markdown(page) body = group_markdown(page)
travel_to(1.hour.from_now) do travel_to(1.hour.from_now) do
expect_noop_group_update(group_id: page.id) do expect_noop_group_update(group_id: page.id) do
update_group(body:, request_headers: { 'X-Transfer-Code' => editor.inheritance_code }) update_group(body:, request_headers: cookie_headers(editor))
end end
end end
end end
@@ -212,7 +212,7 @@ RSpec.describe 'Tag group Wiki facade writes', type: :request do
it 'records the editor and update time' do it 'records the editor and update time' do
editor = create(:user, :member, name: '編輯者') editor = create(:user, :member, name: '編輯者')
travel_to(Time.current + 1.hour) do travel_to(Time.current + 1.hour) do
update_group(request_headers: { 'X-Transfer-Code' => editor.inheritance_code }) update_group(request_headers: cookie_headers(editor))
expect(response).to have_http_status(:ok) expect(response).to have_http_status(:ok)
current = group_rows('tag_groups', id: page.id).sole current = group_rows('tag_groups', id: page.id).sole
version = group_version(page.id, 2) version = group_version(page.id, 2)
@@ -271,7 +271,7 @@ RSpec.describe 'Tag group Wiki facade writes', type: :request do
when :guest then create(:user, :guest) when :guest then create(:user, :guest)
when :banned then create(:user, :member, :banned) when :banned then create(:user, :member, :banned)
end end
request_headers = actor ? { 'X-Transfer-Code' => actor.inheritance_code } : { } request_headers = actor ? cookie_headers(actor) : { }
before_state = group_state before_state = group_state
write_group(operation, request_headers) write_group(operation, request_headers)
expect(response).to have_http_status(role ? :forbidden : :unauthorized) expect(response).to have_http_status(role ? :forbidden : :unauthorized)
+2 -2
ファイルの表示
@@ -8,7 +8,7 @@ RSpec.describe 'Tag group write integrity', type: :request do
let!(:user) { create_member_user! } let!(:user) { create_member_user! }
let!(:tag) { group_tag('洗澡鹿') } let!(:tag) { group_tag('洗澡鹿') }
let!(:other_tag) { group_tag('伊地知ニジカ') } let!(:other_tag) { group_tag('伊地知ニジカ') }
let(:headers) { { 'X-Transfer-Code' => user.inheritance_code } } let(:headers) { cookie_headers(user) }
let(:members) { [{ 'tag_id' => tag.id, 'version_no' => tag.version_no }] } let(:members) { [{ 'tag_id' => tag.id, 'version_no' => tag.version_no }] }
let(:placements) { [{ 'tag_id' => tag.id, 'children' => [] }] } let(:placements) { [{ 'tag_id' => tag.id, 'children' => [] }] }
@@ -141,7 +141,7 @@ RSpec.describe 'Tag group write integrity', type: :request do
end end
let!(:id) { seed_group(name: '更新', members:, placements:) } let!(:id) { seed_group(name: '更新', members:, placements:) }
let!(:editor) { create(:user, :member) } let!(:editor) { create(:user, :member) }
let(:headers) { { 'X-Transfer-Code' => editor.inheritance_code } } let(:headers) { cookie_headers(editor) }
it 'keeps all membership and placement audit fields on a name-only edit' do it 'keeps all membership and placement audit fields on a name-only edit' do
before_rows = group_state.slice('tag_group_tags', 'tag_group_groups', 'tag_group_placements') before_rows = group_state.slice('tag_group_tags', 'tag_group_groups', 'tag_group_placements')
+4 -4
ファイルの表示
@@ -47,7 +47,7 @@ RSpec.describe 'Tag group native API', type: :request do
when :guest then create(:user, :guest) when :guest then create(:user, :guest)
when :banned then create(:user, :member, :banned) when :banned then create(:user, :member, :banned)
end end
headers = actor ? { 'X-Transfer-Code' => actor.inheritance_code } : { } headers = actor ? cookie_headers(actor) : { }
before_state = group_state before_state = group_state
if operation == :create if operation == :create
post '/tag_groups', params: { }, headers:, as: :json post '/tag_groups', params: { }, headers:, as: :json
@@ -61,7 +61,7 @@ RSpec.describe 'Tag group native API', type: :request do
it "rejects a banned IP for native #{ operation }" do it "rejects a banned IP for native #{ operation }" do
IpAddress.create!(ip_address: IPAddr.new('127.0.0.1').hton, banned_at: Time.current) IpAddress.create!(ip_address: IPAddr.new('127.0.0.1').hton, banned_at: Time.current)
headers = { 'X-Transfer-Code' => user.inheritance_code } headers = cookie_headers(user)
before_state = group_state before_state = group_state
if operation == :create if operation == :create
post '/tag_groups', params: { }, headers:, as: :json post '/tag_groups', params: { }, headers:, as: :json
@@ -77,7 +77,7 @@ RSpec.describe 'Tag group native API', type: :request do
let!(:tag) { group_tag('掲載タグ') } let!(:tag) { group_tag('掲載タグ') }
let!(:unplaced) { group_tag('未掲載タグ') } let!(:unplaced) { group_tag('未掲載タグ') }
let!(:child_id) { seed_group(name: '子', description: '子の説明') } let!(:child_id) { seed_group(name: '子', description: '子の説明') }
let(:headers) { { 'X-Transfer-Code' => user.inheritance_code } } let(:headers) { cookie_headers(user) }
let(:members) do let(:members) do
[{ 'tag_id' => tag.id }, { 'tag_group_id' => child_id }, { 'tag_id' => unplaced.id }] [{ 'tag_id' => tag.id }, { 'tag_group_id' => child_id }, { 'tag_id' => unplaced.id }]
end end
@@ -300,7 +300,7 @@ RSpec.describe 'Tag group native API', type: :request do
travel_to(1.hour.from_now) do travel_to(1.hour.from_now) do
expect_noop_group_update(group_id:) do expect_noop_group_update(group_id:) do
put "/tag_groups/#{ group_id }", params: synchronised_payload, put "/tag_groups/#{ group_id }", params: synchronised_payload,
headers: { 'X-Transfer-Code' => editor.inheritance_code }, as: :json headers: cookie_headers(editor), as: :json
end end
end end
end end
+1 -1
ファイルの表示
@@ -3,7 +3,7 @@ require 'rails_helper'
RSpec.describe 'Tag resource ID allocation', type: :request do RSpec.describe 'Tag resource ID allocation', type: :request do
include WikiRecords include WikiRecords
let!(:user) { create_member_user! } let!(:user) { create_member_user! }
let(:headers) { { 'X-Transfer-Code' => user.inheritance_code } } let(:headers) { cookie_headers(user) }
def create_description name def create_description name
tag = create(:tag, primary_name: name) tag = create(:tag, primary_name: name)
+113 -4
ファイルの表示
@@ -10,7 +10,31 @@ RSpec.describe 'Users', type: :request do
end end
def auth_headers(user) def auth_headers(user)
{ 'X-Transfer-Code' => user.inheritance_code } cookie_headers(user)
end
describe 'GET /users/session' do
it 'does not accept the retired transfer-code header' do
user = create(:user)
get '/users/session', headers: { 'X-Transfer-Code' => user.inheritance_code }
expect(response).to have_http_status(:unauthorized)
end
it 'requires authentication' do
get '/users/session'
expect(response).to have_http_status(:unauthorized)
end
it 'returns public user fields without recording a visit' do
user = create(:user, name: 'SSR user')
expect {
get '/users/session', headers: auth_headers(user)
}.not_to change(UserIp, :count)
expect(response).to have_http_status(:ok)
expect(json).to eq('id' => user.id, 'name' => user.name, 'role' => user.role)
expect(response.body).not_to include(user.inheritance_code)
end
end end
describe 'POST /users' do describe 'POST /users' do
@@ -233,17 +257,102 @@ RSpec.describe 'Users', type: :request do
end end
end end
describe 'Cookie sessions with CSRF enabled', csrf: true do
def csrf_headers
get '/users/csrf'
expect(response).to have_http_status(:ok)
{ 'X-CSRF-Token' => json.fetch('token'), 'Origin' => 'http://www.example.com' }
end
def establish_session(user)
post '/users/verify', params: { code: user.inheritance_code }, headers: csrf_headers
expect(response).to have_http_status(:ok)
expect(json.fetch('valid')).to be(true)
end
it 'rejects a write without a token before creating an account' do
expect { post '/users' }.not_to change(User, :count)
expect(response).to have_http_status(:forbidden)
expect(response.headers['X-CSRF-Invalid']).to eq('1')
end
it 'uses the encrypted persistent Cookie for CSR and SSR session reads' do
user = create(:user)
establish_session(user)
cookie = Array(response.headers['set-cookie']).join
expect(cookie).to include('httponly', 'samesite=lax')
expect(cookie).not_to include(user.inheritance_code)
expires = Time.httpdate(cookie.match(/expires=([^;]+)/i)[1])
expect(expires).to be > 19.years.from_now
get '/users/session'
expect(json.fetch('id')).to eq(user.id)
expect(response.body).not_to include(user.inheritance_code)
get '/users/me'
expect(json.fetch('inheritance_code')).to eq(user.inheritance_code)
expect(response.headers['Cache-Control']).to include('no-store')
end
it 'rejects a cross-origin write with an otherwise valid token' do
headers = csrf_headers.merge('Origin' => 'https://other.invalid')
expect { post '/users', headers: }.not_to change(User, :count)
expect(response).to have_http_status(:forbidden)
end
it 'does not replace an existing Cookie identity during recovery' do
current = create(:user)
old = create(:user)
establish_session(current)
post '/users/verify', params: { code: old.inheritance_code, restore: true },
headers: csrf_headers
expect(json.fetch('user').fetch('id')).to eq(current.id)
get '/users/session'
expect(json.fetch('id')).to eq(current.id)
end
it 'allows explicit transfer and refreshes the active session' do
first = create(:user)
second = create(:user)
establish_session(first)
establish_session(second)
get '/users/session'
expect(json.fetch('id')).to eq(second.id)
end
it 'keeps the renewed session and rejects the previous credential' do
user = create(:user)
establish_session(user)
old_cookie = Array(response.headers['set-cookie']).first.split(';').first
old_code = user.inheritance_code
post '/users/code/renew', headers: csrf_headers
expect(response).to have_http_status(:ok)
expect(json.fetch('code')).not_to eq(old_code)
get '/users/session'
expect(json.fetch('id')).to eq(user.id)
other_browser = ActionDispatch::Integration::Session.new(Rails.application)
other_browser.get('/users/session', headers: { 'Cookie' => old_cookie })
expect(other_browser.response).to have_http_status(:unauthorized)
end
it 'clears browser authentication on session deletion' do
establish_session(create(:user))
delete '/users/session', headers: csrf_headers
expect(response).to have_http_status(:no_content)
get '/users/session'
expect(response).to have_http_status(:unauthorized)
end
end
describe 'GET /users/me' do describe 'GET /users/me' do
it 'returns 404 when code not found' do it 'requires a session even when a code is supplied in the query' do
get '/users/me', params: { code: 'nope' } get '/users/me', params: { code: 'nope' }
expect(response).to have_http_status(:not_found) expect(response).to have_http_status(:unauthorized)
end end
it 'returns user slice when found' do it 'returns user slice when found' do
user = create(:user, inheritance_code: SecureRandom.uuid, name: 'me', role: 'guest') user = create(:user, inheritance_code: SecureRandom.uuid, name: 'me', role: 'guest')
get '/users/me', params: { code: user.inheritance_code } get '/users/me', headers: cookie_headers(user)
expect(response).to have_http_status(:ok) expect(response).to have_http_status(:ok)
expect(json['id']).to eq(user.id) expect(json['id']).to eq(user.id)
+1 -1
ファイルの表示
@@ -5,7 +5,7 @@ RSpec.describe 'Wiki conflict handling', type: :request do
let!(:user) { create_member_user! } let!(:user) { create_member_user! }
def auth_headers user def auth_headers user
{ 'X-Transfer-Code' => user.inheritance_code } cookie_headers(user)
end end
it 'returns 409 when base_revision_id is stale' do it 'returns 409 when base_revision_id is stale' do
+2 -2
ファイルの表示
@@ -5,7 +5,7 @@ RSpec.describe 'Wiki creation freeze', type: :request do
include WikiRecords include WikiRecords
let!(:user) { create_member_user! } let!(:user) { create_member_user! }
let(:headers) { { 'X-Transfer-Code' => user.inheritance_code } } let(:headers) { cookie_headers(user) }
context 'existing description write protection' do context 'existing description write protection' do
@@ -18,7 +18,7 @@ RSpec.describe 'Wiki creation freeze', type: :request do
it "rejects #{ role || 'anonymous' } without changing either history" do it "rejects #{ role || 'anonymous' } without changing either history" do
actor = role && create(:user, role == :banned ? :member : role) actor = role && create(:user, role == :banned ? :member : role)
actor.update!(banned_at: Time.current) if role == :banned actor.update!(banned_at: Time.current) if role == :banned
request_headers = actor ? { 'X-Transfer-Code' => actor.inheritance_code } : { } request_headers = actor ? cookie_headers(actor) : { }
before_state = group_state before_state = group_state
put "/wiki/#{ page.id }", params: { title: tag.name('ja'), body: 'after' }, put "/wiki/#{ page.id }", params: { title: tag.name('ja'), body: 'after' },
headers: request_headers headers: request_headers
+42 -24
ファイルの表示
@@ -8,7 +8,7 @@ RSpec.describe 'Wiki and native API locale contracts', type: :request do
let!(:user) { create_member_user! } let!(:user) { create_member_user! }
let!(:tag) { create(:tag, primary_name: 'japanese_title') } let!(:tag) { create(:tag, primary_name: 'japanese_title') }
let!(:ainu_name) { add_ainu_name(tag, 'ainu_title') } let!(:ainu_name) { add_ainu_name(tag, 'ainu_title') }
let(:headers) { { 'X-Transfer-Code' => user.inheritance_code } } let(:headers) { cookie_headers(user) }
let!(:japanese) do let!(:japanese) do
TagDescriptionOperation.create!( TagDescriptionOperation.create!(
Locale.nipponese, tag:, body: 'Japanese body', created_by_user: user) Locale.nipponese, tag:, body: 'Japanese body', created_by_user: user)
@@ -20,12 +20,12 @@ RSpec.describe 'Wiki and native API locale contracts', type: :request do
['/wiki', '/wiki/search'].each do |path| ['/wiki', '/wiki/search'].each do |path|
it "filters #{ path } by the selected language" do it "filters #{ path } by the selected language" do
get path, params: { locale: 'ain' } get path, params: { locale: 'ain-Kana' }
expect(response).to have_http_status(:ok) expect(response).to have_http_status(:ok)
expect(json.map { _1.fetch('id') }).to eq([ainu.id]) expect(json.map { _1.fetch('id') }).to eq([ainu.id])
expect(json.sole).to include('title' => 'ainu_title', 'body' => 'Ainu body') expect(json.sole).to include('title' => 'ainu_title', 'body' => 'Ainu body')
get path, params: { locale: 'ain', title: 'japanese' } get path, params: { locale: 'ain-Kana', title: 'japanese' }
expect(response).to have_http_status(:ok) expect(response).to have_http_status(:ok)
expect(json).to eq([]) expect(json).to eq([])
end end
@@ -39,32 +39,50 @@ RSpec.describe 'Wiki and native API locale contracts', type: :request do
it 'resolves titles and aliases only in the selected language in both APIs' do it 'resolves titles and aliases only in the selected language in both APIs' do
['/wiki/title/japanese_title', '/tags/name/japanese_title/description'].each do |path| ['/wiki/title/japanese_title', '/tags/name/japanese_title/description'].each do |path|
get path, params: { locale: 'ain' } get path, params: { locale: 'ain-Kana' }
expect(response).to have_http_status(:not_found) expect(response).to have_http_status(:not_found)
end end
['/wiki/title/ainu_title', '/tags/name/ainu_title/description'].each do |path| ['/wiki/title/ainu_title', '/tags/name/ainu_title/description'].each do |path|
get path, params: { locale: 'ain' } get path, params: { locale: 'ain-Kana' }
expect(response).to have_http_status(:ok) expect(response).to have_http_status(:ok)
expect(json.fetch('body')).to eq('Ainu body') expect(json.fetch('body')).to eq('Ainu body')
end end
get '/wiki/title/ainu_title/exists', params: { locale: 'ain' } get '/wiki/title/ainu_title/exists', params: { locale: 'ain-Kana' }
expect(response).to have_http_status(:no_content) expect(response).to have_http_status(:no_content)
get '/wiki/title/japanese_title/exists', params: { locale: 'ain' } get '/wiki/title/japanese_title/exists', params: { locale: 'ain-Kana' }
expect(response).to have_http_status(:not_found) expect(response).to have_http_status(:not_found)
end end
it 'does not disclose another language through ID, existence, diff or history' do it 'resolves a foreign description ID to only the requested language body' do
get "/wiki/#{ japanese.id }", params: { locale: 'ain-Kana' }
expect(response).to have_http_status(:ok)
expect(json).to include('id' => ainu.id, 'body' => 'Ainu body')
expect(response.body).not_to include('Japanese body')
end
it 'does not reuse foreign revision IDs when resolving a description' do
get "/wiki/#{ japanese.id }", params: {
locale: 'ain-Kana', version: current_description_version(japanese).id }
expect(response).to have_http_status(:not_found)
expect(response.body).not_to include('Japanese body')
end
it 'keeps existence and diff scoped to the specified language and description ID' do
revision = current_description_version(japanese) revision = current_description_version(japanese)
["/wiki/#{ japanese.id }", "/wiki/#{ japanese.id }/exists", ["/wiki/#{ japanese.id }/exists", "/wiki/#{ japanese.id }/diff"].each do |path|
"/wiki/#{ japanese.id }/diff"].each do |path| get path, params: { locale: 'ain-Kana', from: revision.id }
get path, params: { locale: 'ain', from: revision.id }
expect(response).to have_http_status(:not_found) expect(response).to have_http_status(:not_found)
expect(response.body).not_to include('Japanese body') expect(response.body).not_to include('Japanese body')
end end
get '/wiki/changes', params: { locale: 'ain', id: japanese.id } get "/wiki/#{ ainu.id }/exists", params: { locale: 'ain-Kana' }
expect(response).to have_http_status(:no_content)
end
it 'keeps history scoped to the requested language without foreign revisions' do
get '/wiki/changes', params: { locale: 'ain-Kana', id: japanese.id }
expect(response).to have_http_status(:ok) expect(response).to have_http_status(:ok)
expect(json).to eq([]) expect(json).to eq([])
get '/wiki/changes', params: { locale: 'ain' } get '/wiki/changes', params: { locale: 'ain-Kana' }
expect(json.map { _1.fetch('revision_id') }) expect(json.map { _1.fetch('revision_id') })
.to eq([current_description_version(ainu).id]) .to eq([current_description_version(ainu).id])
end end
@@ -73,7 +91,7 @@ RSpec.describe 'Wiki and native API locale contracts', type: :request do
snapshot = [japanese.attributes, ainu.attributes, ainu_name.attributes] snapshot = [japanese.attributes, ainu.attributes, ainu_name.attributes]
expect do expect do
put "/wiki/#{ japanese.id }", params: { put "/wiki/#{ japanese.id }", params: {
locale: 'ain', title: 'renamed', body: 'wrong target' }, headers: headers locale: 'ain-Kana', title: 'renamed', body: 'wrong target' }, headers: headers
end.not_to change(TagDescriptionVersion, :count) end.not_to change(TagDescriptionVersion, :count)
expect(response).to have_http_status(:not_found) expect(response).to have_http_status(:not_found)
expect([japanese.reload.attributes, ainu.reload.attributes, ainu_name.reload.attributes]) expect([japanese.reload.attributes, ainu.reload.attributes, ainu_name.reload.attributes])
@@ -89,10 +107,10 @@ RSpec.describe 'Wiki and native API locale contracts', type: :request do
snapshot = original.attributes snapshot = original.attributes
expect do expect do
if api == :wiki if api == :wiki
post '/wiki', params: { locale: 'ain', title: 'other_ainu', body: 'created' }, post '/wiki', params: { locale: 'ain-Kana', title: 'other_ainu', body: 'created' },
headers: headers headers: headers
else else
put "/tags/#{ other.id }/description", params: { locale: 'ain', body: 'created' }, put "/tags/#{ other.id }/description", params: { locale: 'ain-Kana', body: 'created' },
headers: headers headers: headers
end end
end.to change(TagDescription, :count).by(1) end.to change(TagDescription, :count).by(1)
@@ -108,20 +126,20 @@ RSpec.describe 'Wiki and native API locale contracts', type: :request do
it 'shares selected-language edits and revision identity across both APIs' do it 'shares selected-language edits and revision identity across both APIs' do
snapshot = japanese.attributes snapshot = japanese.attributes
first = current_description_version(ainu) first = current_description_version(ainu)
put "/tags/#{ tag.id }/description", params: { locale: 'ain', body: 'native edit' }, put "/tags/#{ tag.id }/description", params: { locale: 'ain-Kana', body: 'native edit' },
headers: headers headers: headers
expect(response).to have_http_status(:ok) expect(response).to have_http_status(:ok)
second = current_description_version(ainu) second = current_description_version(ainu)
expect(second).to have_attributes(language_code: 'ain', version_no: 2) expect(second).to have_attributes(language_code: 'ain', version_no: 2)
expect(second.id).not_to eq(first.id) expect(second.id).not_to eq(first.id)
get "/wiki/#{ ainu.id }", params: { locale: 'ain' } get "/wiki/#{ ainu.id }", params: { locale: 'ain-Kana' }
expect(json).to include('body' => 'native edit', 'revision_id' => second.id) expect(json).to include('body' => 'native edit', 'revision_id' => second.id)
put "/wiki/#{ ainu.id }", params: { put "/wiki/#{ ainu.id }", params: {
locale: 'ain', title: 'ainu_title', body: 'facade edit', locale: 'ain-Kana', title: 'ainu_title', body: 'facade edit',
base_revision_id: second.id }, headers: headers base_revision_id: second.id }, headers: headers
expect(response).to have_http_status(:ok) expect(response).to have_http_status(:ok)
get "/tags/#{ tag.id }/description", params: { locale: 'ain' } get "/tags/#{ tag.id }/description", params: { locale: 'ain-Kana' }
expect(json).to include('body' => 'facade edit', 'language_code' => 'ain', 'version_no' => 3) expect(json).to include('body' => 'facade edit', 'language_code' => 'ain', 'version_no' => 3)
expect(japanese.reload.attributes).to eq(snapshot) expect(japanese.reload.attributes).to eq(snapshot)
end end
@@ -133,7 +151,7 @@ RSpec.describe 'Wiki and native API locale contracts', type: :request do
snapshot = [ainu.reload.attributes, ainu_name.attributes, japanese.attributes] snapshot = [ainu.reload.attributes, ainu_name.attributes, japanese.attributes]
expect do expect do
put "/wiki/#{ ainu.id }", params: { put "/wiki/#{ ainu.id }", params: {
locale: 'ain', title: 'conflicting_name', body: 'current', locale: 'ain-Kana', title: 'conflicting_name', body: 'current',
base_revision_id: stale.id }, headers: headers base_revision_id: stale.id }, headers: headers
end.to change(TagDescriptionVersion, :count).by(0).and change(TagVersion, :count).by(0) end.to change(TagDescriptionVersion, :count).by(0).and change(TagVersion, :count).by(0)
expect(response).to have_http_status(:conflict) expect(response).to have_http_status(:conflict)
@@ -145,14 +163,14 @@ RSpec.describe 'Wiki and native API locale contracts', type: :request do
first = current_description_version(ainu) first = current_description_version(ainu)
TagDescriptionOperation.update!( TagDescriptionOperation.update!(
ainu_locale, tag:, body: 'later', updated_by_user: user) ainu_locale, tag:, body: 'later', updated_by_user: user)
get "/wiki/#{ ainu.id }", params: { locale: 'ain', version: first.id } get "/wiki/#{ ainu.id }", params: { locale: 'ain-Kana', version: first.id }
expect(response).to have_http_status(:ok) expect(response).to have_http_status(:ok)
expect(json).to include('body' => 'Ainu body', 'revision_id' => first.id) expect(json).to include('body' => 'Ainu body', 'revision_id' => first.id)
get "/wiki/#{ ainu.id }/diff", params: { locale: 'ain', from: first.id } get "/wiki/#{ ainu.id }/diff", params: { locale: 'ain-Kana', from: first.id }
expect(response).to have_http_status(:ok) expect(response).to have_http_status(:ok)
expect(json.fetch('newer_revision_id')).to eq(current_description_version(ainu).id) expect(json.fetch('newer_revision_id')).to eq(current_description_version(ainu).id)
get "/wiki/#{ ainu.id }/diff", params: { get "/wiki/#{ ainu.id }/diff", params: {
locale: 'ain', from: current_description_version(japanese).id } locale: 'ain-Kana', from: current_description_version(japanese).id }
expect(response).to have_http_status(:not_found) expect(response).to have_http_status(:not_found)
end end
end end
+3 -3
ファイルの表示
@@ -6,7 +6,7 @@ require 'securerandom'
RSpec.describe 'Wiki API', type: :request do RSpec.describe 'Wiki API', type: :request do
include WikiRecords include WikiRecords
def auth_headers(user) def auth_headers(user)
{ 'X-Transfer-Code' => user.inheritance_code } cookie_headers(user)
end end
let!(:user) { create_member_user! } let!(:user) { create_member_user! }
@@ -80,7 +80,7 @@ RSpec.describe 'Wiki API', type: :request do
let(:guest) { create(:user, role: 'guest') } let(:guest) { create(:user, role: 'guest') }
def auth_headers(user) def auth_headers(user)
{ 'X-Transfer-Code' => user.inheritance_code } cookie_headers(user)
end end
context 'when not logged in' do context 'when not logged in' do
@@ -126,7 +126,7 @@ RSpec.describe 'Wiki API', type: :request do
let(:guest) { create(:user, role: 'guest', inheritance_code: SecureRandom.hex(16)) } let(:guest) { create(:user, role: 'guest', inheritance_code: SecureRandom.hex(16)) }
def auth_headers(user) def auth_headers(user)
{ 'X-Transfer-Code' => user.inheritance_code } cookie_headers(user)
end end
let!(:test_tag_name) { create(:tag, primary_name: 'TestPage').tag_name('ja') } let!(:test_tag_name) { create(:tag, primary_name: 'TestPage').tag_name('ja') }
+1 -1
ファイルの表示
@@ -5,7 +5,7 @@ RSpec.describe 'Wiki title collision', type: :request do
let!(:user) { create_member_user! } let!(:user) { create_member_user! }
def auth_headers user def auth_headers user
{ 'X-Transfer-Code' => user.inheritance_code } cookie_headers(user)
end end
def create_wiki_page title:, body: def create_wiki_page title:, body:
+1 -1
ファイルの表示
@@ -17,7 +17,7 @@ RSpec.describe 'Removed standalone Wiki resources', type: :request do
user user
before_state = group_state before_state = group_state
post '/wiki', params: { title:, body: 'body' }, post '/wiki', params: { title:, body: 'body' },
headers: { 'X-Transfer-Code' => user.inheritance_code } headers: cookie_headers(user)
expect(response).to have_http_status(:unprocessable_entity) expect(response).to have_http_status(:unprocessable_entity)
expect(group_state).to eq(before_state) expect(group_state).to eq(before_state)
end end
+116
ファイルの表示
@@ -0,0 +1,116 @@
require 'rails_helper'
require 'tempfile'
RSpec.describe CertifiedTagNameTranslations do
let!(:language) { Language.create!(code: 'ain', name: 'ain') }
let!(:script) { Script.create!(code: 'Kana', name: 'Kana') }
let!(:tag) { create(:tag, primary_name: 'source') }
let(:entries) { [{ 'ja' => 'source', 'ain-Kana' => 'レ_ハイタ' }] }
let(:document) do
{ 'format' => described_class::FORMAT,
'source_locale' => 'ja', 'target_locale' => 'ain-Kana', 'entries' => entries }
end
let(:locale) do
Locale.new(code: 'ain-Kana', language_code: 'ain', script_code: 'Kana',
region_code: 392, variants: '', name: 'Ainu')
end
around do |example|
Tempfile.create(['certified-names', '.json']) do |file|
@path = file.path
example.run
end
end
def write_document
File.write(@path, JSON.generate(document))
locale.certified_tag_names_path = @path
end
it 'registers certified names before automatic names and records their provenance' do
other = create(:tag, primary_name: 'レ_ハイタ')
write_document
locale.save!
expect(tag.reload.tag_name('ain')).to have_attributes(
name: 'レ_ハイタ', primary_flg: true, auto_generated: false)
expect(other.reload.tag_name('ain')).to have_attributes(
name: "Tag_##{ other.id }", auto_generated: true)
version = tag.tag_versions.order(:version_no).last
expect(version.created_by_user_id).to be_nil
expect(version.names.fetch('ain')).to include(
'name' => 'レ_ハイタ', 'auto_generated' => false)
end
it 'skips missing sources without rolling back the locale or matching entries' do
entries.unshift({ 'ja' => 'missing', 'ain-Kana' => 'unused' })
write_document
locale.save!
expect(locale).to be_persisted
expect(tag.reload.name('ain')).to eq('レ_ハイタ')
expect(TagName.exists?(language_code: 'ain', name: 'unused')).to be(false)
end
it 'matches source names exactly rather than using a case-insensitive collation' do
entries.first['ja'] = 'SOURCE'
write_document
expect(described_class.load!(@path, locale:)).to eq({})
end
it 'rejects ambiguous primary matches' do
write_document
relation = instance_double(ActiveRecord::Relation)
allow(TagName).to receive(:where).with(
language_code: 'ja', primary_flg: true, name: ['source']).and_return(relation)
allow(relation).to receive(:pluck).with(:name, :tag_id)
.and_return([['source', tag.id], ['source', tag.id + 1]])
expect { described_class.load!(@path, locale:) }
.to raise_error(described_class::Error, /source.*exactly one tag/)
end
it 'rejects duplicate sources and identifies the offending name' do
entries.push({ 'ja' => 'source', 'ain-Kana' => 'another' })
write_document
expect { locale.save! }.to raise_error(described_class::Error, /duplicate source.*source/)
expect(Locale.exists?(code: 'ain-Kana')).to be(false)
end
it 'rejects duplicate targets even when a source is missing' do
entries.push({ 'ja' => 'missing', 'ain-Kana' => 'レ_ハイタ' })
write_document
expect { locale.save! }.to raise_error(described_class::Error, /duplicate target.*レ_ハイタ/)
expect(Locale.exists?(code: 'ain-Kana')).to be(false)
end
it 'rolls back names and histories if a later certified name is invalid' do
create(:tag, primary_name: 'second')
entries.push({ 'ja' => 'second', 'ain-Kana' => 'nico:invalid' })
write_document
original_versions = TagVersion.order(:id).map(&:attributes)
expect { locale.save! }.to raise_error(described_class::Error, /nico:invalid/)
expect(Locale.exists?(code: 'ain-Kana')).to be(false)
expect(TagName.where(language_code: 'ain')).to be_empty
expect(TagVersion.order(:id).map(&:attributes)).to eq(original_versions)
end
it 'does not add names or versions on repeated completion' do
write_document
locale.save!
original_names = TagName.order(:id).map(&:attributes)
original_versions = TagVersion.order(:id).map(&:attributes)
TagNameCompletion.for_locale!(locale)
expect(TagName.order(:id).map(&:attributes)).to eq(original_names)
expect(TagVersion.order(:id).map(&:attributes)).to eq(original_versions)
end
end
+2 -3
ファイルの表示
@@ -1,7 +1,7 @@
require 'rails_helper' require 'rails_helper'
RSpec.describe PostBulkCreator do RSpec.describe PostBulkCreator do
include_context 'English locale' include_context 'Ainu locale'
it 'limits workers to two and keeps failures in their request slots' do it 'limits workers to two and keeps failures in their request slots' do
locale = Locale.nipponese locale = Locale.nipponese
@@ -67,8 +67,7 @@ RSpec.describe PostBulkCreator do
expect(results[3].dig(:post, :id)).to eq(4) expect(results[3].dig(:post, :id)).to eq(4)
end end
it 'passes a future English locale to preflight and creation in every worker' do it 'passes a future Ainu locale to preflight and creation in every worker' do
pending '日本語以外の Locale 有効化は別課題で実装する'
locale locale
actor = instance_double(User, id: 123) actor = instance_double(User, id: 123)
allow(User).to receive(:find).with(123).and_return(actor) allow(User).to receive(:find).with(123).and_return(actor)
+21 -23
ファイルの表示
@@ -11,23 +11,22 @@ RSpec.describe PostCreatePlan do
end end
context 'with an explicit locale' do context 'with an explicit locale' do
include_context 'English locale' include_context 'Ainu locale'
it 'resolves the matching language identity and its parents' do it 'resolves the matching language identity and its parents' do
pending '日本語以外の Locale 有効化は別課題で実装する'
locale locale
english_name = create(:tag_name, name: 'shared_name', ainu_name = create(:tag_name, name: 'shared_name',
language_code: 'en', script_code: 'Latn') language_code: 'ain', script_code: 'Kana')
english = create(:tag, primary_tag_name: english_name, category: :character) ainu = create(:tag, primary_tag_name: ainu_name, category: :character)
primary_tag_name_for(ainu, 'ja').update!(name: 'ainu_in_japanese', auto_generated: false)
nipponese = create(:tag, primary_name: 'temporary_nipponese', category: :general) nipponese = create(:tag, primary_name: 'temporary_nipponese', category: :general)
# The DB permits this identity; validation has its own contract spec. primary_tag_name_for(nipponese, 'ja').update!(name: 'shared_name')
primary_tag_name_for(nipponese, 'ja').update_columns(name: 'shared_name') ainu_parent_name = create(:tag_name, name: 'ainu_parent',
english_parent_name = create(:tag_name, name: 'english_parent', language_code: 'ain', script_code: 'Kana')
language_code: 'en', script_code: 'Latn') ainu_parent = create(:tag, primary_tag_name: ainu_parent_name,
english_parent = create(:tag, primary_tag_name: english_parent_name,
category: :material) category: :material)
nipponese_parent = create(:tag, primary_name: 'nipponese_parent', category: :general) nipponese_parent = create(:tag, primary_name: 'nipponese_parent', category: :general)
TagImplication.create!(tag: english, parent_tag: english_parent) TagImplication.create!(tag: ainu, parent_tag: ainu_parent)
TagImplication.create!(tag: nipponese, parent_tag: nipponese_parent) TagImplication.create!(tag: nipponese, parent_tag: nipponese_parent)
plan = described_class.new(attributes: { tags: 'shared_name' }).build!(locale) plan = described_class.new(attributes: { tags: 'shared_name' }).build!(locale)
@@ -35,35 +34,34 @@ RSpec.describe PostCreatePlan do
expect(plan[:direct_tag_specs]).to eq( expect(plan[:direct_tag_specs]).to eq(
[{ name: 'shared_name', category: :character }]) [{ name: 'shared_name', category: :character }])
expect(plan[:post_tag_specs]).to include( expect(plan[:post_tag_specs]).to include(
{ name: primary_tag_name_for(english_parent, 'en').name, category: :material }) { name: primary_tag_name_for(ainu_parent, 'ain').name, category: :material })
expect(plan[:post_tag_specs].pluck(:name)) expect(plan[:post_tag_specs].pluck(:name))
.not_to include(primary_tag_name_for(nipponese_parent, 'ja').name) .not_to include(primary_tag_name_for(nipponese_parent, 'ja').name)
end end
it 'resolves aliases within the requested language' do it 'resolves aliases within the requested language' do
pending '日本語以外の Locale 有効化は別課題で実装する'
locale locale
nipponese = create(:tag, primary_name: 'nipponese_canonical') nipponese = create(:tag, primary_name: 'nipponese_canonical')
english_name = create(:tag_name, name: 'english_canonical', ainu_name = create(:tag_name, name: 'ainu_canonical',
language_code: 'en', script_code: 'Latn') language_code: 'ain', script_code: 'Kana')
english = create(:tag, primary_tag_name: english_name, category: :character) ainu = create(:tag, primary_tag_name: ainu_name, category: :character)
nipponese_alias = create(:tag_name, :alias, name: 'shared_alias', nipponese_alias = create(:tag_name, :alias, name: 'shared_alias',
tag: nipponese) tag: nipponese)
english_alias = create(:tag_name, :alias, name: 'shared_alias', ainu_alias = create(:tag_name, :alias, name: 'shared_alias',
tag: english, tag: ainu,
language_code: 'en', script_code: 'Latn') language_code: 'ain', script_code: 'Kana')
expect(nipponese_alias.reload).to have_attributes( expect(nipponese_alias.reload).to have_attributes(
tag_id: nipponese.id, tag_id: nipponese.id,
primary_flg: false, language_code: 'ja') primary_flg: false, language_code: 'ja')
expect(english_alias.reload).to have_attributes( expect(ainu_alias.reload).to have_attributes(
tag_id: english.id, tag_id: ainu.id,
primary_flg: false, language_code: 'en') primary_flg: false, language_code: 'ain')
plan = described_class.new(attributes: { tags: 'shared_alias' }).build!(locale) plan = described_class.new(attributes: { tags: 'shared_alias' }).build!(locale)
expect(plan[:direct_tag_specs]).to eq( expect(plan[:direct_tag_specs]).to eq(
[{ name: english_name.name, category: :character }]) [{ name: ainu_name.name, category: :character }])
end end
end end
+5 -5
ファイルの表示
@@ -77,11 +77,10 @@ RSpec.describe PostCreator do
end end
context 'with an explicit locale' do context 'with an explicit locale' do
include_context 'English locale' include_context 'Ainu locale'
[false, true].each do |planned| [false, true].each do |planned|
it "passes locale to tag creation with planned attributes: #{ planned }" do it "passes locale to tag creation with planned attributes: #{ planned }" do
pending '日本語以外の Locale 有効化は別課題で実装する'
locale locale
attributes = { attributes = {
title: 'Locale propagation', title: 'Locale propagation',
@@ -96,9 +95,10 @@ RSpec.describe PostCreator do
post = described_class.new(actor:, attributes:).create!(locale) post = described_class.new(actor:, attributes:).create!(locale)
expect(Tag).to have_received(:find_or_create_by_tag_name!) expect(Tag).to have_received(:find_or_create_by_tag_name!)
.with(locale, 'locale_character', category: :character).at_least(:once) .with(locale, 'locale_character', category: :character,
tag_name = TagName.find_by!(language_code: 'en', name: 'locale_character') created_by_user: actor).at_least(:once)
expect(tag_name).to have_attributes(script_code: 'Latn', primary_flg: true) tag_name = TagName.find_by!(language_code: 'ain', name: 'locale_character')
expect(tag_name).to have_attributes(script_code: 'Kana', primary_flg: true)
expect(post.tags).to include(tag_name.tag) expect(post.tags).to include(tag_name.tag)
end end
end end
+34
ファイルの表示
@@ -141,6 +141,40 @@ RSpec.describe 'Shared resource IDs across MySQL connections', type: :model do
result(first) result(first)
end end
[false, true].each do |inner_rollback|
[false, true].each do |outer_rollback|
it "keeps savepoint=#{ inner_rollback } lock through outer rollback=#{ outer_rollback }" do
connection = ApplicationRecord.connection
database = connection.pool.db_config.database
lock_name = "btrc:ids:#{ Digest::SHA256.hexdigest(database)[0, 16] }:tags"
quoted_lock = connection.quote(lock_name)
owner = connection.select_value('SELECT CONNECTION_ID()').to_i
connection.transaction do
connection.transaction(requires_new: true) do
create_resource(:external)
raise ActiveRecord::Rollback if inner_rollback
end
expect(connection.select_value("SELECT IS_USED_LOCK(#{ quoted_lock })").to_i)
.to eq(owner)
contender = worker do |other|
other.select_value("SELECT GET_LOCK(#{ quoted_lock }, 0)").to_i
end
expect(result(contender)).to eq(0)
SharedResourceIdAllocator.lock_space!(ExternalTag, 'tags')
raise ActiveRecord::Rollback if outer_rollback
end
expect(connection.select_value("SELECT IS_FREE_LOCK(#{ quoted_lock })").to_i)
.to eq(1)
expect(ExternalTag.exists?(name: "#{ @prefix }_external"))
.to eq(!inner_rollback && !outer_rollback)
end
end
end
it 'raises on a real lock acquisition failure without inserting a record' do it 'raises on a real lock acquisition failure without inserting a record' do
first, (_id, _connection_id, lock_name) = hold_creation(:external, rollback: true) first, (_id, _connection_id, lock_name) = hold_creation(:external, rollback: true)
stub_const('SharedResourceIdAllocator::LOCK_TIMEOUT', 0) stub_const('SharedResourceIdAllocator::LOCK_TIMEOUT', 0)
+30 -12
ファイルの表示
@@ -101,7 +101,7 @@ RSpec.describe SharedResourceIdAllocator, 'lock lifecycle' do
let(:record) { build(:external_tag) } let(:record) { build(:external_tag) }
let(:connection) { double('connection') } let(:connection) { double('connection') }
let(:transaction) { double('transaction', open?: true) } let(:transaction) { double('transaction', open?: true) }
let(:callbacks) { { } } let(:callbacks) { [] }
let(:sql) { [] } let(:sql) { [] }
before do before do
@@ -128,19 +128,24 @@ RSpec.describe SharedResourceIdAllocator, 'lock lifecycle' do
end end
end end
allow(ExternalTag).to receive(:connection).and_return(connection) allow(ExternalTag).to receive(:connection).and_return(connection)
allow(ExternalTag).to receive(:current_transaction).and_return(transaction) allow(connection).to receive(:current_transaction).and_return(transaction)
allow(connection).to receive(:add_transaction_record) { callbacks << _1 }
allow(connection).to receive(:disconnect!)
allow(ExternalTag).to receive(:uncached).and_yield allow(ExternalTag).to receive(:uncached).and_yield
[:after_commit, :after_rollback].each do |event|
allow(transaction).to receive(event) { |&callback| callbacks[event] = callback }
end
end end
[:after_commit, :after_rollback].each do |event| def finish_transaction event
allow(transaction).to receive(:open?).and_return(false)
callbacks.shift.public_send(event)
allow(transaction).to receive(:open?).and_return(true)
end
[:committed!, :rolledback!].each do |event|
it "holds the shared lock until #{ event }" do it "holds the shared lock until #{ event }" do
described_class.assign!(record) described_class.assign!(record)
expect(record.id).to eq(9) expect(record.id).to eq(9)
expect(sql.grep(/RELEASE_LOCK/)).to be_empty expect(sql.grep(/RELEASE_LOCK/)).to be_empty
callbacks.fetch(event).call finish_transaction(event)
expect(sql.grep(/RELEASE_LOCK/).size).to eq(1) expect(sql.grep(/RELEASE_LOCK/).size).to eq(1)
end end
end end
@@ -155,7 +160,7 @@ RSpec.describe SharedResourceIdAllocator, 'lock lifecycle' do
it 'releases a successfully acquired lock when allocation fails and rolls back' do it 'releases a successfully acquired lock when allocation fails and rolls back' do
allow(connection).to receive(:select_all).and_raise('query failed') allow(connection).to receive(:select_all).and_raise('query failed')
expect { described_class.assign!(record) }.to raise_error(RuntimeError, 'query failed') expect { described_class.assign!(record) }.to raise_error(RuntimeError, 'query failed')
callbacks.fetch(:after_rollback).call finish_transaction(:rolledback!)
expect(sql.grep(/RELEASE_LOCK/).size).to eq(1) expect(sql.grep(/RELEASE_LOCK/).size).to eq(1)
end end
@@ -164,18 +169,17 @@ RSpec.describe SharedResourceIdAllocator, 'lock lifecycle' do
described_class.assign!(record) described_class.assign!(record)
expect(record.id).to eq(50) expect(record.id).to eq(50)
expect(sql.grep(/FOR UPDATE/)).to be_empty expect(sql.grep(/FOR UPDATE/)).to be_empty
callbacks.fetch(:after_commit).call finish_transaction(:committed!)
end end
it 'uses the same named lock within each space and different locks between spaces' do it 'uses the same named lock within each space and different locks between spaces' do
locks = [Tag, ExternalTag, TagDescription, TagGroup, TagDescriptionVersion, TagGroupVersion].map do |model| locks = [Tag, ExternalTag, TagDescription, TagGroup, TagDescriptionVersion, TagGroupVersion].map do |model|
allow(model).to receive(:connection).and_return(connection) allow(model).to receive(:connection).and_return(connection)
allow(model).to receive(:current_transaction).and_return(transaction)
allow(model).to receive(:uncached).and_yield allow(model).to receive(:uncached).and_yield
record = model.new(id: 50) record = model.new(id: 50)
described_class.assign!(record) described_class.assign!(record)
statement = sql.grep(/GET_LOCK/).last statement = sql.grep(/GET_LOCK/).last
callbacks.fetch(:after_commit).call finish_transaction(:committed!)
statement statement
end end
expect(locks[0]).to eq(locks[1]) expect(locks[0]).to eq(locks[1])
@@ -187,7 +191,21 @@ RSpec.describe SharedResourceIdAllocator, 'lock lifecycle' do
it 'reports lock release failures' do it 'reports lock release failures' do
described_class.assign!(record) described_class.assign!(record)
allow(connection).to receive(:select_value).with(/RELEASE_LOCK/).and_return(nil) allow(connection).to receive(:select_value).with(/RELEASE_LOCK/).and_return(nil)
expect { callbacks.fetch(:after_commit).call } expect { finish_transaction(:committed!) }
.to raise_error(ActiveRecord::ActiveRecordError, /lock release failed/) .to raise_error(ActiveRecord::ActiveRecordError, /lock release failed/)
expect(connection).to have_received(:disconnect!)
end
it 'keeps the lock and transfers cleanup when a savepoint rolls back' do
described_class.assign!(record)
callbacks.shift.rolledback!
expect(callbacks.size).to eq(1)
expect(sql.grep(/RELEASE_LOCK/)).to be_empty
described_class.assign!(record)
expect(sql.grep(/GET_LOCK/).size).to eq(1)
finish_transaction(:committed!)
expect(sql.grep(/RELEASE_LOCK/).size).to eq(1)
end end
end end
+127
ファイルの表示
@@ -0,0 +1,127 @@
require 'rails_helper'
RSpec.describe TagNameCompletion do
let(:author) { create(:user, :member) }
let!(:language) { Language.create!(code: 'ain', name: 'ain') }
let!(:script) { Script.create!(code: 'Kana', name: 'Kana') }
let(:locale) do
Locale.create!(code: 'ain-Kana', language_code: 'ain', script_code: 'Kana',
region_code: 392, variants: '', name: 'ain-Kana')
end
it 'generates a phonetic primary with the required identity and provenance' do
tag = create(:tag, primary_name: '漢字あカ')
locale
expect(tag.reload.tag_name('ain')).to have_attributes(
name: 'カンチ_ア_カ', language_code: 'ain', script_code: 'Kana',
primary_flg: true, auto_generated: true, tag_id: tag.id)
end
it 'uses the exact fallback when the input cannot be converted' do
tag = create(:tag, primary_name: '😀')
locale
expect(tag.reload.name('ain')).to eq("Tag_##{ tag.id }")
end
it 'uses the exact fallback when a generated name collides with a human name' do
other = create(:tag, primary_name: 'other')
create(:tag_name, tag: other, name: 'AbC', language_code: 'ain', script_code: 'Kana')
tag = create(:tag, primary_name: 'AbC')
locale
expect(tag.reload.name('ain')).to eq("Tag_##{ tag.id }")
expect(other.reload.name('ain')).to eq('AbC')
end
it 'uses the exact fallback for a transcription that fails name validation' do
tag = create(:tag, primary_name: 'base')
source = TagName.new(name: 'nico:invalid', language_code: 'ja')
expect(TagName.generate_name(locale, tag, source)).to eq("Tag_##{ tag.id }")
end
it 'preserves human confirmation and is idempotent across repeated completion' do
locale
tag = Tag.find_or_create_by_tag_name!(Locale.nipponese, 'あいぬ', category: :general)
primary = tag.tag_name('ain')
primary.update!(name: 'human_confirmed', auto_generated: false)
snapshot = tag.tag_names.order(:id).map(&:attributes)
2.times { described_class.for_tag!(tag) }
expect(tag.tag_names.order(:id).map(&:attributes)).to eq(snapshot)
end
it 'creates one primary per active language through the production creation path' do
locale
tag = Tag.find_or_create_by_tag_name!(Locale.nipponese, 'アイヌ', category: :general)
expect(tag.tag_names.where(primary_flg: true).pluck(:language_code, :name))
.to contain_exactly(['ja', 'アイヌ'], ['ain', 'アイヌ'])
expect(Tag.find_or_create_by_tag_name!(locale, 'アイヌ', category: :general)).to eq(tag)
end
['ja', 'ain'].each do |language|
it "records one authored create with all names for a #{ language } origin" do
ainu_locale = locale
origin = language == 'ja' ? Locale.nipponese : ainu_locale
tag = nil
expect {
tag = Tag.find_or_create_by_tag_name!(origin, 'アイヌ', category: :general,
created_by_user: author)
}.to change(TagVersion, :count).by(1)
version = tag.tag_versions.sole
expect(version).to have_attributes(
event_type: 'create', version_no: 1, created_by_user_id: author.id)
expect(tag.reload.version_no).to eq(1)
expect(version.names).to eq(
'ja' => { 'name' => 'アイヌ', 'script_code' => 'Jpan',
'auto_generated' => language != 'ja', 'aliases' => [] },
'ain' => { 'name' => 'アイヌ', 'script_code' => 'Kana',
'auto_generated' => language != 'ain', 'aliases' => [] })
expect {
described_class.for_tag!(tag, created_by_user: author)
found = Tag.find_or_create_by_tag_name!(origin, 'アイヌ', category: :general,
created_by_user: author)
expect(found).to eq(tag)
TagVersioning.record_tag_snapshot!(tag, created_by_user: author)
}.not_to change(TagVersion, :count)
expect(version.reload.created_by_user_id).to eq(author.id)
end
end
it 'records the author even when no additional language needs completion' do
tag = build(:tag, primary_name: 'single_language_creation')
described_class.for_tag!(tag, created_by_user: author)
expect(tag).to be_persisted
expect(tag.tag_versions.sole).to have_attributes(
event_type: 'create', version_no: 1, created_by_user_id: author.id)
expect(tag.tag_versions.sole.names.keys).to eq(['ja'])
end
it 'rolls back a new tag and all names when its create history fails' do
locale
author
before_counts = [Tag.count, TagName.count, TagVersion.count]
allow(TagVersioning).to receive(:record!).and_raise('history failed')
expect {
Tag.find_or_create_by_tag_name!(Locale.nipponese, 'atomic_new_history',
category: :general, created_by_user: author)
}.to raise_error('history failed')
expect([Tag.count, TagName.count, TagVersion.count]).to eq(before_counts)
expect(TagName.where(name: 'atomic_new_history')).to be_empty
end
end
+37
ファイルの表示
@@ -0,0 +1,37 @@
require 'rails_helper'
RSpec.describe TagNameTranscription do
{
'AbCdXYZ' => 'AbCdXYZ',
'あいぬ' => 'アイヌ',
'アイヌ' => 'アイヌ',
'日本語' => 'ニチホンコ',
'漢字' => 'カンチ',
'漢あカ' => 'カン_ア_カ',
'漢字あいうカナ' => 'カンチ_アイウ_カナ',
'Ab漢あカZ' => 'Abカン_ア_カZ',
'りゃリュリョ' => 'リヤ_リユリヨ',
'キャット' => 'キヤット',
'コーヒー' => 'コヒ',
'コウコオケイケエカア' => 'コウコオケイケエカア',
'ガギグゲゴダヂヅデド' => 'カキクケコタチチュテト',
'バビブベボパピプペポ' => 'パピプペポパピプペポ',
'ザジズゼゾ' => 'チャチチュチェチョ',
'ジャジュジョ' => 'チャチュチョ',
'アㇰアㇱアッアンアㇺアㇷ゚' => 'アㇰアㇱアッアンアㇺアㇷ゚',
'アㇻイㇼウㇽエㇾオㇿ' => 'アㇻイㇼウㇽエㇾオㇿ',
'漢Aあ' => 'カンAア' }.each do |source, expected|
it "transcribes #{ source } as #{ expected }" do
expect(described_class.ainu(source)).to eq(expected)
end
end
{ '😀' => '', 'ヴ' => '', 'ティ' => 'テ', 'ㇵ' => '',
'魑' => '', '𠮷' => '', 'ー' => '', '々' => '',
'ア😀😀イ' => 'ア_イ', '😀アイ😀' => 'アイ',
'_😀_' => '__', 'ア_😀_イ' => 'ア___イ', '漢魑字' => 'カン_チ' }.each do |source, expected|
it "preserves convertible portions of #{ source }" do
expect(described_class.ainu(source)).to eq(expected)
end
end
end
+20 -23
ファイルの表示
@@ -1,7 +1,7 @@
require 'rails_helper' require 'rails_helper'
RSpec.describe TagVersionRecorder do RSpec.describe TagVersionRecorder do
include_context 'English locale' include_context 'Ainu locale'
let(:user) { create_member_user! } let(:user) { create_member_user! }
let!(:tag) { create(:tag, primary_name: '日本語名') } let!(:tag) { create(:tag, primary_name: '日本語名') }
@@ -17,11 +17,11 @@ RSpec.describe TagVersionRecorder do
{ 'name' => 'alias_z', 'script_code' => 'Jpan' }] } { 'name' => 'alias_z', 'script_code' => 'Jpan' }] }
end end
def prepare_english_names! def prepare_ainu_names!
locale locale
primary_tag_name_for(tag, 'en').update!(name: 'English name', auto_generated: false) primary_tag_name_for(tag, 'ain').update!(name: 'Ainu name', auto_generated: false)
create(:tag_name, :alias, tag:, language_code: 'en', script_code: 'Latn', create(:tag_name, :alias, tag:, language_code: 'ain', script_code: 'Kana',
name: 'English alias') name: 'Ainu alias')
end end
it 'snapshots the Nipponese primary and sorted aliases in grouped names' do it 'snapshots the Nipponese primary and sorted aliases in grouped names' do
@@ -62,40 +62,37 @@ RSpec.describe TagVersionRecorder do
end end
it 'snapshots future languages together without a request locale argument' do it 'snapshots future languages together without a request locale argument' do
pending '日本語以外の Locale 有効化は別課題で実装する' prepare_ainu_names!
prepare_english_names!
version = described_class.record!(tag:, event_type: :create, created_by_user: user) version = described_class.record!(tag:, event_type: :update, created_by_user: user)
expect(version.names).to eq( expect(version.names).to eq(
'ja' => nipponese_snapshot, 'ja' => nipponese_snapshot,
'en' => { 'name' => 'English name', 'script_code' => 'Latn', 'ain' => { 'name' => 'Ainu name', 'script_code' => 'Kana',
'auto_generated' => false, 'auto_generated' => false,
'aliases' => [{ 'name' => 'English alias', 'script_code' => 'Latn' }] }) 'aliases' => [{ 'name' => 'Ainu alias', 'script_code' => 'Kana' }] })
end end
it 'creates a version for an English-only change and preserves prior names' do it 'creates a version for an Ainu-only change and preserves prior names' do
pending '日本語以外の Locale 有効化は別課題で実装する' prepare_ainu_names!
prepare_english_names! original = described_class.record!(tag:, event_type: :update, created_by_user: user)
original = described_class.record!(tag:, event_type: :create, created_by_user: user) primary_tag_name_for(tag, 'ain').update!(name: 'Renamed')
primary_tag_name_for(tag, 'en').update!(name: 'Renamed')
updated = described_class.record!(tag:, event_type: :update, created_by_user: user) updated = described_class.record!(tag:, event_type: :update, created_by_user: user)
expect(updated.version_no).to eq(2) expect(updated.version_no).to eq(original.version_no + 1)
expect(original.reload.names.fetch('en')).to include('name' => 'English name') expect(original.reload.names.fetch('ain')).to include('name' => 'Ainu name')
expect(updated.names.fetch('en')).to include('name' => 'Renamed') expect(updated.names.fetch('ain')).to include('name' => 'Renamed')
expect(updated.names.fetch('ja')).to eq(nipponese_snapshot) expect(updated.names.fetch('ja')).to eq(nipponese_snapshot)
end end
it 'does not create a version for an unchanged multilingual snapshot' do it 'does not create a version for an unchanged multilingual snapshot' do
pending '日本語以外の Locale 有効化は別課題で実装する' prepare_ainu_names!
prepare_english_names! original = described_class.record!(tag:, event_type: :update, created_by_user: user)
described_class.record!(tag:, event_type: :create, created_by_user: user)
unchanged = described_class.record!(tag:, event_type: :update, created_by_user: user) unchanged = described_class.record!(tag:, event_type: :update, created_by_user: user)
expect(unchanged.version_no).to eq(1) expect(unchanged.version_no).to eq(original.version_no)
expect(tag.reload.tag_versions.count).to eq(1) expect(tag.reload.tag_versions.count).to eq(original.version_no)
end end
end end
+7 -4
ファイルの表示
@@ -1,19 +1,22 @@
RSpec.shared_context 'Ainu locale contract' do RSpec.shared_context 'Ainu locale contract' do
let(:ainu_locale) { instance_double(Locale, language_code: 'ain', script_code: 'Jpan') } let(:ainu_locale) do
instance_double(Locale, language_code: 'ain', script_code: 'Kana', deprecated_at: nil)
end
before do before do
# Only the foreign-key reference is seeded; no Locale or enum is enabled. # Keep description contracts independent of locale activation and backfill.
Language.connection.execute(<<~SQL) Language.connection.execute(<<~SQL)
INSERT IGNORE INTO languages (code, name, created_at) INSERT IGNORE INTO languages (code, name, created_at)
VALUES ('ain', 'Ainu', CURRENT_TIMESTAMP) VALUES ('ain', 'Ainu', CURRENT_TIMESTAMP)
SQL SQL
Script.find_or_create_by!(code: 'Kana') { _1.name = 'Kana' }
allow(Locale).to receive(:find_by).and_call_original allow(Locale).to receive(:find_by).and_call_original
allow(Locale).to receive(:find_by).with(code: 'ain').and_return(ainu_locale) allow(Locale).to receive(:find_by).with(code: 'ain-Kana').and_return(ainu_locale)
end end
def add_ainu_name tag, name def add_ainu_name tag, name
TagName.insert_all!([{ TagName.insert_all!([{
tag_id: tag.id, language_code: 'ain', script_code: 'Jpan', name:, tag_id: tag.id, language_code: 'ain', script_code: 'Kana', name:,
primary_flg: true, auto_generated: false, primary_flg: true, auto_generated: false,
created_at: Time.current, updated_at: Time.current }]) created_at: Time.current, updated_at: Time.current }])
tag.tag_names.reset tag.tag_names.reset
+12
ファイルの表示
@@ -1,4 +1,16 @@
module AuthHelper module AuthHelper
def cookie_headers(user)
env = Rails.application.env_config.merge(Rack::MockRequest.env_for('/'))
endpoint = lambda do |request_env|
ActionDispatch::Request.new(request_env).session[:inheritance_code] = user.inheritance_code
[200, {}, []]
end
store = ActionDispatch::Session::CookieStore.new(
endpoint, Rails.application.config.session_options)
headers = ActionDispatch::Cookies.new(store).call(env)[1]
{ 'Cookie' => Array(headers['set-cookie']).map { _1.split(';').first }.join('; ') }
end
def sign_in_as(user) def sign_in_as(user)
allow_any_instance_of(ApplicationController) allow_any_instance_of(ApplicationController)
.to receive(:current_user).and_return(user) .to receive(:current_user).and_return(user)
+10 -15
ファイルの表示
@@ -1,22 +1,17 @@
RSpec.shared_context 'English locale' do RSpec.shared_context 'Ainu locale' do
def prepare_english_reference! def prepare_ainu_reference!
Language.find_or_create_by!(code: 'en') { _1.name = 'English' } Language.find_or_create_by!(code: 'ain') { _1.name = 'ain' }
Script.find_or_create_by!(code: 'Latn') { _1.name = 'Latin' } Script.find_or_create_by!(code: 'Kana') { _1.name = 'Kana' }
region_class = Class.new(ApplicationRecord) { self.table_name = 'regions' }
region_class.find_or_create_by!(code: 826) do |record|
record.alpha_code = 'GB'
record.name = 'United Kingdom'
end
end end
let(:locale) do let(:locale) do
prepare_english_reference! prepare_ainu_reference!
Locale.find_or_create_by!(code: 'en') do |record| Locale.find_or_create_by!(code: 'ain-Kana') do |record|
record.language_code = 'en' record.language_code = 'ain'
record.script_code = 'Latn' record.script_code = 'Kana'
record.region_code = 826 record.region_code = 392
record.variants = '' record.variants = ''
record.name = 'English' record.name = 'Ainu'
end end
end end
end end
+53
ファイルの表示
@@ -37,3 +37,56 @@ npm run test:run
cd backend && bundle exec rspec cd backend && bundle exec rspec
cd ../frontend && npm run test:run && npm run build && npm run lint cd ../frontend && npm run test:run && npm run build && npm run lint
``` ```
### Help SSR in production
Build both `dist/client` and `dist/server` with `npm run build`, deploy them
together, and restart the Node process. `npm run dev` and `npm run preview`
remain CSR servers; use `npm start` for SSR. The old sitemap generator is not
part of this build.
The following environment is a deployment proposal, not an applied VPS change:
```sh
cd frontend
export NODE_ENV=production
export HOST=127.0.0.1
export PORT=5174
export PUBLIC_ORIGIN=https://hub.nizika.monster
export SSR_API_BASE_URL=http://127.0.0.1:3002
npm start
```
Run this command through the VPS's process supervisor. Port 5174 avoids the
Vite development port 5173. Rails remains on 127.0.0.1:3002. `PUBLIC_ORIGIN`
must match the public HTTPS origin, including any non-default port; redirect
alternate hostnames to that origin. `SSR_API_BASE_URL` configures server-side
public API requests only. Keep the browser's existing API configuration pointed
at the public `/api` endpoint.
Inside the existing Apache HTTPS virtual host, with `proxy`, `proxy_http` and
`headers` enabled, preserve the API rules before the catch-all frontend rule:
```apache
ProxyPreserveHost On
RequestHeader set X-Forwarded-Proto "https"
ProxyPass "/api/" "http://127.0.0.1:3002/"
ProxyPassReverse "/api/" "http://127.0.0.1:3002/"
ProxyPass "/" "http://127.0.0.1:5174/"
ProxyPassReverse "/" "http://127.0.0.1:5174/"
```
Retain existing TLS, authentication-header handling and any exact `/api` rule.
Replace the previous SPA catch-all rewrite instead of rewriting Help or missing
URLs to `index.html` before they reach Node. Do not override Node's 404/503
responses with an SPA fallback or cache its HTML. Hashed assets may be cached;
deployments must keep old hashed files available for already-open clients.
Only Help detail pages are server-rendered. Known CSR routes remain client
rendered, old known routes redirect to Japanese URLs, and unknown routes/assets
return 404. A failed Help preload keeps the readable server HTML, retries on
online/focus/pageshow, and permits one automatic reload per bundle and URL in
session storage. If storage is unavailable or the failure persists, the HTML
remains intact; native links and manual reload remain available.
+17 -4
ファイルの表示
@@ -33,9 +33,9 @@ npm run lint
If either command cannot be run or fails, report the exact command and failure. If either command cannot be run or fails, report the exact command and failure.
Do not create, modify, or run tests unless the user explicitly asks for test 明示的な個別指示がない限り、テストの追加・修正・削除・実行を一切行わない。
work. When the user asks for tests, keep working and rerun them until they fixture、mock、snapshot、テスト補助コード及び既存のテスト差分にも触れない。
pass or the remaining failure is clearly blocked. ビルド・lint・静的確認を使い、未確認事項はそのまま報告する。
## TypeScript ## TypeScript
@@ -128,6 +128,19 @@ pass or the remaining failure is clearly blocked.
## Tailwind and UI ## Tailwind and UI
- 原則として、ユーザ向け UI 文言を TSX / TypeScript に直接記述しない。
新しい文言は既存の i18n システムで管理し、適切な既存キーを先に確認する。
不必要な重複定義を避け、新規機能固有の文言は適切な namespace にまとめる。
- 対応 locale の辞書を整合させる。未翻訳項目には既存の日本語 fallback と
`_untranslated` 規約を使い、language metadata を維持する。
翻訳を捏造せず、未翻訳文字列を正式な翻訳として登録しない。
- UI 文言の追加・変更時はビルド、静的検査、実画面の確認等で、辞書と
SSR / hydration の整合及び翻訳キーの非露出を検証する。
i18n の検証を理由にテスト関連作業を開始しない。
- API の内部識別子、パラメータ、ユーザ入力データ、外部 URL は機械的に
翻訳しない。ボタン、項目名、CSV ヘッダ、エラー、空状態、ツールチップ、
アクセシビリティ用ラベルも UI 文言として扱う。
- Tailwind scans `src/**/*.{html,js,ts,jsx,tsx,mdx}`. - Tailwind scans `src/**/*.{html,js,ts,jsx,tsx,mdx}`.
- Use `cn` from `src/lib/utils.ts` for conditional class names and class merging. - Use `cn` from `src/lib/utils.ts` for conditional class names and class merging.
- In JavaScript, JSX, TypeScript, and TSX, use `cn` from `@/lib/utils` - In JavaScript, JSX, TypeScript, and TSX, use `cn` from `@/lib/utils`
@@ -144,7 +157,7 @@ pass or the remaining failure is clearly blocked.
heavy borders, or nested panel decoration unless the surrounding screen heavy borders, or nested panel decoration unless the surrounding screen
already uses them. already uses them.
- Keep operational screens dense and direct; trim explanatory copy and use - Keep operational screens dense and direct; trim explanatory copy and use
short Japanese labels that fit the control. short localised labels that fit the control.
- Preserve existing Japanese tone and orthography in nearby UI text, including - Preserve existing Japanese tone and orthography in nearby UI text, including
old-kana wording where the file already uses it. old-kana wording where the file already uses it.
- Do not add user-facing copy, helper text, descriptions, notes, tooltips, - Do not add user-facing copy, helper text, descriptions, notes, tooltips,

変更されたファイルが多すぎるため、一部のファイルは表示されません さらに表示