コミットを比較
22
コミット
| 作成者 | SHA1 | 日付 | |
|---|---|---|---|
|
|
1de00191ea | ||
|
|
d1bab299ab | ||
|
|
089727c153 | ||
|
|
894f9f9571 | ||
|
|
2490ed91c4 | ||
|
|
2dbd5260ea | ||
|
|
2e25c6e0ba | ||
|
|
a0c2788e00 | ||
|
|
31fc32b377 | ||
|
|
068355720d | ||
|
|
785b5ef8bc | ||
|
|
7f0efc3a46 | ||
|
|
a947032247 | ||
|
|
3115e31fb1 | ||
|
|
f1181e8510 | ||
|
|
38650b5671 | ||
|
|
b7b284c076 | ||
|
|
6c451d260f | ||
|
|
c1902fbc99 | ||
|
|
2f2b6e2afa | ||
|
|
f5b632ed89 | ||
|
|
15619886f2 |
+186
@@ -67,6 +67,192 @@ pass or the remaining failure is clearly blocked.
|
|||||||
Before changing behavior, inspect the matching route, controller, model,
|
Before changing behavior, inspect the matching route, controller, model,
|
||||||
service, representation, and spec.
|
service, representation, and spec.
|
||||||
|
|
||||||
|
## Shared backend systems
|
||||||
|
|
||||||
|
Before adding backend behaviour, search the existing backend first. At minimum,
|
||||||
|
check these locations:
|
||||||
|
|
||||||
|
- `app/controllers`
|
||||||
|
- `app/controllers/concerns`
|
||||||
|
- `app/models`
|
||||||
|
- `app/models/concerns`
|
||||||
|
- `app/representations`
|
||||||
|
- `app/services`
|
||||||
|
- `app/services/*`
|
||||||
|
- `app/jobs`
|
||||||
|
- `lib`
|
||||||
|
- `lib/tasks`
|
||||||
|
- `config/initializers`
|
||||||
|
|
||||||
|
Do not infer commonality from directory names alone. Read the actual
|
||||||
|
responsibility and representative usage sites.
|
||||||
|
|
||||||
|
### Controller reuse
|
||||||
|
|
||||||
|
Before adding logic to a controller, inspect:
|
||||||
|
|
||||||
|
- `ApplicationController` authentication, authorization, BAN, and IP BAN
|
||||||
|
- existing render and validation-error helpers
|
||||||
|
- existing param parsing
|
||||||
|
- controller concerns
|
||||||
|
- the controller for the same resource
|
||||||
|
- existing services
|
||||||
|
- existing representations
|
||||||
|
|
||||||
|
Keep controllers focused on:
|
||||||
|
|
||||||
|
- authentication and authorization
|
||||||
|
- parameter intake
|
||||||
|
- service and model invocation
|
||||||
|
- HTTP status selection
|
||||||
|
- representation selection
|
||||||
|
|
||||||
|
Do not reimplement these per controller when an existing path already owns
|
||||||
|
them:
|
||||||
|
|
||||||
|
- authentication and role checks
|
||||||
|
- validation error JSON
|
||||||
|
- URL normalisation
|
||||||
|
- tag normalisation
|
||||||
|
- thumbnail handling
|
||||||
|
- version recording
|
||||||
|
- complex transactions
|
||||||
|
- external HTTP fetching
|
||||||
|
- response representation assembly
|
||||||
|
|
||||||
|
### Authentication, authorization, and BAN
|
||||||
|
|
||||||
|
Treat these as the canonical backend entrypoints:
|
||||||
|
|
||||||
|
- `ApplicationController#authenticate_user`
|
||||||
|
- `current_user`
|
||||||
|
- `X-Transfer-Code`
|
||||||
|
- `reject_banned_ip_address!`
|
||||||
|
- `reject_banned_user!`
|
||||||
|
- `gte_member?`
|
||||||
|
- `admin?`
|
||||||
|
|
||||||
|
Do not create feature-local permission services, role comparisons, or header
|
||||||
|
parsing when the existing authentication boundary already owns the behaviour.
|
||||||
|
If the current boundary is insufficient, extend it minimally instead of adding
|
||||||
|
another permission path.
|
||||||
|
|
||||||
|
### Representations
|
||||||
|
|
||||||
|
If an endpoint for the same resource already uses `app/representations`, do not
|
||||||
|
assemble a separate JSON shape directly inside the controller without first
|
||||||
|
checking the existing representation contract.
|
||||||
|
|
||||||
|
Inspect at least:
|
||||||
|
|
||||||
|
- `PostRepr`
|
||||||
|
- `TagRepr`
|
||||||
|
- `MaterialRepr`
|
||||||
|
- `TheatreRepr`
|
||||||
|
- `UserRepr`
|
||||||
|
- `WikiPageRepr`
|
||||||
|
- `DeerjikistRepr`
|
||||||
|
|
||||||
|
When a lightweight response is genuinely different in purpose, keep it
|
||||||
|
deliberate and compatible with the surrounding contracts. Do not force every
|
||||||
|
identifier list into a large representation, but do not fork the same resource
|
||||||
|
shape casually either.
|
||||||
|
|
||||||
|
### Domain services
|
||||||
|
|
||||||
|
When work touches multiple models, transactions, external APIs, file handling,
|
||||||
|
history creation, or multi-step workflow, search `app/services` first.
|
||||||
|
|
||||||
|
At minimum, search for existing services in these responsibility areas:
|
||||||
|
|
||||||
|
- version recorder and versioning
|
||||||
|
- wiki commit
|
||||||
|
- YouTube or Google Drive API client
|
||||||
|
- material sync or ZIP export
|
||||||
|
- similarity calculation
|
||||||
|
- theatre selection or skip finalisation
|
||||||
|
- metadata, thumbnail, or file processing
|
||||||
|
- URL normaliser or sanitisation
|
||||||
|
- import or export
|
||||||
|
- preview safety or HTTP fetch
|
||||||
|
|
||||||
|
Do not create a same-responsibility service under another namespace or another
|
||||||
|
name. If an existing service is close, extend that API minimally instead of
|
||||||
|
wrapping it in a feature-local service.
|
||||||
|
|
||||||
|
### Versioning
|
||||||
|
|
||||||
|
When a feature writes history, snapshots, or restore roots, search the existing
|
||||||
|
versioning path first. At minimum, inspect:
|
||||||
|
|
||||||
|
- `VersionRecorder`
|
||||||
|
- `PostVersionRecorder`
|
||||||
|
- `TagVersionRecorder`
|
||||||
|
- `TagVersioning`
|
||||||
|
- `MaterialVersionRecorder`
|
||||||
|
- `NicoTagVersionRecorder`
|
||||||
|
- `WikiVersionRecorder`
|
||||||
|
|
||||||
|
Do not implement history writes in controllers, callbacks, or ad hoc feature
|
||||||
|
services when the recorder layer already owns the transaction boundary and
|
||||||
|
meaning.
|
||||||
|
|
||||||
|
### Normalisation, sanitisation, and parsing
|
||||||
|
|
||||||
|
For URLs, tag names, times, video durations, identifiers, and paths, search the
|
||||||
|
existing normaliser, sanitisation rule, parser, and model-callback path first.
|
||||||
|
|
||||||
|
Do not let frontend, controller, service, and model each invent different rules
|
||||||
|
for the same value. Use one canonical normalisation path and keep input
|
||||||
|
validation distinct from pre-persistence normalisation.
|
||||||
|
|
||||||
|
### External HTTP and URL safety
|
||||||
|
|
||||||
|
When fetching external URLs, reuse the existing preview-safety stack. Search at
|
||||||
|
least for:
|
||||||
|
|
||||||
|
- URL safety
|
||||||
|
- redirect validation
|
||||||
|
- response size limits
|
||||||
|
- timeouts
|
||||||
|
- network failure mapping
|
||||||
|
- HTML metadata extraction
|
||||||
|
- known-site extraction
|
||||||
|
- thumbnail fetching
|
||||||
|
|
||||||
|
Do not add direct `Net::HTTP`, `Faraday`, or equivalent feature-local HTTP code
|
||||||
|
that reimplements SSRF checks, redirect restrictions, size limits, or timeouts.
|
||||||
|
If the current fetcher is insufficient, extend its existing safety contract.
|
||||||
|
|
||||||
|
### Storage, files, and Active Storage
|
||||||
|
|
||||||
|
When handling files, thumbnails, ZIP output, object storage, or Active Storage
|
||||||
|
blobs, inspect existing storage helpers, exporters, thumbnail generators, and
|
||||||
|
checksum helpers first. Do not reimplement the same attach, export path,
|
||||||
|
download, resize, or checksum flow in a controller or one-off service.
|
||||||
|
|
||||||
|
### Concerns
|
||||||
|
|
||||||
|
Do not create controller or model concerns merely because some code is shared.
|
||||||
|
Use a concern only when multiple classes share the same lifecycle, macro,
|
||||||
|
callback, or tightly cohesive behaviour. Utility collections belong in explicit
|
||||||
|
objects or services, not in `CommonConcern`, `SharedMethods`, or `Utils`.
|
||||||
|
|
||||||
|
### Model boundaries
|
||||||
|
|
||||||
|
Model-specific invariants, associations, validations, and normalisation may
|
||||||
|
live in the model. Multi-model workflow, external access, complex transaction
|
||||||
|
flow, and feature orchestration belong in services. Do not hide feature
|
||||||
|
workflow in model callbacks.
|
||||||
|
|
||||||
|
### Transactions, locking, and race handling
|
||||||
|
|
||||||
|
If transactions, locking, idempotency, or race recovery already exist in a
|
||||||
|
service or model method, do not add a second implementation in a controller or
|
||||||
|
new service. Inspect the existing transaction boundary first, avoid wrapping
|
||||||
|
the same operation in needless nested transactions, and handle unique-constraint
|
||||||
|
races according to the target constraint's business meaning.
|
||||||
|
|
||||||
## Ruby style
|
## Ruby style
|
||||||
|
|
||||||
- Prefer precise, minimal changes.
|
- Prefer precise, minimal changes.
|
||||||
|
|||||||
@@ -85,4 +85,11 @@ class ApplicationController < ActionController::API
|
|||||||
base_errors: },
|
base_errors: },
|
||||||
status:
|
status:
|
||||||
end
|
end
|
||||||
|
|
||||||
|
def normalise_json value
|
||||||
|
return nil if value.nil?
|
||||||
|
return JSON.parse(value) if value.is_a?(String)
|
||||||
|
|
||||||
|
value
|
||||||
|
end
|
||||||
end
|
end
|
||||||
|
|||||||
@@ -89,15 +89,13 @@ class MaterialsController < ApplicationController
|
|||||||
|
|
||||||
begin
|
begin
|
||||||
Material.transaction do
|
Material.transaction do
|
||||||
tag_name = TagName.find_undiscard_or_create_by!(name: tag_name_raw)
|
tag = resolve_material_tag!(tag_name_raw)
|
||||||
tag = tag_name.tag
|
|
||||||
tag = Tag.create!(tag_name:, category: :material) unless tag
|
|
||||||
|
|
||||||
material = Material.new(tag:, url:,
|
material = Material.new(tag:, url:,
|
||||||
created_by_user: current_user,
|
created_by_user: current_user,
|
||||||
updated_by_user: current_user)
|
updated_by_user: current_user)
|
||||||
material.file.attach(uploaded_blob) if uploaded_blob
|
material.file.attach(uploaded_blob) if uploaded_blob
|
||||||
material.save!
|
material.save!
|
||||||
|
TagVersioning.record_tag_snapshot!(tag, created_by_user: current_user)
|
||||||
upsert_export_paths!(material)
|
upsert_export_paths!(material)
|
||||||
MaterialVersionRecorder.record!(material:, event_type: :create,
|
MaterialVersionRecorder.record!(material:, event_type: :create,
|
||||||
created_by_user: current_user)
|
created_by_user: current_user)
|
||||||
@@ -139,10 +137,7 @@ class MaterialsController < ApplicationController
|
|||||||
begin
|
begin
|
||||||
Material.transaction do
|
Material.transaction do
|
||||||
MaterialVersionRecorder.ensure_snapshot!(material, created_by_user: current_user)
|
MaterialVersionRecorder.ensure_snapshot!(material, created_by_user: current_user)
|
||||||
tag_name = TagName.find_undiscard_or_create_by!(name: tag_name_raw)
|
tag = resolve_material_tag!(tag_name_raw)
|
||||||
tag = tag_name.tag
|
|
||||||
tag = Tag.create!(tag_name:, category: :material) unless tag
|
|
||||||
|
|
||||||
material.assign_attributes(tag:, url:, updated_by_user: current_user)
|
material.assign_attributes(tag:, url:, updated_by_user: current_user)
|
||||||
if uploaded_blob
|
if uploaded_blob
|
||||||
material.file.attach(uploaded_blob)
|
material.file.attach(uploaded_blob)
|
||||||
@@ -150,6 +145,7 @@ class MaterialsController < ApplicationController
|
|||||||
material.file.detach
|
material.file.detach
|
||||||
end
|
end
|
||||||
material.save!
|
material.save!
|
||||||
|
TagVersioning.record_tag_snapshot!(tag, created_by_user: current_user)
|
||||||
upsert_export_paths!(material)
|
upsert_export_paths!(material)
|
||||||
MaterialVersionRecorder.record!(material:, event_type: :update,
|
MaterialVersionRecorder.record!(material:, event_type: :update,
|
||||||
created_by_user: current_user)
|
created_by_user: current_user)
|
||||||
@@ -240,6 +236,12 @@ class MaterialsController < ApplicationController
|
|||||||
nil
|
nil
|
||||||
end
|
end
|
||||||
|
|
||||||
|
def resolve_material_tag! tag_name_raw
|
||||||
|
tag_name = TagName.find_or_create_by!(name: tag_name_raw)
|
||||||
|
tag = tag_name.tag
|
||||||
|
tag || Tag.create!(tag_name:, category: :material)
|
||||||
|
end
|
||||||
|
|
||||||
def material_index_needs_tag_name? filters
|
def material_index_needs_tag_name? filters
|
||||||
filters[:q].present? || filters[:sort] == 'tag_name'
|
filters[:q].present? || filters[:sort] == 'tag_name'
|
||||||
end
|
end
|
||||||
|
|||||||
@@ -15,30 +15,35 @@ class NicoTagsController < ApplicationController
|
|||||||
limit = 1 if limit < 1
|
limit = 1 if limit < 1
|
||||||
|
|
||||||
post_tag_max_sql =
|
post_tag_max_sql =
|
||||||
PostTag
|
PostExternalTag
|
||||||
.select('tag_id, MAX(created_at) AS max_created_at')
|
.select('external_tag_id, MAX(created_at) AS max_created_at')
|
||||||
.group('tag_id')
|
.group('external_tag_id')
|
||||||
.to_sql
|
.to_sql
|
||||||
|
|
||||||
q = Tag.nico_tags
|
q =
|
||||||
.joins(:tag_name)
|
ExternalTag
|
||||||
.joins("LEFT JOIN (#{ post_tag_max_sql }) post_tag_max " \
|
.joins("LEFT JOIN (#{ post_tag_max_sql }) post_tag_max " \
|
||||||
'ON post_tag_max.tag_id = tags.id')
|
'ON post_tag_max.external_tag_id = external_tags.id')
|
||||||
.includes(:tag_name, tag_name: :wiki_page, linked_tags: { tag_name: :wiki_page })
|
.includes(linked_tags: { tag_name: :wiki_page })
|
||||||
q = q.where('tag_names.name LIKE ?', "%#{ name }%") if name
|
if name
|
||||||
|
q = q.where(('external_tags.name LIKE ? ' +
|
||||||
|
"OR CONCAT(external_tags.platform, ':', external_tags.name) LIKE ?"),
|
||||||
|
"%#{ name }%", "%#{ name }")
|
||||||
|
end
|
||||||
|
|
||||||
if linked_tag
|
if linked_tag
|
||||||
linked_tag_ids =
|
linked_tag_ids =
|
||||||
Tag
|
Tag
|
||||||
.joins(:tag_name)
|
.joins(:tag_name)
|
||||||
.where('tag_names.name LIKE ?', "%#{ linked_tag }%")
|
.where('tag_names.name LIKE ?', "%#{ linked_tag }%")
|
||||||
.pluck(:id)
|
.pluck(:id)
|
||||||
linked_nico_tag_ids = NicoTagRelation.where(tag_id: linked_tag_ids).pluck(:nico_tag_id)
|
linked_nico_tag_ids = NicoTagRelation.where(tag_id: linked_tag_ids).pluck(:nico_tag_id)
|
||||||
q = q.where(id: linked_nico_tag_ids)
|
q = q.where(id: linked_nico_tag_ids)
|
||||||
end
|
end
|
||||||
if link_status.in?(['linked', 'unlinked'])
|
if link_status.in?(['linked', 'unlinked'])
|
||||||
exists_sql =
|
exists_sql =
|
||||||
'EXISTS (SELECT 1 FROM nico_tag_relations ' \
|
'EXISTS (SELECT 1 FROM nico_tag_relations ' \
|
||||||
'WHERE nico_tag_relations.nico_tag_id = tags.id)'
|
'WHERE nico_tag_relations.nico_tag_id = external_tags.id)'
|
||||||
q = link_status == 'linked' ? q.where(exists_sql) : q.where("NOT #{ exists_sql }")
|
q = link_status == 'linked' ? q.where(exists_sql) : q.where("NOT #{ exists_sql }")
|
||||||
end
|
end
|
||||||
|
|
||||||
@@ -46,21 +51,21 @@ class NicoTagsController < ApplicationController
|
|||||||
sort_sql =
|
sort_sql =
|
||||||
case order[0]
|
case order[0]
|
||||||
when 'name'
|
when 'name'
|
||||||
'tag_names.name'
|
'external_tags.name'
|
||||||
when 'updated_at'
|
when 'updated_at'
|
||||||
'post_tag_max.max_created_at'
|
'post_tag_max.max_created_at'
|
||||||
else
|
else
|
||||||
"tags.#{ order[0] }"
|
"external_tags.#{ order[0] }"
|
||||||
end
|
end
|
||||||
tags = q.reselect('tags.*',
|
tags = q.reselect('external_tags.*',
|
||||||
Arel.sql('post_tag_max.max_created_at AS recent_post_tag_created_at'))
|
Arel.sql('post_tag_max.max_created_at AS recent_post_tag_created_at'))
|
||||||
.order(Arel.sql("#{ sort_sql } #{ order[1] }, tags.id #{ order[1] }"))
|
.order(Arel.sql("#{ sort_sql } #{ order[1] }, external_tags.id #{ order[1] }"))
|
||||||
.limit(limit)
|
.limit(limit)
|
||||||
.offset((page - 1) * limit)
|
.offset((page - 1) * limit)
|
||||||
.to_a
|
.to_a
|
||||||
|
|
||||||
render json: { tags: tags.map { |tag|
|
render json: { tags: tags.map { |tag|
|
||||||
TagRepr.base(tag).merge(
|
external_tag_json(tag).merge(
|
||||||
recent_post_tag_created_at: tag.recent_post_tag_created_at,
|
recent_post_tag_created_at: tag.recent_post_tag_created_at,
|
||||||
linked_tags: tag.linked_tags.map { |lt| TagRepr.base(lt) })
|
linked_tags: tag.linked_tags.map { |lt| TagRepr.base(lt) })
|
||||||
}, count: }
|
}, count: }
|
||||||
@@ -72,8 +77,7 @@ class NicoTagsController < ApplicationController
|
|||||||
|
|
||||||
id = params[:id].to_i
|
id = params[:id].to_i
|
||||||
|
|
||||||
tag = Tag.find(id)
|
tag = ExternalTag.find(id)
|
||||||
return render_bad_request('ニコニコ・タグを指定してください.') unless tag.nico?
|
|
||||||
|
|
||||||
linked_tag_names = params[:tags].to_s.split
|
linked_tag_names = params[:tags].to_s.split
|
||||||
linked_tags = nil
|
linked_tags = nil
|
||||||
@@ -81,16 +85,15 @@ class NicoTagsController < ApplicationController
|
|||||||
ApplicationRecord.transaction do
|
ApplicationRecord.transaction do
|
||||||
linked_tags = Tag.normalise_tags!(linked_tag_names, with_tagme: false,
|
linked_tags = Tag.normalise_tags!(linked_tag_names, with_tagme: false,
|
||||||
with_no_deerjikist: false)
|
with_no_deerjikist: false)
|
||||||
if linked_tags.any? { |t| t.nico? }
|
|
||||||
raise Tag::NicoTagNormalisationError
|
|
||||||
end
|
|
||||||
|
|
||||||
TagVersioning.record_tag_snapshots!(linked_tags, created_by_user: current_user)
|
TagVersioning.record_tag_snapshots!(linked_tags, created_by_user: current_user)
|
||||||
|
|
||||||
tag.linked_tags = linked_tags
|
tag.linked_tags = linked_tags
|
||||||
tag.save!
|
tag.save!
|
||||||
|
|
||||||
NicoTagVersionRecorder.record!(tag:, event_type: :update, created_by_user: current_user)
|
NicoTagVersionRecorder.record!(external_tag: tag,
|
||||||
|
event_type: :update,
|
||||||
|
created_by_user: current_user)
|
||||||
end
|
end
|
||||||
|
|
||||||
render json: tag.linked_tags.map { |t| TagRepr.base(t) }, status: :ok
|
render json: tag.linked_tags.map { |t| TagRepr.base(t) }, status: :ok
|
||||||
@@ -102,6 +105,21 @@ class NicoTagsController < ApplicationController
|
|||||||
|
|
||||||
private
|
private
|
||||||
|
|
||||||
|
def external_tag_json tag
|
||||||
|
{ id: tag.id,
|
||||||
|
name: "#{ tag.platform }:#{ tag.name }",
|
||||||
|
category: 'nico',
|
||||||
|
post_count: tag.post_count,
|
||||||
|
created_at: tag.created_at,
|
||||||
|
updated_at: tag.created_at,
|
||||||
|
deprecated_at: nil,
|
||||||
|
aliases: [],
|
||||||
|
parents: [],
|
||||||
|
has_wiki: false,
|
||||||
|
material_id: nil,
|
||||||
|
has_deerjikists: false }
|
||||||
|
end
|
||||||
|
|
||||||
def render_nico_tag_form_record_invalid record
|
def render_nico_tag_form_record_invalid record
|
||||||
if record.is_a?(TagName) || record.is_a?(Tag)
|
if record.is_a?(TagName) || record.is_a?(Tag)
|
||||||
render_validation_error fields: { tags: record.errors.full_messages.map { |message|
|
render_validation_error fields: { tags: record.errors.full_messages.map { |message|
|
||||||
|
|||||||
@@ -1,7 +1,10 @@
|
|||||||
class PostVersionsController < ApplicationController
|
class PostVersionsController < ApplicationController
|
||||||
def index
|
def index
|
||||||
post_id = params[:post].presence
|
post_id = params[:post].presence
|
||||||
tag_id = params[:tag].presence
|
tag_id = params[:tag].presence&.to_i
|
||||||
|
external_tag_id = params[:external_tag].presence&.to_i
|
||||||
|
return head :bad_request if tag_id && external_tag_id
|
||||||
|
|
||||||
page = (params[:page].presence || 1).to_i
|
page = (params[:page].presence || 1).to_i
|
||||||
limit = (params[:limit].presence || 20).to_i
|
limit = (params[:limit].presence || 20).to_i
|
||||||
|
|
||||||
@@ -10,12 +13,6 @@ class PostVersionsController < ApplicationController
|
|||||||
|
|
||||||
offset = (page - 1) * limit
|
offset = (page - 1) * limit
|
||||||
|
|
||||||
tag_name =
|
|
||||||
if tag_id
|
|
||||||
TagName.joins(:tag).find_by(tag: { id: tag_id })
|
|
||||||
end
|
|
||||||
return render json: { versions: [], count: 0 } if tag_id && tag_name.blank?
|
|
||||||
|
|
||||||
q = PostVersion.joins(<<~SQL.squish)
|
q = PostVersion.joins(<<~SQL.squish)
|
||||||
LEFT JOIN
|
LEFT JOIN
|
||||||
post_versions prev
|
post_versions prev
|
||||||
@@ -23,17 +20,23 @@ class PostVersionsController < ApplicationController
|
|||||||
prev.post_id = post_versions.post_id
|
prev.post_id = post_versions.post_id
|
||||||
AND prev.version_no = post_versions.version_no - 1
|
AND prev.version_no = post_versions.version_no - 1
|
||||||
SQL
|
SQL
|
||||||
.select('post_versions.*', 'prev.title AS prev_title', 'prev.url AS prev_url',
|
.select('post_versions.*',
|
||||||
'prev.thumbnail_base AS prev_thumbnail_base', 'prev.tags AS prev_tags',
|
'prev.title AS prev_title',
|
||||||
|
'prev.url AS prev_url',
|
||||||
|
'prev.thumbnail_base AS prev_thumbnail_base',
|
||||||
|
'prev.tags_json AS prev_tags_json',
|
||||||
'prev.video_ms AS prev_video_ms',
|
'prev.video_ms AS prev_video_ms',
|
||||||
'prev.original_created_from AS prev_original_created_from',
|
'prev.original_created_from AS prev_original_created_from',
|
||||||
'prev.original_created_before AS prev_original_created_before')
|
'prev.original_created_before AS prev_original_created_before')
|
||||||
q = q.where('post_versions.post_id = ?', post_id) if post_id
|
q = q.where('post_versions.post_id = ?', post_id) if post_id
|
||||||
if tag_name
|
if external_tag_id || (tag_id && !(Tag.exists?(id: tag_id)))
|
||||||
escaped = ActiveRecord::Base.sanitize_sql_like(tag_name.name)
|
q = q.where('JSON_CONTAINS(post_versions.tags_json,' +
|
||||||
q = q.where(("CONCAT(' ', post_versions.tags, ' ') LIKE :kw " +
|
"JSON_OBJECT('external_tag_id', #{ external_tag_id || tag_id })) " +
|
||||||
"OR CONCAT(' ', prev.tags, ' ') LIKE :kw"),
|
'OR JSON_CONTAINS(prev.tags_json,' +
|
||||||
kw: "% #{ escaped } %")
|
"JSON_OBJECT('external_tag_id', #{ external_tag_id || tag_id }))")
|
||||||
|
elsif tag_id
|
||||||
|
q = q.where("JSON_CONTAINS(post_versions.tags_json, JSON_OBJECT('tag_id', #{ tag_id })) " +
|
||||||
|
"OR JSON_CONTAINS(prev.tags_json, JSON_OBJECT('tag_id', #{ tag_id }))")
|
||||||
end
|
end
|
||||||
|
|
||||||
count = q.except(:select, :order, :limit, :offset).count
|
count = q.except(:select, :order, :limit, :offset).count
|
||||||
@@ -48,77 +51,91 @@ class PostVersionsController < ApplicationController
|
|||||||
private
|
private
|
||||||
|
|
||||||
def serialise_versions rows
|
def serialise_versions rows
|
||||||
|
rows = rows.to_a
|
||||||
user_ids = rows.map(&:created_by_user_id).compact.uniq
|
user_ids = rows.map(&:created_by_user_id).compact.uniq
|
||||||
users_by_id = User.where(id: user_ids).pluck(:id, :name).to_h
|
users_by_id = User.where(id: user_ids).pluck(:id, :name).to_h
|
||||||
|
snapshots = rows.flat_map { |row|
|
||||||
|
[normalise_json(row.tags_json),
|
||||||
|
normalise_json(row.attributes['prev_tags_json']) || []]
|
||||||
|
}
|
||||||
|
external_tag_names = external_tag_names_for(snapshots)
|
||||||
|
|
||||||
rows.map do |row|
|
rows.map do |row|
|
||||||
cur_tags = split_tags(row.tags)
|
cur_tags = snapshot_tag_literals(normalise_json(row.tags_json), external_tag_names)
|
||||||
prev_tags = split_tags(row.attributes['prev_tags'])
|
prev_tags = snapshot_tag_literals(
|
||||||
|
normalise_json(row.attributes['prev_tags_json']) || [], external_tag_names)
|
||||||
|
|
||||||
{
|
{ post_id: row.post_id,
|
||||||
post_id: row.post_id,
|
|
||||||
version_no: row.version_no,
|
version_no: row.version_no,
|
||||||
event_type: row.event_type,
|
event_type: row.event_type,
|
||||||
title: {
|
title: { current: row.title, prev: row.attributes['prev_title'] },
|
||||||
current: row.title,
|
url: { current: row.url, prev: row.attributes['prev_url'] },
|
||||||
prev: row.attributes['prev_title']
|
thumbnail: { current: nil, prev: nil },
|
||||||
},
|
thumbnail_base: { current: row.thumbnail_base,
|
||||||
url: {
|
prev: row.attributes['prev_thumbnail_base'] },
|
||||||
current: row.url,
|
video_ms: { current: row.video_ms, prev: row.attributes['prev_video_ms'] },
|
||||||
prev: row.attributes['prev_url']
|
|
||||||
},
|
|
||||||
thumbnail: {
|
|
||||||
current: nil,
|
|
||||||
prev: nil
|
|
||||||
},
|
|
||||||
thumbnail_base: {
|
|
||||||
current: row.thumbnail_base,
|
|
||||||
prev: row.attributes['prev_thumbnail_base']
|
|
||||||
},
|
|
||||||
video_ms: {
|
|
||||||
current: row.video_ms,
|
|
||||||
prev: row.attributes['prev_video_ms']
|
|
||||||
},
|
|
||||||
tags: build_version_tags(cur_tags, prev_tags),
|
tags: build_version_tags(cur_tags, prev_tags),
|
||||||
original_created_from: {
|
original_created_from: {
|
||||||
current: row.original_created_from&.iso8601,
|
current: row.original_created_from&.iso8601,
|
||||||
prev: row.attributes['prev_original_created_from']&.iso8601
|
prev: row.attributes['prev_original_created_from']&.iso8601 },
|
||||||
},
|
|
||||||
original_created_before: {
|
original_created_before: {
|
||||||
current: row.original_created_before&.iso8601,
|
current: row.original_created_before&.iso8601,
|
||||||
prev: row.attributes['prev_original_created_before']&.iso8601
|
prev: row.attributes['prev_original_created_before']&.iso8601 },
|
||||||
},
|
|
||||||
created_at: row.created_at.iso8601,
|
created_at: row.created_at.iso8601,
|
||||||
created_by_user:
|
created_by_user:
|
||||||
if row.created_by_user_id
|
if row.created_by_user_id
|
||||||
{
|
{ id: row.created_by_user_id,
|
||||||
id: row.created_by_user_id,
|
name: users_by_id[row.created_by_user_id] }
|
||||||
name: users_by_id[row.created_by_user_id]
|
end }
|
||||||
}
|
|
||||||
end
|
|
||||||
}
|
|
||||||
end
|
end
|
||||||
end
|
end
|
||||||
|
|
||||||
def build_version_tags(cur_tags, prev_tags)
|
def external_tag_names_for snapshots
|
||||||
|
ids = snapshots.flatten.filter_map { _1['external_tag_id'] }.uniq
|
||||||
|
names = ExternalTag.where(id: ids).pluck(:id, :platform, :name).to_h { |id, platform, name|
|
||||||
|
[id, "#{ platform }:#{ name }"]
|
||||||
|
}
|
||||||
|
missing_ids = ids - names.keys
|
||||||
|
|
||||||
|
NicoTagVersion
|
||||||
|
.where(tag_id: missing_ids)
|
||||||
|
.order(:tag_id, version_no: :desc)
|
||||||
|
.pluck(:tag_id, :name)
|
||||||
|
.each { |id, name| names[id] ||= name }
|
||||||
|
|
||||||
|
names
|
||||||
|
end
|
||||||
|
|
||||||
|
def snapshot_tag_literals snapshots, external_tag_names
|
||||||
|
snapshots
|
||||||
|
.filter_map { |snapshot|
|
||||||
|
if snapshot.key?('tag_id')
|
||||||
|
[tag_category_order(snapshot['category']),
|
||||||
|
Post.tag_snapshot_literal(snapshot)]
|
||||||
|
elsif external_tag_names[snapshot['external_tag_id']]
|
||||||
|
[6, external_tag_names[snapshot['external_tag_id']]]
|
||||||
|
end
|
||||||
|
}
|
||||||
|
.sort_by { |order, name| [order, name.downcase] }
|
||||||
|
.map(&:second)
|
||||||
|
end
|
||||||
|
|
||||||
|
def tag_category_order category
|
||||||
|
['deerjikist', 'meme', 'character', 'general', 'material', 'meta'].index(category) || 6
|
||||||
|
end
|
||||||
|
|
||||||
|
def build_version_tags cur_tags, prev_tags
|
||||||
(cur_tags | prev_tags).map do |name|
|
(cur_tags | prev_tags).map do |name|
|
||||||
type =
|
type =
|
||||||
if cur_tags.include?(name) && prev_tags.include?(name)
|
if cur_tags.include?(name) && prev_tags.include?(name)
|
||||||
'context'
|
'context'
|
||||||
elsif cur_tags.include?(name)
|
elsif cur_tags.include?(name)
|
||||||
'added'
|
'added'
|
||||||
else
|
else
|
||||||
'removed'
|
'removed'
|
||||||
end
|
end
|
||||||
|
|
||||||
{
|
{ name:, type: }
|
||||||
name:,
|
|
||||||
type:
|
|
||||||
}
|
|
||||||
end
|
end
|
||||||
end
|
end
|
||||||
|
|
||||||
def split_tags(tags)
|
|
||||||
tags.to_s.split(/\s+/).reject(&:blank?)
|
|
||||||
end
|
|
||||||
end
|
end
|
||||||
|
|||||||
@@ -1,5 +1,6 @@
|
|||||||
class PostsController < ApplicationController
|
class PostsController < ApplicationController
|
||||||
Event = Struct.new(:post, :tag, :user, :change_type, :timestamp, keyword_init: true)
|
Event = Struct.new(:post, :tag, :user, :change_type, :timestamp, keyword_init: true)
|
||||||
|
MAX_BULK_REQUEST_BYTES = 40 * 1024 * 1024
|
||||||
|
|
||||||
class VideoMsParseError < ArgumentError
|
class VideoMsParseError < ArgumentError
|
||||||
;
|
;
|
||||||
@@ -35,8 +36,8 @@ class PostsController < ApplicationController
|
|||||||
offset = (page - 1) * limit
|
offset = (page - 1) * limit
|
||||||
|
|
||||||
pt_max_sql =
|
pt_max_sql =
|
||||||
PostTag
|
PostVersion
|
||||||
.select('post_id, MAX(updated_at) AS max_updated_at')
|
.select('post_id, MAX(created_at) AS max_updated_at')
|
||||||
.group('post_id')
|
.group('post_id')
|
||||||
.to_sql
|
.to_sql
|
||||||
|
|
||||||
@@ -48,10 +49,9 @@ class PostsController < ApplicationController
|
|||||||
filtered_posts
|
filtered_posts
|
||||||
.joins("LEFT JOIN (#{ pt_max_sql }) pt_max ON pt_max.post_id = posts.id")
|
.joins("LEFT JOIN (#{ pt_max_sql }) pt_max ON pt_max.post_id = posts.id")
|
||||||
.reselect('posts.*', Arel.sql("#{ updated_at_all_sql } AS updated_at_all"))
|
.reselect('posts.*', Arel.sql("#{ updated_at_all_sql } AS updated_at_all"))
|
||||||
.preload(:uploaded_user, :parents, :children,
|
.preload(:external_tags, :uploaded_user, :parents, :children,
|
||||||
active_post_tags: [:sections,
|
post_tags: [:sections, { tag: [:deerjikists, :materials,
|
||||||
{ tag: [:deerjikists, :materials,
|
{ tag_name: :wiki_page }] }])
|
||||||
{ tag_name: :wiki_page }] }])
|
|
||||||
.with_attached_thumbnail
|
.with_attached_thumbnail
|
||||||
|
|
||||||
q = q.where('posts.url LIKE ?', "%#{ url }%") if url
|
q = q.where('posts.url LIKE ?', "%#{ url }%") if url
|
||||||
@@ -102,25 +102,96 @@ class PostsController < ApplicationController
|
|||||||
end
|
end
|
||||||
|
|
||||||
def random
|
def random
|
||||||
post = filtered_posts.preload(:uploaded_user, :parents, :children,
|
post =
|
||||||
active_post_tags: [:sections,
|
filtered_posts
|
||||||
{ tag: [:deerjikists, :materials,
|
.preload(:uploaded_user, :parents, :children,
|
||||||
{ tag_name: :wiki_page }] }])
|
post_tags: [:sections, { tag: [:deerjikists, :materials,
|
||||||
.with_attached_thumbnail
|
{ tag_name: :wiki_page }] }])
|
||||||
.order('RAND()')
|
.with_attached_thumbnail
|
||||||
.first
|
.order('RAND()')
|
||||||
|
.first
|
||||||
return head :not_found unless post
|
return head :not_found unless post
|
||||||
|
|
||||||
render json: PostRepr.base(post, current_user)
|
render json: PostRepr.base(post, current_user)
|
||||||
end
|
end
|
||||||
|
|
||||||
|
def metadata
|
||||||
|
return head :unauthorized unless current_user
|
||||||
|
return head :forbidden unless current_user.gte_member?
|
||||||
|
return render_bad_request('URL は必須です.') if params[:url].blank?
|
||||||
|
|
||||||
|
normal_url = PostUrlNormaliser.normalise(params[:url].to_s)
|
||||||
|
return render_validation_error(fields: { url: ['URL が不正です.'] }) if normal_url.blank?
|
||||||
|
|
||||||
|
Preview::UrlSafety.validate(normal_url)
|
||||||
|
existing_post = Post.with_attached_thumbnail.find_by(url: normal_url)
|
||||||
|
if existing_post.present?
|
||||||
|
return render json: {
|
||||||
|
url: normal_url,
|
||||||
|
title: nil,
|
||||||
|
thumbnail_base: nil,
|
||||||
|
tags: nil,
|
||||||
|
display_tags: [],
|
||||||
|
original_created_from: nil,
|
||||||
|
original_created_before: nil,
|
||||||
|
duration: nil,
|
||||||
|
video_ms: nil,
|
||||||
|
field_warnings: { },
|
||||||
|
base_warnings: [],
|
||||||
|
existing_post_id: existing_post.id,
|
||||||
|
existing_post: compact_post(existing_post.id) }
|
||||||
|
end
|
||||||
|
|
||||||
|
metadata = PostMetadataFetcher.fetch(normal_url)
|
||||||
|
field_warnings = { }
|
||||||
|
field_warnings[:title] = ['タイトルを取得できませんでした.'] if metadata[:title].blank?
|
||||||
|
if metadata[:thumbnail_base].blank?
|
||||||
|
field_warnings[:thumbnail_base] = ['サムネールを取得できませんでした.']
|
||||||
|
end
|
||||||
|
|
||||||
|
render json: {
|
||||||
|
url: normal_url,
|
||||||
|
title: metadata[:title],
|
||||||
|
thumbnail_base: metadata[:thumbnail_base],
|
||||||
|
tags: metadata[:tags],
|
||||||
|
display_tags: metadata[:display_tags],
|
||||||
|
original_created_from: metadata[:original_created_from],
|
||||||
|
original_created_before: metadata[:original_created_before],
|
||||||
|
duration: metadata[:duration],
|
||||||
|
video_ms: metadata[:video_ms],
|
||||||
|
field_warnings: field_warnings,
|
||||||
|
base_warnings: [],
|
||||||
|
existing_post_id: nil,
|
||||||
|
existing_post: nil }
|
||||||
|
rescue ArgumentError => e
|
||||||
|
render_bad_request e.message
|
||||||
|
rescue Preview::UrlSafety::UnsafeUrl => e
|
||||||
|
render_validation_error fields: { url: [e.message] }
|
||||||
|
rescue Preview::HttpFetcher::FetchFailed,
|
||||||
|
Preview::HttpFetcher::FetchTimeout,
|
||||||
|
Preview::HttpFetcher::ResponseTooLarge
|
||||||
|
render json: {
|
||||||
|
url: normal_url,
|
||||||
|
title: nil,
|
||||||
|
thumbnail_base: nil,
|
||||||
|
tags: nil,
|
||||||
|
display_tags: [],
|
||||||
|
original_created_from: nil,
|
||||||
|
original_created_before: nil,
|
||||||
|
duration: nil,
|
||||||
|
video_ms: nil,
|
||||||
|
field_warnings: { url: ['自動取得に失敗しました.'] },
|
||||||
|
base_warnings: [],
|
||||||
|
existing_post_id: nil,
|
||||||
|
existing_post: nil }
|
||||||
|
end
|
||||||
|
|
||||||
def show
|
def show
|
||||||
post =
|
post =
|
||||||
Post
|
Post
|
||||||
.includes(:uploaded_user, :parents, :children,
|
.includes(:uploaded_user, :parents, :children,
|
||||||
active_post_tags: [:sections,
|
post_tags: [:sections, { tag: [:deerjikists, :materials,
|
||||||
{ tag: [:deerjikists, :materials,
|
{ tag_name: :wiki_page }] }])
|
||||||
{ tag_name: :wiki_page }] }])
|
|
||||||
.with_attached_thumbnail
|
.with_attached_thumbnail
|
||||||
.find_by(id: params[:id])
|
.find_by(id: params[:id])
|
||||||
return head :not_found unless post
|
return head :not_found unless post
|
||||||
@@ -142,48 +213,51 @@ class PostsController < ApplicationController
|
|||||||
return head :unauthorized unless current_user
|
return head :unauthorized unless current_user
|
||||||
return head :forbidden unless current_user.gte_member?
|
return head :forbidden unless current_user.gte_member?
|
||||||
|
|
||||||
# TODO: サイトに応じて thumbnail_base 設定
|
preflight = PostCreatePreflight.new(
|
||||||
title = params[:title].presence
|
attributes: post_create_attributes,
|
||||||
url = params[:url]
|
thumbnail: params[:thumbnail],
|
||||||
thumbnail = params[:thumbnail]
|
host: request.base_url).run
|
||||||
tag_names = params[:tags].to_s.split
|
return render json: dry_run_json(preflight) if bool?(:dry)
|
||||||
original_created_from = params[:original_created_from]
|
if preflight[:existing_post_id].present?
|
||||||
original_created_before = params[:original_created_before]
|
post = Post.new(url: preflight[:url])
|
||||||
parent_post_ids = parse_parent_post_ids
|
post.errors.add :url, :taken
|
||||||
resized_thumbnail = thumbnail.present? ? Post.resized_thumbnail_attachment(thumbnail) : nil
|
return render_post_form_record_invalid post
|
||||||
|
|
||||||
post = Post.new(title:, url:, thumbnail_base: nil, uploaded_user: current_user,
|
|
||||||
original_created_from:, original_created_before:)
|
|
||||||
post.thumbnail.attach(resized_thumbnail) if resized_thumbnail
|
|
||||||
|
|
||||||
ApplicationRecord.transaction do
|
|
||||||
post.save!
|
|
||||||
|
|
||||||
Tag.normalise_tags!(tag_names, deny_deprecated: true, with_sections: true) =>
|
|
||||||
{ tags:, sections: }
|
|
||||||
TagVersioning.record_tag_snapshots!(tags, created_by_user: current_user)
|
|
||||||
|
|
||||||
tags = Tag.expand_parent_tags(tags).reject(&:deprecated?)
|
|
||||||
post.video_ms = normalise_video_ms(tags)
|
|
||||||
validate_video_sections!(post.video_ms, sections)
|
|
||||||
post.save!
|
|
||||||
sync_post_tags!(post, tags, sections)
|
|
||||||
|
|
||||||
sync_parent_posts!(post, parent_post_ids)
|
|
||||||
|
|
||||||
PostVersionRecorder.record!(post:, event_type: :create, created_by_user: current_user)
|
|
||||||
end
|
end
|
||||||
|
|
||||||
|
post = PostCreator.new(actor: current_user,
|
||||||
|
attributes: post_create_attributes.merge(
|
||||||
|
preflight.slice(
|
||||||
|
:url,
|
||||||
|
:title,
|
||||||
|
:thumbnail_base,
|
||||||
|
:tags,
|
||||||
|
:parent_post_ids,
|
||||||
|
:original_created_from,
|
||||||
|
:original_created_before,
|
||||||
|
:duration,
|
||||||
|
:video_ms,
|
||||||
|
:direct_tag_specs,
|
||||||
|
:default_tag_specs,
|
||||||
|
:snapshot_tag_specs,
|
||||||
|
:post_tag_specs,
|
||||||
|
:tag_sections,
|
||||||
|
:normalised_parent_post_ids).symbolize_keys).merge(
|
||||||
|
thumbnail: params[:thumbnail])).create!
|
||||||
|
|
||||||
post.reload
|
post.reload
|
||||||
render json: PostRepr.base(post), status: :created
|
render json: PostRepr.base(post), status: :created
|
||||||
|
rescue PostCreatePreflight::ValidationFailed => e
|
||||||
|
render_validation_error fields: e.fields, base: e.base_errors
|
||||||
rescue Tag::NicoTagNormalisationError
|
rescue Tag::NicoTagNormalisationError
|
||||||
render_validation_error fields: { tags: 'ニコニコ・タグは直接指定できません.' }
|
render_validation_error fields: { tags: 'ニコニコ・タグは直接指定できません.' }
|
||||||
rescue Tag::DeprecatedTagNormalisationError
|
rescue Tag::DeprecatedTagNormalisationError
|
||||||
render_unprocessable_entity '廃止済みタグは付与できません.', field: :tags
|
render_unprocessable_entity '廃止済みタグは付与できません.', field: :tags
|
||||||
rescue Tag::SectionLiteralParseError
|
rescue Tag::SectionLiteralParseError
|
||||||
render_validation_error fields: { tags: ['タグ区間の記法が不正です.'] }
|
render_validation_error fields: { tags: ['タグ区間の記法が不正です.'] }
|
||||||
rescue VideoMsParseError
|
rescue PostCreator::VideoMsParseError
|
||||||
render_validation_error fields: { video_ms: ['動画時間の記法が不正です.'] }
|
render_validation_error fields: { video_ms: ['動画時間の記法が不正です.'] }
|
||||||
|
rescue Post::RemoteThumbnailFetchFailed
|
||||||
|
render_validation_error fields: { thumbnail_base: ['サムネイル画像の取得に失敗しました.'] }
|
||||||
rescue MiniMagick::Error
|
rescue MiniMagick::Error
|
||||||
render_validation_error fields: { thumbnail: ['サムネイル画像の変換に失敗しました.'] }
|
render_validation_error fields: { thumbnail: ['サムネイル画像の変換に失敗しました.'] }
|
||||||
rescue ArgumentError => e
|
rescue ArgumentError => e
|
||||||
@@ -192,6 +266,25 @@ class PostsController < ApplicationController
|
|||||||
render_post_form_record_invalid e.record
|
render_post_form_record_invalid e.record
|
||||||
end
|
end
|
||||||
|
|
||||||
|
def bulk
|
||||||
|
return head :unauthorized unless current_user
|
||||||
|
return head :forbidden unless current_user.gte_member?
|
||||||
|
return head :unsupported_media_type unless request.content_mime_type == Mime[:multipart_form]
|
||||||
|
return head :payload_too_large if request.content_length.to_i > MAX_BULK_REQUEST_BYTES
|
||||||
|
posts = parse_bulk_posts_manifest
|
||||||
|
thumbnails = parse_bulk_thumbnails(posts.length)
|
||||||
|
result = PostBulkCreator.new(
|
||||||
|
actor: current_user,
|
||||||
|
posts:,
|
||||||
|
thumbnails:,
|
||||||
|
host: request.base_url).run
|
||||||
|
render json: result
|
||||||
|
rescue JSON::ParserError
|
||||||
|
render_bad_request 'posts manifest の JSON が不正です.'
|
||||||
|
rescue ArgumentError => e
|
||||||
|
render_validation_error base: [e.message]
|
||||||
|
end
|
||||||
|
|
||||||
def viewed
|
def viewed
|
||||||
return head :unauthorized unless current_user
|
return head :unauthorized unless current_user
|
||||||
|
|
||||||
@@ -290,50 +383,6 @@ class PostsController < ApplicationController
|
|||||||
render_post_form_record_invalid e.record
|
render_post_form_record_invalid e.record
|
||||||
end
|
end
|
||||||
|
|
||||||
def changes
|
|
||||||
id = params[:id].presence
|
|
||||||
tag_id = params[:tag].presence
|
|
||||||
page = (params[:page].presence || 1).to_i
|
|
||||||
limit = (params[:limit].presence || 20).to_i
|
|
||||||
|
|
||||||
page = 1 if page < 1
|
|
||||||
limit = 1 if limit < 1
|
|
||||||
|
|
||||||
offset = (page - 1) * limit
|
|
||||||
|
|
||||||
pts = PostTag.with_discarded
|
|
||||||
pts = pts.where(post_id: id) if id.present?
|
|
||||||
pts = pts.where(tag_id:) if tag_id.present?
|
|
||||||
pts = pts.includes(:post, :created_user, :deleted_user,
|
|
||||||
tag: [:deerjikists, :materials, { tag_name: :wiki_page }])
|
|
||||||
|
|
||||||
events = []
|
|
||||||
pts.each do |pt|
|
|
||||||
tag = TagRepr.base(pt.tag)
|
|
||||||
post = pt.post
|
|
||||||
|
|
||||||
events << Event.new(
|
|
||||||
post:,
|
|
||||||
tag:,
|
|
||||||
user: pt.created_user && { id: pt.created_user.id, name: pt.created_user.name },
|
|
||||||
change_type: 'add',
|
|
||||||
timestamp: pt.created_at)
|
|
||||||
|
|
||||||
if pt.discarded_at
|
|
||||||
events << Event.new(
|
|
||||||
post:,
|
|
||||||
tag:,
|
|
||||||
user: pt.deleted_user && { id: pt.deleted_user.id, name: pt.deleted_user.name },
|
|
||||||
change_type: 'remove',
|
|
||||||
timestamp: pt.discarded_at)
|
|
||||||
end
|
|
||||||
end
|
|
||||||
events.sort_by!(&:timestamp)
|
|
||||||
events.reverse!
|
|
||||||
|
|
||||||
render json: { changes: (events.slice(offset, limit) || []).as_json, count: events.size }
|
|
||||||
end
|
|
||||||
|
|
||||||
private
|
private
|
||||||
|
|
||||||
def filtered_posts
|
def filtered_posts
|
||||||
@@ -379,8 +428,20 @@ class PostsController < ApplicationController
|
|||||||
end
|
end
|
||||||
end
|
end
|
||||||
|
|
||||||
def tagged_post_ids_for(name) =
|
def tagged_post_ids_for(name)
|
||||||
Post.joins(tags: :tag_name).where(tag_names: { name: }).select(:id)
|
posts_by_internal_tags =
|
||||||
|
Post
|
||||||
|
.joins(tags: :tag_name)
|
||||||
|
.where(tag_names: { name: })
|
||||||
|
.select(:id)
|
||||||
|
|
||||||
|
posts_by_external_tags =
|
||||||
|
Post
|
||||||
|
.joins(:external_tags)
|
||||||
|
.where("CONCAT(external_tags.platform, ':', external_tags.name) = ?", name)
|
||||||
|
|
||||||
|
(posts_by_internal_tags + posts_by_external_tags).uniq(&:id)
|
||||||
|
end
|
||||||
|
|
||||||
def sync_post_tags! post, desired_tags, sections
|
def sync_post_tags! post, desired_tags, sections
|
||||||
desired_tags.each do |t|
|
desired_tags.each do |t|
|
||||||
@@ -408,13 +469,13 @@ class PostsController < ApplicationController
|
|||||||
end
|
end
|
||||||
end
|
end
|
||||||
|
|
||||||
PostTag.where(post_id: post.id, tag_id: to_remove.to_a).kept.find_each do |pt|
|
PostTag.where(post_id: post.id, tag_id: to_remove.to_a).find_each do |pt|
|
||||||
pt.discard_by!(current_user)
|
pt.destroy!
|
||||||
end
|
end
|
||||||
end
|
end
|
||||||
|
|
||||||
def build_tag_tree_for post
|
def build_tag_tree_for post
|
||||||
post_tags = post.active_post_tags.reject { |post_tag| post_tag.tag.deprecated? }
|
post_tags = post.post_tags.reject { |post_tag| post_tag.tag.deprecated? }
|
||||||
tags = post_tags.map(&:tag)
|
tags = post_tags.map(&:tag)
|
||||||
tag_ids = tags.map(&:id)
|
tag_ids = tags.map(&:id)
|
||||||
|
|
||||||
@@ -458,7 +519,10 @@ class PostsController < ApplicationController
|
|||||||
memo[tag_id] = TagRepr.inline(tag).merge(children:, sections:)
|
memo[tag_id] = TagRepr.inline(tag).merge(children:, sections:)
|
||||||
end
|
end
|
||||||
|
|
||||||
root_ids.filter_map { |id| build_node.call(id, []) }
|
internal_tags = root_ids.filter_map { |id| build_node.call(id, []) }
|
||||||
|
external_tags =
|
||||||
|
post.external_tags.map { ExternalTagRepr.inline(_1).merge(children: [], sections: []) }
|
||||||
|
internal_tags + external_tags
|
||||||
end
|
end
|
||||||
|
|
||||||
def sibling_posts_by_parent parent_post_ids
|
def sibling_posts_by_parent parent_post_ids
|
||||||
@@ -486,6 +550,79 @@ class PostsController < ApplicationController
|
|||||||
}.uniq
|
}.uniq
|
||||||
end
|
end
|
||||||
|
|
||||||
|
def post_create_attributes
|
||||||
|
{ title: params[:title],
|
||||||
|
url: params[:url],
|
||||||
|
thumbnail_base: params[:thumbnail_base],
|
||||||
|
tags: params[:tags],
|
||||||
|
original_created_from: params[:original_created_from],
|
||||||
|
original_created_before: params[:original_created_before],
|
||||||
|
parent_post_ids: parse_parent_post_ids,
|
||||||
|
video_ms: params[:video_ms],
|
||||||
|
duration: params[:duration] }
|
||||||
|
end
|
||||||
|
|
||||||
|
def parse_bulk_posts_manifest
|
||||||
|
manifest = params[:posts]
|
||||||
|
raise ArgumentError, 'posts は必須です.' if manifest.blank?
|
||||||
|
raise ArgumentError, 'posts は JSON 文字列で指定してください.' unless manifest.is_a?(String)
|
||||||
|
|
||||||
|
posts = JSON.parse(manifest)
|
||||||
|
raise ArgumentError, 'posts は配列で指定してください.' unless posts.is_a?(Array)
|
||||||
|
raise ArgumentError, '投稿件数は 1 件以上必要です.' if posts.empty?
|
||||||
|
raise ArgumentError, '投稿件数が多すぎます.' if posts.length > 100
|
||||||
|
raise ArgumentError, 'posts 要素の形式が不正です.' unless posts.all? { _1.is_a?(Hash) }
|
||||||
|
|
||||||
|
posts
|
||||||
|
end
|
||||||
|
|
||||||
|
def parse_bulk_thumbnails post_count
|
||||||
|
thumbnails = { }
|
||||||
|
raw = params[:thumbnails]
|
||||||
|
return thumbnails if raw.blank?
|
||||||
|
raise ArgumentError, 'thumbnail key が不正です.' unless raw.respond_to?(:to_unsafe_h)
|
||||||
|
|
||||||
|
raw.to_unsafe_h.each do |key, value|
|
||||||
|
raise ArgumentError, 'thumbnail key が不正です.' unless key.to_s.match?(/\A\d+\z/)
|
||||||
|
|
||||||
|
index = Integer(key, 10)
|
||||||
|
raise ArgumentError, 'thumbnail index が範囲外です.' if index.negative? || index >= post_count
|
||||||
|
raise ArgumentError, 'thumbnail index が重複しています.' if thumbnails.key?(index)
|
||||||
|
unless value.is_a?(ActionDispatch::Http::UploadedFile)
|
||||||
|
raise ArgumentError, 'thumbnail upload が不正です.'
|
||||||
|
end
|
||||||
|
|
||||||
|
thumbnails[index] = value
|
||||||
|
end
|
||||||
|
|
||||||
|
thumbnails
|
||||||
|
end
|
||||||
|
|
||||||
|
def compact_post post_id
|
||||||
|
return nil if post_id.blank?
|
||||||
|
|
||||||
|
post = Post.with_attached_thumbnail.find_by(id: post_id)
|
||||||
|
PostCompactRepr.base(post, host: request.base_url)
|
||||||
|
end
|
||||||
|
|
||||||
|
def dry_run_json preflight
|
||||||
|
preflight.slice(
|
||||||
|
:url,
|
||||||
|
:title,
|
||||||
|
:thumbnail_base,
|
||||||
|
:tags,
|
||||||
|
:display_tags,
|
||||||
|
:parent_post_ids,
|
||||||
|
:original_created_from,
|
||||||
|
:original_created_before,
|
||||||
|
:duration,
|
||||||
|
:video_ms,
|
||||||
|
:field_warnings,
|
||||||
|
:base_warnings,
|
||||||
|
:existing_post_id,
|
||||||
|
:existing_post)
|
||||||
|
end
|
||||||
|
|
||||||
def sync_parent_posts! post, parent_post_ids
|
def sync_parent_posts! post, parent_post_ids
|
||||||
if parent_post_ids.include?(post.id)
|
if parent_post_ids.include?(post.id)
|
||||||
post.errors.add :parent_post_ids, '自分自身を親投稿にはできません.'
|
post.errors.add :parent_post_ids, '自分自身を親投稿にはできません.'
|
||||||
@@ -532,7 +669,10 @@ class PostsController < ApplicationController
|
|||||||
end
|
end
|
||||||
|
|
||||||
def editable_tag_names_from_version version
|
def editable_tag_names_from_version version
|
||||||
version.tags.to_s.split.reject { |name| name.downcase.start_with?('nico:') }.sort
|
version.tags_json
|
||||||
|
.select { _1.key?('tag_id') }
|
||||||
|
.map { Post.tag_snapshot_literal(_1) }
|
||||||
|
.sort
|
||||||
end
|
end
|
||||||
|
|
||||||
def post_snapshot_from_record post
|
def post_snapshot_from_record post
|
||||||
@@ -547,9 +687,7 @@ class PostsController < ApplicationController
|
|||||||
def editable_tag_names_from_post post
|
def editable_tag_names_from_post post
|
||||||
post
|
post
|
||||||
.post_tags
|
.post_tags
|
||||||
.kept
|
|
||||||
.joins(tag: :tag_name)
|
.joins(tag: :tag_name)
|
||||||
.merge(Tag.not_nico)
|
|
||||||
.merge(Tag.where(deprecated_at: nil))
|
.merge(Tag.where(deprecated_at: nil))
|
||||||
.includes(:sections, tag: :tag_name)
|
.includes(:sections, tag: :tag_name)
|
||||||
.order('tag_names.name')
|
.order('tag_names.name')
|
||||||
@@ -565,9 +703,9 @@ class PostsController < ApplicationController
|
|||||||
|
|
||||||
def post_incoming_snapshot title:, original_created_from:, original_created_before:,
|
def post_incoming_snapshot title:, original_created_from:, original_created_before:,
|
||||||
tag_names:, video_ms_param:, duration_param:, parent_post_ids:
|
tag_names:, video_ms_param:, duration_param:, parent_post_ids:
|
||||||
|
validate_original_created_values!(original_created_from, original_created_before)
|
||||||
Tag.normalise_tags!(tag_names, with_tagme: false, deny_deprecated: true,
|
Tag.normalise_tags!(tag_names, with_tagme: false, deny_deprecated: true,
|
||||||
with_sections: true) =>
|
with_sections: true) => { tags:, sections: }
|
||||||
{ tags:, sections: }
|
|
||||||
|
|
||||||
tags = Tag.expand_parent_tags(tags).reject(&:deprecated?)
|
tags = Tag.expand_parent_tags(tags).reject(&:deprecated?)
|
||||||
video_ms = normalise_video_ms(tags, video_ms_param:, duration_param:)
|
video_ms = normalise_video_ms(tags, video_ms_param:, duration_param:)
|
||||||
@@ -602,6 +740,23 @@ class PostsController < ApplicationController
|
|||||||
value.to_s
|
value.to_s
|
||||||
end
|
end
|
||||||
|
|
||||||
|
def validate_original_created_values! original_created_from, original_created_before
|
||||||
|
candidate = Post.new(
|
||||||
|
url: 'https://example.invalid/original-created-validation',
|
||||||
|
original_created_from:,
|
||||||
|
original_created_before:)
|
||||||
|
candidate.valid?
|
||||||
|
fields = [:original_created_from, :original_created_before, :original_created_at]
|
||||||
|
relevant_errors = candidate.errors.select { fields.include?(_1.attribute) }
|
||||||
|
return if relevant_errors.empty?
|
||||||
|
|
||||||
|
invalid_post = Post.new
|
||||||
|
relevant_errors.each { |error|
|
||||||
|
invalid_post.errors.add(error.attribute, error.message)
|
||||||
|
}
|
||||||
|
raise ActiveRecord::RecordInvalid, invalid_post
|
||||||
|
end
|
||||||
|
|
||||||
def section_literal section
|
def section_literal section
|
||||||
"[#{ Post.ms_to_time(section[0]) }-#{ section[1] ? Post.ms_to_time(section[1]) : '' }]"
|
"[#{ Post.ms_to_time(section[0]) }-#{ section[1] ? Post.ms_to_time(section[1]) : '' }]"
|
||||||
end
|
end
|
||||||
@@ -693,15 +848,12 @@ class PostsController < ApplicationController
|
|||||||
original_created_from: snapshot[:original_created_from],
|
original_created_from: snapshot[:original_created_from],
|
||||||
original_created_before: snapshot[:original_created_before])
|
original_created_before: snapshot[:original_created_before])
|
||||||
|
|
||||||
Tag.normalise_tags!(snapshot[:tag_names], with_tagme: false,
|
Tag.normalise_tags!(snapshot[:tag_names],
|
||||||
deny_deprecated: true,
|
with_tagme: false,
|
||||||
with_sections: true) =>
|
deny_deprecated: true,
|
||||||
{ tags: editable_tags, sections: }
|
with_sections: true) => { tags:, sections: }
|
||||||
TagVersioning.record_tag_snapshots!(editable_tags, created_by_user: current_user)
|
TagVersioning.record_tag_snapshots!(tags, created_by_user: current_user)
|
||||||
|
|
||||||
readonly_tags = post.tags.nico.to_a
|
|
||||||
|
|
||||||
tags = readonly_tags + editable_tags
|
|
||||||
tags = Tag.expand_parent_tags(tags).reject(&:deprecated?)
|
tags = Tag.expand_parent_tags(tags).reject(&:deprecated?)
|
||||||
|
|
||||||
post.video_ms = tags.any? { _1.id == Tag.video.id } ? snapshot[:video_ms] : nil
|
post.video_ms = tags.any? { _1.id == Tag.video.id } ? snapshot[:video_ms] : nil
|
||||||
|
|||||||
@@ -1,50 +1,55 @@
|
|||||||
class PreviewController < ApplicationController
|
class PreviewController < ApplicationController
|
||||||
|
before_action :require_member!
|
||||||
|
|
||||||
def title
|
def title
|
||||||
# TODO: # 既知サイトなら決まったフォーマットで title 取得するやぅに.
|
return render_bad_request('URL は必須です.') if params[:url].blank?
|
||||||
return head :unauthorized unless current_user
|
|
||||||
|
|
||||||
url = params[:url]
|
render json: { title: Preview::ThumbnailFetcher.title(params[:url]) }
|
||||||
return render_bad_request('URL は必須です.') unless url.present?
|
rescue Preview::UrlSafety::UnsafeUrl => e
|
||||||
|
|
||||||
unless url.start_with?(/http(s)?:\/\//)
|
|
||||||
url = 'http://' + url
|
|
||||||
end
|
|
||||||
|
|
||||||
html = URI.open(url, open_timeout: 5, read_timeout: 5).read
|
|
||||||
doc = Nokogiri::HTML.parse(html)
|
|
||||||
title = doc.at('title')&.text&.strip
|
|
||||||
|
|
||||||
render json: { title: title }
|
|
||||||
rescue => e
|
|
||||||
render_bad_request(e.message)
|
render_bad_request(e.message)
|
||||||
|
rescue Preview::HttpFetcher::FetchTimeout => e
|
||||||
|
render_preview_error(e.message, :gateway_timeout)
|
||||||
|
rescue Preview::HttpFetcher::ResponseTooLarge => e
|
||||||
|
render_preview_error(e.message, :payload_too_large)
|
||||||
|
rescue Preview::HttpFetcher::FetchFailed => e
|
||||||
|
render_preview_error(e.message, :bad_gateway)
|
||||||
end
|
end
|
||||||
|
|
||||||
def thumbnail
|
def thumbnail
|
||||||
# TODO: 既知ドメインであれば指定のアドレスからサムネールを取得するやぅにする.
|
return render_bad_request('URL は必須です.') if params[:url].blank?
|
||||||
|
|
||||||
|
attachment =
|
||||||
|
Post.resized_thumbnail_attachment(
|
||||||
|
StringIO.new(Preview::ThumbnailFetcher.fetch(params[:url])))
|
||||||
|
send_data attachment[:io].read,
|
||||||
|
type: attachment[:content_type],
|
||||||
|
disposition: 'inline'
|
||||||
|
rescue Preview::UrlSafety::UnsafeUrl => e
|
||||||
|
render_bad_request(e.message)
|
||||||
|
rescue Preview::HttpFetcher::FetchTimeout => e
|
||||||
|
render_preview_error(e.message, :gateway_timeout)
|
||||||
|
rescue Preview::HttpFetcher::ResponseTooLarge => e
|
||||||
|
render_preview_error(e.message, :payload_too_large)
|
||||||
|
rescue Preview::HttpFetcher::FetchFailed => e
|
||||||
|
render_preview_error(e.message, :bad_gateway)
|
||||||
|
rescue Preview::ThumbnailFetcher::GenerationFailed, MiniMagick::Error => e
|
||||||
|
render_unprocessable_entity(e.message)
|
||||||
|
end
|
||||||
|
|
||||||
|
private
|
||||||
|
|
||||||
|
def require_member!
|
||||||
return head :unauthorized unless current_user
|
return head :unauthorized unless current_user
|
||||||
|
return if current_user.gte_member?
|
||||||
|
|
||||||
url = params[:url]
|
head :forbidden
|
||||||
return render_bad_request('URL は必須です.') if url.blank?
|
end
|
||||||
|
|
||||||
unless url.start_with?(/http(s)?:\/\//)
|
def render_preview_error(message, status)
|
||||||
url = 'http://' + url
|
render json: { type: status.to_s,
|
||||||
end
|
message:,
|
||||||
|
errors: { },
|
||||||
path = Rails.root.join('tmp', "thumb_#{ SecureRandom.hex }.png")
|
base_errors: [message] },
|
||||||
system("node #{ Rails.root }/lib/screenshot.js #{ Shellwords.escape(url) } #{ path }")
|
status:
|
||||||
|
|
||||||
if File.exist?(path)
|
|
||||||
image = MiniMagick::Image.open(path)
|
|
||||||
image.resize '180x180'
|
|
||||||
File.delete(path) rescue nil
|
|
||||||
send_file image.path, type: 'image/png', disposition: 'inline'
|
|
||||||
else
|
|
||||||
render json: { type: 'internal_server_error',
|
|
||||||
message: 'サムネールを生成できませんでした.',
|
|
||||||
errors: { },
|
|
||||||
base_errors: ['サムネールを生成できませんでした.'] },
|
|
||||||
status: :internal_server_error
|
|
||||||
end
|
|
||||||
end
|
end
|
||||||
end
|
end
|
||||||
|
|||||||
@@ -10,7 +10,6 @@ class TagChildrenController < ApplicationController
|
|||||||
|
|
||||||
parent = Tag.find(parent_id)
|
parent = Tag.find(parent_id)
|
||||||
child = Tag.find(child_id)
|
child = Tag.find(child_id)
|
||||||
return render_bad_request('ニコニコ・タグの階層は変更できません.') if parent.nico? || child.nico?
|
|
||||||
|
|
||||||
ApplicationRecord.transaction do
|
ApplicationRecord.transaction do
|
||||||
TagVersioning.ensure_snapshot!(child, created_by_user: current_user)
|
TagVersioning.ensure_snapshot!(child, created_by_user: current_user)
|
||||||
@@ -33,7 +32,6 @@ class TagChildrenController < ApplicationController
|
|||||||
|
|
||||||
parent = Tag.find(parent_id)
|
parent = Tag.find(parent_id)
|
||||||
child = Tag.find(child_id)
|
child = Tag.find(child_id)
|
||||||
return render_bad_request('ニコニコ・タグの階層は変更できません.') if parent.nico? || child.nico?
|
|
||||||
|
|
||||||
ApplicationRecord.transaction do
|
ApplicationRecord.transaction do
|
||||||
TagVersioning.ensure_snapshot!(child, created_by_user: current_user)
|
TagVersioning.ensure_snapshot!(child, created_by_user: current_user)
|
||||||
|
|||||||
@@ -34,49 +34,155 @@ class TagsController < ApplicationController
|
|||||||
|
|
||||||
offset = (page - 1) * limit
|
offset = (page - 1) * limit
|
||||||
|
|
||||||
q =
|
tags =
|
||||||
if post_id.present?
|
if post_id.present?
|
||||||
Tag.joins(:posts, :tag_name)
|
Tag.joins(:posts, :tag_name).where(posts: { id: post_id })
|
||||||
else
|
else
|
||||||
Tag.joins(:tag_name)
|
Tag.joins(:tag_name)
|
||||||
end
|
end
|
||||||
.includes(:tag_name, :materials, tag_name: :wiki_page)
|
|
||||||
q = q.where(posts: { id: post_id }) if post_id.present?
|
|
||||||
|
|
||||||
q = q.where('tag_names.name LIKE ?', "%#{ name }%") if name
|
external_tags =
|
||||||
q = q.where(category:) if category
|
if post_id.present?
|
||||||
q = q.where('tags.post_count >= ?', post_count_between[0]) if post_count_between[0]
|
ExternalTag.joins(:posts).where(posts: { id: post_id })
|
||||||
q = q.where('tags.post_count <= ?', post_count_between[1]) if post_count_between[1]
|
else
|
||||||
q = q.where('tags.created_at >= ?', created_between[0]) if created_between[0]
|
ExternalTag.all
|
||||||
q = q.where('tags.created_at <= ?', created_between[1]) if created_between[1]
|
end
|
||||||
q = q.where('tags.updated_at >= ?', updated_between[0]) if updated_between[0]
|
|
||||||
q = q.where('tags.updated_at <= ?', updated_between[1]) if updated_between[1]
|
if name
|
||||||
if deprecated_given
|
tags = tags.where('tag_names.name LIKE ?', "%#{ name }%")
|
||||||
q = deprecated ? q.where.not(deprecated_at: nil) : q.where(deprecated_at: nil)
|
external_tags =
|
||||||
|
external_tags.where("CONCAT(external_tags.platform, ':', external_tags.name) LIKE ?",
|
||||||
|
"%#{ name }%")
|
||||||
end
|
end
|
||||||
|
|
||||||
|
if category == 'nico'
|
||||||
|
tags = tags.none
|
||||||
|
elsif category
|
||||||
|
tags = tags.where(category:)
|
||||||
|
external_tags = external_tags.none
|
||||||
|
end
|
||||||
|
|
||||||
|
if post_count_between[0]
|
||||||
|
tags = tags.where('tags.post_count >= ?', post_count_between[0])
|
||||||
|
external_tags = external_tags.where('external_tags.post_count >= ?', post_count_between[0])
|
||||||
|
end
|
||||||
|
|
||||||
|
if post_count_between[1]
|
||||||
|
tags = tags.where('tags.post_count <= ?', post_count_between[1])
|
||||||
|
external_tags = external_tags.where('external_tags.post_count <= ?', post_count_between[1])
|
||||||
|
end
|
||||||
|
|
||||||
|
if created_between[0]
|
||||||
|
tags = tags.where('tags.created_at >= ?', created_between[0])
|
||||||
|
external_tags = external_tags.where('external_tags.created_at >= ?', created_between[0])
|
||||||
|
end
|
||||||
|
|
||||||
|
if created_between[1]
|
||||||
|
tags = tags.where('tags.created_at <= ?', created_between[1])
|
||||||
|
external_tags = external_tags.where('external_tags.created_at <= ?', created_between[1])
|
||||||
|
end
|
||||||
|
|
||||||
|
if updated_between[0]
|
||||||
|
tags = tags.where('tags.updated_at >= ?', updated_between[0])
|
||||||
|
external_tags = external_tags.where('external_tags.created_at >= ?', updated_between[0])
|
||||||
|
end
|
||||||
|
|
||||||
|
if updated_between[1]
|
||||||
|
tags = tags.where('tags.updated_at <= ?', updated_between[1])
|
||||||
|
external_tags = external_tags.where('external_tags.created_at <= ?', updated_between[1])
|
||||||
|
end
|
||||||
|
|
||||||
|
if deprecated_given
|
||||||
|
if deprecated
|
||||||
|
tags = tags.where.not(deprecated_at: nil)
|
||||||
|
external_tags = external_tags.none
|
||||||
|
else
|
||||||
|
tags = tags.where(deprecated_at: nil)
|
||||||
|
end
|
||||||
|
end
|
||||||
|
|
||||||
|
tag_sql = tags.select("'tag' AS source",
|
||||||
|
'tags.id',
|
||||||
|
'tag_names.name',
|
||||||
|
'tags.category',
|
||||||
|
'tags.post_count',
|
||||||
|
'tags.created_at',
|
||||||
|
'tags.updated_at',
|
||||||
|
'tags.deprecated_at').to_sql
|
||||||
|
|
||||||
|
external_tag_sql = external_tags.select(
|
||||||
|
"'external_tag' AS source",
|
||||||
|
'external_tags.id',
|
||||||
|
"CONCAT(external_tags.platform, ':', external_tags.name) AS name",
|
||||||
|
"'nico' AS category",
|
||||||
|
'external_tags.post_count',
|
||||||
|
'external_tags.created_at',
|
||||||
|
'external_tags.created_at AS updated_at',
|
||||||
|
'NULL AS deprecated_at').to_sql
|
||||||
|
|
||||||
|
union_sql = "#{ tag_sql } UNION ALL #{ external_tag_sql }"
|
||||||
|
|
||||||
sort_sql =
|
sort_sql =
|
||||||
case order[0]
|
if order[0] == 'category'
|
||||||
when 'name'
|
'CASE category ' +
|
||||||
'tag_names.name'
|
|
||||||
when 'category'
|
|
||||||
'CASE tags.category ' +
|
|
||||||
"WHEN 'deerjikist' THEN 0 " +
|
"WHEN 'deerjikist' THEN 0 " +
|
||||||
"WHEN 'meme' THEN 1 " +
|
"WHEN 'meme' THEN 1 " +
|
||||||
"WHEN 'character' THEN 2 " +
|
"WHEN 'character' THEN 2 " +
|
||||||
"WHEN 'general' THEN 3 " +
|
"WHEN 'general' THEN 3 " +
|
||||||
"WHEN 'material' THEN 4 " +
|
"WHEN 'material' THEN 4 " +
|
||||||
"WHEN 'meta' THEN 5 " +
|
"WHEN 'meta' THEN 5 " +
|
||||||
"WHEN 'nico' THEN 6 END"
|
"WHEN 'nico' THEN 6 " +
|
||||||
|
'END'
|
||||||
else
|
else
|
||||||
"tags.#{ order[0] }"
|
order[0]
|
||||||
end
|
end
|
||||||
tags = q.order(Arel.sql("#{ sort_sql } #{ order[1] }, tags.id #{ order[1] }"))
|
|
||||||
.limit(limit)
|
|
||||||
.offset(offset)
|
|
||||||
.to_a
|
|
||||||
|
|
||||||
render json: { tags: TagRepr.many(tags), count: q.size }
|
connection = ApplicationRecord.connection
|
||||||
|
|
||||||
|
count = connection.select_value(<<~SQL)
|
||||||
|
SELECT
|
||||||
|
COUNT(0)
|
||||||
|
FROM
|
||||||
|
(#{ union_sql }) legacy_tags
|
||||||
|
SQL
|
||||||
|
|
||||||
|
rows = connection.select_all(<<~SQL).to_a
|
||||||
|
SELECT
|
||||||
|
source
|
||||||
|
, id
|
||||||
|
FROM
|
||||||
|
(#{ union_sql }) legacy_tags
|
||||||
|
ORDER BY
|
||||||
|
#{ sort_sql } #{ order[1] }
|
||||||
|
, id #{ order[1] }
|
||||||
|
, source #{ order[1] }
|
||||||
|
LIMIT
|
||||||
|
#{ limit }
|
||||||
|
OFFSET
|
||||||
|
#{ offset }
|
||||||
|
SQL
|
||||||
|
|
||||||
|
tag_ids = rows.filter { _1['source'] == 'tag' }.map { _1['id'] }
|
||||||
|
external_tag_ids = rows.filter { _1['source'] == 'external_tag' }.map { _1['id'] }
|
||||||
|
|
||||||
|
tags_by_id =
|
||||||
|
Tag
|
||||||
|
.joins(:tag_name)
|
||||||
|
.includes(:tag_name, :materials, tag_name: :wiki_page)
|
||||||
|
.where(id: tag_ids)
|
||||||
|
.index_by(&:id)
|
||||||
|
external_tags_by_id =
|
||||||
|
ExternalTag
|
||||||
|
.where(id: external_tag_ids)
|
||||||
|
.index_by(&:id)
|
||||||
|
|
||||||
|
render json: { tags: rows.map { |row|
|
||||||
|
if row['source'] == 'tag'
|
||||||
|
TagRepr.base(tags_by_id.fetch(row['id']))
|
||||||
|
else
|
||||||
|
external_tag_json(external_tags_by_id.fetch(row['id']))
|
||||||
|
end
|
||||||
|
}, count: }
|
||||||
end
|
end
|
||||||
|
|
||||||
def with_depth
|
def with_depth
|
||||||
@@ -100,13 +206,14 @@ class TagsController < ApplicationController
|
|||||||
|
|
||||||
def autocomplete
|
def autocomplete
|
||||||
q = params[:q].to_s.strip.sub(/\Anot:/i, '')
|
q = params[:q].to_s.strip.sub(/\Anot:/i, '')
|
||||||
|
prefix = "#{ ActiveRecord::Base.sanitize_sql_like(q) }%"
|
||||||
|
|
||||||
with_nico = bool?(:nico, default: true)
|
with_nico = bool?(:nico, default: true)
|
||||||
present_only = bool?(:present, default: true)
|
present_only = bool?(:present, default: true)
|
||||||
|
|
||||||
alias_rows =
|
alias_rows =
|
||||||
TagName
|
TagName
|
||||||
.where('name LIKE ?', "#{ q }%")
|
.where('name LIKE ?', prefix)
|
||||||
.where.not(canonical_id: nil)
|
.where.not(canonical_id: nil)
|
||||||
.pluck(:canonical_id, :name)
|
.pluck(:canonical_id, :name)
|
||||||
|
|
||||||
@@ -118,29 +225,43 @@ class TagsController < ApplicationController
|
|||||||
matched_alias_by_tag_name_id[canonical_id] ||= alias_name
|
matched_alias_by_tag_name_id[canonical_id] ||= alias_name
|
||||||
end
|
end
|
||||||
|
|
||||||
base = Tag.joins(:tag_name)
|
base =
|
||||||
.includes(:tag_name, :materials, tag_name: :wiki_page)
|
Tag
|
||||||
.where(deprecated_at: nil)
|
.joins(:tag_name)
|
||||||
|
.includes(:tag_name, :materials, tag_name: :wiki_page)
|
||||||
|
.where(deprecated_at: nil)
|
||||||
base = base.where('tags.post_count > 0') if present_only
|
base = base.where('tags.post_count > 0') if present_only
|
||||||
|
|
||||||
canonical_hit =
|
canonical_hit = base.where('tag_names.name LIKE ?', prefix)
|
||||||
base
|
|
||||||
.where(((with_nico ? '(tags.category = ? AND tag_names.name LIKE ?) OR ' : '') +
|
|
||||||
'tag_names.name LIKE ?'),
|
|
||||||
*(with_nico ? ['nico', "nico:#{ q }%"] : []), "#{ q }%")
|
|
||||||
|
|
||||||
tags =
|
internal_tags = canonical_hit.or(base.where(tag_name_id: canonical_ids.uniq))
|
||||||
if canonical_ids.present?
|
|
||||||
canonical_hit.or(base.where(tag_name_id: canonical_ids.uniq))
|
|
||||||
else
|
|
||||||
canonical_hit
|
|
||||||
end
|
|
||||||
|
|
||||||
tags = tags.order(Arel.sql('post_count DESC, tag_names.name')).limit(20).to_a
|
internal_rows =
|
||||||
|
internal_tags
|
||||||
|
.order(Arel.sql('tags.post_count DESC, tag_names.name'))
|
||||||
|
.limit(20)
|
||||||
|
.map { |tag|
|
||||||
|
TagRepr.base(tag).merge(matched_alias: matched_alias_by_tag_name_id[tag.tag_name_id])
|
||||||
|
}
|
||||||
|
|
||||||
render json: tags.map { |tag|
|
return render json: internal_rows unless with_nico
|
||||||
TagRepr.base(tag).merge(matched_alias: matched_alias_by_tag_name_id[tag.tag_name_id])
|
|
||||||
}
|
external_base = ExternalTag.all
|
||||||
|
external_base = external_base.where('post_count > 0') if present_only
|
||||||
|
|
||||||
|
external_rows =
|
||||||
|
external_base
|
||||||
|
.where("CONCAT(platform, ':', name) LIKE ? OR name LIKE ?", prefix, prefix)
|
||||||
|
.order(post_count: :desc, name: :asc)
|
||||||
|
.limit(20)
|
||||||
|
.map { external_tag_json(_1) }
|
||||||
|
|
||||||
|
rows =
|
||||||
|
(internal_rows + external_rows)
|
||||||
|
.sort_by { |row| [-row['post_count'], row['name']] }
|
||||||
|
.first(20)
|
||||||
|
|
||||||
|
render json: rows
|
||||||
end
|
end
|
||||||
|
|
||||||
def show
|
def show
|
||||||
@@ -158,14 +279,20 @@ class TagsController < ApplicationController
|
|||||||
name = params[:name].to_s.strip
|
name = params[:name].to_s.strip
|
||||||
return render_bad_request('name は必須です.') if name.blank?
|
return render_bad_request('name は必須です.') if name.blank?
|
||||||
|
|
||||||
tag = Tag.joins(:tag_name)
|
tag =
|
||||||
.includes(:tag_name, :materials, tag_name: :wiki_page)
|
Tag
|
||||||
.find_by(tag_names: { name: })
|
.joins(:tag_name)
|
||||||
if tag
|
.includes(:tag_name, :materials, tag_name: :wiki_page)
|
||||||
render json: TagRepr.base(tag)
|
.find_by(tag_names: { name: })
|
||||||
else
|
return render json: TagRepr.base(tag) if tag
|
||||||
head :not_found
|
|
||||||
end
|
platform, external_name = name.split(':', 2)
|
||||||
|
return head :not_found unless external_name
|
||||||
|
|
||||||
|
external_tag = ExternalTag.find_by(platform:, name: external_name)
|
||||||
|
return head :not_found unless external_tag
|
||||||
|
|
||||||
|
render json: ExternalTagRepr.base(external_tag)
|
||||||
end
|
end
|
||||||
|
|
||||||
def deerjikists
|
def deerjikists
|
||||||
@@ -200,14 +327,45 @@ class TagsController < ApplicationController
|
|||||||
.find_by(id: params[:id])
|
.find_by(id: params[:id])
|
||||||
return head :not_found unless tag
|
return head :not_found unless tag
|
||||||
|
|
||||||
|
rows = normalise_deerjikist_rows(tag)
|
||||||
|
return if performed?
|
||||||
|
|
||||||
ApplicationRecord.transaction do
|
ApplicationRecord.transaction do
|
||||||
tag.deerjikists = []
|
tag.lock!
|
||||||
params[:_json].each.with_index do |item, i|
|
|
||||||
platform = item[:platform]
|
requested_keys = rows.map { |row| [row[:platform], row[:code]] }.uniq
|
||||||
code = normalise_deerjikist_code(platform, item[:code])
|
row_indexes_by_key = rows_by_key(rows)
|
||||||
deerjikist = Deerjikist.find_or_initialize_by(platform:, code:)
|
locked_deerjikists = lock_deerjikists_for_tag_update(tag.id, requested_keys)
|
||||||
deerjikist.tag = tag
|
current_deerjikists = locked_deerjikists.filter { |deerjikist|
|
||||||
render_deerjikist_form_record_invalid(deerjikist, i) unless deerjikist.save
|
deerjikist.tag_id == tag.id
|
||||||
|
}
|
||||||
|
requested_deerjikists = locked_deerjikists.filter { |deerjikist|
|
||||||
|
row_indexes_by_key.key?([deerjikist.platform, deerjikist.code])
|
||||||
|
}
|
||||||
|
|
||||||
|
render_deerjikist_conflicts(requested_deerjikists, row_indexes_by_key, tag)
|
||||||
|
raise ActiveRecord::Rollback if performed?
|
||||||
|
|
||||||
|
requested_keys_set = requested_keys.to_set
|
||||||
|
current_deerjikists.each do |deerjikist|
|
||||||
|
key = [deerjikist.platform, deerjikist.code]
|
||||||
|
deerjikist.destroy! unless requested_keys_set.include?(key)
|
||||||
|
end
|
||||||
|
|
||||||
|
existing_keys = requested_deerjikists.to_h { |deerjikist|
|
||||||
|
[[deerjikist.platform, deerjikist.code], true]
|
||||||
|
}
|
||||||
|
requested_keys.each do |platform, code|
|
||||||
|
next if existing_keys[[platform, code]]
|
||||||
|
|
||||||
|
deerjikist = Deerjikist.new(platform:, code:, tag:)
|
||||||
|
row_index = row_indexes_by_key[[platform, code]].first
|
||||||
|
begin
|
||||||
|
render_deerjikist_form_record_invalid(deerjikist, row_index) unless deerjikist.save
|
||||||
|
rescue ActiveRecord::RecordNotUnique
|
||||||
|
conflicts = lock_deerjikists_for_tag_update(tag.id, [[platform, code]])
|
||||||
|
render_deerjikist_conflicts(conflicts, row_indexes_by_key, tag)
|
||||||
|
end
|
||||||
raise ActiveRecord::Rollback if performed?
|
raise ActiveRecord::Rollback if performed?
|
||||||
end
|
end
|
||||||
end
|
end
|
||||||
@@ -251,11 +409,7 @@ class TagsController < ApplicationController
|
|||||||
parent_names = params[:parent_tags].to_s.split.uniq
|
parent_names = params[:parent_tags].to_s.split.uniq
|
||||||
deprecated = bool?(:deprecated)
|
deprecated = bool?(:deprecated)
|
||||||
|
|
||||||
if tag.nico? && deprecated
|
if category == 'nico'
|
||||||
return render_unprocessable_entity 'ニコタグは廃止できません.', field: :deprecated
|
|
||||||
end
|
|
||||||
|
|
||||||
if tag.nico? || category == 'nico'
|
|
||||||
return render_unprocessable_entity 'ニコタグは変更できません.', field: :category
|
return render_unprocessable_entity 'ニコタグは変更できません.', field: :category
|
||||||
end
|
end
|
||||||
|
|
||||||
@@ -303,13 +457,9 @@ class TagsController < ApplicationController
|
|||||||
|
|
||||||
tag = Tag.find(params[:id])
|
tag = Tag.find(params[:id])
|
||||||
|
|
||||||
if tag.nico? && deprecated_given && deprecated
|
|
||||||
return render_unprocessable_entity 'ニコタグは廃止できません.', field: :deprecated
|
|
||||||
end
|
|
||||||
|
|
||||||
return unless validate_tag_rename(tag, name)
|
return unless validate_tag_rename(tag, name)
|
||||||
|
|
||||||
if tag.nico? || (category.present? && category == 'nico')
|
if category.present? && category == 'nico'
|
||||||
return render_unprocessable_entity 'ニコタグは変更できません.', field: :category
|
return render_unprocessable_entity 'ニコタグは変更できません.', field: :category
|
||||||
end
|
end
|
||||||
|
|
||||||
@@ -516,11 +666,6 @@ class TagsController < ApplicationController
|
|||||||
end
|
end
|
||||||
|
|
||||||
def record_tag_version! tag, event_type:, created_by_user:, name_changed: false, wiki_page: nil
|
def record_tag_version! tag, event_type:, created_by_user:, name_changed: false, wiki_page: nil
|
||||||
if tag.nico?
|
|
||||||
NicoTagVersionRecorder.record!(tag:, event_type:, created_by_user:)
|
|
||||||
return
|
|
||||||
end
|
|
||||||
|
|
||||||
TagVersionRecorder.record!(tag:, event_type:, created_by_user:)
|
TagVersionRecorder.record!(tag:, event_type:, created_by_user:)
|
||||||
|
|
||||||
return unless name_changed
|
return unless name_changed
|
||||||
@@ -542,7 +687,7 @@ class TagsController < ApplicationController
|
|||||||
return false
|
return false
|
||||||
end
|
end
|
||||||
|
|
||||||
target_tag_name = TagName.with_discarded.find_by(name:)
|
target_tag_name = TagName.find_by(name:)
|
||||||
return true if target_tag_name.nil?
|
return true if target_tag_name.nil?
|
||||||
return true if target_tag_name.canonical_id?
|
return true if target_tag_name.canonical_id?
|
||||||
|
|
||||||
@@ -554,17 +699,14 @@ class TagsController < ApplicationController
|
|||||||
return if name == tag.name
|
return if name == tag.name
|
||||||
|
|
||||||
current_tag_name = tag.tag_name
|
current_tag_name = tag.tag_name
|
||||||
target_tag_name = TagName.with_discarded.find_by(name:)
|
target_tag_name = TagName.find_by(name:)
|
||||||
|
|
||||||
if target_tag_name.nil?
|
if target_tag_name.nil?
|
||||||
current_tag_name.update!(name:)
|
current_tag_name.update!(name:)
|
||||||
return
|
return
|
||||||
end
|
end
|
||||||
|
|
||||||
promote_tag_alias!(
|
promote_tag_alias!(tag, current_tag_name:, promoted_tag_name: target_tag_name)
|
||||||
tag,
|
|
||||||
current_tag_name:,
|
|
||||||
promoted_tag_name: target_tag_name)
|
|
||||||
end
|
end
|
||||||
|
|
||||||
def promote_tag_alias! tag, current_tag_name:, promoted_tag_name:
|
def promote_tag_alias! tag, current_tag_name:, promoted_tag_name:
|
||||||
@@ -574,11 +716,9 @@ class TagsController < ApplicationController
|
|||||||
TagVersioning.ensure_snapshot!(old_owner_tag, created_by_user: current_user)
|
TagVersioning.ensure_snapshot!(old_owner_tag, created_by_user: current_user)
|
||||||
end
|
end
|
||||||
|
|
||||||
promoted_tag_name.undiscard! if promoted_tag_name.discarded?
|
|
||||||
promoted_tag_name.update!(canonical: nil)
|
promoted_tag_name.update!(canonical: nil)
|
||||||
|
|
||||||
TagName.with_discarded
|
TagName.where(canonical_id: current_tag_name.id)
|
||||||
.where(canonical_id: current_tag_name.id)
|
|
||||||
.where.not(id: promoted_tag_name.id)
|
.where.not(id: promoted_tag_name.id)
|
||||||
.find_each do |alias_tag_name|
|
.find_each do |alias_tag_name|
|
||||||
alias_tag_name.update!(canonical: promoted_tag_name)
|
alias_tag_name.update!(canonical: promoted_tag_name)
|
||||||
@@ -609,7 +749,7 @@ class TagsController < ApplicationController
|
|||||||
end
|
end
|
||||||
|
|
||||||
alias_names.each do |alias_name|
|
alias_names.each do |alias_name|
|
||||||
alias_tag_name = TagName.find_undiscard_or_create_by!(name: alias_name)
|
alias_tag_name = TagName.find_or_create_by!(name: alias_name)
|
||||||
affected_tags << alias_tag_name.canonical&.tag
|
affected_tags << alias_tag_name.canonical&.tag
|
||||||
end
|
end
|
||||||
|
|
||||||
@@ -624,7 +764,7 @@ class TagsController < ApplicationController
|
|||||||
end
|
end
|
||||||
|
|
||||||
alias_names.each do |alias_name|
|
alias_names.each do |alias_name|
|
||||||
alias_tag_name = TagName.find_undiscard_or_create_by!(name: alias_name)
|
alias_tag_name = TagName.find_or_create_by!(name: alias_name)
|
||||||
alias_tag_name.update!(canonical: tag.tag_name)
|
alias_tag_name.update!(canonical: tag.tag_name)
|
||||||
end
|
end
|
||||||
|
|
||||||
@@ -635,8 +775,7 @@ class TagsController < ApplicationController
|
|||||||
|
|
||||||
def update_parent_tags! tag, parent_names
|
def update_parent_tags! tag, parent_names
|
||||||
parent_tags = Tag.normalise_tags!(parent_names, with_tagme: false,
|
parent_tags = Tag.normalise_tags!(parent_names, with_tagme: false,
|
||||||
with_no_deerjikist: false,
|
with_no_deerjikist: false)
|
||||||
deny_nico: true)
|
|
||||||
|
|
||||||
old_parent_tags = tag.parents.to_a
|
old_parent_tags = tag.parents.to_a
|
||||||
|
|
||||||
@@ -653,6 +792,7 @@ class TagsController < ApplicationController
|
|||||||
end
|
end
|
||||||
|
|
||||||
def normalise_deerjikist_code platform, code
|
def normalise_deerjikist_code platform, code
|
||||||
|
code = code.to_s
|
||||||
return code if platform != 'youtube' || code[0] != '@'
|
return code if platform != 'youtube' || code[0] != '@'
|
||||||
|
|
||||||
url = "https://www.youtube.com/#{ code }"
|
url = "https://www.youtube.com/#{ code }"
|
||||||
@@ -669,6 +809,82 @@ class TagsController < ApplicationController
|
|||||||
nil
|
nil
|
||||||
end
|
end
|
||||||
|
|
||||||
|
def normalise_deerjikist_rows tag
|
||||||
|
rows = []
|
||||||
|
|
||||||
|
params[:_json].each.with_index do |item, index|
|
||||||
|
platform = item[:platform]
|
||||||
|
unless Deerjikist.platforms.key?(platform)
|
||||||
|
render_deerjikist_platform_invalid(index)
|
||||||
|
return rows
|
||||||
|
end
|
||||||
|
|
||||||
|
code = normalise_deerjikist_code(platform, item[:code])
|
||||||
|
deerjikist = Deerjikist.new(platform:, code:, tag:)
|
||||||
|
unless deerjikist.valid?
|
||||||
|
render_deerjikist_form_record_invalid(deerjikist, index)
|
||||||
|
return rows
|
||||||
|
end
|
||||||
|
|
||||||
|
rows << { index:, platform:, code: }
|
||||||
|
end
|
||||||
|
|
||||||
|
rows
|
||||||
|
end
|
||||||
|
|
||||||
|
def lock_deerjikists_for_tag_update tag_id, keys
|
||||||
|
clauses = ['tag_id = ?']
|
||||||
|
values = [tag_id]
|
||||||
|
|
||||||
|
keys.each do |platform, code|
|
||||||
|
clauses << '(platform = ? AND code = ?)'
|
||||||
|
values << platform << code
|
||||||
|
end
|
||||||
|
|
||||||
|
Deerjikist
|
||||||
|
.where(clauses.join(' OR '), *values)
|
||||||
|
.order(:platform, :code)
|
||||||
|
.lock
|
||||||
|
.to_a
|
||||||
|
end
|
||||||
|
|
||||||
|
def render_deerjikist_conflicts deerjikists, row_indexes_by_key, tag
|
||||||
|
conflicts = deerjikists.filter { |deerjikist| deerjikist.tag_id != tag.id }
|
||||||
|
return if conflicts.empty?
|
||||||
|
|
||||||
|
tag_names_by_id = Tag
|
||||||
|
.joins(:tag_name)
|
||||||
|
.where(id: conflicts.map(&:tag_id).uniq)
|
||||||
|
.pluck('tags.id', 'tag_names.name')
|
||||||
|
.to_h
|
||||||
|
fields = { }
|
||||||
|
|
||||||
|
conflicts.each do |deerjikist|
|
||||||
|
message = "この情報は既に「#{ tag_names_by_id[deerjikist.tag_id] }」に紐づいてゐます."
|
||||||
|
row_indexes_by_key[[deerjikist.platform, deerjikist.code]].each do |index|
|
||||||
|
field = :"deerjikists.#{ index }.code"
|
||||||
|
fields[field] ||= []
|
||||||
|
fields[field] << message
|
||||||
|
end
|
||||||
|
end
|
||||||
|
|
||||||
|
render_validation_error fields:
|
||||||
|
end
|
||||||
|
|
||||||
|
def render_deerjikist_platform_invalid index
|
||||||
|
render_validation_error fields: {
|
||||||
|
:"deerjikists.#{ index }.platform" => ['値が不正です.'],
|
||||||
|
}
|
||||||
|
end
|
||||||
|
|
||||||
|
def rows_by_key rows
|
||||||
|
rows.each_with_object({ }) do |row, result|
|
||||||
|
key = [row[:platform], row[:code]]
|
||||||
|
result[key] ||= []
|
||||||
|
result[key] << row[:index]
|
||||||
|
end
|
||||||
|
end
|
||||||
|
|
||||||
def render_deerjikist_form_record_invalid deerjikist, index
|
def render_deerjikist_form_record_invalid deerjikist, index
|
||||||
fields = { }
|
fields = { }
|
||||||
|
|
||||||
@@ -687,4 +903,20 @@ class TagsController < ApplicationController
|
|||||||
|
|
||||||
render_validation_error fields:
|
render_validation_error fields:
|
||||||
end
|
end
|
||||||
|
|
||||||
|
def external_tag_json tag
|
||||||
|
{ 'id' => tag.id,
|
||||||
|
'name' => "#{ tag.platform }:#{ tag.name }",
|
||||||
|
'category' => 'nico',
|
||||||
|
'deprecated_at' => nil,
|
||||||
|
'created_at' => tag.created_at,
|
||||||
|
'updated_at' => tag.created_at,
|
||||||
|
'post_count' => tag.post_count,
|
||||||
|
'matched_alias' => nil,
|
||||||
|
'aliases' => [],
|
||||||
|
'parents' => [],
|
||||||
|
'has_wiki' => false,
|
||||||
|
'material_id' => nil,
|
||||||
|
'has_deerjikists' => false }
|
||||||
|
end
|
||||||
end
|
end
|
||||||
|
|||||||
@@ -0,0 +1,61 @@
|
|||||||
|
class UserSettingsController < ApplicationController
|
||||||
|
wrap_parameters false
|
||||||
|
|
||||||
|
def show
|
||||||
|
return head :unauthorized unless current_user
|
||||||
|
|
||||||
|
render json: current_setting.serializable_hash
|
||||||
|
end
|
||||||
|
|
||||||
|
def update
|
||||||
|
return head :unauthorized unless current_user
|
||||||
|
|
||||||
|
raw_attributes = editable_raw_attributes
|
||||||
|
field_errors = validate_raw_attributes(raw_attributes)
|
||||||
|
return render_validation_error fields: field_errors if field_errors.present?
|
||||||
|
|
||||||
|
setting = current_setting
|
||||||
|
setting.assign_attributes(raw_attributes)
|
||||||
|
|
||||||
|
if setting.save
|
||||||
|
render json: setting.serializable_hash, status: :ok
|
||||||
|
else
|
||||||
|
render_validation_error setting
|
||||||
|
end
|
||||||
|
end
|
||||||
|
|
||||||
|
private
|
||||||
|
|
||||||
|
def current_setting
|
||||||
|
Setting.find_or_create_by!(user: current_user) do |setting|
|
||||||
|
setting.assign_attributes(Setting.defaults)
|
||||||
|
end
|
||||||
|
rescue ActiveRecord::RecordNotUnique
|
||||||
|
Setting.find_by!(user: current_user)
|
||||||
|
end
|
||||||
|
|
||||||
|
def editable_raw_attributes
|
||||||
|
request.request_parameters.slice(*Setting::EDITABLE_ATTRIBUTES)
|
||||||
|
end
|
||||||
|
|
||||||
|
def validate_raw_attributes raw_attributes
|
||||||
|
raw_attributes.each_with_object({ }) do |(key, value), errors|
|
||||||
|
next if value_matches_type?(key, value)
|
||||||
|
|
||||||
|
errors[key.to_sym] = ['値の型が不正です.']
|
||||||
|
end
|
||||||
|
end
|
||||||
|
|
||||||
|
def value_matches_type? key, value
|
||||||
|
case Setting::TYPE_BY_ATTRIBUTE.fetch(key)
|
||||||
|
when :string
|
||||||
|
value.is_a?(String)
|
||||||
|
when :integer
|
||||||
|
value.is_a?(Integer)
|
||||||
|
when :boolean
|
||||||
|
value == true || value == false
|
||||||
|
else
|
||||||
|
false
|
||||||
|
end
|
||||||
|
end
|
||||||
|
end
|
||||||
@@ -0,0 +1,38 @@
|
|||||||
|
class UserThemeSlotsController < ApplicationController
|
||||||
|
wrap_parameters false
|
||||||
|
|
||||||
|
def index
|
||||||
|
return head :unauthorized unless current_user
|
||||||
|
|
||||||
|
render json: current_user
|
||||||
|
.theme_slots
|
||||||
|
.order(:base_theme, :slot_no)
|
||||||
|
.map { |slot| slot.serializable_hash }
|
||||||
|
end
|
||||||
|
|
||||||
|
def update
|
||||||
|
return head :unauthorized unless current_user
|
||||||
|
|
||||||
|
base_theme = params[:base_theme].to_s
|
||||||
|
slot_no = params[:slot_no].to_i
|
||||||
|
tokens = params[:tokens]
|
||||||
|
|
||||||
|
unless UserThemeSlot::BASE_THEMES.include?(base_theme)
|
||||||
|
return render_validation_error fields: { base_theme: ['値が不正です.'] }
|
||||||
|
end
|
||||||
|
unless UserThemeSlot::SLOT_NOS.include?(slot_no)
|
||||||
|
return render_validation_error fields: { slot_no: ['値が不正です.'] }
|
||||||
|
end
|
||||||
|
unless tokens.is_a?(ActionController::Parameters) || tokens.is_a?(Hash)
|
||||||
|
return render_validation_error fields: { tokens: ['JSON object で指定してください.'] }
|
||||||
|
end
|
||||||
|
|
||||||
|
slot = UserThemeSlot.find_or_initialize_by(user: current_user,
|
||||||
|
base_theme:,
|
||||||
|
slot_no:)
|
||||||
|
slot.tokens = tokens.is_a?(ActionController::Parameters) ? tokens.to_unsafe_h : tokens
|
||||||
|
slot.save!
|
||||||
|
|
||||||
|
render json: slot.serializable_hash, status: :ok
|
||||||
|
end
|
||||||
|
end
|
||||||
@@ -94,7 +94,7 @@ class WikiPagesController < ApplicationController
|
|||||||
return render_unprocessable_entity('タイトルは必須です.', field: :title) if title.blank?
|
return render_unprocessable_entity('タイトルは必須です.', field: :title) if title.blank?
|
||||||
return render_unprocessable_entity('本文は必須です.', field: :body) if body.blank?
|
return render_unprocessable_entity('本文は必須です.', field: :body) if body.blank?
|
||||||
|
|
||||||
tag_name = TagName.find_undiscard_or_create_by!(name: title)
|
tag_name = TagName.find_or_create_by!(name: title)
|
||||||
|
|
||||||
page =
|
page =
|
||||||
Wiki::Commit.create_content!(
|
Wiki::Commit.create_content!(
|
||||||
|
|||||||
@@ -0,0 +1,20 @@
|
|||||||
|
class ExternalTag < ApplicationRecord
|
||||||
|
enum :platform, nico: 'nico'
|
||||||
|
validates :platform, presence: true, inclusion: { in: ExternalTag.platforms.keys }
|
||||||
|
|
||||||
|
has_many :post_external_tags, dependent: :delete_all
|
||||||
|
has_many :posts, through: :post_external_tags
|
||||||
|
|
||||||
|
has_many :nico_tag_versions, foreign_key: :tag_id, inverse_of: :external_tag
|
||||||
|
|
||||||
|
has_many :nico_tag_relations,
|
||||||
|
foreign_key: :nico_tag_id,
|
||||||
|
inverse_of: :nico_tag,
|
||||||
|
dependent: :destroy
|
||||||
|
|
||||||
|
has_many :linked_tags, through: :nico_tag_relations, source: :tag
|
||||||
|
|
||||||
|
def snapshot_linked_tag_names
|
||||||
|
linked_tags.joins(:tag_name).order('tag_names.name').pluck('tag_names.name')
|
||||||
|
end
|
||||||
|
end
|
||||||
@@ -1,24 +1,10 @@
|
|||||||
class NicoTagRelation < ApplicationRecord
|
class NicoTagRelation < ApplicationRecord
|
||||||
belongs_to :nico_tag, class_name: 'Tag'
|
belongs_to :nico_tag,
|
||||||
belongs_to :tag, class_name: 'Tag'
|
class_name: 'ExternalTag',
|
||||||
|
foreign_key: :nico_tag_id,
|
||||||
|
inverse_of: :nico_tag_relations
|
||||||
|
belongs_to :tag, class_name: 'Tag', foreign_key: :tag_id
|
||||||
|
|
||||||
validates :nico_tag_id, presence: true
|
validates :nico_tag_id, presence: true
|
||||||
validates :tag_id, presence: true
|
validates :tag_id, presence: true
|
||||||
|
|
||||||
validate :nico_tag_must_be_nico
|
|
||||||
validate :tag_mustnt_be_nico
|
|
||||||
|
|
||||||
private
|
|
||||||
|
|
||||||
def nico_tag_must_be_nico
|
|
||||||
if nico_tag && nico_tag.category != 'nico'
|
|
||||||
errors.add :nico_tag_id, 'タグのカテゴリがニコニコである必要があります.'
|
|
||||||
end
|
|
||||||
end
|
|
||||||
|
|
||||||
def tag_mustnt_be_nico
|
|
||||||
if tag && tag.category == 'nico'
|
|
||||||
errors.add :tag_id, '連携先タグのカテゴリはニコニコであってはなりません.'
|
|
||||||
end
|
|
||||||
end
|
|
||||||
end
|
end
|
||||||
|
|||||||
@@ -1,7 +1,7 @@
|
|||||||
class NicoTagVersion < ApplicationRecord
|
class NicoTagVersion < ApplicationRecord
|
||||||
include VersionRecord
|
include VersionRecord
|
||||||
|
|
||||||
belongs_to :tag
|
belongs_to :external_tag, foreign_key: :tag_id, inverse_of: :nico_tag_versions
|
||||||
|
|
||||||
validates :name, presence: true
|
validates :name, presence: true
|
||||||
end
|
end
|
||||||
|
|||||||
+412
-30
@@ -1,42 +1,81 @@
|
|||||||
class Post < ApplicationRecord
|
class Post < ApplicationRecord
|
||||||
|
require 'date'
|
||||||
require 'mini_magick'
|
require 'mini_magick'
|
||||||
|
require 'nokogiri'
|
||||||
require 'stringio'
|
require 'stringio'
|
||||||
|
require 'timeout'
|
||||||
|
|
||||||
def self.resized_thumbnail_attachment(upload)
|
class RemoteThumbnailFetchFailed < StandardError; end
|
||||||
|
|
||||||
|
ORIGINAL_CREATED_INVALID_MESSAGE = 'オリジナルの作成日時の形式が不正です.'.freeze
|
||||||
|
ORIGINAL_CREATED_MINUTE_PRECISION_MESSAGE =
|
||||||
|
'オリジナルの作成日時は分単位で入力してください.'.freeze
|
||||||
|
ORIGINAL_CREATED_ORDER_MESSAGE = 'オリジナルの作成日時の順番がをかしぃです.'.freeze
|
||||||
|
ORIGINAL_CREATED_MINIMUM_RANGE_MESSAGE =
|
||||||
|
'オリジナルの作成日時の範囲は1分以上必要です.'.freeze
|
||||||
|
REMOTE_SVG_CONTENT_TYPE = 'image/svg+xml'.freeze
|
||||||
|
MAX_SVG_DIMENSION = 4_096
|
||||||
|
MAX_SVG_PIXELS = 16_777_216
|
||||||
|
THUMBNAIL_PROCESS_TIMEOUT = 5.seconds
|
||||||
|
def self.resized_thumbnail_attachment(upload, content_type: nil)
|
||||||
upload.rewind
|
upload.rewind
|
||||||
image = MiniMagick::Image.read(upload.read)
|
bytes = upload.read
|
||||||
image.resize '180x180'
|
blob = Timeout.timeout(THUMBNAIL_PROCESS_TIMEOUT) do
|
||||||
image.format 'jpg'
|
image = image_for_thumbnail_upload(bytes, content_type:)
|
||||||
|
image.auto_orient
|
||||||
|
image.resize '180x180'
|
||||||
|
image.format 'jpg'
|
||||||
|
image.to_blob
|
||||||
|
end
|
||||||
|
|
||||||
{ io: StringIO.new(image.to_blob),
|
{ io: StringIO.new(blob),
|
||||||
filename: 'resized_thumbnail.jpg',
|
filename: 'resized_thumbnail.jpg',
|
||||||
content_type: 'image/jpeg' }
|
content_type: 'image/jpeg' }
|
||||||
|
rescue Timeout::Error
|
||||||
|
raise MiniMagick::Error, 'サムネイル画像の変換に失敗しました.'
|
||||||
ensure
|
ensure
|
||||||
upload.rewind
|
upload.rewind
|
||||||
end
|
end
|
||||||
|
|
||||||
|
def self.remote_thumbnail_attachment(raw_url)
|
||||||
|
response = Preview::ThumbnailFetcher.fetch_image_response(raw_url)
|
||||||
|
resized_thumbnail_attachment(
|
||||||
|
StringIO.new(response.body),
|
||||||
|
content_type: response.content_type)
|
||||||
|
rescue Preview::UrlSafety::UnsafeUrl,
|
||||||
|
Preview::ThumbnailFetcher::GenerationFailed,
|
||||||
|
Preview::HttpFetcher::FetchFailed,
|
||||||
|
Preview::HttpFetcher::FetchTimeout,
|
||||||
|
Preview::HttpFetcher::ResponseTooLarge,
|
||||||
|
Timeout::Error,
|
||||||
|
MiniMagick::Error => e
|
||||||
|
raise RemoteThumbnailFetchFailed, e.message
|
||||||
|
end
|
||||||
|
|
||||||
belongs_to :uploaded_user, class_name: 'User', optional: true
|
belongs_to :uploaded_user, class_name: 'User', optional: true
|
||||||
|
|
||||||
has_many :post_tags, dependent: :destroy, inverse_of: :post
|
has_many :post_tags, dependent: :destroy, inverse_of: :post
|
||||||
has_many :active_post_tags, -> { kept }, class_name: 'PostTag', inverse_of: :post
|
has_many :tags, through: :post_tags
|
||||||
has_many :post_tags_with_discarded, -> { with_discarded }, class_name: 'PostTag'
|
|
||||||
has_many :tags, through: :active_post_tags
|
|
||||||
has_many :active_tags, -> { where(tags: { deprecated_at: nil }) },
|
has_many :active_tags, -> { where(tags: { deprecated_at: nil }) },
|
||||||
through: :active_post_tags, source: :tag
|
through: :post_tags,
|
||||||
|
source: :tag
|
||||||
|
|
||||||
has_many :user_post_views, dependent: :delete_all
|
has_many :user_post_views, dependent: :delete_all
|
||||||
has_many :post_similarities, dependent: :delete_all
|
has_many :post_similarities, dependent: :delete_all
|
||||||
has_many :post_versions
|
has_many :post_versions
|
||||||
|
|
||||||
has_many :gekanator_guessed_games,
|
has_many :gekanator_guessed_games,
|
||||||
class_name: 'GekanatorGame',
|
class_name: 'GekanatorGame',
|
||||||
foreign_key: :guessed_post_id,
|
foreign_key: :guessed_post_id,
|
||||||
dependent: :delete_all,
|
dependent: :delete_all,
|
||||||
inverse_of: :guessed_post
|
inverse_of: :guessed_post
|
||||||
|
|
||||||
has_many :gekanator_correct_games,
|
has_many :gekanator_correct_games,
|
||||||
class_name: 'GekanatorGame',
|
class_name: 'GekanatorGame',
|
||||||
foreign_key: :correct_post_id,
|
foreign_key: :correct_post_id,
|
||||||
dependent: :delete_all,
|
dependent: :delete_all,
|
||||||
inverse_of: :correct_post
|
inverse_of: :correct_post
|
||||||
|
|
||||||
has_many :gekanator_question_examples, dependent: :delete_all
|
has_many :gekanator_question_examples, dependent: :delete_all
|
||||||
|
|
||||||
has_many :parent_post_implications,
|
has_many :parent_post_implications,
|
||||||
@@ -53,13 +92,16 @@ class Post < ApplicationRecord
|
|||||||
inverse_of: :parent_post
|
inverse_of: :parent_post
|
||||||
has_many :children, through: :child_post_implications, source: :post
|
has_many :children, through: :child_post_implications, source: :post
|
||||||
|
|
||||||
|
has_many :post_external_tags, dependent: :destroy
|
||||||
|
has_many :external_tags, through: :post_external_tags
|
||||||
|
|
||||||
has_one_attached :thumbnail
|
has_one_attached :thumbnail
|
||||||
|
|
||||||
attribute :version_no, :integer, default: 1
|
attribute :version_no, :integer, default: 1
|
||||||
|
|
||||||
before_validation :normalise_url
|
before_validation :normalise_url, if: :will_save_change_to_url?
|
||||||
|
|
||||||
validates :url, presence: true, uniqueness: true
|
validates :url, presence: true, uniqueness: true, length: { maximum: 768 }
|
||||||
validates :video_ms, numericality: { only_integer: true, greater_than: 0 }, allow_nil: true
|
validates :video_ms, numericality: { only_integer: true, greater_than: 0 }, allow_nil: true
|
||||||
|
|
||||||
validate :validate_original_created_range
|
validate :validate_original_created_range
|
||||||
@@ -86,7 +128,6 @@ class Post < ApplicationRecord
|
|||||||
|
|
||||||
def snapshot_tag_names
|
def snapshot_tag_names
|
||||||
post_tags
|
post_tags
|
||||||
.kept
|
|
||||||
.joins(tag: :tag_name)
|
.joins(tag: :tag_name)
|
||||||
.includes(:sections, tag: :tag_name)
|
.includes(:sections, tag: :tag_name)
|
||||||
.order('tag_names.name')
|
.order('tag_names.name')
|
||||||
@@ -100,8 +141,46 @@ class Post < ApplicationRecord
|
|||||||
end
|
end
|
||||||
end
|
end
|
||||||
|
|
||||||
|
def self.tag_snapshot_literal tag
|
||||||
|
sections = tag.fetch('sections', []).map do |sec|
|
||||||
|
begin_ms = sec.fetch('begin_ms')
|
||||||
|
end_ms = sec['end_ms']
|
||||||
|
|
||||||
|
"[#{ Post.ms_to_time(begin_ms) }-#{ end_ms ? Post.ms_to_time(end_ms) : '' }]"
|
||||||
|
end
|
||||||
|
|
||||||
|
"#{ tag.fetch('name') }#{ sections.join }"
|
||||||
|
end
|
||||||
|
|
||||||
|
def snapshot_tags_json
|
||||||
|
tag_snapshots =
|
||||||
|
post_tags
|
||||||
|
.joins(tag: :tag_name)
|
||||||
|
.includes(:sections, tag: :tag_name)
|
||||||
|
.order('tags.id')
|
||||||
|
.map { |pt|
|
||||||
|
{ 'tag_id' => pt.tag.id,
|
||||||
|
'version_no' => pt.tag.version_no,
|
||||||
|
'name' => pt.tag.name,
|
||||||
|
'category' => pt.tag.category,
|
||||||
|
'sections' => pt.sections.sort_by(&:begin_ms).map {
|
||||||
|
{ 'begin_ms' => _1.begin_ms, 'end_ms' => _1.end_ms }
|
||||||
|
} }
|
||||||
|
}
|
||||||
|
|
||||||
|
external_tag_snapshots =
|
||||||
|
post_external_tags.order(:external_tag_id).map {
|
||||||
|
{ 'external_tag_id' => _1.external_tag_id }
|
||||||
|
}
|
||||||
|
|
||||||
|
tag_snapshots + external_tag_snapshots
|
||||||
|
end
|
||||||
|
|
||||||
def self.section_literal section
|
def self.section_literal section
|
||||||
"[#{ Post.ms_to_time(section.begin_ms) }-#{ section.end_ms ? Post.ms_to_time(section.end_ms) : '' }]"
|
end_ms =
|
||||||
|
section.end_ms ? Post.ms_to_time(section.end_ms) : ''
|
||||||
|
|
||||||
|
"[#{ Post.ms_to_time(section.begin_ms) }-#{ end_ms }]"
|
||||||
end
|
end
|
||||||
|
|
||||||
def self.ms_to_time ms
|
def self.ms_to_time ms
|
||||||
@@ -118,7 +197,11 @@ class Post < ApplicationRecord
|
|||||||
'%d:%02d' % [min, s]
|
'%d:%02d' % [min, s]
|
||||||
end
|
end
|
||||||
|
|
||||||
remainder_ms.positive? ? "#{ base }.#{ remainder_ms.to_s.rjust(3, '0') }" : base
|
if remainder_ms.positive?
|
||||||
|
"#{ base }.#{ remainder_ms.to_s.rjust(3, '0') }"
|
||||||
|
else
|
||||||
|
base
|
||||||
|
end
|
||||||
end
|
end
|
||||||
|
|
||||||
def snapshot_parent_post_ids = parents.order(:id).pluck(:id)
|
def snapshot_parent_post_ids = parents.order(:id).pluck(:id)
|
||||||
@@ -140,19 +223,26 @@ class Post < ApplicationRecord
|
|||||||
thumbnail.attach(self.class.resized_thumbnail_attachment(StringIO.new(thumbnail.download)))
|
thumbnail.attach(self.class.resized_thumbnail_attachment(StringIO.new(thumbnail.download)))
|
||||||
end
|
end
|
||||||
|
|
||||||
|
def attach_thumbnail_from_url! raw_url
|
||||||
|
thumbnail.attach(self.class.remote_thumbnail_attachment(raw_url))
|
||||||
|
end
|
||||||
|
|
||||||
private
|
private
|
||||||
|
|
||||||
def validate_original_created_range
|
def validate_original_created_range
|
||||||
f = original_created_from
|
return if skip_original_created_validation?
|
||||||
b = original_created_before
|
|
||||||
return if f.blank? || b.blank?
|
|
||||||
|
|
||||||
f = Time.zone.parse(f) if String === f
|
f = parse_original_created_value(:original_created_from)
|
||||||
b = Time.zone.parse(b) if String === b
|
b = parse_original_created_value(:original_created_before)
|
||||||
return if !(f) || !(b)
|
return if f.nil? || b.nil?
|
||||||
|
|
||||||
if f >= b
|
if b <= f
|
||||||
errors.add :original_created_at, 'オリジナルの作成日時の順番がをかしぃです.'
|
errors.add :original_created_at, ORIGINAL_CREATED_ORDER_MESSAGE
|
||||||
|
return
|
||||||
|
end
|
||||||
|
|
||||||
|
if b - f < 1.minute
|
||||||
|
errors.add :original_created_at, ORIGINAL_CREATED_MINIMUM_RANGE_MESSAGE
|
||||||
end
|
end
|
||||||
end
|
end
|
||||||
|
|
||||||
@@ -173,15 +263,307 @@ class Post < ApplicationRecord
|
|||||||
def normalise_url
|
def normalise_url
|
||||||
return if url.blank?
|
return if url.blank?
|
||||||
|
|
||||||
self.url = url.strip
|
self.url = PostUrlNormaliser.normalise(url) || url.strip
|
||||||
|
end
|
||||||
|
|
||||||
u = URI.parse(url)
|
def self.image_for_thumbnail_upload(bytes, content_type: nil)
|
||||||
return unless u in URI::HTTP
|
if svg_content_type?(content_type) || svg_document_bytes?(bytes)
|
||||||
|
return decode_svg_thumbnail(bytes)
|
||||||
|
end
|
||||||
|
|
||||||
u.host = u.host.downcase if u.host
|
raise MiniMagick::Error, 'サムネイル画像の形式が不正です.' unless raster_thumbnail_bytes?(bytes)
|
||||||
u.path = u.path.sub(/\/\Z/, '') if u.path.present?
|
|
||||||
self.url = u.to_s
|
decode_raster_thumbnail(bytes)
|
||||||
rescue URI::InvalidURIError
|
end
|
||||||
;
|
|
||||||
|
def self.raster_thumbnail_bytes?(bytes)
|
||||||
|
raster_thumbnail_format(bytes).present?
|
||||||
|
end
|
||||||
|
|
||||||
|
def self.remote_thumbnail_image_bytes?(bytes, content_type: nil)
|
||||||
|
return true if svg_content_type?(content_type) || svg_document_bytes?(bytes)
|
||||||
|
|
||||||
|
raster_thumbnail_bytes?(bytes)
|
||||||
|
end
|
||||||
|
|
||||||
|
def self.svg_content_type?(content_type)
|
||||||
|
content_type.to_s.split(';', 2).first.to_s.downcase.strip == REMOTE_SVG_CONTENT_TYPE
|
||||||
|
end
|
||||||
|
|
||||||
|
def self.svg_document_bytes?(bytes)
|
||||||
|
document = Nokogiri::XML(
|
||||||
|
bytes,
|
||||||
|
nil,
|
||||||
|
nil,
|
||||||
|
Nokogiri::XML::ParseOptions::STRICT |
|
||||||
|
Nokogiri::XML::ParseOptions::NONET)
|
||||||
|
document.root&.name == 'svg'
|
||||||
|
rescue Nokogiri::XML::SyntaxError
|
||||||
|
false
|
||||||
|
end
|
||||||
|
|
||||||
|
def self.decode_raster_thumbnail(bytes)
|
||||||
|
MiniMagick::Image.read(bytes)
|
||||||
|
end
|
||||||
|
|
||||||
|
def self.decode_svg_thumbnail(bytes)
|
||||||
|
MiniMagick::Image.read(sanitised_svg_bytes(bytes))
|
||||||
|
end
|
||||||
|
|
||||||
|
def self.raster_thumbnail_format(bytes)
|
||||||
|
binary = bytes.to_s.b
|
||||||
|
return 'jpeg' if binary.start_with?("\xFF\xD8\xFF".b)
|
||||||
|
return 'png' if binary.start_with?("\x89PNG\r\n\x1A\n".b)
|
||||||
|
return 'gif' if binary.start_with?('GIF87a'.b) || binary.start_with?('GIF89a'.b)
|
||||||
|
return 'webp' if binary.bytesize >= 12 &&
|
||||||
|
binary.start_with?('RIFF'.b) &&
|
||||||
|
binary.byteslice(8, 4) == 'WEBP'
|
||||||
|
|
||||||
|
nil
|
||||||
|
end
|
||||||
|
|
||||||
|
def self.sanitised_svg_bytes(bytes)
|
||||||
|
parse_options =
|
||||||
|
Nokogiri::XML::ParseOptions::STRICT |
|
||||||
|
Nokogiri::XML::ParseOptions::NONET
|
||||||
|
document = Nokogiri::XML(
|
||||||
|
bytes,
|
||||||
|
nil,
|
||||||
|
nil,
|
||||||
|
parse_options)
|
||||||
|
root = document.root
|
||||||
|
raise MiniMagick::Error, 'SVG が不正です.' if root == nil || root.name != 'svg'
|
||||||
|
raise MiniMagick::Error, 'SVG が不正です.' if document.internal_subset != nil
|
||||||
|
raise MiniMagick::Error, 'SVG が不正です.' if svg_uses_disallowed_features?(document)
|
||||||
|
|
||||||
|
width, height = svg_dimensions(root)
|
||||||
|
raise MiniMagick::Error, 'SVG が大きすぎます.' if width == nil || height == nil
|
||||||
|
if width > MAX_SVG_DIMENSION || height > MAX_SVG_DIMENSION || width * height > MAX_SVG_PIXELS
|
||||||
|
raise MiniMagick::Error, 'SVG が大きすぎます.'
|
||||||
|
end
|
||||||
|
|
||||||
|
document.to_xml
|
||||||
|
rescue Nokogiri::XML::SyntaxError
|
||||||
|
raise MiniMagick::Error, 'SVG が不正です.'
|
||||||
|
end
|
||||||
|
|
||||||
|
def self.svg_uses_disallowed_features?(document)
|
||||||
|
document.traverse.any? do |node|
|
||||||
|
next false unless node.element?
|
||||||
|
|
||||||
|
name = node.name.to_s.downcase
|
||||||
|
next true if name == 'script' || name == 'foreignobject'
|
||||||
|
next style_contains_disallowed_urls?(node.text.to_s) if name == 'style'
|
||||||
|
|
||||||
|
node.attribute_nodes.any? do |attribute|
|
||||||
|
attribute_name = attribute.name.to_s.downcase
|
||||||
|
attribute_value = attribute.value.to_s
|
||||||
|
attribute_name.start_with?('on') ||
|
||||||
|
external_svg_reference?(attribute_name, attribute_value)
|
||||||
|
end
|
||||||
|
end
|
||||||
|
end
|
||||||
|
|
||||||
|
def self.external_svg_reference?(attribute_name, attribute_value)
|
||||||
|
if ['href', 'xlink:href', 'src'].include?(attribute_name)
|
||||||
|
return external_svg_url?(attribute_value)
|
||||||
|
end
|
||||||
|
return style_contains_disallowed_urls?(attribute_value) if attribute_name == 'style'
|
||||||
|
return svg_url_function_disallowed?(attribute_value) if attribute_value.match?(/url\s*\(/i)
|
||||||
|
|
||||||
|
false
|
||||||
|
end
|
||||||
|
|
||||||
|
def self.external_svg_url?(value)
|
||||||
|
stripped = value.to_s.strip
|
||||||
|
return false if stripped.blank? || stripped.start_with?('#')
|
||||||
|
|
||||||
|
true
|
||||||
|
end
|
||||||
|
|
||||||
|
def self.style_contains_disallowed_urls?(value)
|
||||||
|
text = value.to_s
|
||||||
|
text.match?(/@import/i) || svg_url_function_disallowed?(text)
|
||||||
|
end
|
||||||
|
|
||||||
|
def self.svg_url_function_disallowed?(value)
|
||||||
|
value.to_s.scan(/url\s*\(([^)]*)\)/i).flatten.any? do |entry|
|
||||||
|
reference =
|
||||||
|
entry.to_s.strip
|
||||||
|
.delete_prefix("'")
|
||||||
|
.delete_prefix('"')
|
||||||
|
.delete_suffix("'")
|
||||||
|
.delete_suffix('"')
|
||||||
|
reference.present? && !(reference.start_with?('#'))
|
||||||
|
end
|
||||||
|
end
|
||||||
|
|
||||||
|
def self.svg_dimensions(root)
|
||||||
|
width = svg_length_to_pixels(root['width'])
|
||||||
|
height = svg_length_to_pixels(root['height'])
|
||||||
|
return [width, height] if width && height
|
||||||
|
|
||||||
|
view_box = root['viewBox'].to_s.strip.split(/\s+/).map { Float(_1) rescue nil }
|
||||||
|
return [nil, nil] if view_box.length != 4 || view_box.any?(&:nil?)
|
||||||
|
return [nil, nil] unless view_box[2].finite? && view_box[2].positive?
|
||||||
|
return [nil, nil] unless view_box[3].finite? && view_box[3].positive?
|
||||||
|
|
||||||
|
[view_box[2], view_box[3]]
|
||||||
|
end
|
||||||
|
|
||||||
|
def self.svg_length_to_pixels(value)
|
||||||
|
return nil if value.blank?
|
||||||
|
|
||||||
|
matched = /\A([0-9]+(?:\.[0-9]+)?)(px)?\z/i.match(value.to_s.strip)
|
||||||
|
return nil if matched == nil
|
||||||
|
|
||||||
|
pixels = Float(matched[1])
|
||||||
|
return nil unless pixels.finite? && pixels.positive?
|
||||||
|
|
||||||
|
pixels
|
||||||
|
rescue ArgumentError
|
||||||
|
nil
|
||||||
|
end
|
||||||
|
|
||||||
|
private_class_method :image_for_thumbnail_upload,
|
||||||
|
:svg_content_type?,
|
||||||
|
:decode_raster_thumbnail,
|
||||||
|
:decode_svg_thumbnail,
|
||||||
|
:raster_thumbnail_format,
|
||||||
|
:sanitised_svg_bytes,
|
||||||
|
:svg_uses_disallowed_features?,
|
||||||
|
:external_svg_reference?,
|
||||||
|
:external_svg_url?,
|
||||||
|
:style_contains_disallowed_urls?,
|
||||||
|
:svg_url_function_disallowed?,
|
||||||
|
:svg_dimensions,
|
||||||
|
:svg_length_to_pixels
|
||||||
|
|
||||||
|
def parse_original_created_value field
|
||||||
|
raw_value = public_send("#{ field }_before_type_cast")
|
||||||
|
value = public_send(field)
|
||||||
|
return nil if raw_value.blank? && value.blank?
|
||||||
|
|
||||||
|
time =
|
||||||
|
case raw_value
|
||||||
|
when String
|
||||||
|
parse_original_created_string(raw_value)
|
||||||
|
when Time, ActiveSupport::TimeWithZone
|
||||||
|
raw_value.in_time_zone
|
||||||
|
else
|
||||||
|
value&.in_time_zone
|
||||||
|
end
|
||||||
|
if time.nil?
|
||||||
|
errors.add field, ORIGINAL_CREATED_INVALID_MESSAGE
|
||||||
|
return nil
|
||||||
|
end
|
||||||
|
unless minute_precision_time?(time)
|
||||||
|
errors.add field, ORIGINAL_CREATED_MINUTE_PRECISION_MESSAGE
|
||||||
|
return nil
|
||||||
|
end
|
||||||
|
time
|
||||||
|
end
|
||||||
|
|
||||||
|
def parse_original_created_string raw_value
|
||||||
|
value = raw_value.to_s.strip
|
||||||
|
return nil if value.blank?
|
||||||
|
|
||||||
|
match = value.match(/\A(\d{4})-(\d{2})-(\d{2})T(\d{2}):(\d{2})\z/)
|
||||||
|
if match
|
||||||
|
year = match[1].to_i
|
||||||
|
month = match[2].to_i
|
||||||
|
day = match[3].to_i
|
||||||
|
hour = match[4].to_i
|
||||||
|
minute = match[5].to_i
|
||||||
|
return nil unless valid_original_created_components?(year, month, day, hour, minute, 0)
|
||||||
|
|
||||||
|
return Time.zone.local(year, month, day, hour, minute)
|
||||||
|
end
|
||||||
|
|
||||||
|
match =
|
||||||
|
value.match(
|
||||||
|
/
|
||||||
|
\A(\d{4})-(\d{2})-(\d{2})T(\d{2}):(\d{2})
|
||||||
|
(?::(\d{2})(?:\.(\d+))?)?
|
||||||
|
(Z|[+-]\d{2}:?\d{2})?
|
||||||
|
\z
|
||||||
|
/x)
|
||||||
|
return nil if match.nil?
|
||||||
|
|
||||||
|
year = match[1].to_i
|
||||||
|
month = match[2].to_i
|
||||||
|
day = match[3].to_i
|
||||||
|
hour = match[4].to_i
|
||||||
|
minute = match[5].to_i
|
||||||
|
second = match[6]&.to_i || 0
|
||||||
|
fraction = match[7]
|
||||||
|
offset = match[8]
|
||||||
|
return nil unless valid_original_created_components?(year, month, day, hour, minute, second)
|
||||||
|
return nil if offset.present? && !(valid_original_created_offset?(offset))
|
||||||
|
|
||||||
|
if offset.present?
|
||||||
|
return Time.new(
|
||||||
|
year,
|
||||||
|
month,
|
||||||
|
day,
|
||||||
|
hour,
|
||||||
|
minute,
|
||||||
|
second + Rational(parse_original_created_nanoseconds(fraction), 1_000_000_000),
|
||||||
|
normalise_original_created_offset(offset)).in_time_zone
|
||||||
|
end
|
||||||
|
|
||||||
|
Time.zone.local(
|
||||||
|
year,
|
||||||
|
month,
|
||||||
|
day,
|
||||||
|
hour,
|
||||||
|
minute,
|
||||||
|
second).change(nsec: parse_original_created_nanoseconds(fraction))
|
||||||
|
rescue ArgumentError, TypeError
|
||||||
|
nil
|
||||||
|
end
|
||||||
|
|
||||||
|
def minute_precision_time? value
|
||||||
|
value.sec.zero? && value.nsec.zero?
|
||||||
|
end
|
||||||
|
|
||||||
|
def valid_original_created_components? year, month, day, hour, minute, second
|
||||||
|
return false unless Date.valid_date?(year, month, day)
|
||||||
|
return false unless hour.between?(0, 23)
|
||||||
|
return false unless minute.between?(0, 59)
|
||||||
|
return false unless second.between?(0, 59)
|
||||||
|
|
||||||
|
true
|
||||||
|
end
|
||||||
|
|
||||||
|
def valid_original_created_offset? value
|
||||||
|
match = value.match(/\A([+-])(\d{2}):?(\d{2})\z/)
|
||||||
|
return true if value == 'Z'
|
||||||
|
return false if match.nil?
|
||||||
|
|
||||||
|
hours = match[2].to_i
|
||||||
|
minutes = match[3].to_i
|
||||||
|
hours.between?(0, 23) && minutes.between?(0, 59)
|
||||||
|
end
|
||||||
|
|
||||||
|
def parse_original_created_nanoseconds value
|
||||||
|
return 0 if value.blank?
|
||||||
|
|
||||||
|
digits = value[0, 9].ljust(9, '0')
|
||||||
|
Integer(digits, 10)
|
||||||
|
end
|
||||||
|
|
||||||
|
def normalise_original_created_offset value
|
||||||
|
return '+00:00' if value == 'Z'
|
||||||
|
|
||||||
|
value.match?(/\A[+-]\d{2}:\d{2}\z/) ? value : "#{ value[0, 3] }:#{ value[3, 2] }"
|
||||||
|
end
|
||||||
|
|
||||||
|
def skip_original_created_validation?
|
||||||
|
return false if new_record?
|
||||||
|
return false if will_save_change_to_original_created_from?
|
||||||
|
return false if will_save_change_to_original_created_before?
|
||||||
|
|
||||||
|
true
|
||||||
end
|
end
|
||||||
end
|
end
|
||||||
|
|||||||
@@ -0,0 +1,4 @@
|
|||||||
|
class PostExternalTag < ApplicationRecord
|
||||||
|
belongs_to :post
|
||||||
|
belongs_to :external_tag, counter_cache: :post_count
|
||||||
|
end
|
||||||
@@ -1,14 +1,7 @@
|
|||||||
class PostTag < ApplicationRecord
|
class PostTag < ApplicationRecord
|
||||||
include Discard::Model
|
|
||||||
|
|
||||||
before_destroy do
|
|
||||||
raise ActiveRecord::ReadOnlyRecord, '消さないでください.'
|
|
||||||
end
|
|
||||||
|
|
||||||
belongs_to :post
|
belongs_to :post
|
||||||
belongs_to :tag, counter_cache: :post_count
|
belongs_to :tag, counter_cache: :post_count
|
||||||
belongs_to :created_user, class_name: 'User', optional: true
|
belongs_to :created_user, class_name: 'User', optional: true
|
||||||
belongs_to :deleted_user, class_name: 'User', optional: true
|
|
||||||
|
|
||||||
has_many :sections, -> { order(:begin_ms) }, class_name: 'PostTagSection',
|
has_many :sections, -> { order(:begin_ms) }, class_name: 'PostTagSection',
|
||||||
foreign_key: [:post_id, :tag_id],
|
foreign_key: [:post_id, :tag_id],
|
||||||
@@ -18,18 +11,5 @@ class PostTag < ApplicationRecord
|
|||||||
|
|
||||||
validates :post_id, presence: true
|
validates :post_id, presence: true
|
||||||
validates :tag_id, presence: true
|
validates :tag_id, presence: true
|
||||||
validates :post_id, uniqueness: {
|
validates :post_id, uniqueness: { scope: :tag_id }
|
||||||
scope: :tag_id,
|
|
||||||
conditions: -> { where(discarded_at: nil) } }
|
|
||||||
|
|
||||||
def discard_by! deleted_user
|
|
||||||
return self if discarded?
|
|
||||||
|
|
||||||
transaction do
|
|
||||||
update!(discarded_at: Time.current, deleted_user:)
|
|
||||||
Tag.where(id: tag_id).update_all('post_count = GREATEST(post_count - 1, 0)')
|
|
||||||
end
|
|
||||||
|
|
||||||
self
|
|
||||||
end
|
|
||||||
end
|
end
|
||||||
|
|||||||
@@ -4,10 +4,10 @@ class PostTagSection < ApplicationRecord
|
|||||||
belongs_to :post
|
belongs_to :post
|
||||||
belongs_to :tag
|
belongs_to :tag
|
||||||
|
|
||||||
belongs_to :post_tag, -> { kept }, foreign_key: [:post_id, :tag_id],
|
belongs_to :post_tag, foreign_key: [:post_id, :tag_id],
|
||||||
primary_key: [:post_id, :tag_id],
|
primary_key: [:post_id, :tag_id],
|
||||||
inverse_of: :sections,
|
inverse_of: :sections,
|
||||||
optional: true
|
optional: true
|
||||||
|
|
||||||
validates :post_id, presence: true
|
validates :post_id, presence: true
|
||||||
validates :tag_id, presence: true
|
validates :tag_id, presence: true
|
||||||
|
|||||||
@@ -0,0 +1,163 @@
|
|||||||
|
class PostUrlSanitisationRule < ApplicationRecord
|
||||||
|
include Discard::Model
|
||||||
|
|
||||||
|
class InvalidUrlError < StandardError
|
||||||
|
attr_reader :invalid_rows
|
||||||
|
|
||||||
|
def initialize(invalid_rows)
|
||||||
|
@invalid_rows = invalid_rows
|
||||||
|
ids = invalid_rows.map { _1.fetch(:post_id) }.join(', ')
|
||||||
|
super("post URL sanitisation produced invalid URLs for posts #{ ids }")
|
||||||
|
end
|
||||||
|
end
|
||||||
|
|
||||||
|
class UrlConflictError < StandardError
|
||||||
|
attr_reader :conflicts
|
||||||
|
|
||||||
|
def initialize(conflicts)
|
||||||
|
@conflicts = conflicts
|
||||||
|
urls = conflicts.map { _1.fetch(:url) }.uniq.join(', ')
|
||||||
|
super("post URL sanitisation conflicts detected for #{ urls }")
|
||||||
|
end
|
||||||
|
end
|
||||||
|
|
||||||
|
self.primary_key = :priority
|
||||||
|
|
||||||
|
default_scope -> { kept }
|
||||||
|
|
||||||
|
validates :source_pattern, presence: true, uniqueness: true
|
||||||
|
|
||||||
|
validate :source_pattern_must_be_regexp
|
||||||
|
|
||||||
|
class << self
|
||||||
|
def sanitise(url) = sanitise_with_rules(url, rules)
|
||||||
|
|
||||||
|
def apply!
|
||||||
|
rewrites = nil
|
||||||
|
|
||||||
|
Post.transaction do
|
||||||
|
compiled_rules = rules
|
||||||
|
|
||||||
|
rewrites = Post.order(:id)
|
||||||
|
.lock('FOR UPDATE')
|
||||||
|
.pluck(:id, :url)
|
||||||
|
.map do |post_id, original_url|
|
||||||
|
{ post_id:,
|
||||||
|
original_url:,
|
||||||
|
sanitised_url: sanitise_with_rules(original_url, compiled_rules) }
|
||||||
|
end
|
||||||
|
|
||||||
|
invalid_rows = rewrites.filter { invalid_sanitised_url?(_1.fetch(:sanitised_url)) }
|
||||||
|
.map { { post_id: _1.fetch(:post_id),
|
||||||
|
original_url: _1.fetch(:original_url),
|
||||||
|
sanitised_url: _1.fetch(:sanitised_url) } }
|
||||||
|
raise InvalidUrlError.new(invalid_rows) if invalid_rows.present?
|
||||||
|
|
||||||
|
conflicts = build_conflicts(rewrites)
|
||||||
|
raise UrlConflictError.new(conflicts) if conflicts.present?
|
||||||
|
|
||||||
|
changed = rewrites.filter { _1.fetch(:original_url) != _1.fetch(:sanitised_url) }
|
||||||
|
return if changed.empty?
|
||||||
|
|
||||||
|
token = SecureRandom.hex(6)
|
||||||
|
|
||||||
|
changed.each do |row|
|
||||||
|
Post.where(id: row.fetch(:post_id))
|
||||||
|
.update_all(url: temporary_url_for(row.fetch(:post_id), token))
|
||||||
|
end
|
||||||
|
|
||||||
|
changed.each do |row|
|
||||||
|
Post.where(id: row.fetch(:post_id))
|
||||||
|
.update_all(url: row.fetch(:sanitised_url))
|
||||||
|
end
|
||||||
|
end
|
||||||
|
rescue ActiveRecord::RecordNotUnique => error
|
||||||
|
conflicts = build_persisted_conflicts(rewrites)
|
||||||
|
raise error if conflicts.empty?
|
||||||
|
|
||||||
|
raise UrlConflictError.new(conflicts), cause: error
|
||||||
|
end
|
||||||
|
|
||||||
|
private
|
||||||
|
|
||||||
|
def rules = kept.order(:priority).map { |r| [Regexp.new(r.source_pattern), r.replacement] }
|
||||||
|
|
||||||
|
def sanitise_with_rules(url, compiled_rules)
|
||||||
|
compiled_rules.reduce(url.dup) do |value, (pattern, replacement)|
|
||||||
|
value.sub(pattern, replacement)
|
||||||
|
end
|
||||||
|
end
|
||||||
|
|
||||||
|
def temporary_url_for(post_id, token) =
|
||||||
|
"https://post-url-sanitising.invalid/#{ token }/#{ post_id }"
|
||||||
|
|
||||||
|
def invalid_sanitised_url?(url)
|
||||||
|
return true if url.blank?
|
||||||
|
return true if url.length > 768
|
||||||
|
|
||||||
|
parsed = URI.parse(url)
|
||||||
|
return true if !(parsed in URI::HTTP)
|
||||||
|
return true if parsed.host.blank?
|
||||||
|
|
||||||
|
false
|
||||||
|
rescue URI::InvalidURIError
|
||||||
|
true
|
||||||
|
end
|
||||||
|
|
||||||
|
def build_conflicts(rewrites)
|
||||||
|
rewrites
|
||||||
|
.group_by { _1.fetch(:sanitised_url).downcase }
|
||||||
|
.values
|
||||||
|
.filter { _1.size > 1 }
|
||||||
|
.flatten
|
||||||
|
.map { { url: _1.fetch(:sanitised_url),
|
||||||
|
post_id: _1.fetch(:post_id),
|
||||||
|
original_url: _1.fetch(:original_url) } }
|
||||||
|
end
|
||||||
|
|
||||||
|
def build_persisted_conflicts(rewrites)
|
||||||
|
return [] if rewrites.blank?
|
||||||
|
|
||||||
|
target_rows = rewrites.filter { _1.fetch(:original_url) != _1.fetch(:sanitised_url) }
|
||||||
|
target_keys = target_rows.map { _1.fetch(:sanitised_url).downcase }.uniq
|
||||||
|
return [] if target_keys.empty?
|
||||||
|
|
||||||
|
target_pairs = target_rows.to_h do |row|
|
||||||
|
[row.fetch(:post_id), row.fetch(:sanitised_url).downcase]
|
||||||
|
end
|
||||||
|
|
||||||
|
persisted_rows = Post.order(:id)
|
||||||
|
.where('LOWER(url) IN (?)', target_keys)
|
||||||
|
.pluck(:id, :url)
|
||||||
|
.reject { |post_id, original_url| target_pairs[post_id] == original_url.downcase }
|
||||||
|
.map { |post_id, original_url|
|
||||||
|
{ url: original_url,
|
||||||
|
post_id:,
|
||||||
|
original_url:,
|
||||||
|
conflict_key: original_url.downcase }
|
||||||
|
}
|
||||||
|
|
||||||
|
target_conflicts = target_rows.map { { url: _1.fetch(:sanitised_url),
|
||||||
|
post_id: _1.fetch(:post_id),
|
||||||
|
original_url: _1.fetch(:original_url),
|
||||||
|
conflict_key: _1.fetch(:sanitised_url).downcase } }
|
||||||
|
|
||||||
|
(persisted_rows + target_conflicts)
|
||||||
|
.group_by { _1.fetch(:conflict_key) }
|
||||||
|
.values
|
||||||
|
.filter { _1.size > 1 }
|
||||||
|
.flatten
|
||||||
|
.map { _1.except(:conflict_key) }
|
||||||
|
end
|
||||||
|
end
|
||||||
|
|
||||||
|
private
|
||||||
|
|
||||||
|
def source_pattern_must_be_regexp
|
||||||
|
return if source_pattern.blank?
|
||||||
|
|
||||||
|
Regexp.new(source_pattern)
|
||||||
|
rescue RegexpError
|
||||||
|
errors.add :source_pattern, '変な正規表現だね〜(笑)'
|
||||||
|
end
|
||||||
|
end
|
||||||
@@ -1,7 +1,52 @@
|
|||||||
class Setting < ApplicationRecord
|
class Setting < ApplicationRecord
|
||||||
|
THEMES = ['system', 'light', 'dark'].freeze
|
||||||
|
|
||||||
|
# These remain in the typed settings schema to preserve existing backend
|
||||||
|
# work. The common `/users/settings` page currently surfaces only `theme`.
|
||||||
|
AUTO_FETCH_MODES = ['auto', 'manual', 'off'].freeze
|
||||||
|
WIKI_EDITOR_MODES = ['split', 'write', 'preview'].freeze
|
||||||
|
|
||||||
|
STRING_ATTRIBUTES = [
|
||||||
|
'theme',
|
||||||
|
'auto_fetch_title',
|
||||||
|
'auto_fetch_thumbnail',
|
||||||
|
'wiki_editor_mode',
|
||||||
|
].freeze
|
||||||
|
INTEGER_ATTRIBUTES = [].freeze
|
||||||
|
BOOLEAN_ATTRIBUTES = [].freeze
|
||||||
|
EDITABLE_ATTRIBUTES =
|
||||||
|
(STRING_ATTRIBUTES + INTEGER_ATTRIBUTES + BOOLEAN_ATTRIBUTES).freeze
|
||||||
|
TYPE_BY_ATTRIBUTE = {
|
||||||
|
'theme' => :string,
|
||||||
|
'auto_fetch_title' => :string,
|
||||||
|
'auto_fetch_thumbnail' => :string,
|
||||||
|
'wiki_editor_mode' => :string,
|
||||||
|
}.freeze
|
||||||
|
|
||||||
belongs_to :user
|
belongs_to :user
|
||||||
|
|
||||||
validates :user_id, presence: true
|
validates :user_id, presence: true
|
||||||
validates :key, presence: true, length: { maximum: 255 }
|
validates :user_id, uniqueness: true
|
||||||
validates :value, presence: true
|
|
||||||
|
validates :theme, inclusion: { in: THEMES }
|
||||||
|
validates :auto_fetch_title, inclusion: { in: AUTO_FETCH_MODES }
|
||||||
|
validates :auto_fetch_thumbnail, inclusion: { in: AUTO_FETCH_MODES }
|
||||||
|
validates :wiki_editor_mode, inclusion: { in: WIKI_EDITOR_MODES }
|
||||||
|
|
||||||
|
def self.defaults
|
||||||
|
{
|
||||||
|
theme: 'system',
|
||||||
|
auto_fetch_title: 'manual',
|
||||||
|
auto_fetch_thumbnail: 'manual',
|
||||||
|
wiki_editor_mode: 'split',
|
||||||
|
}
|
||||||
|
end
|
||||||
|
|
||||||
|
def self.serializable_attributes
|
||||||
|
EDITABLE_ATTRIBUTES.map(&:to_sym)
|
||||||
|
end
|
||||||
|
|
||||||
|
def serializable_hash(options = nil)
|
||||||
|
super({ only: self.class.serializable_attributes }.merge(options || { }))
|
||||||
|
end
|
||||||
end
|
end
|
||||||
|
|||||||
+19
-42
@@ -2,8 +2,6 @@ require 'set'
|
|||||||
|
|
||||||
|
|
||||||
class Tag < ApplicationRecord
|
class Tag < ApplicationRecord
|
||||||
include MyDiscard
|
|
||||||
|
|
||||||
class NicoTagNormalisationError < ArgumentError
|
class NicoTagNormalisationError < ArgumentError
|
||||||
;
|
;
|
||||||
end
|
end
|
||||||
@@ -28,15 +26,12 @@ class Tag < ApplicationRecord
|
|||||||
end
|
end
|
||||||
|
|
||||||
has_many :post_tags, inverse_of: :tag
|
has_many :post_tags, inverse_of: :tag
|
||||||
has_many :active_post_tags, -> { kept }, class_name: 'PostTag', inverse_of: :tag
|
has_many :posts, through: :post_tags
|
||||||
has_many :post_tags_with_discarded, -> { with_discarded }, class_name: 'PostTag'
|
|
||||||
has_many :posts, through: :active_post_tags
|
|
||||||
|
|
||||||
has_many :nico_tag_relations, foreign_key: :nico_tag_id, dependent: :destroy
|
|
||||||
has_many :linked_tags, through: :nico_tag_relations, source: :tag
|
|
||||||
|
|
||||||
has_many :reversed_nico_tag_relations,
|
has_many :reversed_nico_tag_relations,
|
||||||
class_name: 'NicoTagRelation', foreign_key: :tag_id, dependent: :destroy
|
class_name: 'NicoTagRelation',
|
||||||
|
foreign_key: :tag_id,
|
||||||
|
dependent: :destroy
|
||||||
has_many :linked_nico_tags, through: :reversed_nico_tag_relations, source: :nico_tag
|
has_many :linked_nico_tags, through: :reversed_nico_tag_relations, source: :nico_tag
|
||||||
|
|
||||||
has_many :tag_implications, foreign_key: :parent_tag_id, dependent: :destroy
|
has_many :tag_implications, foreign_key: :parent_tag_id, dependent: :destroy
|
||||||
@@ -54,7 +49,6 @@ class Tag < ApplicationRecord
|
|||||||
has_many :materials
|
has_many :materials
|
||||||
|
|
||||||
has_many :tag_versions
|
has_many :tag_versions
|
||||||
has_many :nico_tag_versions
|
|
||||||
|
|
||||||
belongs_to :tag_name
|
belongs_to :tag_name
|
||||||
delegate :wiki_page, to: :tag_name
|
delegate :wiki_page, to: :tag_name
|
||||||
@@ -69,17 +63,13 @@ class Tag < ApplicationRecord
|
|||||||
character: 'character',
|
character: 'character',
|
||||||
general: 'general',
|
general: 'general',
|
||||||
material: 'material',
|
material: 'material',
|
||||||
nico: 'nico',
|
|
||||||
meta: 'meta'
|
meta: 'meta'
|
||||||
|
|
||||||
validates :category, presence: true, inclusion: { in: Tag.categories.keys }
|
validates :category, presence: true, inclusion: { in: Tag.categories.keys }
|
||||||
|
|
||||||
validate :nico_tag_name_must_start_with_nico
|
validate :tag_name_mustnt_start_with_nico
|
||||||
validate :tag_name_must_be_canonical
|
validate :tag_name_must_be_canonical
|
||||||
validate :category_must_be_deerjikist_with_deerjikists
|
validate :category_must_be_deerjikist_with_deerjikists
|
||||||
validate :nico_tags_cannot_be_deprecated
|
|
||||||
|
|
||||||
scope :nico_tags, -> { nico }
|
|
||||||
|
|
||||||
CATEGORY_PREFIXES = {
|
CATEGORY_PREFIXES = {
|
||||||
'general:' => :general,
|
'general:' => :general,
|
||||||
@@ -114,10 +104,9 @@ class Tag < ApplicationRecord
|
|||||||
|
|
||||||
def self.normalise_tags! tag_names, with_tagme: true,
|
def self.normalise_tags! tag_names, with_tagme: true,
|
||||||
with_no_deerjikist: true,
|
with_no_deerjikist: true,
|
||||||
deny_nico: true,
|
|
||||||
deny_deprecated: false,
|
deny_deprecated: false,
|
||||||
with_sections: false
|
with_sections: false
|
||||||
if deny_nico && tag_names.any? { |n| n.downcase.start_with?('nico:') }
|
if tag_names.any? { |n| n.downcase.start_with?('nico:') }
|
||||||
raise NicoTagNormalisationError
|
raise NicoTagNormalisationError
|
||||||
end
|
end
|
||||||
|
|
||||||
@@ -234,10 +223,10 @@ class Tag < ApplicationRecord
|
|||||||
end
|
end
|
||||||
|
|
||||||
def self.find_or_create_by_tag_name! name, category:
|
def self.find_or_create_by_tag_name! name, category:
|
||||||
tn = TagName.find_undiscard_or_create_by!(name: name.to_s.strip)
|
tn = TagName.find_or_create_by!(name: name.to_s.strip)
|
||||||
tn = tn.canonical if tn.canonical_id?
|
tn = tn.canonical if tn.canonical_id?
|
||||||
|
|
||||||
Tag.find_undiscard_or_create_by!(tag_name_id: tn.id) do |t|
|
Tag.find_or_create_by!(tag_name_id: tn.id) do |t|
|
||||||
t.category = category
|
t.category = category
|
||||||
end
|
end
|
||||||
rescue ActiveRecord::RecordNotUnique
|
rescue ActiveRecord::RecordNotUnique
|
||||||
@@ -259,11 +248,11 @@ class Tag < ApplicationRecord
|
|||||||
|
|
||||||
TagVersioning.ensure_snapshot!(source_tag, created_by_user:)
|
TagVersioning.ensure_snapshot!(source_tag, created_by_user:)
|
||||||
|
|
||||||
source_tag.post_tags.kept.find_each do |source_pt|
|
source_tag.post_tags.find_each do |source_pt|
|
||||||
post_id = source_pt.post_id
|
post_id = source_pt.post_id
|
||||||
affected_post_ids << post_id
|
affected_post_ids << post_id
|
||||||
source_pt.discard_by!(created_by_user)
|
source_pt.destroy!
|
||||||
unless PostTag.kept.exists?(post_id:, tag: target_tag)
|
unless PostTag.exists?(post_id:, tag: target_tag)
|
||||||
PostTag.create!(post_id:, tag: target_tag)
|
PostTag.create!(post_id:, tag: target_tag)
|
||||||
end
|
end
|
||||||
end
|
end
|
||||||
@@ -275,14 +264,10 @@ class Tag < ApplicationRecord
|
|||||||
end
|
end
|
||||||
|
|
||||||
TagVersioning.record!(source_tag, event_type: :discard, created_by_user:)
|
TagVersioning.record!(source_tag, event_type: :discard, created_by_user:)
|
||||||
source_tag.discard!
|
source_tag.destroy!
|
||||||
|
|
||||||
if source_tag.nico?
|
source_tag_name.update_columns(canonical_id: target_tag.tag_name_id,
|
||||||
source_tag_name.discard!
|
updated_at: Time.current)
|
||||||
else
|
|
||||||
source_tag_name.update_columns(canonical_id: target_tag.tag_name_id,
|
|
||||||
updated_at: Time.current)
|
|
||||||
end
|
|
||||||
|
|
||||||
TagVersioning.record!(target_tag, event_type: :update, created_by_user:)
|
TagVersioning.record!(target_tag, event_type: :update, created_by_user:)
|
||||||
end
|
end
|
||||||
@@ -293,13 +278,13 @@ class Tag < ApplicationRecord
|
|||||||
end
|
end
|
||||||
|
|
||||||
# 投稿件数を再集計
|
# 投稿件数を再集計
|
||||||
target_tag.update_columns(post_count: PostTag.kept.where(tag: target_tag).count)
|
target_tag.update_columns(post_count: PostTag.where(tag: target_tag).count)
|
||||||
end
|
end
|
||||||
|
|
||||||
target_tag.reload
|
target_tag.reload
|
||||||
end
|
end
|
||||||
|
|
||||||
def snapshot_aliases = tag_name.aliases.kept.order(:name).pluck(:name)
|
def snapshot_aliases = tag_name.aliases.order(:name).pluck(:name)
|
||||||
|
|
||||||
def snapshot_parent_tag_ids = parents.order(:id).pluck(:id)
|
def snapshot_parent_tag_ids = parents.order(:id).pluck(:id)
|
||||||
|
|
||||||
@@ -309,11 +294,9 @@ class Tag < ApplicationRecord
|
|||||||
|
|
||||||
private
|
private
|
||||||
|
|
||||||
def nico_tag_name_must_start_with_nico
|
def tag_name_mustnt_start_with_nico
|
||||||
n = name.to_s
|
if name.to_s.downcase.start_with?('nico:')
|
||||||
if ((nico? && !(n.downcase.start_with?('nico:'))) ||
|
errors.add :name, 'タグの命名規則に反してゐます.'
|
||||||
(!(nico?) && n.downcase.start_with?('nico:')))
|
|
||||||
errors.add :name, 'ニコニコ・タグの命名規則に反してゐます.'
|
|
||||||
end
|
end
|
||||||
end
|
end
|
||||||
|
|
||||||
@@ -354,10 +337,4 @@ class Tag < ApplicationRecord
|
|||||||
|
|
||||||
total_s * 1_000 + match[:ms].to_s.ljust(3, '0')[0, 3].to_i
|
total_s * 1_000 + match[:ms].to_s.ljust(3, '0')[0, 3].to_i
|
||||||
end
|
end
|
||||||
|
|
||||||
def nico_tags_cannot_be_deprecated
|
|
||||||
if nico? && deprecated_at.present?
|
|
||||||
errors.add :deprecated_at, 'ニコタグは廃止できません.'
|
|
||||||
end
|
|
||||||
end
|
|
||||||
end
|
end
|
||||||
|
|||||||
@@ -1,6 +1,4 @@
|
|||||||
class TagName < ApplicationRecord
|
class TagName < ApplicationRecord
|
||||||
include MyDiscard
|
|
||||||
|
|
||||||
has_one :tag
|
has_one :tag
|
||||||
has_one :wiki_page
|
has_one :wiki_page
|
||||||
|
|
||||||
|
|||||||
@@ -10,8 +10,9 @@ class TagNameSanitisationRule < ApplicationRecord
|
|||||||
validate :source_pattern_must_be_regexp
|
validate :source_pattern_must_be_regexp
|
||||||
|
|
||||||
class << self
|
class << self
|
||||||
def sanitise(name) =
|
def sanitise(name)
|
||||||
rules.reduce(name.dup) { |name, (pattern, replacement)| name.gsub(pattern, replacement) }
|
rules.reduce(name.dup) { |name, (pattern, replacement)| name.gsub(pattern, replacement) }
|
||||||
|
end
|
||||||
|
|
||||||
def apply!
|
def apply!
|
||||||
TagName.find_each do |tn|
|
TagName.find_each do |tn|
|
||||||
@@ -32,7 +33,7 @@ class TagNameSanitisationRule < ApplicationRecord
|
|||||||
elsif source_tag
|
elsif source_tag
|
||||||
source_tag.update_columns(tag_name_id: existing_tn.id, updated_at: Time.current)
|
source_tag.update_columns(tag_name_id: existing_tn.id, updated_at: Time.current)
|
||||||
end
|
end
|
||||||
tn.discard!
|
tn.destroy!
|
||||||
|
|
||||||
next
|
next
|
||||||
end
|
end
|
||||||
|
|||||||
@@ -7,7 +7,8 @@ class User < ApplicationRecord
|
|||||||
|
|
||||||
has_many :created_posts,
|
has_many :created_posts,
|
||||||
class_name: 'Post', foreign_key: :uploaded_user_id, dependent: :nullify
|
class_name: 'Post', foreign_key: :uploaded_user_id, dependent: :nullify
|
||||||
has_many :settings
|
has_one :setting, dependent: :destroy
|
||||||
|
has_many :theme_slots, class_name: 'UserThemeSlot', dependent: :destroy
|
||||||
has_many :user_ips, dependent: :destroy
|
has_many :user_ips, dependent: :destroy
|
||||||
has_many :ip_addresses, through: :user_ips
|
has_many :ip_addresses, through: :user_ips
|
||||||
has_many :user_post_views, dependent: :destroy
|
has_many :user_post_views, dependent: :destroy
|
||||||
|
|||||||
@@ -0,0 +1,27 @@
|
|||||||
|
class UserThemeSlot < ApplicationRecord
|
||||||
|
BASE_THEMES = ['light', 'dark'].freeze
|
||||||
|
SLOT_NOS = [1, 2, 3].freeze
|
||||||
|
|
||||||
|
belongs_to :user
|
||||||
|
|
||||||
|
validates :user_id, presence: true
|
||||||
|
validates :base_theme, presence: true, inclusion: { in: BASE_THEMES }
|
||||||
|
validates :slot_no, presence: true, inclusion: { in: SLOT_NOS }
|
||||||
|
validates :tokens, presence: true
|
||||||
|
validates :user_id, uniqueness: { scope: [:base_theme, :slot_no] }
|
||||||
|
|
||||||
|
validate :tokens_must_be_object
|
||||||
|
|
||||||
|
def serializable_hash(options = nil)
|
||||||
|
hash = { only: [:base_theme, :slot_no, :tokens, :created_at, :updated_at] }
|
||||||
|
super(hash.merge(options || {}))
|
||||||
|
end
|
||||||
|
|
||||||
|
private
|
||||||
|
|
||||||
|
def tokens_must_be_object
|
||||||
|
return if tokens.is_a?(Hash)
|
||||||
|
|
||||||
|
errors.add(:tokens, 'は JSON object で指定してください.')
|
||||||
|
end
|
||||||
|
end
|
||||||
@@ -0,0 +1,20 @@
|
|||||||
|
module ExternalTagRepr
|
||||||
|
module_function
|
||||||
|
|
||||||
|
def base tag
|
||||||
|
{ 'id' => tag.id,
|
||||||
|
'name' => "#{ tag.platform }:#{ tag.name }",
|
||||||
|
'category' => 'nico',
|
||||||
|
'post_count' => tag.post_count,
|
||||||
|
'created_at' => tag.created_at,
|
||||||
|
'updated_at' => tag.created_at,
|
||||||
|
'deprecated_at' => nil,
|
||||||
|
'aliases' => [],
|
||||||
|
'parents' => [],
|
||||||
|
'has_wiki' => false,
|
||||||
|
'material_id' => nil,
|
||||||
|
'has_deerjikists' => false }
|
||||||
|
end
|
||||||
|
|
||||||
|
def inline(tag) = base(tag)
|
||||||
|
end
|
||||||
@@ -0,0 +1,19 @@
|
|||||||
|
# frozen_string_literal: true
|
||||||
|
|
||||||
|
|
||||||
|
module PostCompactRepr
|
||||||
|
module_function
|
||||||
|
|
||||||
|
def base post, host: nil
|
||||||
|
return nil if post.nil?
|
||||||
|
|
||||||
|
PostRepr
|
||||||
|
.common(post, host:)
|
||||||
|
.slice(
|
||||||
|
'id',
|
||||||
|
'title',
|
||||||
|
'url',
|
||||||
|
'thumbnail',
|
||||||
|
'thumbnail_base')
|
||||||
|
end
|
||||||
|
end
|
||||||
@@ -17,8 +17,13 @@ module PostRepr
|
|||||||
|
|
||||||
module_function
|
module_function
|
||||||
|
|
||||||
def base post, current_user = nil
|
def base post, current_user = nil, host: nil
|
||||||
json = common(post)
|
json =
|
||||||
|
if host.present?
|
||||||
|
common(post, host:)
|
||||||
|
else
|
||||||
|
common(post)
|
||||||
|
end
|
||||||
json['tags'] = tag_json(post)
|
json['tags'] = tag_json(post)
|
||||||
json['uploaded_user'] = post.uploaded_user && UserRepr.base(post.uploaded_user)
|
json['uploaded_user'] = post.uploaded_user && UserRepr.base(post.uploaded_user)
|
||||||
json['viewed'] = current_user ? current_user.viewed?(post) : false
|
json['viewed'] = current_user ? current_user.viewed?(post) : false
|
||||||
@@ -26,50 +31,98 @@ module PostRepr
|
|||||||
end
|
end
|
||||||
|
|
||||||
def detail post, current_user = nil, parent_posts: [], child_posts: [],
|
def detail post, current_user = nil, parent_posts: [], child_posts: [],
|
||||||
sibling_posts: { }, related: []
|
sibling_posts: { }, related: [], host: nil
|
||||||
base(post, current_user).merge(
|
if host.present?
|
||||||
'parent_posts' => cards(parent_posts),
|
base(post, current_user, host:).merge(
|
||||||
'child_posts' => cards(child_posts),
|
'parent_posts' => cards(parent_posts, host:),
|
||||||
'sibling_posts' => sibling_posts.transform_keys(&:to_s).transform_values { |posts|
|
'child_posts' => cards(child_posts, host:),
|
||||||
cards(posts)
|
'sibling_posts' => sibling_posts.transform_keys(&:to_s).transform_values { |posts|
|
||||||
},
|
cards(posts, host:)
|
||||||
'related' => cards(related))
|
},
|
||||||
|
'related' => cards(related, host:))
|
||||||
|
else
|
||||||
|
base(post, current_user).merge(
|
||||||
|
'parent_posts' => cards(parent_posts),
|
||||||
|
'child_posts' => cards(child_posts),
|
||||||
|
'sibling_posts' => sibling_posts.transform_keys(&:to_s).transform_values { |posts|
|
||||||
|
cards(posts)
|
||||||
|
},
|
||||||
|
'related' => cards(related))
|
||||||
|
end
|
||||||
end
|
end
|
||||||
|
|
||||||
def card post
|
def card post, host: nil
|
||||||
common(post).merge('parent_posts' => [], 'child_posts' => [])
|
if host.present?
|
||||||
|
common(post, host:).merge('parent_posts' => [], 'child_posts' => [])
|
||||||
|
else
|
||||||
|
common(post).merge('parent_posts' => [], 'child_posts' => [])
|
||||||
|
end
|
||||||
end
|
end
|
||||||
|
|
||||||
def cards posts
|
def cards posts, host: nil
|
||||||
posts.map { |post| card(post) }
|
if host.present?
|
||||||
|
posts.map { |post| card(post, host:) }
|
||||||
|
else
|
||||||
|
posts.map { |post| card(post) }
|
||||||
|
end
|
||||||
end
|
end
|
||||||
|
|
||||||
def many posts, current_user = nil
|
def many posts, current_user = nil, host: nil
|
||||||
posts.map { |p| base(p, current_user) }
|
if host.present?
|
||||||
|
posts.map { |p| base(p, current_user, host:) }
|
||||||
|
else
|
||||||
|
posts.map { |p| base(p, current_user) }
|
||||||
|
end
|
||||||
end
|
end
|
||||||
|
|
||||||
def common post
|
def common post, host: nil
|
||||||
BASE_FIELDS.to_h { |field| [field.to_s, post.public_send(field)] }
|
BASE_FIELDS.to_h { |field| [field.to_s, post.public_send(field)] }
|
||||||
.merge('thumbnail' => thumbnail_url(post))
|
.merge(
|
||||||
|
'thumbnail' =>
|
||||||
|
if host.present?
|
||||||
|
thumbnail_url(post, host:)
|
||||||
|
else
|
||||||
|
thumbnail_url(post)
|
||||||
|
end)
|
||||||
end
|
end
|
||||||
|
|
||||||
def tag_json post
|
def tag_json post
|
||||||
post
|
internal_tags =
|
||||||
.active_post_tags
|
post
|
||||||
|
.post_tags
|
||||||
.reject { _1.tag.deprecated? }
|
.reject { _1.tag.deprecated? }
|
||||||
.sort_by { _1.tag.name }
|
.sort_by { _1.tag.name }
|
||||||
.map { |post_tag|
|
.map { |post_tag|
|
||||||
TagRepr.inline(post_tag.tag).merge(
|
TagRepr.inline(post_tag.tag).merge(
|
||||||
'children' => [],
|
'children' => [],
|
||||||
'sections' => post_tag.sections.as_json(only: [:begin_ms, :end_ms]))
|
'sections' => post_tag.sections.as_json(only: [:begin_ms, :end_ms]))
|
||||||
}
|
}
|
||||||
|
|
||||||
|
external_tags =
|
||||||
|
post
|
||||||
|
.external_tags
|
||||||
|
.sort_by { _1.name }
|
||||||
|
.map { ExternalTagRepr.base(_1).merge('children' => [], 'sections' => []) }
|
||||||
|
|
||||||
|
internal_tags + external_tags
|
||||||
end
|
end
|
||||||
|
|
||||||
def thumbnail_url post
|
def thumbnail_url post, host: nil
|
||||||
return nil unless post.thumbnail.attached?
|
return nil unless post.thumbnail.attached?
|
||||||
|
|
||||||
Rails.application.routes.url_helpers.rails_blob_url(post.thumbnail, only_path: false)
|
options = { only_path: false }
|
||||||
rescue
|
options[:host] = host if host.present?
|
||||||
|
|
||||||
|
Rails.application.routes.url_helpers.rails_storage_proxy_url(post.thumbnail, **options)
|
||||||
|
rescue ActionController::UrlGenerationError, ArgumentError, URI::InvalidURIError => e
|
||||||
|
payload = {
|
||||||
|
post_id: post.id,
|
||||||
|
attachment_id: post.thumbnail.attachment&.id,
|
||||||
|
blob_id: post.thumbnail.blob&.id,
|
||||||
|
error_class: e.class,
|
||||||
|
message: e.message }
|
||||||
|
|
||||||
|
Rails.logger.warn("PostRepr.thumbnail_url failed #{ payload.to_json }")
|
||||||
nil
|
nil
|
||||||
end
|
end
|
||||||
end
|
end
|
||||||
|
|||||||
@@ -1,19 +1,22 @@
|
|||||||
class NicoTagVersionRecorder < VersionRecorder
|
class NicoTagVersionRecorder < VersionRecorder
|
||||||
def self.record! tag:, event_type:, created_by_user:
|
def self.record! external_tag:, event_type:, created_by_user:
|
||||||
new(tag:, event_type:, created_by_user:).record!
|
new(external_tag:, event_type:, created_by_user:).record!
|
||||||
end
|
end
|
||||||
|
|
||||||
def initialize tag:, event_type:, created_by_user:
|
def initialize external_tag:, event_type:, created_by_user:
|
||||||
super(record: tag, event_type:, created_by_user:)
|
super(record: external_tag, event_type:, created_by_user:)
|
||||||
end
|
end
|
||||||
|
|
||||||
private
|
private
|
||||||
|
|
||||||
def version_class = NicoTagVersion
|
def version_class = NicoTagVersion
|
||||||
def version_association = :nico_tag_versions
|
def version_association = :nico_tag_versions
|
||||||
def record_key = :tag
|
def record_key = :external_tag
|
||||||
|
|
||||||
def snapshot_attributes
|
def snapshot_attributes
|
||||||
{ name: @record.name, linked_tags: @record.snapshot_linked_tag_names.join(' ') }
|
{ name: "#{ @record.platform }:#{ @record.name }",
|
||||||
|
linked_tags: @record.snapshot_linked_tag_names.join(' ') }
|
||||||
end
|
end
|
||||||
|
|
||||||
|
def tracks_version_no_on_record? = false
|
||||||
end
|
end
|
||||||
|
|||||||
@@ -0,0 +1,202 @@
|
|||||||
|
class PostBulkCreator
|
||||||
|
def initialize actor:, posts:, thumbnails:, host: nil
|
||||||
|
@actor_id = actor.id
|
||||||
|
@posts = posts
|
||||||
|
@thumbnails = thumbnails
|
||||||
|
@host = host
|
||||||
|
end
|
||||||
|
|
||||||
|
def run
|
||||||
|
results = Array.new(@posts.length)
|
||||||
|
mutex = Mutex.new
|
||||||
|
next_index = 0
|
||||||
|
|
||||||
|
workers = Array.new(2) do
|
||||||
|
Thread.new do
|
||||||
|
Rails.application.executor.wrap do
|
||||||
|
ActiveRecord::Base.connection_pool.with_connection do
|
||||||
|
actor = User.find(@actor_id)
|
||||||
|
loop do
|
||||||
|
index = nil
|
||||||
|
begin
|
||||||
|
index = mutex.synchronize do
|
||||||
|
current = next_index
|
||||||
|
next_index += 1
|
||||||
|
current
|
||||||
|
end
|
||||||
|
break if index >= @posts.length
|
||||||
|
|
||||||
|
attributes = @posts[index]
|
||||||
|
results[index] = create_row(actor, attributes, index)
|
||||||
|
rescue StandardError => e
|
||||||
|
Rails.logger.error(
|
||||||
|
"post_bulk_creator_worker_failure #{ { error: e.class.name,
|
||||||
|
message: e.message,
|
||||||
|
index: }.to_json }")
|
||||||
|
results[index] = {
|
||||||
|
status: 'failed',
|
||||||
|
recoverable: false,
|
||||||
|
errors: { base: ['登録中にエラーが発生しました.'] },
|
||||||
|
base_errors: [] }
|
||||||
|
end
|
||||||
|
end
|
||||||
|
end
|
||||||
|
end
|
||||||
|
end
|
||||||
|
end
|
||||||
|
workers.each(&:join)
|
||||||
|
|
||||||
|
results.each_index do |index|
|
||||||
|
next if results[index].present?
|
||||||
|
|
||||||
|
results[index] = {
|
||||||
|
status: 'failed',
|
||||||
|
recoverable: false,
|
||||||
|
errors: { base: ['登録中にエラーが発生しました.'] },
|
||||||
|
base_errors: [] }
|
||||||
|
end
|
||||||
|
|
||||||
|
{ results: }
|
||||||
|
end
|
||||||
|
|
||||||
|
private
|
||||||
|
|
||||||
|
def create_row actor, attributes, index
|
||||||
|
preflight =
|
||||||
|
PostCreatePreflight.new(
|
||||||
|
attributes: attributes,
|
||||||
|
thumbnail: thumbnail_for(index, attributes),
|
||||||
|
host: @host).run
|
||||||
|
if preflight[:existing_post_id].present?
|
||||||
|
return {
|
||||||
|
status: 'skipped',
|
||||||
|
existing_post_id: preflight[:existing_post_id],
|
||||||
|
existing_post: preflight[:existing_post] }
|
||||||
|
end
|
||||||
|
|
||||||
|
post = PostCreator.new(
|
||||||
|
actor: actor,
|
||||||
|
attributes: normalised_attributes(attributes, preflight, index)).create!
|
||||||
|
result = {
|
||||||
|
status: 'created',
|
||||||
|
post: { id: post.id } }
|
||||||
|
result[:field_warnings] = preflight[:field_warnings] if preflight[:field_warnings].present?
|
||||||
|
result[:base_warnings] = preflight[:base_warnings] if preflight[:base_warnings].present?
|
||||||
|
result
|
||||||
|
rescue PostCreatePreflight::ValidationFailed => e
|
||||||
|
{
|
||||||
|
status: 'failed',
|
||||||
|
recoverable: true,
|
||||||
|
errors: e.fields,
|
||||||
|
base_errors: e.base_errors }
|
||||||
|
rescue ActiveRecord::RecordInvalid => e
|
||||||
|
existing_post = existing_post_for_race(attributes, e.record)
|
||||||
|
if existing_post.present?
|
||||||
|
return {
|
||||||
|
status: 'skipped',
|
||||||
|
existing_post_id: existing_post[:id],
|
||||||
|
existing_post: existing_post }
|
||||||
|
end
|
||||||
|
|
||||||
|
{
|
||||||
|
status: 'failed',
|
||||||
|
recoverable: true,
|
||||||
|
errors: e.record.errors.to_hash,
|
||||||
|
base_errors: e.record.errors[:base] }
|
||||||
|
rescue ActiveRecord::RecordNotUnique => e
|
||||||
|
if e.message.include?('index_posts_on_url')
|
||||||
|
existing_post = existing_post_for_race(attributes)
|
||||||
|
return {
|
||||||
|
status: 'skipped',
|
||||||
|
existing_post_id: existing_post[:id],
|
||||||
|
existing_post: existing_post } if existing_post.present?
|
||||||
|
end
|
||||||
|
|
||||||
|
Rails.logger.error(
|
||||||
|
"post_bulk_creator_record_not_unique #{ { error: e.class.name,
|
||||||
|
message: e.message }.to_json }")
|
||||||
|
{
|
||||||
|
status: 'failed',
|
||||||
|
recoverable: false,
|
||||||
|
errors: { base: ['登録中にエラーが発生しました.'] },
|
||||||
|
base_errors: [] }
|
||||||
|
rescue Tag::NicoTagNormalisationError
|
||||||
|
{
|
||||||
|
status: 'failed',
|
||||||
|
recoverable: true,
|
||||||
|
errors: { tags: ['ニコニコ・タグは直接指定できません.'] },
|
||||||
|
base_errors: [] }
|
||||||
|
rescue Tag::DeprecatedTagNormalisationError
|
||||||
|
{
|
||||||
|
status: 'failed',
|
||||||
|
recoverable: true,
|
||||||
|
errors: { tags: ['廃止済みタグは付与できません.'] },
|
||||||
|
base_errors: [] }
|
||||||
|
rescue PostCreator::VideoMsParseError
|
||||||
|
{
|
||||||
|
status: 'failed',
|
||||||
|
recoverable: true,
|
||||||
|
errors: { video_ms: ['動画時間の記法が不正です.'] },
|
||||||
|
base_errors: [] }
|
||||||
|
rescue Post::RemoteThumbnailFetchFailed
|
||||||
|
{
|
||||||
|
status: 'failed',
|
||||||
|
recoverable: true,
|
||||||
|
errors: { thumbnail_base: ['サムネイル画像の取得に失敗しました.'] },
|
||||||
|
base_errors: [] }
|
||||||
|
rescue ArgumentError => e
|
||||||
|
{
|
||||||
|
status: 'failed',
|
||||||
|
recoverable: true,
|
||||||
|
errors: { base: [e.message] },
|
||||||
|
base_errors: [] }
|
||||||
|
rescue StandardError => e
|
||||||
|
Rails.logger.error(
|
||||||
|
"post_bulk_creator_failure #{ { error: e.class.name,
|
||||||
|
message: e.message }.to_json }")
|
||||||
|
{
|
||||||
|
status: 'failed',
|
||||||
|
recoverable: false,
|
||||||
|
errors: { base: ['登録中にエラーが発生しました.'] },
|
||||||
|
base_errors: [] }
|
||||||
|
end
|
||||||
|
|
||||||
|
def normalised_attributes attributes, preflight, index
|
||||||
|
{
|
||||||
|
url: preflight[:url],
|
||||||
|
title: preflight[:title],
|
||||||
|
thumbnail_base: preflight[:thumbnail_base],
|
||||||
|
thumbnail: thumbnail_for(index, attributes),
|
||||||
|
tags: preflight[:tags],
|
||||||
|
parent_post_ids: preflight[:parent_post_ids],
|
||||||
|
original_created_from: preflight[:original_created_from],
|
||||||
|
original_created_before: preflight[:original_created_before],
|
||||||
|
duration: preflight[:duration],
|
||||||
|
video_ms: preflight[:video_ms],
|
||||||
|
direct_tag_specs: preflight[:direct_tag_specs],
|
||||||
|
default_tag_specs: preflight[:default_tag_specs],
|
||||||
|
snapshot_tag_specs: preflight[:snapshot_tag_specs],
|
||||||
|
post_tag_specs: preflight[:post_tag_specs],
|
||||||
|
tag_sections: preflight[:tag_sections],
|
||||||
|
normalised_parent_post_ids: preflight[:normalised_parent_post_ids] }
|
||||||
|
end
|
||||||
|
|
||||||
|
def thumbnail_for index, attributes
|
||||||
|
return nil if attributes['thumbnail_base'].present? || attributes[:thumbnail_base].present?
|
||||||
|
|
||||||
|
@thumbnails[index]
|
||||||
|
end
|
||||||
|
|
||||||
|
def existing_post_for_race attributes, record = nil
|
||||||
|
return nil if record.present? && !(record.errors.of_kind?(:url, :taken))
|
||||||
|
|
||||||
|
normal_url = PostUrlNormaliser.normalise(attributes['url'] || attributes[:url])
|
||||||
|
return nil if normal_url.blank?
|
||||||
|
|
||||||
|
compact_existing_post(Post.with_attached_thumbnail.find_by(url: normal_url))
|
||||||
|
end
|
||||||
|
|
||||||
|
def compact_existing_post post
|
||||||
|
PostCompactRepr.base(post, host: @host)
|
||||||
|
end
|
||||||
|
end
|
||||||
@@ -0,0 +1,269 @@
|
|||||||
|
class PostCreatePlan
|
||||||
|
VIDEO_TAG_NAME = '動画'.freeze
|
||||||
|
TAGME_TAG_NAME = 'タグ希望'.freeze
|
||||||
|
NO_DEERJIKIST_TAG_NAME = 'ニジラー情報不詳'.freeze
|
||||||
|
|
||||||
|
def initialize attributes:
|
||||||
|
@attributes = attributes.symbolize_keys
|
||||||
|
@existing_tags_by_name = nil
|
||||||
|
end
|
||||||
|
|
||||||
|
def build!
|
||||||
|
direct_tag_specs, tag_sections = parse_direct_tag_specs
|
||||||
|
default_tag_specs = build_default_tag_specs(direct_tag_specs)
|
||||||
|
snapshot_tag_specs = merge_tag_specs(direct_tag_specs + default_tag_specs)
|
||||||
|
preload_existing_tags_by_name!(snapshot_tag_specs.map { _1[:name] })
|
||||||
|
validate_new_tag_specs!(snapshot_tag_specs)
|
||||||
|
post_tag_specs = expand_parent_tag_specs(snapshot_tag_specs)
|
||||||
|
video_ms = normalise_video_ms(snapshot_tag_specs)
|
||||||
|
validate_video_sections!(video_ms, tag_sections)
|
||||||
|
parent_post_ids = normalise_parent_post_ids
|
||||||
|
validate_parent_post_ids!(parent_post_ids)
|
||||||
|
|
||||||
|
{ url: @attributes[:url],
|
||||||
|
title: @attributes[:title].to_s,
|
||||||
|
thumbnail_base: @attributes[:thumbnail_base].presence,
|
||||||
|
original_created_from: @attributes[:original_created_from].presence,
|
||||||
|
original_created_before: @attributes[:original_created_before].presence,
|
||||||
|
tags: serialised_tags(direct_tag_specs, tag_sections),
|
||||||
|
display_tags: display_tags(direct_tag_specs, tag_sections),
|
||||||
|
duration: @attributes[:duration].to_s,
|
||||||
|
video_ms: video_ms,
|
||||||
|
parent_post_ids: parent_post_ids.join(' '),
|
||||||
|
direct_tag_specs: direct_tag_specs,
|
||||||
|
default_tag_specs: default_tag_specs,
|
||||||
|
snapshot_tag_specs: snapshot_tag_specs,
|
||||||
|
post_tag_specs: post_tag_specs,
|
||||||
|
tag_sections: tag_sections,
|
||||||
|
normalised_parent_post_ids: parent_post_ids }
|
||||||
|
end
|
||||||
|
|
||||||
|
private
|
||||||
|
|
||||||
|
def tag_names = @attributes[:tags].to_s.split
|
||||||
|
|
||||||
|
def parse_direct_tag_specs
|
||||||
|
tag_sections = { }
|
||||||
|
direct_tag_specs = []
|
||||||
|
|
||||||
|
tag_names.each do |raw_name|
|
||||||
|
tag_name, category, sections = parse_raw_tag_name(raw_name)
|
||||||
|
existing_tag = existing_tags_by_name[tag_name]
|
||||||
|
raise Tag::DeprecatedTagNormalisationError, [existing_tag.name] if existing_tag&.deprecated?
|
||||||
|
|
||||||
|
direct_tag_specs << {
|
||||||
|
name: tag_name,
|
||||||
|
category: (category || existing_tag&.category || 'general').to_sym }
|
||||||
|
if sections.present?
|
||||||
|
tag_sections[tag_name] ||= []
|
||||||
|
tag_sections[tag_name].concat(sections)
|
||||||
|
tag_sections[tag_name] = Tag.merge_section_ranges(tag_sections[tag_name])
|
||||||
|
tag_sections.delete(tag_name) if tag_sections[tag_name] == [[0, nil]]
|
||||||
|
end
|
||||||
|
end
|
||||||
|
|
||||||
|
[merge_tag_specs(direct_tag_specs), tag_sections]
|
||||||
|
end
|
||||||
|
|
||||||
|
def parse_raw_tag_name raw_name
|
||||||
|
name = raw_name.to_s
|
||||||
|
prefix, category =
|
||||||
|
Tag::CATEGORY_PREFIXES.find {
|
||||||
|
name.downcase.start_with?(_1[0])
|
||||||
|
} || ['', nil]
|
||||||
|
name = name.sub(/\A#{ prefix }/i, '')
|
||||||
|
|
||||||
|
sections = []
|
||||||
|
while (match = name.match(/\A(\S*?)\[([^\[\]\s]*)-([^\[\]\s]*)\](\S*)\z/))
|
||||||
|
name = "#{ match[1] }#{ match[4] }"
|
||||||
|
next if match[2].empty? && match[3].empty?
|
||||||
|
|
||||||
|
sections << Tag.normalise_section_range!(
|
||||||
|
begin_raw: match[2],
|
||||||
|
end_raw: match[3],
|
||||||
|
tag_name: name)
|
||||||
|
end
|
||||||
|
if name.include?('[') || name.include?(']')
|
||||||
|
raise Tag::SectionLiteralParseError.new(raw_name, raw_name)
|
||||||
|
end
|
||||||
|
|
||||||
|
[resolved_tag_name(name), category&.to_sym, sections]
|
||||||
|
end
|
||||||
|
|
||||||
|
def build_default_tag_specs direct_tag_specs
|
||||||
|
default_tag_specs = []
|
||||||
|
if direct_tag_specs.length < 10 && direct_tag_specs.none? { _1[:name] == TAGME_TAG_NAME }
|
||||||
|
default_tag_specs << {
|
||||||
|
name: TAGME_TAG_NAME,
|
||||||
|
category: :meta }
|
||||||
|
end
|
||||||
|
if direct_tag_specs.none? { deerjikist_tag_spec?(_1) }
|
||||||
|
default_tag_specs << {
|
||||||
|
name: NO_DEERJIKIST_TAG_NAME,
|
||||||
|
category: :meta }
|
||||||
|
end
|
||||||
|
|
||||||
|
default_tag_specs
|
||||||
|
end
|
||||||
|
|
||||||
|
def validate_new_tag_specs! specs
|
||||||
|
Array(specs).each do |spec|
|
||||||
|
next if existing_tags_by_name.key?(spec[:name])
|
||||||
|
|
||||||
|
validate_new_tag_spec!(spec)
|
||||||
|
end
|
||||||
|
end
|
||||||
|
|
||||||
|
def validate_new_tag_spec! spec
|
||||||
|
tag_name = TagName.new(name: spec[:name])
|
||||||
|
tag = Tag.new(category: spec[:category], tag_name:)
|
||||||
|
return if tag_name.valid? && tag.valid?
|
||||||
|
|
||||||
|
post = Post.new
|
||||||
|
tag_name.errors[:name].each do |message|
|
||||||
|
post.errors.add :tags, "#{ spec[:name] }: #{ message }"
|
||||||
|
end
|
||||||
|
tag.errors.each do |error|
|
||||||
|
next if error.attribute == :tag_name
|
||||||
|
|
||||||
|
post.errors.add :tags, "#{ spec[:name] }: #{ error.message }"
|
||||||
|
end
|
||||||
|
raise ActiveRecord::RecordInvalid, post
|
||||||
|
end
|
||||||
|
|
||||||
|
def expand_parent_tag_specs snapshot_tag_specs
|
||||||
|
existing_snapshot_tags = snapshot_tag_specs.filter_map { existing_tags_by_name[_1[:name]] }
|
||||||
|
expanded_parent_specs =
|
||||||
|
Tag.expand_parent_tags(existing_snapshot_tags)
|
||||||
|
.reject(&:deprecated?)
|
||||||
|
.map { |tag|
|
||||||
|
{ name: tag.name,
|
||||||
|
category: tag.category.to_sym }
|
||||||
|
}
|
||||||
|
merge_tag_specs(snapshot_tag_specs + expanded_parent_specs)
|
||||||
|
end
|
||||||
|
|
||||||
|
def merge_tag_specs specs
|
||||||
|
specs.each_with_object({ }) { |spec, merged|
|
||||||
|
merged[spec[:name]] =
|
||||||
|
if merged.key?(spec[:name]) && merged[spec[:name]][:category] != :general
|
||||||
|
merged[spec[:name]]
|
||||||
|
else
|
||||||
|
{
|
||||||
|
name: spec[:name],
|
||||||
|
category: spec[:category] }
|
||||||
|
end
|
||||||
|
}.values.sort_by { _1[:name] }
|
||||||
|
end
|
||||||
|
|
||||||
|
def existing_tags_by_name
|
||||||
|
@existing_tags_by_name ||= begin
|
||||||
|
names = tag_names.map { canonical_tag_name_without_sections(_1) }.uniq
|
||||||
|
Tag.joins(:tag_name).where(tag_names: { name: names }).index_by(&:name)
|
||||||
|
end
|
||||||
|
end
|
||||||
|
|
||||||
|
def preload_existing_tags_by_name! names
|
||||||
|
wanted_names = Array(names).map { _1.to_s }.reject(&:blank?).uniq
|
||||||
|
missing_names = wanted_names - existing_tags_by_name.keys
|
||||||
|
return if missing_names.empty?
|
||||||
|
|
||||||
|
existing_tags_by_name.merge!(
|
||||||
|
Tag.joins(:tag_name)
|
||||||
|
.where(tag_names: { name: missing_names })
|
||||||
|
.index_by(&:name))
|
||||||
|
end
|
||||||
|
|
||||||
|
def canonical_tag_name_without_sections raw_name
|
||||||
|
name, = parse_raw_tag_name(raw_name)
|
||||||
|
name
|
||||||
|
end
|
||||||
|
|
||||||
|
def deerjikist_tag_spec? spec
|
||||||
|
return true if spec[:category] == :deerjikist
|
||||||
|
|
||||||
|
existing_tags_by_name[spec[:name]]&.deerjikist?
|
||||||
|
end
|
||||||
|
|
||||||
|
def normalise_parent_post_ids
|
||||||
|
Array(@attributes[:parent_post_ids]).flat_map { _1.to_s.split }.map { |token|
|
||||||
|
id = Integer(token, exception: false)
|
||||||
|
raise ArgumentError, "親投稿 Id. が不正です: #{ token }" if id.nil? || id <= 0
|
||||||
|
|
||||||
|
id
|
||||||
|
}.uniq.sort
|
||||||
|
end
|
||||||
|
|
||||||
|
def validate_parent_post_ids! ids
|
||||||
|
missing = ids - Post.where(id: ids).pluck(:id)
|
||||||
|
raise ArgumentError, "存在しない親投稿 Id. があります: #{ missing.join(' ') }" if missing.present?
|
||||||
|
end
|
||||||
|
|
||||||
|
def serialised_tags direct_tag_specs, tag_sections
|
||||||
|
direct_tag_specs.map { |spec|
|
||||||
|
"#{ spec[:name] }#{ tag_sections[spec[:name]].to_a.map { section_literal(_1) }.join }"
|
||||||
|
}.sort.join(' ')
|
||||||
|
end
|
||||||
|
|
||||||
|
def display_tags direct_tag_specs, tag_sections
|
||||||
|
direct_tag_specs.map { |spec|
|
||||||
|
{
|
||||||
|
name: spec[:name],
|
||||||
|
category: spec[:category].to_s,
|
||||||
|
section_literals: tag_sections[spec[:name]].to_a.map { section_literal(_1) } }
|
||||||
|
}.sort_by { _1[:name] }
|
||||||
|
end
|
||||||
|
|
||||||
|
def section_literal range
|
||||||
|
begin_ms, end_ms = range
|
||||||
|
"[#{ Post.ms_to_time(begin_ms) }-#{ end_ms ? Post.ms_to_time(end_ms) : '' }]"
|
||||||
|
end
|
||||||
|
|
||||||
|
def normalise_video_ms snapshot_tag_specs
|
||||||
|
return nil unless snapshot_tag_specs.any? { _1[:name] == VIDEO_TAG_NAME }
|
||||||
|
|
||||||
|
video_ms = @attributes[:video_ms]
|
||||||
|
if video_ms.present?
|
||||||
|
value = Integer(video_ms, exception: false)
|
||||||
|
raise PostCreator::VideoMsParseError unless value&.positive?
|
||||||
|
|
||||||
|
return value
|
||||||
|
end
|
||||||
|
|
||||||
|
duration = @attributes[:duration]
|
||||||
|
return nil if duration.blank?
|
||||||
|
|
||||||
|
value = Tag.time_to_ms!(duration.to_s, tag_name: '動画時間')
|
||||||
|
raise PostCreator::VideoMsParseError unless value.positive?
|
||||||
|
|
||||||
|
value
|
||||||
|
rescue Tag::SectionLiteralParseError
|
||||||
|
raise PostCreator::VideoMsParseError
|
||||||
|
end
|
||||||
|
|
||||||
|
def validate_video_sections! video_ms, tag_sections
|
||||||
|
return unless video_ms
|
||||||
|
|
||||||
|
tag_sections.each_value do |ranges|
|
||||||
|
ranges.each do |begin_ms, end_ms|
|
||||||
|
if begin_ms >= video_ms
|
||||||
|
post = Post.new
|
||||||
|
post.errors.add :video_ms, 'タグ区間の開始が動画時間以上です.'
|
||||||
|
raise ActiveRecord::RecordInvalid, post
|
||||||
|
end
|
||||||
|
if end_ms && end_ms > video_ms
|
||||||
|
post = Post.new
|
||||||
|
post.errors.add :video_ms, 'タグ区間の終端が動画時間を超えてゐます.'
|
||||||
|
raise ActiveRecord::RecordInvalid, post
|
||||||
|
end
|
||||||
|
end
|
||||||
|
end
|
||||||
|
end
|
||||||
|
|
||||||
|
def resolved_tag_name name
|
||||||
|
tag_name = TagName.includes(:canonical).find_by(name:)
|
||||||
|
return name if tag_name.nil?
|
||||||
|
|
||||||
|
(tag_name.canonical || tag_name).name
|
||||||
|
end
|
||||||
|
end
|
||||||
@@ -0,0 +1,141 @@
|
|||||||
|
class PostCreatePreflight
|
||||||
|
class ValidationFailed < StandardError
|
||||||
|
attr_reader :fields, :base_errors
|
||||||
|
|
||||||
|
def initialize fields: { }, base_errors: []
|
||||||
|
super('入力内容を確認してください.')
|
||||||
|
@fields = fields
|
||||||
|
@base_errors = base_errors
|
||||||
|
end
|
||||||
|
end
|
||||||
|
|
||||||
|
def initialize attributes:, thumbnail: nil, host: nil
|
||||||
|
@attributes = attributes.symbolize_keys
|
||||||
|
@thumbnail = thumbnail
|
||||||
|
@host = host
|
||||||
|
end
|
||||||
|
|
||||||
|
def run
|
||||||
|
preview = PostImportPreviewer.new.preview_rows(
|
||||||
|
rows: [preview_row],
|
||||||
|
fetch_metadata: false).first
|
||||||
|
if preview[:existing_post_id].present?
|
||||||
|
return {
|
||||||
|
url: preview[:url],
|
||||||
|
title: preview[:attributes]['title'],
|
||||||
|
thumbnail_base: preview[:attributes]['thumbnail_base'],
|
||||||
|
tags: preview[:attributes]['tags'],
|
||||||
|
parent_post_ids: preview[:attributes]['parent_post_ids'],
|
||||||
|
original_created_from: preview[:attributes]['original_created_from'],
|
||||||
|
original_created_before: preview[:attributes]['original_created_before'],
|
||||||
|
duration: preview[:attributes]['duration'],
|
||||||
|
video_ms: preview[:attributes]['video_ms'],
|
||||||
|
display_tags: preview[:display_tags] || [],
|
||||||
|
field_warnings: final_field_warnings(preview[:field_warnings] || { }),
|
||||||
|
base_warnings: preview[:base_warnings],
|
||||||
|
existing_post_id: preview[:existing_post_id],
|
||||||
|
existing_post: existing_post_compact(preview[:existing_post_id]) }
|
||||||
|
end
|
||||||
|
|
||||||
|
if preview[:validation_errors].present?
|
||||||
|
raise ValidationFailed.new(fields: preview[:validation_errors])
|
||||||
|
end
|
||||||
|
|
||||||
|
validate_thumbnail_upload!
|
||||||
|
|
||||||
|
plan = PostCreatePlan.new(
|
||||||
|
attributes: {
|
||||||
|
url: preview[:url],
|
||||||
|
title: preview[:attributes]['title'],
|
||||||
|
thumbnail_base: preview[:attributes]['thumbnail_base'],
|
||||||
|
tags: preview[:attributes]['tags'],
|
||||||
|
parent_post_ids: preview[:attributes]['parent_post_ids'],
|
||||||
|
original_created_from: preview[:attributes]['original_created_from'],
|
||||||
|
original_created_before: preview[:attributes]['original_created_before'],
|
||||||
|
duration: preview[:attributes]['duration'],
|
||||||
|
video_ms: preview[:attributes]['video_ms'] }).build!
|
||||||
|
|
||||||
|
{
|
||||||
|
url: plan[:url],
|
||||||
|
title: plan[:title],
|
||||||
|
thumbnail_base: plan[:thumbnail_base],
|
||||||
|
tags: plan[:tags],
|
||||||
|
parent_post_ids: plan[:parent_post_ids],
|
||||||
|
original_created_from: plan[:original_created_from],
|
||||||
|
original_created_before: plan[:original_created_before],
|
||||||
|
duration: plan[:duration],
|
||||||
|
video_ms: plan[:video_ms],
|
||||||
|
display_tags: plan[:display_tags],
|
||||||
|
direct_tag_specs: plan[:direct_tag_specs],
|
||||||
|
default_tag_specs: plan[:default_tag_specs],
|
||||||
|
snapshot_tag_specs: plan[:snapshot_tag_specs],
|
||||||
|
post_tag_specs: plan[:post_tag_specs],
|
||||||
|
tag_sections: plan[:tag_sections],
|
||||||
|
normalised_parent_post_ids: plan[:normalised_parent_post_ids],
|
||||||
|
field_warnings: final_field_warnings(preview[:field_warnings] || { }),
|
||||||
|
base_warnings: preview[:base_warnings],
|
||||||
|
existing_post_id: preview[:existing_post_id],
|
||||||
|
existing_post: existing_post_compact(preview[:existing_post_id]) }
|
||||||
|
end
|
||||||
|
|
||||||
|
private
|
||||||
|
|
||||||
|
def preview_row
|
||||||
|
{
|
||||||
|
source_row: 1,
|
||||||
|
url: @attributes[:url].to_s,
|
||||||
|
attributes: {
|
||||||
|
'title' => @attributes[:title].to_s,
|
||||||
|
'thumbnail_base' => @attributes[:thumbnail_base].to_s,
|
||||||
|
'original_created_from' => @attributes[:original_created_from].to_s,
|
||||||
|
'original_created_before' => @attributes[:original_created_before].to_s,
|
||||||
|
'duration' => @attributes[:duration].to_s,
|
||||||
|
'video_ms' => @attributes[:video_ms],
|
||||||
|
'tags' => @attributes[:tags].to_s,
|
||||||
|
'parent_post_ids' => parent_post_ids_text },
|
||||||
|
provenance: {
|
||||||
|
'url' => 'manual',
|
||||||
|
'title' => 'manual',
|
||||||
|
'thumbnail_base' => 'manual',
|
||||||
|
'original_created_from' => 'manual',
|
||||||
|
'original_created_before' => 'manual',
|
||||||
|
'duration' => 'manual',
|
||||||
|
'video_ms' => 'manual',
|
||||||
|
'tags' => 'manual',
|
||||||
|
'parent_post_ids' => 'manual' },
|
||||||
|
tag_sources: {
|
||||||
|
'automatic' => '',
|
||||||
|
'manual' => @attributes[:tags].to_s } }
|
||||||
|
end
|
||||||
|
|
||||||
|
def parent_post_ids_text
|
||||||
|
Array(@attributes[:parent_post_ids]).flat_map { _1.to_s.split }.join(' ')
|
||||||
|
end
|
||||||
|
|
||||||
|
def validate_thumbnail_upload!
|
||||||
|
return if @attributes[:thumbnail_base].present?
|
||||||
|
return if @thumbnail.blank?
|
||||||
|
|
||||||
|
PostThumbnailUploadValidator.validate!(@thumbnail)
|
||||||
|
rescue PostThumbnailUploadValidator::InvalidUpload => e
|
||||||
|
raise ValidationFailed.new(fields: { thumbnail: [e.message] })
|
||||||
|
end
|
||||||
|
|
||||||
|
def existing_post_compact post_id
|
||||||
|
return nil if post_id.blank?
|
||||||
|
|
||||||
|
post = Post.with_attached_thumbnail.find_by(id: post_id)
|
||||||
|
PostCompactRepr.base(post, host: @host)
|
||||||
|
end
|
||||||
|
|
||||||
|
def final_field_warnings field_warnings
|
||||||
|
thumbnail_warnings = (field_warnings['thumbnail_base'] || []).reject { _1 == 'サムネールなし' }
|
||||||
|
if @attributes[:thumbnail_base].blank? && @thumbnail.blank?
|
||||||
|
thumbnail_warnings = (thumbnail_warnings + ['サムネールなし']).uniq
|
||||||
|
end
|
||||||
|
|
||||||
|
next_warnings = field_warnings.except('thumbnail_base')
|
||||||
|
next_warnings['thumbnail_base'] = thumbnail_warnings if thumbnail_warnings.present?
|
||||||
|
next_warnings
|
||||||
|
end
|
||||||
|
end
|
||||||
@@ -0,0 +1,155 @@
|
|||||||
|
class PostCreator
|
||||||
|
class VideoMsParseError < ArgumentError; end
|
||||||
|
|
||||||
|
attr_reader :field_warnings
|
||||||
|
|
||||||
|
def initialize actor:, attributes:
|
||||||
|
@actor = actor
|
||||||
|
@attributes = attributes.symbolize_keys
|
||||||
|
@field_warnings = { }
|
||||||
|
end
|
||||||
|
|
||||||
|
def create!
|
||||||
|
thumbnail_attachment = prepare_thumbnail_attachment
|
||||||
|
post = Post.new(title: @attributes[:title].presence,
|
||||||
|
url: @attributes[:url],
|
||||||
|
thumbnail_base: @attributes[:thumbnail_base].presence,
|
||||||
|
uploaded_user: @actor,
|
||||||
|
original_created_from: @attributes[:original_created_from].presence,
|
||||||
|
original_created_before: @attributes[:original_created_before].presence)
|
||||||
|
|
||||||
|
ApplicationRecord.transaction do
|
||||||
|
post.save!
|
||||||
|
post.thumbnail.attach(thumbnail_attachment) if thumbnail_attachment.present?
|
||||||
|
snapshot_tags = planned_snapshot_tags
|
||||||
|
post_tags = planned_post_tags
|
||||||
|
sections = planned_sections
|
||||||
|
TagVersioning.record_tag_snapshots!(snapshot_tags, created_by_user: @actor)
|
||||||
|
post.video_ms = planned_video_ms
|
||||||
|
post.save!
|
||||||
|
sync_post_tags!(post, post_tags, sections)
|
||||||
|
sync_parent_posts!(post, planned_parent_post_ids)
|
||||||
|
PostVersionRecorder.record!(post:, event_type: :create, created_by_user: @actor)
|
||||||
|
end
|
||||||
|
post
|
||||||
|
rescue StandardError
|
||||||
|
post&.thumbnail&.purge if post&.thumbnail&.attached?
|
||||||
|
raise
|
||||||
|
end
|
||||||
|
|
||||||
|
private
|
||||||
|
|
||||||
|
def prepare_thumbnail_attachment
|
||||||
|
PostThumbnailAttachmentBuilder.build(
|
||||||
|
thumbnail: @attributes[:thumbnail],
|
||||||
|
thumbnail_base: @attributes[:thumbnail_base].presence)
|
||||||
|
end
|
||||||
|
|
||||||
|
def planned_snapshot_tags = planned_create_attributes[:snapshot_tags]
|
||||||
|
|
||||||
|
def planned_post_tags = planned_create_attributes[:post_tags]
|
||||||
|
|
||||||
|
def planned_sections = planned_create_attributes[:tag_sections]
|
||||||
|
|
||||||
|
def planned_parent_post_ids = planned_create_attributes[:normalised_parent_post_ids]
|
||||||
|
|
||||||
|
def planned_video_ms
|
||||||
|
planned_create_attributes[:video_ms]
|
||||||
|
end
|
||||||
|
|
||||||
|
def planned_create_attributes
|
||||||
|
@planned_create_attributes ||= begin
|
||||||
|
if @attributes.key?(:snapshot_tag_specs)
|
||||||
|
snapshot_tags = materialise_tags(@attributes[:snapshot_tag_specs] || [])
|
||||||
|
post_tags = materialise_tags(@attributes[:post_tag_specs] || [])
|
||||||
|
{
|
||||||
|
snapshot_tags: snapshot_tags,
|
||||||
|
post_tags: post_tags,
|
||||||
|
tag_sections: materialise_sections(
|
||||||
|
@attributes[:tag_sections] || { },
|
||||||
|
snapshot_tags,
|
||||||
|
post_tags),
|
||||||
|
normalised_parent_post_ids: @attributes[:normalised_parent_post_ids] || [],
|
||||||
|
video_ms: @attributes[:video_ms] }
|
||||||
|
else
|
||||||
|
build_materialised_plan
|
||||||
|
end
|
||||||
|
end
|
||||||
|
end
|
||||||
|
|
||||||
|
def build_materialised_plan
|
||||||
|
plan = PostCreatePlan.new(attributes: @attributes).build!
|
||||||
|
snapshot_tags = materialise_tags(plan[:snapshot_tag_specs] || [])
|
||||||
|
post_tags = materialise_tags(plan[:post_tag_specs] || [])
|
||||||
|
{
|
||||||
|
snapshot_tags: snapshot_tags,
|
||||||
|
post_tags: post_tags,
|
||||||
|
tag_sections: materialise_sections(
|
||||||
|
plan[:tag_sections] || { },
|
||||||
|
snapshot_tags,
|
||||||
|
post_tags),
|
||||||
|
normalised_parent_post_ids: plan[:normalised_parent_post_ids] || [],
|
||||||
|
video_ms: plan[:video_ms] }
|
||||||
|
end
|
||||||
|
|
||||||
|
def materialise_tags specs
|
||||||
|
Array(specs).each_with_object({ }) do |spec, tags|
|
||||||
|
name = spec[:name] || spec['name']
|
||||||
|
category = spec[:category] || spec['category']
|
||||||
|
next if name.blank? || category.blank?
|
||||||
|
|
||||||
|
tag = Tag.find_or_create_by_tag_name!(name, category:)
|
||||||
|
tag.update!(category:) if tag.category.to_sym != category.to_sym
|
||||||
|
tags[name] ||= tag
|
||||||
|
end.values
|
||||||
|
end
|
||||||
|
|
||||||
|
def materialise_sections sections_by_name, snapshot_tags, post_tags
|
||||||
|
tags_by_name = post_tags.index_by(&:name)
|
||||||
|
snapshot_tags.each do |tag|
|
||||||
|
tags_by_name[tag.name] ||= tag
|
||||||
|
end
|
||||||
|
|
||||||
|
sections_by_name.each_with_object({ }) do |(tag_name, ranges), sections|
|
||||||
|
tag = tags_by_name[tag_name.to_s]
|
||||||
|
next if tag.nil?
|
||||||
|
|
||||||
|
sections[tag.id] = Array(ranges).map { |range| [range[0], range[1]] }
|
||||||
|
end
|
||||||
|
end
|
||||||
|
|
||||||
|
def sync_post_tags! post, desired_tags, sections
|
||||||
|
desired_ids = desired_tags.map(&:id).to_set
|
||||||
|
current_ids = post.tags.pluck(:id).to_set
|
||||||
|
|
||||||
|
Tag.where(id: desired_ids - current_ids).find_each do |tag|
|
||||||
|
PostTag.create_or_find_by!(post:, tag:, created_user: @actor)
|
||||||
|
end
|
||||||
|
|
||||||
|
PostTagSection.where(post_id: post.id).destroy_all
|
||||||
|
sections.each do |tag_id, ranges|
|
||||||
|
ranges.each do |begin_ms, end_ms|
|
||||||
|
PostTagSection.create!(post_id: post.id,
|
||||||
|
tag_id:,
|
||||||
|
begin_ms:,
|
||||||
|
end_ms:)
|
||||||
|
end
|
||||||
|
end
|
||||||
|
|
||||||
|
PostTag.where(post_id: post.id,
|
||||||
|
tag_id: (current_ids - desired_ids).to_a).destroy_all
|
||||||
|
end
|
||||||
|
|
||||||
|
def sync_parent_posts! post, ids
|
||||||
|
if ids.include?(post.id)
|
||||||
|
post.errors.add :parent_post_ids, '自分自身を親投稿にはできません.'
|
||||||
|
raise ActiveRecord::RecordInvalid, post
|
||||||
|
end
|
||||||
|
missing = ids - Post.where(id: ids).pluck(:id)
|
||||||
|
if missing.present?
|
||||||
|
post.errors.add :parent_post_ids, "存在しない親投稿 Id. があります: #{ missing.join(' ') }"
|
||||||
|
raise ActiveRecord::RecordInvalid, post
|
||||||
|
end
|
||||||
|
ids.each { |parent_post_id| PostImplication.create_or_find_by!(post:, parent_post_id:) }
|
||||||
|
end
|
||||||
|
end
|
||||||
@@ -0,0 +1,522 @@
|
|||||||
|
require 'time'
|
||||||
|
require 'timeout'
|
||||||
|
|
||||||
|
class PostImportPreviewer
|
||||||
|
FIELDS = [
|
||||||
|
'title',
|
||||||
|
'thumbnail_base',
|
||||||
|
'original_created_from',
|
||||||
|
'original_created_before',
|
||||||
|
'video_ms',
|
||||||
|
'duration',
|
||||||
|
'tags',
|
||||||
|
'parent_post_ids'].freeze
|
||||||
|
FETCH_WARNING_FIELDS = ['url', 'title', 'thumbnail_base'].freeze
|
||||||
|
TITLE_FETCH_WARNING = 'タイトルを取得できませんでした.'.freeze
|
||||||
|
THUMBNAIL_FETCH_WARNING = 'サムネールを取得できませんでした.'.freeze
|
||||||
|
METADATA_FETCH_WARNING = '自動取得に失敗しました.'.freeze
|
||||||
|
|
||||||
|
def preview_rows rows:, fetch_metadata: true, metadata_cache: { }
|
||||||
|
prepared_rows = rows.map { prepare_row(_1) }
|
||||||
|
url_counts = prepared_rows.filter_map { _1[:normal_url] }.tally
|
||||||
|
existing_posts =
|
||||||
|
Post.where(url: prepared_rows.map { _1[:normal_url] }.compact.uniq).index_by(&:url)
|
||||||
|
existing_parent_ids = preload_parent_ids(prepared_rows)
|
||||||
|
preload_metadata!(
|
||||||
|
prepared_rows,
|
||||||
|
fetch_metadata,
|
||||||
|
metadata_cache,
|
||||||
|
existing_posts,
|
||||||
|
url_counts)
|
||||||
|
known_tags =
|
||||||
|
preload_known_tags(
|
||||||
|
prepared_rows,
|
||||||
|
fetch_metadata,
|
||||||
|
metadata_cache,
|
||||||
|
existing_posts,
|
||||||
|
url_counts)
|
||||||
|
prepared_rows.map { |row|
|
||||||
|
preview_row(row,
|
||||||
|
fetch_metadata:,
|
||||||
|
metadata_cache:,
|
||||||
|
existing_posts:,
|
||||||
|
url_counts:,
|
||||||
|
known_tags:,
|
||||||
|
existing_parent_ids:)
|
||||||
|
}
|
||||||
|
end
|
||||||
|
|
||||||
|
def normalised_url value
|
||||||
|
PostUrlNormaliser.normalise(value)
|
||||||
|
end
|
||||||
|
|
||||||
|
private
|
||||||
|
|
||||||
|
def prepare_row row
|
||||||
|
source = row.symbolize_keys
|
||||||
|
url = source[:url].to_s.strip
|
||||||
|
normal_url = normalised_url(url)
|
||||||
|
source.merge(url_text: url, normal_url:, url_error: validate_url_safety(normal_url))
|
||||||
|
end
|
||||||
|
|
||||||
|
def preview_row row,
|
||||||
|
fetch_metadata:,
|
||||||
|
metadata_cache:,
|
||||||
|
existing_posts:,
|
||||||
|
url_counts:,
|
||||||
|
known_tags:,
|
||||||
|
existing_parent_ids:
|
||||||
|
attributes = initial_attributes(row)
|
||||||
|
provenance = initial_provenance(row)
|
||||||
|
tag_sources = initial_tag_sources(row, attributes, provenance)
|
||||||
|
field_warnings = initial_field_warnings(row)
|
||||||
|
base_warnings = initial_base_warnings(row)
|
||||||
|
url = row[:url_text]
|
||||||
|
provenance['url'] = 'manual'
|
||||||
|
normal_url = row[:normal_url]
|
||||||
|
url_for_metadata = normal_url || url
|
||||||
|
existing_post = normal_url.present? ? existing_posts[normal_url] : nil
|
||||||
|
|
||||||
|
validation_errors = {}
|
||||||
|
validation_errors[:url] = ['URL が不正です.'] if normal_url.blank?
|
||||||
|
if row[:url_error].present?
|
||||||
|
validation_errors[:url] = [row[:url_error]]
|
||||||
|
end
|
||||||
|
if normal_url.present? && url_counts[normal_url].to_i > 1
|
||||||
|
validation_errors[:url] = ['URL が重複しています.']
|
||||||
|
end
|
||||||
|
|
||||||
|
if row[:metadata_url].present? && row[:metadata_url] != url_for_metadata
|
||||||
|
clear_automatic_values!(attributes, provenance, tag_sources)
|
||||||
|
field_warnings = { }
|
||||||
|
base_warnings = [ ]
|
||||||
|
end
|
||||||
|
|
||||||
|
if validation_errors.blank? && normal_url.present? && existing_post
|
||||||
|
attributes['tags'] = merged_tags(tag_sources, provenance['tags'])
|
||||||
|
warnings_present = field_warnings.values.any?(&:present?) || base_warnings.present?
|
||||||
|
return { source_row: row[:source_row],
|
||||||
|
url: normal_url,
|
||||||
|
attributes:,
|
||||||
|
provenance:,
|
||||||
|
tag_sources:,
|
||||||
|
metadata_url: url_for_metadata,
|
||||||
|
skip_reason: 'existing',
|
||||||
|
existing_post_id: existing_post.id,
|
||||||
|
field_warnings:,
|
||||||
|
base_warnings:,
|
||||||
|
validation_errors:,
|
||||||
|
status: warnings_present ? 'warning' : 'ready' }
|
||||||
|
end
|
||||||
|
|
||||||
|
should_fetch = validation_errors.blank?
|
||||||
|
should_fetch &&= should_fetch_metadata?(fetch_metadata, row[:source_row].to_i)
|
||||||
|
if should_fetch
|
||||||
|
clear_fetch_warnings!(field_warnings)
|
||||||
|
metadata = metadata_for(url_for_metadata, metadata_cache)
|
||||||
|
if metadata[:validation_errors].present?
|
||||||
|
validation_errors.merge!(metadata[:validation_errors])
|
||||||
|
else
|
||||||
|
apply_metadata!(attributes, provenance, tag_sources, metadata[:data])
|
||||||
|
apply_fetch_warnings!(field_warnings, metadata[:warnings])
|
||||||
|
end
|
||||||
|
end
|
||||||
|
|
||||||
|
attributes['url'] = url
|
||||||
|
validate_basic_data(attributes, validation_errors)
|
||||||
|
validate_preview_tags(merged_tags(tag_sources, provenance['tags']),
|
||||||
|
validation_errors,
|
||||||
|
known_tags)
|
||||||
|
validate_parents(attributes['parent_post_ids'], validation_errors, existing_parent_ids)
|
||||||
|
attributes.delete('url')
|
||||||
|
attributes['tags'] = merged_tags(tag_sources, provenance['tags'])
|
||||||
|
|
||||||
|
warnings_present = field_warnings.values.any?(&:present?) || base_warnings.present?
|
||||||
|
{ source_row: row[:source_row],
|
||||||
|
url: validation_errors[:url].present? ? url : (normal_url || url),
|
||||||
|
attributes:,
|
||||||
|
provenance:,
|
||||||
|
tag_sources:,
|
||||||
|
metadata_url: url_for_metadata,
|
||||||
|
skip_reason: nil,
|
||||||
|
existing_post_id: nil,
|
||||||
|
field_warnings:,
|
||||||
|
base_warnings:,
|
||||||
|
validation_errors:,
|
||||||
|
status: validation_errors.present? ? 'error' : (warnings_present ? 'warning' : 'ready') }
|
||||||
|
end
|
||||||
|
|
||||||
|
def should_fetch_metadata? fetch_metadata, source_row
|
||||||
|
case fetch_metadata
|
||||||
|
when true then true
|
||||||
|
when Integer then fetch_metadata == source_row
|
||||||
|
when false, nil then false
|
||||||
|
else raise ArgumentError, '取得対象が不正です.'
|
||||||
|
end
|
||||||
|
end
|
||||||
|
|
||||||
|
def initial_attributes row
|
||||||
|
attributes = row[:attributes]&.stringify_keys || { }
|
||||||
|
FIELDS.to_h { |field|
|
||||||
|
value = attributes[field]
|
||||||
|
normalised =
|
||||||
|
if field == 'duration'
|
||||||
|
normalise_duration_attribute(value)
|
||||||
|
elsif field == 'video_ms'
|
||||||
|
value.nil? ? '' : value.to_s
|
||||||
|
else
|
||||||
|
value.to_s
|
||||||
|
end
|
||||||
|
[field, normalised] }
|
||||||
|
end
|
||||||
|
|
||||||
|
def initial_provenance row
|
||||||
|
provenance = row[:provenance]&.stringify_keys || { }
|
||||||
|
FIELDS.to_h { |field| [field, provenance[field].presence || 'automatic'] }
|
||||||
|
.merge('url' => 'manual')
|
||||||
|
end
|
||||||
|
|
||||||
|
def initial_tag_sources row, attributes, provenance
|
||||||
|
sources = row[:tag_sources]&.stringify_keys || { 'automatic' => '', 'manual' => '' }
|
||||||
|
sources['automatic'] = sources['automatic'].to_s
|
||||||
|
sources['manual'] =
|
||||||
|
provenance['tags'] == 'manual' ? attributes['tags'].to_s : sources['manual'].to_s
|
||||||
|
sources
|
||||||
|
end
|
||||||
|
|
||||||
|
def initial_field_warnings row
|
||||||
|
(row[:field_warnings] || { })
|
||||||
|
.stringify_keys
|
||||||
|
.transform_values { |value| Array(value).map(&:to_s) }
|
||||||
|
end
|
||||||
|
|
||||||
|
def initial_base_warnings row
|
||||||
|
Array(row[:base_warnings]).map(&:to_s)
|
||||||
|
end
|
||||||
|
|
||||||
|
def clear_automatic_values! attributes, provenance, tag_sources
|
||||||
|
['title', 'thumbnail_base', 'original_created_from',
|
||||||
|
'original_created_before', 'video_ms', 'duration'].each do |field|
|
||||||
|
attributes[field] = '' if provenance[field] == 'automatic'
|
||||||
|
end
|
||||||
|
tag_sources['automatic'] = ''
|
||||||
|
attributes['tags'] = merged_tags(tag_sources, provenance['tags'])
|
||||||
|
end
|
||||||
|
|
||||||
|
def clear_fetch_warnings! field_warnings
|
||||||
|
FETCH_WARNING_FIELDS.each do |field|
|
||||||
|
field_warnings.delete(field)
|
||||||
|
end
|
||||||
|
end
|
||||||
|
|
||||||
|
def metadata_for url, cache
|
||||||
|
cache[url] ||= fetch_metadata(url)
|
||||||
|
end
|
||||||
|
|
||||||
|
def fetch_metadata url
|
||||||
|
return { data: { }, warnings: { 'url' => ['URL が空です.'] },
|
||||||
|
validation_errors: { } } if url.blank?
|
||||||
|
|
||||||
|
data = sanitise_metadata(PostMetadataFetcher.fetch(url).stringify_keys.compact)
|
||||||
|
warnings = { }
|
||||||
|
add_field_warning!(warnings, 'title', TITLE_FETCH_WARNING) if data['title'].blank?
|
||||||
|
if data['thumbnail_base'].blank?
|
||||||
|
add_field_warning!(warnings, 'thumbnail_base', THUMBNAIL_FETCH_WARNING)
|
||||||
|
end
|
||||||
|
{ data:, warnings:, validation_errors: { } }
|
||||||
|
rescue Preview::UrlSafety::UnsafeUrl => e
|
||||||
|
payload = { error: e.class.name, message: e.message }
|
||||||
|
Rails.logger.info(
|
||||||
|
"post_import_metadata_fetch_unsafe_url #{ payload.to_json }")
|
||||||
|
{ data: { }, warnings: { }, validation_errors: { url: [e.message] } }
|
||||||
|
rescue Preview::HttpFetcher::FetchFailed,
|
||||||
|
Preview::HttpFetcher::ResponseTooLarge => e
|
||||||
|
payload = { error: e.class.name, message: e.message }
|
||||||
|
Rails.logger.info(
|
||||||
|
"post_import_metadata_fetch_failure #{ payload.to_json }")
|
||||||
|
{ data: { }, warnings: { 'url' => [METADATA_FETCH_WARNING] }, validation_errors: { } }
|
||||||
|
end
|
||||||
|
|
||||||
|
def sanitise_metadata metadata
|
||||||
|
{
|
||||||
|
'title' => sanitise_metadata_title(metadata['title']),
|
||||||
|
'thumbnail_base' => sanitise_metadata_url(metadata['thumbnail_base']),
|
||||||
|
'original_created_from' => sanitise_metadata_time(metadata['original_created_from']),
|
||||||
|
'original_created_before' => sanitise_metadata_time(metadata['original_created_before']),
|
||||||
|
'duration' => sanitise_metadata_duration(metadata['duration']),
|
||||||
|
'tags' => metadata['tags'].to_s.presence }.compact
|
||||||
|
end
|
||||||
|
|
||||||
|
def sanitise_metadata_title value
|
||||||
|
value.is_a?(String) ? value.presence : nil
|
||||||
|
end
|
||||||
|
|
||||||
|
def sanitise_metadata_url value
|
||||||
|
return nil unless value.is_a?(String)
|
||||||
|
|
||||||
|
stripped = value.strip
|
||||||
|
return nil if stripped.blank?
|
||||||
|
|
||||||
|
uri = URI.parse(stripped)
|
||||||
|
return nil unless uri.is_a?(URI::HTTP) && uri.host.present?
|
||||||
|
|
||||||
|
stripped
|
||||||
|
rescue URI::InvalidURIError
|
||||||
|
nil
|
||||||
|
end
|
||||||
|
|
||||||
|
def sanitise_metadata_time value
|
||||||
|
return nil unless value.is_a?(String)
|
||||||
|
|
||||||
|
Time.iso8601(value).in_time_zone.change(sec: 0, nsec: 0).iso8601
|
||||||
|
rescue ArgumentError, TypeError
|
||||||
|
nil
|
||||||
|
end
|
||||||
|
|
||||||
|
def sanitise_metadata_duration value
|
||||||
|
return nil unless value.is_a?(String)
|
||||||
|
|
||||||
|
value.presence
|
||||||
|
end
|
||||||
|
|
||||||
|
def normalise_duration_attribute value
|
||||||
|
return '' if value.nil?
|
||||||
|
return value if value.is_a?(String)
|
||||||
|
|
||||||
|
milliseconds = Integer(value, exception: false)
|
||||||
|
return value.to_s if milliseconds.nil? || milliseconds <= 0
|
||||||
|
|
||||||
|
seconds_string = (milliseconds / 1_000.0).to_s
|
||||||
|
seconds_string.end_with?('.0') ? seconds_string.delete_suffix('.0') : seconds_string
|
||||||
|
end
|
||||||
|
|
||||||
|
def preload_metadata! prepared_rows, fetch_metadata, metadata_cache, existing_posts, url_counts
|
||||||
|
urls = prepared_rows.filter_map { |row|
|
||||||
|
next unless row[:normal_url].present?
|
||||||
|
next if row[:url_error].present?
|
||||||
|
next unless should_fetch_metadata?(fetch_metadata, row[:source_row].to_i)
|
||||||
|
next if url_counts[row[:normal_url]].to_i > 1
|
||||||
|
next if existing_posts.key?(row[:normal_url])
|
||||||
|
|
||||||
|
row[:normal_url]
|
||||||
|
}.uniq
|
||||||
|
urls = urls.reject { metadata_cache.key?(_1) }
|
||||||
|
return if urls.empty?
|
||||||
|
|
||||||
|
url_queue = Queue.new
|
||||||
|
result_queue = Queue.new
|
||||||
|
urls.each { url_queue << _1 }
|
||||||
|
workers = [urls.length, 4].min.times.map {
|
||||||
|
Thread.new {
|
||||||
|
Rails.application.executor.wrap do
|
||||||
|
loop do
|
||||||
|
url = url_queue.pop(true)
|
||||||
|
result_queue << [url, safe_fetch_metadata(url)]
|
||||||
|
rescue ThreadError
|
||||||
|
break
|
||||||
|
end
|
||||||
|
end
|
||||||
|
}
|
||||||
|
}
|
||||||
|
Timeout.timeout(15) { workers.each(&:join) }
|
||||||
|
rescue Timeout::Error
|
||||||
|
workers&.each(&:kill)
|
||||||
|
ensure
|
||||||
|
workers&.each(&:join)
|
||||||
|
while result_queue&.size.to_i.positive?
|
||||||
|
url, result = result_queue.pop
|
||||||
|
metadata_cache[url] = result
|
||||||
|
end
|
||||||
|
urls&.each do |url|
|
||||||
|
metadata_cache[url] ||= { data: { }, warnings: { 'url' => [METADATA_FETCH_WARNING] } }
|
||||||
|
end
|
||||||
|
end
|
||||||
|
|
||||||
|
def safe_fetch_metadata url
|
||||||
|
fetch_metadata(url)
|
||||||
|
rescue Preview::UrlSafety::UnsafeUrl => e
|
||||||
|
payload = { error: e.class.name, message: e.message }
|
||||||
|
Rails.logger.info(
|
||||||
|
"post_import_metadata_fetch_unsafe_url #{ payload.to_json }")
|
||||||
|
{ data: { }, warnings: { }, validation_errors: { url: [e.message] } }
|
||||||
|
rescue StandardError => e
|
||||||
|
payload = { error: e.class.name, message: e.message }
|
||||||
|
Rails.logger.error(
|
||||||
|
"post_import_metadata_fetch_unexpected_failure #{ payload.to_json }")
|
||||||
|
{ data: { }, warnings: { 'url' => [METADATA_FETCH_WARNING] }, validation_errors: { } }
|
||||||
|
end
|
||||||
|
|
||||||
|
def preload_known_tags prepared_rows, fetch_metadata, metadata_cache, existing_posts, url_counts
|
||||||
|
names = prepared_rows.flat_map { |row|
|
||||||
|
attributes = initial_attributes(row)
|
||||||
|
provenance = initial_provenance(row)
|
||||||
|
tag_sources = initial_tag_sources(row, attributes, provenance)
|
||||||
|
if metadata_url_changed?(row)
|
||||||
|
clear_automatic_values!(attributes, provenance, tag_sources)
|
||||||
|
end
|
||||||
|
if should_apply_metadata_to_row?(row, fetch_metadata, existing_posts, url_counts)
|
||||||
|
metadata = metadata_for(row[:normal_url], metadata_cache)
|
||||||
|
apply_metadata!(attributes, provenance, tag_sources, metadata[:data])
|
||||||
|
end
|
||||||
|
preview_tag_names(merged_tags(tag_sources, provenance['tags']))
|
||||||
|
}.compact.uniq
|
||||||
|
return { } if names.empty?
|
||||||
|
|
||||||
|
Tag.joins(:tag_name)
|
||||||
|
.where(tag_names: { name: names })
|
||||||
|
.includes(:tag_name)
|
||||||
|
.to_a
|
||||||
|
.index_by(&:name)
|
||||||
|
end
|
||||||
|
|
||||||
|
def preload_parent_ids prepared_rows
|
||||||
|
ids = prepared_rows.flat_map { |row|
|
||||||
|
attributes = initial_attributes(row)
|
||||||
|
preview_parent_ids(attributes['parent_post_ids'])
|
||||||
|
}.uniq
|
||||||
|
return { } if ids.empty?
|
||||||
|
|
||||||
|
Post.where(id: ids).pluck(:id).to_h { [_1, true] }
|
||||||
|
end
|
||||||
|
|
||||||
|
def metadata_url_changed? row
|
||||||
|
row[:metadata_url].present? && row[:metadata_url] != (row[:normal_url] || row[:url_text])
|
||||||
|
end
|
||||||
|
|
||||||
|
def should_apply_metadata_to_row? row, fetch_metadata, existing_posts, url_counts
|
||||||
|
normal_url = row[:normal_url]
|
||||||
|
return false if normal_url.blank?
|
||||||
|
return false if row[:url_error].present?
|
||||||
|
return false if url_counts[normal_url].to_i > 1
|
||||||
|
return false if existing_posts.key?(normal_url)
|
||||||
|
|
||||||
|
should_fetch_metadata?(fetch_metadata, row[:source_row].to_i)
|
||||||
|
end
|
||||||
|
|
||||||
|
def validate_url_safety normal_url
|
||||||
|
return nil if normal_url.blank?
|
||||||
|
|
||||||
|
Preview::UrlSafety.validate(normal_url)
|
||||||
|
nil
|
||||||
|
rescue Preview::UrlSafety::UnsafeUrl => e
|
||||||
|
e.message
|
||||||
|
end
|
||||||
|
|
||||||
|
def apply_metadata! attributes, provenance, tag_sources, metadata
|
||||||
|
metadata.each do |field, value|
|
||||||
|
if field == 'tags'
|
||||||
|
next if provenance['tags'] == 'manual'
|
||||||
|
|
||||||
|
tag_sources['automatic'] = value.to_s
|
||||||
|
attributes['tags'] = merged_tags(tag_sources)
|
||||||
|
next
|
||||||
|
end
|
||||||
|
next unless provenance[field] == 'automatic'
|
||||||
|
|
||||||
|
attributes[field] = value
|
||||||
|
provenance[field] = 'automatic'
|
||||||
|
end
|
||||||
|
end
|
||||||
|
|
||||||
|
def apply_fetch_warnings! field_warnings, warnings
|
||||||
|
warnings.each do |field, values|
|
||||||
|
values.each do |value|
|
||||||
|
add_field_warning!(field_warnings, field, value)
|
||||||
|
end
|
||||||
|
end
|
||||||
|
end
|
||||||
|
|
||||||
|
def add_field_warning! field_warnings, field, message
|
||||||
|
field_warnings[field] ||= []
|
||||||
|
field_warnings[field] << message unless field_warnings[field].include?(message)
|
||||||
|
end
|
||||||
|
|
||||||
|
def merged_tags sources, origin = nil
|
||||||
|
return sources['manual'].to_s if origin == 'manual'
|
||||||
|
|
||||||
|
sources['automatic'].to_s
|
||||||
|
end
|
||||||
|
|
||||||
|
def preview_tag_names raw
|
||||||
|
names = raw.to_s.split
|
||||||
|
return [] if names.empty?
|
||||||
|
if names.any? { _1.downcase.start_with?('nico:') }
|
||||||
|
return []
|
||||||
|
end
|
||||||
|
|
||||||
|
names.map { |name| TagName.canonicalise(name.sub(/\[.*\]\z/, '')).first }
|
||||||
|
rescue Tag::SectionLiteralParseError
|
||||||
|
[]
|
||||||
|
end
|
||||||
|
|
||||||
|
def validate_preview_tags raw, errors, known_tags
|
||||||
|
names = raw.to_s.split
|
||||||
|
return if names.empty?
|
||||||
|
if names.any? { _1.downcase.start_with?('nico:') }
|
||||||
|
errors[:tags] = ['ニコニコ・タグは直接指定できません.']
|
||||||
|
return
|
||||||
|
end
|
||||||
|
|
||||||
|
parsed = names.map { |name| TagName.canonicalise(name.sub(/\[.*\]\z/, '')).first }
|
||||||
|
existing = parsed.filter_map { known_tags[_1] }
|
||||||
|
deprecated = existing.select(&:deprecated?).map(&:name)
|
||||||
|
errors[:tags] = ["廃止済みタグがあります: #{ deprecated.join(' ') }"] if deprecated.present?
|
||||||
|
rescue Tag::SectionLiteralParseError
|
||||||
|
errors[:tags] = ['タグ区間の記法が不正です.']
|
||||||
|
end
|
||||||
|
|
||||||
|
def validate_basic_data attributes, errors
|
||||||
|
post = Post.new(title: attributes['title'].presence,
|
||||||
|
url: attributes['url'],
|
||||||
|
thumbnail_base: attributes['thumbnail_base'].presence,
|
||||||
|
original_created_from: attributes['original_created_from'].presence,
|
||||||
|
original_created_before: attributes['original_created_before'].presence,
|
||||||
|
video_ms: parse_video_ms(attributes, errors))
|
||||||
|
post.valid?
|
||||||
|
post.errors.each do |error|
|
||||||
|
next if error.attribute == :url && error.type == :taken
|
||||||
|
|
||||||
|
(errors[error.attribute] ||= []) << error.message
|
||||||
|
end
|
||||||
|
end
|
||||||
|
|
||||||
|
def parse_video_ms attributes, errors
|
||||||
|
return nil unless attributes['tags'].to_s.split.include?('動画')
|
||||||
|
|
||||||
|
video_ms = attributes['video_ms']
|
||||||
|
if video_ms.present?
|
||||||
|
value = Integer(video_ms, exception: false)
|
||||||
|
if value&.positive?
|
||||||
|
return value
|
||||||
|
end
|
||||||
|
|
||||||
|
errors[:video_ms] = ['動画時間の記法が不正です.']
|
||||||
|
return nil
|
||||||
|
end
|
||||||
|
|
||||||
|
value = attributes['duration']
|
||||||
|
return nil if value.blank?
|
||||||
|
|
||||||
|
Tag.time_to_ms!(value.to_s, tag_name: '動画時間')
|
||||||
|
rescue Tag::SectionLiteralParseError
|
||||||
|
errors[:video_ms] = ['動画時間の記法が不正です.']
|
||||||
|
nil
|
||||||
|
end
|
||||||
|
|
||||||
|
def preview_parent_ids raw
|
||||||
|
raw.to_s.split.map { Integer(_1, exception: false) }.compact
|
||||||
|
end
|
||||||
|
|
||||||
|
def validate_parents raw, errors, existing_parent_ids
|
||||||
|
ids = raw.to_s.split.map { Integer(_1, exception: false) }
|
||||||
|
return if ids.compact.empty? && raw.to_s.blank?
|
||||||
|
|
||||||
|
if ids.any? { _1.nil? || _1 <= 0 }
|
||||||
|
errors[:parent_post_ids] = ['親投稿 Id. が不正です.']
|
||||||
|
return
|
||||||
|
end
|
||||||
|
if ids.uniq.any? { !existing_parent_ids[_1] }
|
||||||
|
errors[:parent_post_ids] = ['存在しない親投稿 Id. があります.']
|
||||||
|
end
|
||||||
|
end
|
||||||
|
end
|
||||||
@@ -0,0 +1,198 @@
|
|||||||
|
require 'time'
|
||||||
|
require 'date'
|
||||||
|
|
||||||
|
class PostMetadataFetcher
|
||||||
|
TIMESTAMP_PATTERN =
|
||||||
|
/\A(\d{4})-(\d{2})-(\d{2})T(\d{2})
|
||||||
|
(?::(\d{2})(?::(\d{2})(?:\.(\d+))?)?)?
|
||||||
|
(Z|[+-]\d{2}:?\d{2})?\z/x
|
||||||
|
|
||||||
|
def self.fetch raw_url
|
||||||
|
uri, = Preview::UrlSafety.validate(raw_url)
|
||||||
|
response = Preview::HttpFetcher.fetch(
|
||||||
|
uri.to_s,
|
||||||
|
max_bytes: Preview::ThumbnailFetcher::HTML_MAX_BYTES)
|
||||||
|
metadata = Preview::HtmlMetadataExtractor.extract(response)
|
||||||
|
document = Nokogiri::HTML.parse(response.body)
|
||||||
|
content = lambda { |name|
|
||||||
|
document
|
||||||
|
.at_css("meta[property='#{ name }'], meta[name='#{ name }']")
|
||||||
|
&.[]('content')
|
||||||
|
&.strip
|
||||||
|
&.presence
|
||||||
|
}
|
||||||
|
duration = content.call('og:video:duration') || content.call('video:duration')
|
||||||
|
published = content.call('article:published_time') || content.call('date')
|
||||||
|
created_range = original_created_range(published)
|
||||||
|
platform_tags = platform_tags(uri)
|
||||||
|
{ title: metadata[:title],
|
||||||
|
thumbnail_base:
|
||||||
|
Preview::KnownSiteExtractor.thumbnail_url(uri) || metadata[:image_url],
|
||||||
|
original_created_from: serialise_time(created_range&.first),
|
||||||
|
original_created_before: serialise_time(created_range&.last),
|
||||||
|
duration: serialise_duration(duration),
|
||||||
|
display_tags: display_tags(platform_tags),
|
||||||
|
tags: platform_tags.join(' ') }
|
||||||
|
end
|
||||||
|
|
||||||
|
def self.platform_tags uri
|
||||||
|
return ['動画', 'YouTube'] if Preview::KnownSiteExtractor.youtube_video_id(uri)
|
||||||
|
return ['動画', 'ニコニコ'] if Preview::KnownSiteExtractor.niconico_video_id(uri)
|
||||||
|
|
||||||
|
[]
|
||||||
|
end
|
||||||
|
|
||||||
|
def self.display_tags names
|
||||||
|
return [] if names.blank?
|
||||||
|
|
||||||
|
existing_tags =
|
||||||
|
Tag
|
||||||
|
.joins(:tag_name)
|
||||||
|
.where(tag_names: { name: names })
|
||||||
|
.index_by(&:name)
|
||||||
|
|
||||||
|
names.map { |name|
|
||||||
|
tag = existing_tags[name]
|
||||||
|
{
|
||||||
|
name: name,
|
||||||
|
category: (tag&.category || 'meta'),
|
||||||
|
section_literals: [] }
|
||||||
|
}
|
||||||
|
end
|
||||||
|
|
||||||
|
def self.original_created_range value
|
||||||
|
return nil if value.blank?
|
||||||
|
|
||||||
|
raw = value.to_s.strip
|
||||||
|
from, before =
|
||||||
|
case raw
|
||||||
|
when /\A(\d{4})\z/
|
||||||
|
year = Regexp.last_match(1).to_i
|
||||||
|
from = Time.zone.local(year, 1, 1)
|
||||||
|
[from, from + 1.year]
|
||||||
|
when /\A(\d{4})-(\d{2})\z/
|
||||||
|
year = Regexp.last_match(1).to_i
|
||||||
|
month = Regexp.last_match(2).to_i
|
||||||
|
return nil unless month.between?(1, 12)
|
||||||
|
|
||||||
|
from = Time.zone.local(year, month, 1)
|
||||||
|
[from, from + 1.month]
|
||||||
|
when /\A(\d{4})-(\d{2})-(\d{2})\z/
|
||||||
|
year = Regexp.last_match(1).to_i
|
||||||
|
month = Regexp.last_match(2).to_i
|
||||||
|
day = Regexp.last_match(3).to_i
|
||||||
|
return nil unless Date.valid_date?(year, month, day)
|
||||||
|
|
||||||
|
from = Time.zone.local(year, month, day)
|
||||||
|
[from, from + 1.day]
|
||||||
|
else
|
||||||
|
parse_timestamp_range(raw)
|
||||||
|
end
|
||||||
|
return nil if from.nil? || before.nil?
|
||||||
|
|
||||||
|
[from, before]
|
||||||
|
rescue ArgumentError, TypeError
|
||||||
|
nil
|
||||||
|
end
|
||||||
|
|
||||||
|
def self.parse_timestamp_range raw
|
||||||
|
match = raw.match(TIMESTAMP_PATTERN)
|
||||||
|
return nil unless match
|
||||||
|
|
||||||
|
year = match[1].to_i
|
||||||
|
month = match[2].to_i
|
||||||
|
day = match[3].to_i
|
||||||
|
hour = match[4].to_i
|
||||||
|
minute = match[5]&.to_i || 0
|
||||||
|
second = match[6]&.to_i || 0
|
||||||
|
fraction = match[7]
|
||||||
|
offset = match[8]
|
||||||
|
return nil unless valid_timestamp_components?(year, month, day, hour, minute, second)
|
||||||
|
return nil unless valid_offset?(offset)
|
||||||
|
|
||||||
|
nanoseconds = parse_nanoseconds(fraction)
|
||||||
|
timestamp =
|
||||||
|
if offset.present?
|
||||||
|
Time.new(
|
||||||
|
year,
|
||||||
|
month,
|
||||||
|
day,
|
||||||
|
hour,
|
||||||
|
minute,
|
||||||
|
second + Rational(nanoseconds, 1_000_000_000),
|
||||||
|
parse_offset(offset)).in_time_zone
|
||||||
|
else
|
||||||
|
Time.zone.local(year, month, day, hour, minute, second).change(nsec: nanoseconds)
|
||||||
|
end
|
||||||
|
|
||||||
|
from = timestamp.change(sec: 0, nsec: 0)
|
||||||
|
before =
|
||||||
|
if match[5].nil?
|
||||||
|
from + 1.hour
|
||||||
|
else
|
||||||
|
from + 1.minute
|
||||||
|
end
|
||||||
|
[from, before]
|
||||||
|
end
|
||||||
|
|
||||||
|
def self.parse_nanoseconds value
|
||||||
|
return 0 if value.blank?
|
||||||
|
|
||||||
|
digits = value[0, 9].ljust(9, '0')
|
||||||
|
Integer(digits, 10)
|
||||||
|
end
|
||||||
|
|
||||||
|
def self.parse_offset value
|
||||||
|
return '+00:00' if value == 'Z'
|
||||||
|
|
||||||
|
value.match?(/\A[+-]\d{2}:\d{2}\z/) ? value : "#{ value[0, 3] }:#{ value[3, 2] }"
|
||||||
|
end
|
||||||
|
|
||||||
|
def self.serialise_duration value
|
||||||
|
seconds = Float(value)
|
||||||
|
return nil unless seconds.positive?
|
||||||
|
|
||||||
|
milliseconds = (seconds * 1_000).round
|
||||||
|
seconds_string = (milliseconds / 1_000.0).to_s
|
||||||
|
seconds_string.end_with?('.0') ? seconds_string.delete_suffix('.0') : seconds_string
|
||||||
|
rescue ArgumentError, TypeError
|
||||||
|
nil
|
||||||
|
end
|
||||||
|
|
||||||
|
def self.valid_timestamp_components? year, month, day, hour, minute, second
|
||||||
|
return false unless Date.valid_date?(year, month, day)
|
||||||
|
return false unless hour.between?(0, 23)
|
||||||
|
return false unless minute.between?(0, 59)
|
||||||
|
return false unless second.between?(0, 59)
|
||||||
|
|
||||||
|
true
|
||||||
|
end
|
||||||
|
|
||||||
|
def self.valid_offset? value
|
||||||
|
return true if value.nil? || value == 'Z'
|
||||||
|
|
||||||
|
match = value.match(/\A([+-])(\d{2}):?(\d{2})\z/)
|
||||||
|
return false if match.nil?
|
||||||
|
|
||||||
|
hours = match[2].to_i
|
||||||
|
minutes = match[3].to_i
|
||||||
|
hours.between?(0, 23) && minutes.between?(0, 59)
|
||||||
|
end
|
||||||
|
|
||||||
|
def self.serialise_time value
|
||||||
|
return nil if value.nil?
|
||||||
|
|
||||||
|
value.nsec.zero? ? value.iso8601 : value.iso8601(9)
|
||||||
|
end
|
||||||
|
|
||||||
|
private_class_method :platform_tags,
|
||||||
|
:display_tags,
|
||||||
|
:original_created_range,
|
||||||
|
:parse_timestamp_range,
|
||||||
|
:parse_nanoseconds,
|
||||||
|
:parse_offset,
|
||||||
|
:serialise_duration,
|
||||||
|
:valid_timestamp_components?,
|
||||||
|
:valid_offset?,
|
||||||
|
:serialise_time
|
||||||
|
end
|
||||||
@@ -0,0 +1,11 @@
|
|||||||
|
class PostThumbnailAttachmentBuilder
|
||||||
|
def self.build thumbnail:, thumbnail_base:
|
||||||
|
if thumbnail_base.present?
|
||||||
|
return Post.remote_thumbnail_attachment(thumbnail_base)
|
||||||
|
end
|
||||||
|
|
||||||
|
return nil if thumbnail.blank?
|
||||||
|
|
||||||
|
Post.resized_thumbnail_attachment(thumbnail)
|
||||||
|
end
|
||||||
|
end
|
||||||
@@ -0,0 +1,34 @@
|
|||||||
|
class PostThumbnailUploadValidator
|
||||||
|
MAX_THUMBNAIL_BYTES = 20 * 1024 * 1024
|
||||||
|
ALLOWED_CONTENT_TYPES = Preview::ThumbnailFetcher::RASTER_IMAGE_CONTENT_TYPES.freeze
|
||||||
|
|
||||||
|
class InvalidUpload < StandardError; end
|
||||||
|
|
||||||
|
def self.validate! thumbnail
|
||||||
|
return if thumbnail.blank?
|
||||||
|
unless thumbnail.is_a?(ActionDispatch::Http::UploadedFile)
|
||||||
|
raise InvalidUpload, 'thumbnail upload が不正です.'
|
||||||
|
end
|
||||||
|
raise InvalidUpload, 'thumbnail file size が大きすぎます.' if thumbnail.size > MAX_THUMBNAIL_BYTES
|
||||||
|
raise InvalidUpload, 'サムネイル画像の形式が不正です.' unless allowed_content_type?(thumbnail.content_type)
|
||||||
|
bytes = thumbnail.read
|
||||||
|
raise InvalidUpload, 'サムネイル画像の形式が不正です.' if Post.svg_document_bytes?(bytes)
|
||||||
|
unless Post.raster_thumbnail_bytes?(bytes)
|
||||||
|
raise InvalidUpload, 'サムネイル画像の形式が不正です.'
|
||||||
|
end
|
||||||
|
|
||||||
|
attachment = Post.resized_thumbnail_attachment(
|
||||||
|
StringIO.new(bytes),
|
||||||
|
content_type: thumbnail.content_type)
|
||||||
|
attachment[:io].close if attachment[:io].respond_to?(:close)
|
||||||
|
rescue MiniMagick::Error, Timeout::Error
|
||||||
|
raise InvalidUpload, 'サムネイル画像の変換に失敗しました.'
|
||||||
|
ensure
|
||||||
|
thumbnail&.rewind if thumbnail.respond_to?(:rewind)
|
||||||
|
end
|
||||||
|
|
||||||
|
def self.allowed_content_type? content_type
|
||||||
|
mime_type = content_type.to_s.split(';', 2).first.to_s.downcase.strip
|
||||||
|
ALLOWED_CONTENT_TYPES.include?(mime_type)
|
||||||
|
end
|
||||||
|
end
|
||||||
@@ -0,0 +1,13 @@
|
|||||||
|
class PostUrlNormaliser
|
||||||
|
def self.normalise raw_url
|
||||||
|
value = raw_url.to_s.strip
|
||||||
|
uri = URI.parse(value)
|
||||||
|
return nil unless uri.is_a?(URI::HTTP) && uri.host.present?
|
||||||
|
|
||||||
|
uri.host = uri.host.downcase
|
||||||
|
uri.path = uri.path.sub(/\/\z/, '') if uri.path.present?
|
||||||
|
PostUrlSanitisationRule.sanitise(uri.to_s)
|
||||||
|
rescue URI::InvalidURIError
|
||||||
|
nil
|
||||||
|
end
|
||||||
|
end
|
||||||
@@ -25,6 +25,7 @@ class PostVersionRecorder < VersionRecorder
|
|||||||
thumbnail_base: @record.thumbnail_base,
|
thumbnail_base: @record.thumbnail_base,
|
||||||
video_ms: @record.video_ms,
|
video_ms: @record.video_ms,
|
||||||
tags: @record.snapshot_tag_names.join(' '),
|
tags: @record.snapshot_tag_names.join(' '),
|
||||||
|
tags_json: @record.snapshot_tags_json,
|
||||||
parent_post_ids: @record.snapshot_parent_post_ids.join(' '),
|
parent_post_ids: @record.snapshot_parent_post_ids.join(' '),
|
||||||
original_created_from: @record.original_created_from,
|
original_created_from: @record.original_created_from,
|
||||||
original_created_before: @record.original_created_before }
|
original_created_before: @record.original_created_before }
|
||||||
|
|||||||
@@ -0,0 +1,21 @@
|
|||||||
|
module Preview
|
||||||
|
class HtmlMetadataExtractor
|
||||||
|
IMAGE_SELECTORS = [
|
||||||
|
'meta[property="og:image"]',
|
||||||
|
'meta[name="twitter:image"]',
|
||||||
|
'meta[name="thumbnail"]'
|
||||||
|
].freeze
|
||||||
|
|
||||||
|
def self.extract(response)
|
||||||
|
document = Nokogiri::HTML.parse(response.body)
|
||||||
|
image_url = IMAGE_SELECTORS.filter_map {
|
||||||
|
document.at_css(_1)&.[]('content')&.strip.presence
|
||||||
|
}.first
|
||||||
|
|
||||||
|
{ title: document.at_css('title')&.text&.strip,
|
||||||
|
image_url: image_url ? URI.join(response.url, image_url).to_s : nil }
|
||||||
|
rescue URI::InvalidURIError
|
||||||
|
{ title: document.at_css('title')&.text&.strip, image_url: nil }
|
||||||
|
end
|
||||||
|
end
|
||||||
|
end
|
||||||
@@ -0,0 +1,123 @@
|
|||||||
|
require 'net/http'
|
||||||
|
require 'json'
|
||||||
|
|
||||||
|
module Preview
|
||||||
|
class HttpFetcher
|
||||||
|
class FetchFailed < StandardError; end
|
||||||
|
class FetchTimeout < FetchFailed; end
|
||||||
|
class ResponseTooLarge < FetchFailed; end
|
||||||
|
|
||||||
|
MAX_REDIRECTS = 5
|
||||||
|
DEFAULT_MAX_BYTES = 5.megabytes
|
||||||
|
|
||||||
|
Response = Data.define(:body, :content_type, :url)
|
||||||
|
|
||||||
|
def self.fetch(raw_url, max_bytes: DEFAULT_MAX_BYTES, redirects: MAX_REDIRECTS)
|
||||||
|
uri, addresses = UrlSafety.validate(raw_url)
|
||||||
|
response = request(uri, addresses.first, max_bytes)
|
||||||
|
|
||||||
|
if response.is_a?(Net::HTTPRedirection)
|
||||||
|
location = response['location']
|
||||||
|
if redirects.zero?
|
||||||
|
log_failure(:redirect_limit,
|
||||||
|
url: uri.to_s,
|
||||||
|
redirects:,
|
||||||
|
location:,
|
||||||
|
content_type: response['content-type'],
|
||||||
|
content_length: response['content-length'])
|
||||||
|
raise FetchFailed, 'redirect が多すぎます.'
|
||||||
|
end
|
||||||
|
|
||||||
|
if location.blank?
|
||||||
|
log_failure(:blank_redirect_location,
|
||||||
|
url: uri.to_s,
|
||||||
|
redirects:,
|
||||||
|
content_type: response['content-type'],
|
||||||
|
content_length: response['content-length'])
|
||||||
|
raise FetchFailed, 'redirect 先が不正です.'
|
||||||
|
end
|
||||||
|
|
||||||
|
redirect_url =
|
||||||
|
begin
|
||||||
|
URI.join(uri, location).to_s
|
||||||
|
rescue URI::InvalidURIError => e
|
||||||
|
log_failure(:invalid_redirect_location,
|
||||||
|
url: uri.to_s,
|
||||||
|
redirects:,
|
||||||
|
location:,
|
||||||
|
error: e.class.name,
|
||||||
|
message: e.message)
|
||||||
|
raise FetchFailed, 'redirect 先が不正です.'
|
||||||
|
end
|
||||||
|
|
||||||
|
return fetch(redirect_url, max_bytes:, redirects: redirects - 1)
|
||||||
|
end
|
||||||
|
|
||||||
|
unless response.is_a?(Net::HTTPSuccess)
|
||||||
|
log_failure(:http_status,
|
||||||
|
url: uri.to_s,
|
||||||
|
code: response.code,
|
||||||
|
content_type: response['content-type'],
|
||||||
|
content_length: response['content-length'])
|
||||||
|
raise FetchFailed, "外部サーバーが HTTP #{ response.code } を返しました."
|
||||||
|
end
|
||||||
|
|
||||||
|
Response.new(response.body, response['content-type'].to_s, uri.to_s)
|
||||||
|
rescue Net::OpenTimeout, Net::ReadTimeout, Timeout::Error => e
|
||||||
|
log_failure(:timeout, url: uri&.to_s || raw_url, error: e.class.name, message: e.message)
|
||||||
|
raise FetchTimeout, e.message
|
||||||
|
rescue SocketError, SystemCallError, OpenSSL::SSL::SSLError, EOFError => e
|
||||||
|
log_failure(:network_error, url: uri&.to_s || raw_url, error: e.class.name,
|
||||||
|
message: e.message)
|
||||||
|
raise FetchFailed, e.message
|
||||||
|
end
|
||||||
|
|
||||||
|
def self.request(uri, ip_address, max_bytes)
|
||||||
|
http = Net::HTTP.new(uri.host, uri.port)
|
||||||
|
http.ipaddr = ip_address
|
||||||
|
http.use_ssl = uri.scheme == 'https'
|
||||||
|
http.open_timeout = 5
|
||||||
|
http.read_timeout = 8
|
||||||
|
http.write_timeout = 5
|
||||||
|
|
||||||
|
request = Net::HTTP::Get.new(uri)
|
||||||
|
request['User-Agent'] = 'BTRC-Hub thumbnail preview'
|
||||||
|
request['Accept'] = 'text/html,image/*;q=0.9,*/*;q=0.1'
|
||||||
|
|
||||||
|
http.request(request) do |response|
|
||||||
|
length = response['content-length'].to_i
|
||||||
|
if length > max_bytes
|
||||||
|
log_failure(:response_too_large,
|
||||||
|
url: uri.to_s,
|
||||||
|
content_type: response['content-type'],
|
||||||
|
content_length: response['content-length'],
|
||||||
|
max_bytes:)
|
||||||
|
raise ResponseTooLarge, '外部データが大きすぎます.'
|
||||||
|
end
|
||||||
|
|
||||||
|
body = +''
|
||||||
|
response.read_body do |chunk|
|
||||||
|
body << chunk
|
||||||
|
next unless body.bytesize > max_bytes
|
||||||
|
|
||||||
|
log_failure(:response_too_large,
|
||||||
|
url: uri.to_s,
|
||||||
|
content_type: response['content-type'],
|
||||||
|
content_length: response['content-length'],
|
||||||
|
bytes_read: body.bytesize,
|
||||||
|
max_bytes:)
|
||||||
|
raise ResponseTooLarge, '外部データが大きすぎます.'
|
||||||
|
end
|
||||||
|
response.instance_variable_set(:@body, body)
|
||||||
|
response.instance_variable_set(:@read, true)
|
||||||
|
return response
|
||||||
|
end
|
||||||
|
end
|
||||||
|
|
||||||
|
def self.log_failure(reason, **payload)
|
||||||
|
Rails.logger.warn("preview_http_fetcher_failure #{ { reason:, **payload }.to_json }")
|
||||||
|
end
|
||||||
|
|
||||||
|
private_class_method :request, :log_failure
|
||||||
|
end
|
||||||
|
end
|
||||||
@@ -0,0 +1,34 @@
|
|||||||
|
module Preview
|
||||||
|
class KnownSiteExtractor
|
||||||
|
def self.thumbnail_url(uri)
|
||||||
|
youtube_thumbnail(uri)
|
||||||
|
end
|
||||||
|
|
||||||
|
def self.youtube_video_id(uri)
|
||||||
|
case uri.host&.downcase
|
||||||
|
when 'youtu.be'
|
||||||
|
uri.path.split('/').reject(&:blank?).first
|
||||||
|
when 'www.youtube.com', 'youtube.com', 'm.youtube.com'
|
||||||
|
uri.path == '/watch' ? URI.decode_www_form(uri.query.to_s).to_h['v'] : nil
|
||||||
|
end&.then { _1 if _1.match?(/\A[A-Za-z0-9_-]{6,20}\z/) }
|
||||||
|
end
|
||||||
|
|
||||||
|
def self.niconico_video_id(uri)
|
||||||
|
case uri.host&.downcase
|
||||||
|
when 'www.nicovideo.jp', 'nicovideo.jp'
|
||||||
|
uri.path[%r{\A/watch/(sm\d+)\z}, 1]
|
||||||
|
when 'nico.ms'
|
||||||
|
uri.path[%r{\A/(sm\d+)\z}, 1]
|
||||||
|
end
|
||||||
|
end
|
||||||
|
|
||||||
|
def self.youtube_thumbnail(uri)
|
||||||
|
id = youtube_video_id(uri)
|
||||||
|
return unless id
|
||||||
|
|
||||||
|
"https://i.ytimg.com/vi/#{ id }/hqdefault.jpg"
|
||||||
|
end
|
||||||
|
|
||||||
|
private_class_method :youtube_thumbnail
|
||||||
|
end
|
||||||
|
end
|
||||||
@@ -0,0 +1,102 @@
|
|||||||
|
module Preview
|
||||||
|
class ThumbnailFetcher
|
||||||
|
class GenerationFailed < StandardError; end
|
||||||
|
RASTER_IMAGE_CONTENT_TYPES = [
|
||||||
|
'image/jpeg', 'image/png', 'image/gif', 'image/webp'
|
||||||
|
].freeze
|
||||||
|
HTML_MAX_BYTES = 1.megabyte
|
||||||
|
NICONICO_XML_MAX_BYTES = 256.kilobytes
|
||||||
|
|
||||||
|
def self.fetch(raw_url)
|
||||||
|
uri, = UrlSafety.validate(raw_url)
|
||||||
|
|
||||||
|
known_url = KnownSiteExtractor.thumbnail_url(uri)
|
||||||
|
image = fetch_image_or_nil(known_url) if known_url
|
||||||
|
return image if image
|
||||||
|
|
||||||
|
niconico_url = niconico_thumbnail_url(uri)
|
||||||
|
image = fetch_image_or_nil(niconico_url) if niconico_url
|
||||||
|
return image if image
|
||||||
|
|
||||||
|
page = HttpFetcher.fetch(uri.to_s, max_bytes: HTML_MAX_BYTES)
|
||||||
|
metadata = HtmlMetadataExtractor.extract(page)
|
||||||
|
raise GenerationFailed, 'サムネール画像が見つかりませんでした.' if metadata[:image_url].blank?
|
||||||
|
|
||||||
|
fetch_image!(metadata[:image_url])
|
||||||
|
end
|
||||||
|
|
||||||
|
def self.fetch_image_response(raw_url)
|
||||||
|
uri, = UrlSafety.validate(raw_url)
|
||||||
|
response = HttpFetcher.fetch(uri.to_s)
|
||||||
|
unless Post.remote_thumbnail_image_bytes?(
|
||||||
|
response.body,
|
||||||
|
content_type: response.content_type)
|
||||||
|
raise GenerationFailed, 'サムネール画像が見つかりませんでした.'
|
||||||
|
end
|
||||||
|
|
||||||
|
response
|
||||||
|
rescue HttpFetcher::FetchTimeout
|
||||||
|
raise
|
||||||
|
rescue HttpFetcher::ResponseTooLarge
|
||||||
|
raise
|
||||||
|
rescue HttpFetcher::FetchFailed
|
||||||
|
raise GenerationFailed, 'サムネール画像を取得できませんでした.'
|
||||||
|
end
|
||||||
|
|
||||||
|
def self.title(raw_url)
|
||||||
|
uri, = UrlSafety.validate(raw_url)
|
||||||
|
HtmlMetadataExtractor.extract(
|
||||||
|
HttpFetcher.fetch(uri.to_s, max_bytes: HTML_MAX_BYTES))[:title]
|
||||||
|
end
|
||||||
|
|
||||||
|
def self.fetch_image_or_nil(url)
|
||||||
|
return nil if url.blank?
|
||||||
|
|
||||||
|
response = HttpFetcher.fetch(url)
|
||||||
|
return nil unless Post.remote_thumbnail_image_bytes?(
|
||||||
|
response.body,
|
||||||
|
content_type: response.content_type)
|
||||||
|
|
||||||
|
response.body
|
||||||
|
rescue HttpFetcher::FetchTimeout
|
||||||
|
raise
|
||||||
|
rescue HttpFetcher::FetchFailed
|
||||||
|
nil
|
||||||
|
end
|
||||||
|
|
||||||
|
def self.fetch_image!(url)
|
||||||
|
fetch_image_response(url).body
|
||||||
|
end
|
||||||
|
|
||||||
|
def self.niconico_thumbnail_url(uri)
|
||||||
|
video_id = KnownSiteExtractor.niconico_video_id(uri)
|
||||||
|
return nil if video_id.blank?
|
||||||
|
|
||||||
|
response = HttpFetcher.fetch("https://ext.nicovideo.jp/api/getthumbinfo/#{ video_id }",
|
||||||
|
max_bytes: NICONICO_XML_MAX_BYTES)
|
||||||
|
xml = Nokogiri::XML(response.body)
|
||||||
|
return nil unless xml.at_xpath('/nicovideo_thumb_response/@status')&.value == 'ok'
|
||||||
|
|
||||||
|
xml.at_xpath('//thumbnail_url')&.text&.strip.presence
|
||||||
|
rescue HttpFetcher::FetchFailed, HttpFetcher::FetchTimeout => e
|
||||||
|
payload = {
|
||||||
|
url: uri.to_s,
|
||||||
|
video_id:,
|
||||||
|
error: e.class.name,
|
||||||
|
message: e.message }
|
||||||
|
Rails.logger.info("preview_niconico_getthumbinfo_fallback #{ payload.to_json }")
|
||||||
|
nil
|
||||||
|
rescue Nokogiri::XML::SyntaxError => e
|
||||||
|
payload = {
|
||||||
|
url: uri.to_s,
|
||||||
|
video_id:,
|
||||||
|
error: e.class.name,
|
||||||
|
message: e.message }
|
||||||
|
Rails.logger.info("preview_niconico_getthumbinfo_fallback #{ payload.to_json }")
|
||||||
|
nil
|
||||||
|
end
|
||||||
|
|
||||||
|
private_class_method :fetch_image_or_nil, :fetch_image!,
|
||||||
|
:niconico_thumbnail_url
|
||||||
|
end
|
||||||
|
end
|
||||||
@@ -0,0 +1,55 @@
|
|||||||
|
require 'resolv'
|
||||||
|
require 'ipaddr'
|
||||||
|
require 'uri'
|
||||||
|
|
||||||
|
module Preview
|
||||||
|
class UrlSafety
|
||||||
|
class UnsafeUrl < StandardError; end
|
||||||
|
|
||||||
|
FORBIDDEN_NETWORKS = [
|
||||||
|
'0.0.0.0/8', '10.0.0.0/8', '100.64.0.0/10', '127.0.0.0/8',
|
||||||
|
'169.254.0.0/16', '172.16.0.0/12', '192.0.0.0/24',
|
||||||
|
'192.0.2.0/24', '192.168.0.0/16', '198.18.0.0/15',
|
||||||
|
'198.51.100.0/24', '203.0.113.0/24', '224.0.0.0/4',
|
||||||
|
'240.0.0.0/4', '::/128', '::1/128', 'fc00::/7', 'fe80::/10',
|
||||||
|
'ff00::/8', '2001:db8::/32', '::ffff:0:0/96'
|
||||||
|
].map { IPAddr.new(_1) }.freeze
|
||||||
|
|
||||||
|
def self.validate(raw_url)
|
||||||
|
value = raw_url.to_s.strip
|
||||||
|
if value.match?(/\A[a-z][a-z0-9+\-.]*:/i)
|
||||||
|
unless value.match?(/\Ahttps?:\/\//i)
|
||||||
|
raise UnsafeUrl, 'http または https の URL を指定してください.'
|
||||||
|
end
|
||||||
|
else
|
||||||
|
value = "http://#{ value }"
|
||||||
|
end
|
||||||
|
uri = URI.parse(value)
|
||||||
|
|
||||||
|
unless ['http', 'https'].include?(uri.scheme&.downcase) && uri.host.present?
|
||||||
|
raise UnsafeUrl, 'http または https の URL を指定してください.'
|
||||||
|
end
|
||||||
|
raise UnsafeUrl, 'userinfo つき URL は使用できません.' if uri.userinfo.present?
|
||||||
|
|
||||||
|
addresses = Resolv.getaddresses(uri.host)
|
||||||
|
raise UnsafeUrl, 'URL のホストを解決できません.' if addresses.empty?
|
||||||
|
|
||||||
|
parsed_addresses = addresses.map { IPAddr.new(_1) }
|
||||||
|
if parsed_addresses.any? { |address| forbidden?(address) }
|
||||||
|
raise UnsafeUrl, '安全でない接続先は使用できません.'
|
||||||
|
end
|
||||||
|
|
||||||
|
[uri, parsed_addresses.map(&:to_s)]
|
||||||
|
rescue Resolv::ResolvError
|
||||||
|
raise UnsafeUrl, 'URL のホストを解決できません.'
|
||||||
|
rescue URI::InvalidURIError, IPAddr::InvalidAddressError
|
||||||
|
raise UnsafeUrl, 'URL が不正です.'
|
||||||
|
end
|
||||||
|
|
||||||
|
def self.forbidden?(address)
|
||||||
|
FORBIDDEN_NETWORKS.any? { _1.include?(address) }
|
||||||
|
end
|
||||||
|
|
||||||
|
private_class_method :forbidden?
|
||||||
|
end
|
||||||
|
end
|
||||||
@@ -1,32 +1,16 @@
|
|||||||
class TagVersioning
|
class TagVersioning
|
||||||
def self.record! tag, event_type:, created_by_user:
|
def self.record! tag, event_type:, created_by_user:
|
||||||
if tag.nico?
|
TagVersionRecorder.record!(tag:, event_type:, created_by_user:)
|
||||||
NicoTagVersionRecorder.record!(tag:, event_type:, created_by_user:)
|
|
||||||
else
|
|
||||||
TagVersionRecorder.record!(tag:, event_type:, created_by_user:)
|
|
||||||
end
|
|
||||||
end
|
end
|
||||||
|
|
||||||
def self.ensure_snapshot! tag, created_by_user:
|
def self.ensure_snapshot! tag, created_by_user:
|
||||||
if tag.nico?
|
return if tag.tag_versions.exists?
|
||||||
return if tag.nico_tag_versions.exists?
|
|
||||||
|
|
||||||
NicoTagVersionRecorder.record!(tag:, event_type: :create, created_by_user:)
|
TagVersionRecorder.record!(tag:, event_type: :create, created_by_user:)
|
||||||
else
|
|
||||||
return if tag.tag_versions.exists?
|
|
||||||
|
|
||||||
TagVersionRecorder.record!(tag:, event_type: :create, created_by_user:)
|
|
||||||
end
|
|
||||||
end
|
end
|
||||||
|
|
||||||
def self.record_tag_snapshot! tag, created_by_user:
|
def self.record_tag_snapshot! tag, created_by_user:
|
||||||
event_type =
|
event_type = tag.tag_versions.exists? ? :update : :create
|
||||||
if tag.nico?
|
|
||||||
tag.nico_tag_versions.exists? ? :update : :create
|
|
||||||
else
|
|
||||||
tag.tag_versions.exists? ? :update : :create
|
|
||||||
end
|
|
||||||
|
|
||||||
record!(tag, event_type:, created_by_user:)
|
record!(tag, event_type:, created_by_user:)
|
||||||
end
|
end
|
||||||
|
|
||||||
|
|||||||
@@ -47,10 +47,14 @@ class VersionRecorder
|
|||||||
end
|
end
|
||||||
|
|
||||||
def update_record_version_no! version_no
|
def update_record_version_no! version_no
|
||||||
|
return unless tracks_version_no_on_record?
|
||||||
|
|
||||||
@record.update_columns(version_no:)
|
@record.update_columns(version_no:)
|
||||||
@record.version_no = version_no
|
@record.version_no = version_no
|
||||||
end
|
end
|
||||||
|
|
||||||
|
def tracks_version_no_on_record? = true
|
||||||
|
|
||||||
def validate_version_sequence! latest
|
def validate_version_sequence! latest
|
||||||
if !(latest) && @event_type != 'create'
|
if !(latest) && @event_type != 'create'
|
||||||
raise "#{ version_class.name } first event must be create"
|
raise "#{ version_class.name } first event must be create"
|
||||||
@@ -60,7 +64,7 @@ class VersionRecorder
|
|||||||
raise "#{ version_class.name } create event already exists"
|
raise "#{ version_class.name } create event already exists"
|
||||||
end
|
end
|
||||||
|
|
||||||
return unless latest
|
return if !(latest) || !(tracks_version_no_on_record?)
|
||||||
|
|
||||||
if @record.version_no != latest.version_no
|
if @record.version_no != latest.version_no
|
||||||
raise ("#{ record_class.name }##{ @record.id } version_no is #{ @record.version_no }, " +
|
raise ("#{ record_class.name }##{ @record.id } version_no is #{ @record.version_no }, " +
|
||||||
|
|||||||
@@ -1,4 +1,3 @@
|
|||||||
require 'open-uri'
|
|
||||||
require 'set'
|
require 'set'
|
||||||
require 'time'
|
require 'time'
|
||||||
|
|
||||||
@@ -104,7 +103,7 @@ module Youtube
|
|||||||
end
|
end
|
||||||
|
|
||||||
def sync_post_tags! post, desired_tag_ids, current_tag_ids: nil
|
def sync_post_tags! post, desired_tag_ids, current_tag_ids: nil
|
||||||
current_tag_ids ||= PostTag.kept.where(post_id: post.id).pluck(:tag_id).to_set
|
current_tag_ids ||= PostTag.where(post_id: post.id).pluck(:tag_id).to_set
|
||||||
desired_tag_ids = desired_tag_ids.compact.to_set
|
desired_tag_ids = desired_tag_ids.compact.to_set
|
||||||
|
|
||||||
to_add = desired_tag_ids - current_tag_ids
|
to_add = desired_tag_ids - current_tag_ids
|
||||||
@@ -118,8 +117,8 @@ module Youtube
|
|||||||
end
|
end
|
||||||
end
|
end
|
||||||
|
|
||||||
PostTag.where(post_id: post.id, tag_id: to_remove.to_a).kept.find_each do |pt|
|
PostTag.where(post_id: post.id, tag_id: to_remove.to_a).find_each do |pt|
|
||||||
pt.discard_by!(nil)
|
pt.destroy!
|
||||||
end
|
end
|
||||||
end
|
end
|
||||||
|
|
||||||
@@ -127,12 +126,12 @@ module Youtube
|
|||||||
return if post.thumbnail.attached?
|
return if post.thumbnail.attached?
|
||||||
return if thumbnail_url.blank?
|
return if thumbnail_url.blank?
|
||||||
|
|
||||||
post.thumbnail.attach(
|
post.attach_thumbnail_from_url!(thumbnail_url)
|
||||||
io: URI.open(thumbnail_url),
|
rescue Post::RemoteThumbnailFetchFailed => e
|
||||||
filename: File.basename(URI.parse(thumbnail_url).path),
|
Rails.logger.info("youtube_sync_thumbnail_fetch_failed #{ { post_id: post.id,
|
||||||
content_type: 'image/jpeg')
|
thumbnail_url:,
|
||||||
|
error: e.class.name,
|
||||||
post.resized_thumbnail!
|
message: e.message }.to_json }")
|
||||||
end
|
end
|
||||||
|
|
||||||
def youtube_url_regexp id
|
def youtube_url_regexp id
|
||||||
|
|||||||
@@ -1,4 +1,4 @@
|
|||||||
require "active_support/core_ext/integer/time"
|
require 'active_support/core_ext/integer/time'
|
||||||
|
|
||||||
Rails.application.configure do
|
Rails.application.configure do
|
||||||
# Settings specified here will take precedence over those in config/application.rb.
|
# Settings specified here will take precedence over those in config/application.rb.
|
||||||
@@ -17,8 +17,8 @@ Rails.application.configure do
|
|||||||
|
|
||||||
# Enable/disable Action Controller caching. By default Action Controller caching is disabled.
|
# Enable/disable Action Controller caching. By default Action Controller caching is disabled.
|
||||||
# Run rails dev:cache to toggle Action Controller caching.
|
# Run rails dev:cache to toggle Action Controller caching.
|
||||||
if Rails.root.join("tmp/caching-dev.txt").exist?
|
if Rails.root.join('tmp/caching-dev.txt').exist?
|
||||||
config.public_file_server.headers = { "cache-control" => "public, max-age=#{2.days.to_i}" }
|
config.public_file_server.headers = { 'cache-control' => "public, max-age=#{2.days.to_i}" }
|
||||||
else
|
else
|
||||||
config.action_controller.perform_caching = false
|
config.action_controller.perform_caching = false
|
||||||
end
|
end
|
||||||
@@ -36,7 +36,11 @@ Rails.application.configure do
|
|||||||
config.action_mailer.perform_caching = false
|
config.action_mailer.perform_caching = false
|
||||||
|
|
||||||
# Set localhost to be used by links generated in mailer templates.
|
# Set localhost to be used by links generated in mailer templates.
|
||||||
config.action_mailer.default_url_options = { host: "localhost", port: 3000 }
|
config.action_mailer.default_url_options = { host: 'localhost', port: 3000 }
|
||||||
|
Rails.application.routes.default_url_options.merge!(
|
||||||
|
host: 'localhost',
|
||||||
|
port: 3002,
|
||||||
|
protocol: 'http')
|
||||||
|
|
||||||
# Print deprecation notices to the Rails logger.
|
# Print deprecation notices to the Rails logger.
|
||||||
config.active_support.deprecation = :log
|
config.active_support.deprecation = :log
|
||||||
|
|||||||
@@ -52,8 +52,9 @@ Rails.application.routes.draw do
|
|||||||
|
|
||||||
resources :posts, only: [:index, :show, :create, :update] do
|
resources :posts, only: [:index, :show, :create, :update] do
|
||||||
collection do
|
collection do
|
||||||
|
get :metadata
|
||||||
|
post :bulk
|
||||||
get :random
|
get :random
|
||||||
get :changes
|
|
||||||
get :versions, to: 'post_versions#index'
|
get :versions, to: 'post_versions#index'
|
||||||
end
|
end
|
||||||
|
|
||||||
@@ -81,6 +82,11 @@ Rails.application.routes.draw do
|
|||||||
end
|
end
|
||||||
end
|
end
|
||||||
|
|
||||||
|
get 'users/settings', to: 'user_settings#show'
|
||||||
|
patch 'users/settings', to: 'user_settings#update'
|
||||||
|
get 'users/theme_slots', to: 'user_theme_slots#index'
|
||||||
|
put 'users/theme_slots/:base_theme/:slot_no', to: 'user_theme_slots#update'
|
||||||
|
|
||||||
resources :users, only: [:create, :update] do
|
resources :users, only: [:create, :update] do
|
||||||
collection do
|
collection do
|
||||||
post :verify
|
post :verify
|
||||||
|
|||||||
@@ -0,0 +1,31 @@
|
|||||||
|
class RebuildSettingsAsTypedUserSettings < ActiveRecord::Migration[8.0]
|
||||||
|
def change
|
||||||
|
remove_foreign_key :settings, :users if foreign_key_exists?(:settings, :users)
|
||||||
|
remove_index :settings, :user_id if index_exists?(:settings, :user_id)
|
||||||
|
|
||||||
|
remove_column :settings, :key, :string if column_exists?(:settings, :key)
|
||||||
|
remove_column :settings, :value, :json if column_exists?(:settings, :value)
|
||||||
|
|
||||||
|
change_column_null :settings, :user_id, false
|
||||||
|
|
||||||
|
add_column :settings, :theme, :string, null: false, default: 'system'
|
||||||
|
add_column :settings,
|
||||||
|
:auto_fetch_title,
|
||||||
|
:string,
|
||||||
|
null: false,
|
||||||
|
default: 'manual'
|
||||||
|
add_column :settings,
|
||||||
|
:auto_fetch_thumbnail,
|
||||||
|
:string,
|
||||||
|
null: false,
|
||||||
|
default: 'manual'
|
||||||
|
add_column :settings,
|
||||||
|
:wiki_editor_mode,
|
||||||
|
:string,
|
||||||
|
null: false,
|
||||||
|
default: 'split'
|
||||||
|
|
||||||
|
add_index :settings, :user_id, unique: true
|
||||||
|
add_foreign_key :settings, :users
|
||||||
|
end
|
||||||
|
end
|
||||||
@@ -0,0 +1,22 @@
|
|||||||
|
class CreateUserThemeSlots < ActiveRecord::Migration[8.0]
|
||||||
|
def change
|
||||||
|
create_table :user_theme_slots do |t|
|
||||||
|
t.references :user, null: false, foreign_key: true
|
||||||
|
t.string :base_theme, null: false
|
||||||
|
t.integer :slot_no, null: false
|
||||||
|
t.json :tokens, null: false
|
||||||
|
t.timestamps
|
||||||
|
end
|
||||||
|
|
||||||
|
add_index :user_theme_slots,
|
||||||
|
[:user_id, :base_theme, :slot_no],
|
||||||
|
unique: true,
|
||||||
|
name: 'index_user_theme_slots_on_user_theme_and_slot'
|
||||||
|
add_check_constraint :user_theme_slots,
|
||||||
|
"base_theme IN ('light', 'dark')",
|
||||||
|
name: 'user_theme_slots_base_theme_valid'
|
||||||
|
add_check_constraint :user_theme_slots,
|
||||||
|
'slot_no BETWEEN 1 AND 3',
|
||||||
|
name: 'user_theme_slots_slot_no_valid'
|
||||||
|
end
|
||||||
|
end
|
||||||
@@ -0,0 +1,61 @@
|
|||||||
|
class CreatePostUrlSanitisationRules < ActiveRecord::Migration[8.0]
|
||||||
|
class PostUrlSanitisationRule < ActiveRecord::Base
|
||||||
|
self.table_name = 'post_url_sanitisation_rules'
|
||||||
|
end
|
||||||
|
|
||||||
|
def up
|
||||||
|
create_table :post_url_sanitisation_rules, id: :integer, primary_key: :priority do |t|
|
||||||
|
t.string :source_pattern, null: false
|
||||||
|
t.string :replacement, null: false
|
||||||
|
t.timestamps
|
||||||
|
t.datetime :discarded_at
|
||||||
|
|
||||||
|
t.index :source_pattern, unique: true
|
||||||
|
t.index :discarded_at
|
||||||
|
end
|
||||||
|
|
||||||
|
now = Time.current
|
||||||
|
|
||||||
|
PostUrlSanitisationRule.insert_all!([
|
||||||
|
{ priority: 10,
|
||||||
|
source_pattern: '\Ahttps?://youtu\.be/([^/?#]+)(?:[?#].*)?\z',
|
||||||
|
replacement: 'https://www.youtube.com/watch?v=\1',
|
||||||
|
created_at: now,
|
||||||
|
updated_at: now },
|
||||||
|
{ priority: 20,
|
||||||
|
source_pattern: '\Ahttps?://(?:www\.|m\.)?youtube\.com/live/([^/?#]+)(?:[?#].*)?\z',
|
||||||
|
replacement: 'https://www.youtube.com/watch?v=\1',
|
||||||
|
created_at: now,
|
||||||
|
updated_at: now },
|
||||||
|
{ priority: 30,
|
||||||
|
source_pattern: '\Ahttps?://(?:www\.|m\.)?youtube\.com/shorts/([^/?#]+)(?:[?#].*)?\z',
|
||||||
|
replacement: 'https://www.youtube.com/watch?v=\1',
|
||||||
|
created_at: now,
|
||||||
|
updated_at: now },
|
||||||
|
{ priority: 40,
|
||||||
|
source_pattern: '\Ahttps?://(?:www\.|m\.)?youtube\.com/embed/([^/?#]+)(?:[?#].*)?\z',
|
||||||
|
replacement: 'https://www.youtube.com/watch?v=\1',
|
||||||
|
created_at: now,
|
||||||
|
updated_at: now },
|
||||||
|
{ priority: 50,
|
||||||
|
source_pattern:
|
||||||
|
'\Ahttps?://(?:www\.|m\.)?youtube\.com/watch\?(?:[^#&]+&)*v=([^&#]+)(?:[&#].*)?\z',
|
||||||
|
replacement: 'https://www.youtube.com/watch?v=\1',
|
||||||
|
created_at: now,
|
||||||
|
updated_at: now },
|
||||||
|
{ priority: 60,
|
||||||
|
source_pattern: '\Ahttps?://nico\.ms/([^/?#]+)(?:[?#].*)?\z',
|
||||||
|
replacement: 'https://www.nicovideo.jp/watch/\1',
|
||||||
|
created_at: now,
|
||||||
|
updated_at: now },
|
||||||
|
{ priority: 70,
|
||||||
|
source_pattern: '\Ahttps?://(?:www\.)?nicovideo\.jp/watch/([^?#/]+)(?:[?#].*)?\z',
|
||||||
|
replacement: 'https://www.nicovideo.jp/watch/\1',
|
||||||
|
created_at: now,
|
||||||
|
updated_at: now }])
|
||||||
|
end
|
||||||
|
|
||||||
|
def down
|
||||||
|
drop_table :post_url_sanitisation_rules
|
||||||
|
end
|
||||||
|
end
|
||||||
@@ -0,0 +1,299 @@
|
|||||||
|
class AddTagsJsonToPostVersions < ActiveRecord::Migration[8.0]
|
||||||
|
RESOLUTION_GRACE = 1.second
|
||||||
|
SECTION_LITERAL_PATTERN = /\[[^\[\]\s]*-[^\[\]\s]*\]\z/
|
||||||
|
|
||||||
|
class MigrationPostVersion < ActiveRecord::Base
|
||||||
|
self.table_name = 'post_versions'
|
||||||
|
end
|
||||||
|
|
||||||
|
class MigrationTag < ActiveRecord::Base
|
||||||
|
self.table_name = 'tags'
|
||||||
|
end
|
||||||
|
|
||||||
|
class MigrationTagName < ActiveRecord::Base
|
||||||
|
self.table_name = 'tag_names'
|
||||||
|
end
|
||||||
|
|
||||||
|
class MigrationTagVersion < ActiveRecord::Base
|
||||||
|
self.table_name = 'tag_versions'
|
||||||
|
end
|
||||||
|
|
||||||
|
class MigrationNicoTagVersion < ActiveRecord::Base
|
||||||
|
self.table_name = 'nico_tag_versions'
|
||||||
|
end
|
||||||
|
|
||||||
|
def up
|
||||||
|
add_column :post_versions, :tags_json, :json, after: :tags
|
||||||
|
MigrationPostVersion.reset_column_information
|
||||||
|
|
||||||
|
backfill_missing_initial_tag_versions!
|
||||||
|
intervals_by_name = build_intervals_by_name
|
||||||
|
|
||||||
|
say_with_time 'Backfilling post_versions.tags_json' do
|
||||||
|
MigrationPostVersion.where(tags_json: nil).find_each(batch_size: 500) do |version|
|
||||||
|
version.update_columns(tags_json: build_tags_json(version, intervals_by_name))
|
||||||
|
end
|
||||||
|
end
|
||||||
|
|
||||||
|
change_column_null :post_versions, :tags_json, false
|
||||||
|
|
||||||
|
schema = connection.quote(JSON.generate({
|
||||||
|
type: 'array',
|
||||||
|
items: { type: 'object',
|
||||||
|
properties: { id: { type: 'integer', minimum: 1 },
|
||||||
|
version_no: { type: 'integer', minimum: 1 },
|
||||||
|
name: { type: 'string', minLength: 1 },
|
||||||
|
category: { type: 'string', enum: ['deerjikist',
|
||||||
|
'meme',
|
||||||
|
'character',
|
||||||
|
'general',
|
||||||
|
'material',
|
||||||
|
'meta',
|
||||||
|
'nico'] },
|
||||||
|
sections: { type: 'array',
|
||||||
|
items: { type: 'object',
|
||||||
|
properties: { begin_ms: { type: 'integer',
|
||||||
|
minimum: 0 },
|
||||||
|
end_ms: { type: ['integer',
|
||||||
|
'null'],
|
||||||
|
minimum: 0 } },
|
||||||
|
required: ['begin_ms', 'end_ms'],
|
||||||
|
additionalProperties: false } } },
|
||||||
|
required: ['id', 'version_no', 'name', 'category', 'sections'],
|
||||||
|
additionalProperties: false } }))
|
||||||
|
|
||||||
|
add_check_constraint :post_versions,
|
||||||
|
"JSON_SCHEMA_VALID(#{ schema }, tags_json)",
|
||||||
|
name: 'chk_post_versions_tags_json_schema'
|
||||||
|
end
|
||||||
|
|
||||||
|
def down
|
||||||
|
remove_check_constraint :post_versions, name: 'chk_post_versions_tags_json_schema'
|
||||||
|
|
||||||
|
remove_column :post_versions, :tags_json
|
||||||
|
end
|
||||||
|
|
||||||
|
private
|
||||||
|
|
||||||
|
def backfill_missing_initial_tag_versions!
|
||||||
|
say_with_time 'Backfilling missing initial tag versions' do
|
||||||
|
tag_rows = missing_initial_version_rows(MigrationTagVersion, nico: false)
|
||||||
|
nico_rows = missing_initial_version_rows(MigrationNicoTagVersion, nico: true)
|
||||||
|
|
||||||
|
MigrationTagVersion.insert_all!(tag_rows) if tag_rows.any?
|
||||||
|
MigrationNicoTagVersion.insert_all!(nico_rows) if nico_rows.any?
|
||||||
|
|
||||||
|
tag_rows.length + nico_rows.length
|
||||||
|
end
|
||||||
|
end
|
||||||
|
|
||||||
|
def missing_initial_version_rows version_class, nico:
|
||||||
|
first_versions =
|
||||||
|
version_class
|
||||||
|
.order(:tag_id, :version_no)
|
||||||
|
.to_a
|
||||||
|
.group_by(&:tag_id)
|
||||||
|
.transform_values(&:first)
|
||||||
|
rows = []
|
||||||
|
|
||||||
|
MigrationTag.find_each do |tag|
|
||||||
|
next if (tag.category == 'nico') != nico
|
||||||
|
|
||||||
|
first_version = first_versions[tag.id]
|
||||||
|
next unless first_version
|
||||||
|
next if valid_initial_version?(first_version)
|
||||||
|
|
||||||
|
assert_inferable_initial_version!(tag, first_version)
|
||||||
|
rows << initial_version_row(tag, first_version, nico:)
|
||||||
|
end
|
||||||
|
|
||||||
|
rows
|
||||||
|
end
|
||||||
|
|
||||||
|
def valid_initial_version? version
|
||||||
|
version.version_no == 1 && version.event_type == 'create'
|
||||||
|
end
|
||||||
|
|
||||||
|
def assert_inferable_initial_version! tag, version
|
||||||
|
inferable =
|
||||||
|
version.version_no == 2 &&
|
||||||
|
version.event_type == 'discard' &&
|
||||||
|
tag.created_at < version.created_at
|
||||||
|
return if inferable
|
||||||
|
|
||||||
|
details = [
|
||||||
|
"tag_id=#{ tag.id }",
|
||||||
|
"version_no=#{ version.version_no }",
|
||||||
|
"event_type=#{ version.event_type.inspect }",
|
||||||
|
"tag_created_at=#{ tag.created_at.iso8601(6) }",
|
||||||
|
"version_created_at=#{ version.created_at.iso8601(6) }"]
|
||||||
|
|
||||||
|
raise "Cannot infer initial tag version: #{ details.join(', ') }"
|
||||||
|
end
|
||||||
|
|
||||||
|
def initial_version_row tag, discard_version, nico:
|
||||||
|
row = {
|
||||||
|
tag_id: tag.id,
|
||||||
|
version_no: 1,
|
||||||
|
event_type: 'create',
|
||||||
|
name: discard_version.name,
|
||||||
|
created_at: tag.created_at,
|
||||||
|
created_by_user_id: nil }
|
||||||
|
|
||||||
|
if nico
|
||||||
|
return row.merge(linked_tags: discard_version.linked_tags)
|
||||||
|
end
|
||||||
|
|
||||||
|
row.merge(
|
||||||
|
category: discard_version.category,
|
||||||
|
aliases: discard_version.aliases,
|
||||||
|
parent_tag_ids: discard_version.parent_tag_ids,
|
||||||
|
deprecated_at: discard_version.deprecated_at)
|
||||||
|
end
|
||||||
|
|
||||||
|
def build_intervals_by_name
|
||||||
|
intervals_by_name = Hash.new { |hash, name| hash[name] = [] }
|
||||||
|
versions_by_kind = {
|
||||||
|
tag: versions_by_tag_id(MigrationTagVersion),
|
||||||
|
nico: versions_by_tag_id(MigrationNicoTagVersion) }
|
||||||
|
current_names = current_names_by_tag_id
|
||||||
|
|
||||||
|
MigrationTag.find_each do |tag|
|
||||||
|
nico = tag.category == 'nico'
|
||||||
|
kind = nico ? :nico : :tag
|
||||||
|
versions = versions_by_kind.fetch(kind).fetch(tag.id, [])
|
||||||
|
|
||||||
|
intervals_for(
|
||||||
|
tag,
|
||||||
|
versions,
|
||||||
|
current_name: current_names.fetch(tag.id),
|
||||||
|
nico:).each do |interval|
|
||||||
|
name = interval.delete(:name)
|
||||||
|
intervals_by_name[name] << interval
|
||||||
|
end
|
||||||
|
end
|
||||||
|
|
||||||
|
intervals_by_name
|
||||||
|
end
|
||||||
|
|
||||||
|
def versions_by_tag_id version_class
|
||||||
|
version_class
|
||||||
|
.order(:tag_id, :version_no)
|
||||||
|
.to_a
|
||||||
|
.group_by(&:tag_id)
|
||||||
|
end
|
||||||
|
|
||||||
|
def current_names_by_tag_id
|
||||||
|
MigrationTagName
|
||||||
|
.joins('INNER JOIN tags ON tags.tag_name_id = tag_names.id')
|
||||||
|
.pluck('tags.id', 'tag_names.name')
|
||||||
|
.to_h
|
||||||
|
end
|
||||||
|
|
||||||
|
def intervals_for tag, versions, current_name:, nico:
|
||||||
|
if versions.empty?
|
||||||
|
return [{
|
||||||
|
name: current_name,
|
||||||
|
tag_id: tag.id,
|
||||||
|
version_no: tag.version_no,
|
||||||
|
category: nico ? 'nico' : tag.category,
|
||||||
|
from: tag.created_at,
|
||||||
|
to: tag.discarded_at }]
|
||||||
|
end
|
||||||
|
|
||||||
|
versions.each_with_index.filter_map do |version, index|
|
||||||
|
next if version.event_type == 'discard'
|
||||||
|
|
||||||
|
{
|
||||||
|
name: version.name,
|
||||||
|
tag_id: tag.id,
|
||||||
|
version_no: version.version_no,
|
||||||
|
category: nico ? 'nico' : version.category,
|
||||||
|
from: version.created_at,
|
||||||
|
to: versions[index + 1]&.created_at || tag.discarded_at }
|
||||||
|
end
|
||||||
|
end
|
||||||
|
|
||||||
|
def build_tags_json version, intervals_by_name
|
||||||
|
entries = version.tags.to_s.split.map do |literal|
|
||||||
|
name = tag_name_from_literal(literal)
|
||||||
|
interval = resolve_tag!(intervals_by_name.fetch(name, []), name:, version:)
|
||||||
|
|
||||||
|
{ 'id' => interval.fetch(:tag_id),
|
||||||
|
'version_no' => interval.fetch(:version_no),
|
||||||
|
'name' => name,
|
||||||
|
'category' => interval.fetch(:category),
|
||||||
|
'sections' => [] }
|
||||||
|
end
|
||||||
|
|
||||||
|
assert_unique_tag_ids!(version, entries)
|
||||||
|
|
||||||
|
entries.sort_by { |entry| entry.fetch('id') }
|
||||||
|
end
|
||||||
|
|
||||||
|
def tag_name_from_literal literal
|
||||||
|
name = literal.dup
|
||||||
|
name.sub!(SECTION_LITERAL_PATTERN, '') while name.match?(
|
||||||
|
SECTION_LITERAL_PATTERN)
|
||||||
|
|
||||||
|
if name.empty? || name.include?('[') || name.include?(']')
|
||||||
|
raise "Invalid legacy tag literal: #{ literal.inspect }"
|
||||||
|
end
|
||||||
|
|
||||||
|
name
|
||||||
|
end
|
||||||
|
|
||||||
|
def resolve_tag! intervals, name:, version:
|
||||||
|
time = version.created_at
|
||||||
|
candidates = intervals.select do |interval|
|
||||||
|
interval.fetch(:from) <= time &&
|
||||||
|
(interval[:to].nil? || time < interval.fetch(:to))
|
||||||
|
end
|
||||||
|
|
||||||
|
candidates = future_candidates(intervals, time) if candidates.empty?
|
||||||
|
|
||||||
|
return candidates.first if candidates.one?
|
||||||
|
|
||||||
|
candidate_versions = candidates.map do |candidate|
|
||||||
|
[candidate.fetch(:tag_id), candidate.fetch(:version_no)]
|
||||||
|
end
|
||||||
|
details = [
|
||||||
|
"post_version_id=#{ version.id }",
|
||||||
|
"post_id=#{ version.post_id }",
|
||||||
|
"name=#{ name.inspect }",
|
||||||
|
"created_at=#{ time.iso8601(6) }",
|
||||||
|
"candidates=#{ candidate_versions.inspect }"].join(', ')
|
||||||
|
|
||||||
|
raise "Could not resolve tag snapshot: #{ details }"
|
||||||
|
end
|
||||||
|
|
||||||
|
def future_candidates intervals, time
|
||||||
|
candidates = intervals.select do |interval|
|
||||||
|
interval.fetch(:from) > time &&
|
||||||
|
interval.fetch(:from) <= time + RESOLUTION_GRACE
|
||||||
|
end
|
||||||
|
return [] if candidates.empty?
|
||||||
|
|
||||||
|
nearest_from = candidates.map { |interval| interval.fetch(:from) }.min
|
||||||
|
|
||||||
|
candidates.select do |interval|
|
||||||
|
interval.fetch(:from) == nearest_from
|
||||||
|
end
|
||||||
|
end
|
||||||
|
|
||||||
|
def assert_unique_tag_ids! version, entries
|
||||||
|
duplicate_tag_ids =
|
||||||
|
entries
|
||||||
|
.map { |entry| entry.fetch('id') }
|
||||||
|
.tally
|
||||||
|
.select { |_tag_id, count| count > 1 }
|
||||||
|
.keys
|
||||||
|
return if duplicate_tag_ids.empty?
|
||||||
|
|
||||||
|
details = [
|
||||||
|
"post_version_id=#{ version.id }",
|
||||||
|
"duplicate_tag_ids=#{ duplicate_tag_ids.inspect }"].join(', ')
|
||||||
|
|
||||||
|
raise "Duplicate tag IDs: #{ details }"
|
||||||
|
end
|
||||||
|
end
|
||||||
@@ -0,0 +1,53 @@
|
|||||||
|
class DeleteInactiveRecordsFromPostTags < ActiveRecord::Migration[8.0]
|
||||||
|
def up
|
||||||
|
execute <<~SQL
|
||||||
|
DELETE
|
||||||
|
FROM
|
||||||
|
post_tags
|
||||||
|
WHERE
|
||||||
|
discarded_at IS NOT NULL
|
||||||
|
SQL
|
||||||
|
|
||||||
|
remove_index :post_tags, [:tag_id, :discarded_at]
|
||||||
|
remove_index :post_tags, [:post_id, :discarded_at]
|
||||||
|
remove_index :post_tags, name: 'idx_post_tags_active_unique'
|
||||||
|
remove_index :post_tags, :discarded_at
|
||||||
|
|
||||||
|
remove_foreign_key :post_tags, column: :deleted_user_id
|
||||||
|
remove_index :post_tags, :deleted_user_id
|
||||||
|
|
||||||
|
remove_column :post_tags, :active_unique_key
|
||||||
|
remove_column :post_tags, :is_active
|
||||||
|
remove_column :post_tags, :discarded_at
|
||||||
|
remove_column :post_tags, :deleted_user_id
|
||||||
|
remove_column :post_tags, :updated_at
|
||||||
|
|
||||||
|
execute <<~SQL
|
||||||
|
ALTER TABLE
|
||||||
|
post_tags
|
||||||
|
MODIFY COLUMN
|
||||||
|
id BIGINT NOT NULL
|
||||||
|
SQL
|
||||||
|
|
||||||
|
execute <<~SQL
|
||||||
|
ALTER TABLE
|
||||||
|
post_tags
|
||||||
|
DROP PRIMARY KEY
|
||||||
|
SQL
|
||||||
|
|
||||||
|
remove_column :post_tags, :id
|
||||||
|
|
||||||
|
execute <<~SQL
|
||||||
|
ALTER TABLE
|
||||||
|
post_tags
|
||||||
|
ADD PRIMARY KEY
|
||||||
|
(post_id, tag_id)
|
||||||
|
SQL
|
||||||
|
|
||||||
|
remove_index :post_tags, :post_id
|
||||||
|
end
|
||||||
|
|
||||||
|
def down
|
||||||
|
raise ActiveRecord::IrreversibleMigration, '戻せません.'
|
||||||
|
end
|
||||||
|
end
|
||||||
+11
@@ -0,0 +1,11 @@
|
|||||||
|
class AddForeignKeyOnPostIdAndTagIdInPostTagSections < ActiveRecord::Migration[8.0]
|
||||||
|
def change
|
||||||
|
remove_foreign_key :post_tag_sections, :posts, column: :post_id
|
||||||
|
remove_foreign_key :post_tag_sections, :tags, column: :tag_id
|
||||||
|
|
||||||
|
add_foreign_key :post_tag_sections, :post_tags,
|
||||||
|
column: [:post_id, :tag_id],
|
||||||
|
primary_key: [:post_id, :tag_id],
|
||||||
|
on_delete: :cascade
|
||||||
|
end
|
||||||
|
end
|
||||||
@@ -0,0 +1,46 @@
|
|||||||
|
class DeleteDiscardedRecordsFromTags < ActiveRecord::Migration[8.0]
|
||||||
|
def up
|
||||||
|
remove_foreign_key :tag_versions, :tags, column: :tag_id
|
||||||
|
remove_foreign_key :nico_tag_versions, :tags, column: :tag_id
|
||||||
|
remove_foreign_key :material_versions, :tags, column: :tag_id
|
||||||
|
|
||||||
|
execute <<~SQL
|
||||||
|
DELETE
|
||||||
|
ntr
|
||||||
|
FROM
|
||||||
|
nico_tag_relations ntr
|
||||||
|
INNER JOIN
|
||||||
|
tags t
|
||||||
|
ON
|
||||||
|
t.discarded_at IS NOT NULL
|
||||||
|
AND t.id IN (ntr.tag_id, ntr.nico_tag_id)
|
||||||
|
SQL
|
||||||
|
|
||||||
|
execute <<~SQL
|
||||||
|
DELETE
|
||||||
|
ti
|
||||||
|
FROM
|
||||||
|
tag_implications ti
|
||||||
|
INNER JOIN
|
||||||
|
tags t
|
||||||
|
ON
|
||||||
|
t.discarded_at IS NOT NULL
|
||||||
|
AND t.id IN (ti.tag_id, ti.parent_tag_id)
|
||||||
|
SQL
|
||||||
|
|
||||||
|
execute <<~SQL
|
||||||
|
DELETE
|
||||||
|
FROM
|
||||||
|
tags
|
||||||
|
WHERE
|
||||||
|
discarded_at IS NOT NULL
|
||||||
|
SQL
|
||||||
|
|
||||||
|
remove_index :tags, :discarded_at
|
||||||
|
remove_column :tags, :discarded_at
|
||||||
|
end
|
||||||
|
|
||||||
|
def down
|
||||||
|
raise ActiveRecord::IrreversibleMigration, '戻せません.'
|
||||||
|
end
|
||||||
|
end
|
||||||
@@ -0,0 +1,18 @@
|
|||||||
|
class DeleteDiscardedRecordsFromTagNames < ActiveRecord::Migration[8.0]
|
||||||
|
def up
|
||||||
|
execute <<~SQL
|
||||||
|
DELETE
|
||||||
|
FROM
|
||||||
|
tag_names
|
||||||
|
WHERE
|
||||||
|
discarded_at IS NOT NULL
|
||||||
|
SQL
|
||||||
|
|
||||||
|
remove_index :tag_names, :discarded_at
|
||||||
|
remove_column :tag_names, :discarded_at
|
||||||
|
end
|
||||||
|
|
||||||
|
def down
|
||||||
|
raise ActiveRecord::IrreversibleMigration, '戻せません.'
|
||||||
|
end
|
||||||
|
end
|
||||||
@@ -0,0 +1,54 @@
|
|||||||
|
class CreateExternalTags < ActiveRecord::Migration[8.0]
|
||||||
|
def up
|
||||||
|
create_table :external_tags do |t|
|
||||||
|
t.string :platform, limit: 16, null: false
|
||||||
|
t.string :name, limit: 255, null: false
|
||||||
|
t.integer :post_count, null: false, default: 0
|
||||||
|
t.datetime :created_at, null: false
|
||||||
|
|
||||||
|
t.index [:platform, :name], unique: true
|
||||||
|
end
|
||||||
|
|
||||||
|
execute <<~SQL
|
||||||
|
INSERT INTO
|
||||||
|
external_tags(id, platform, name, post_count, created_at)
|
||||||
|
SELECT
|
||||||
|
t.id
|
||||||
|
, 'nico' AS platform
|
||||||
|
, SUBSTR(tn.name, 6) AS name
|
||||||
|
, t.post_count
|
||||||
|
, t.created_at
|
||||||
|
FROM
|
||||||
|
tags t
|
||||||
|
INNER JOIN
|
||||||
|
tag_names tn
|
||||||
|
ON
|
||||||
|
tn.id = t.tag_name_id
|
||||||
|
AND t.category = 'nico'
|
||||||
|
SQL
|
||||||
|
|
||||||
|
execute <<~SQL
|
||||||
|
INSERT INTO
|
||||||
|
external_tags(id, platform, name, post_count, created_at)
|
||||||
|
SELECT
|
||||||
|
ntv.tag_id
|
||||||
|
, 'nico' AS platform
|
||||||
|
, SUBSTR(ntv.name, 6) AS name
|
||||||
|
, 0 AS post_count
|
||||||
|
, ntv.created_at
|
||||||
|
FROM
|
||||||
|
nico_tag_versions ntv
|
||||||
|
LEFT JOIN
|
||||||
|
external_tags et
|
||||||
|
ON
|
||||||
|
et.id = ntv.tag_id
|
||||||
|
WHERE
|
||||||
|
ntv.version_no = 1
|
||||||
|
AND et.id IS NULL
|
||||||
|
SQL
|
||||||
|
end
|
||||||
|
|
||||||
|
def down
|
||||||
|
drop_table :external_tags
|
||||||
|
end
|
||||||
|
end
|
||||||
@@ -0,0 +1,29 @@
|
|||||||
|
class CreatePostExternalTags < ActiveRecord::Migration[8.0]
|
||||||
|
def up
|
||||||
|
create_table :post_external_tags, primary_key: [:post_id, :external_tag_id] do |t|
|
||||||
|
t.references :post, null: false, index: false, foreign_key: true
|
||||||
|
t.references :external_tag, null: false, foreign_key: true
|
||||||
|
t.datetime :created_at, null: false
|
||||||
|
end
|
||||||
|
|
||||||
|
execute <<~SQL
|
||||||
|
INSERT INTO
|
||||||
|
post_external_tags(post_id, external_tag_id, created_at)
|
||||||
|
SELECT
|
||||||
|
pt.post_id
|
||||||
|
, pt.tag_id AS external_tag_id
|
||||||
|
, pt.created_at
|
||||||
|
FROM
|
||||||
|
post_tags pt
|
||||||
|
INNER JOIN
|
||||||
|
tags t
|
||||||
|
ON
|
||||||
|
pt.tag_id = t.id
|
||||||
|
AND t.category = 'nico'
|
||||||
|
SQL
|
||||||
|
end
|
||||||
|
|
||||||
|
def down
|
||||||
|
drop_table :post_external_tags
|
||||||
|
end
|
||||||
|
end
|
||||||
@@ -0,0 +1,6 @@
|
|||||||
|
class ChangeForeignKeyOnNicoTagRelations < ActiveRecord::Migration[8.0]
|
||||||
|
def change
|
||||||
|
remove_foreign_key :nico_tag_relations, :tags, column: :nico_tag_id
|
||||||
|
add_foreign_key :nico_tag_relations, :external_tags, column: :nico_tag_id
|
||||||
|
end
|
||||||
|
end
|
||||||
@@ -0,0 +1,199 @@
|
|||||||
|
class MigrateExternalTags < ActiveRecord::Migration[8.0]
|
||||||
|
class MigrationPostVersion < ActiveRecord::Base
|
||||||
|
self.table_name = 'post_versions'
|
||||||
|
end
|
||||||
|
|
||||||
|
def up
|
||||||
|
x = connection.select_value(<<~SQL)
|
||||||
|
SELECT
|
||||||
|
COUNT(0)
|
||||||
|
FROM
|
||||||
|
post_tag_sections pts
|
||||||
|
INNER JOIN
|
||||||
|
tags t
|
||||||
|
ON
|
||||||
|
t.id = pts.tag_id
|
||||||
|
WHERE
|
||||||
|
t.category = 'nico'
|
||||||
|
SQL
|
||||||
|
if x > 0
|
||||||
|
raise "post_tag_sections に #{ x } 件のチンカスがあります!"
|
||||||
|
end
|
||||||
|
|
||||||
|
x = connection.select_value(<<~SQL)
|
||||||
|
SELECT
|
||||||
|
COUNT(0)
|
||||||
|
FROM
|
||||||
|
materials m
|
||||||
|
INNER JOIN
|
||||||
|
tags t
|
||||||
|
ON
|
||||||
|
t.id = m.tag_id
|
||||||
|
WHERE
|
||||||
|
t.category = 'nico'
|
||||||
|
SQL
|
||||||
|
if x > 0
|
||||||
|
raise "materials に #{ x } 件のチンカスがあります!"
|
||||||
|
end
|
||||||
|
|
||||||
|
x = connection.select_value(<<~SQL)
|
||||||
|
SELECT
|
||||||
|
COUNT(0)
|
||||||
|
FROM
|
||||||
|
tag_implications ti
|
||||||
|
INNER JOIN
|
||||||
|
tags t
|
||||||
|
ON
|
||||||
|
t.id = ti.tag_id
|
||||||
|
OR t.id = ti.parent_tag_id
|
||||||
|
WHERE
|
||||||
|
t.category = 'nico'
|
||||||
|
SQL
|
||||||
|
if x > 0
|
||||||
|
raise "tag_implications に #{ x } 件のチンカスがあります!"
|
||||||
|
end
|
||||||
|
|
||||||
|
execute <<~SQL
|
||||||
|
DELETE
|
||||||
|
ts
|
||||||
|
FROM
|
||||||
|
tag_similarities ts
|
||||||
|
INNER JOIN
|
||||||
|
tags t
|
||||||
|
ON
|
||||||
|
t.category = 'nico'
|
||||||
|
AND (t.id = ts.tag_id
|
||||||
|
OR t.id = ts.target_tag_id)
|
||||||
|
SQL
|
||||||
|
|
||||||
|
execute <<~SQL
|
||||||
|
DELETE
|
||||||
|
tset
|
||||||
|
FROM
|
||||||
|
theatre_skip_event_tags tset
|
||||||
|
INNER JOIN
|
||||||
|
tags t
|
||||||
|
ON
|
||||||
|
t.category = 'nico'
|
||||||
|
AND t.id = tset.tag_id
|
||||||
|
SQL
|
||||||
|
|
||||||
|
remove_check_constraint :post_versions, name: 'chk_post_versions_tags_json_schema'
|
||||||
|
|
||||||
|
say_with_time 'Migrate post_versions.tags_json' do
|
||||||
|
count = 0
|
||||||
|
|
||||||
|
MigrationPostVersion.find_each(batch_size: 500) do |version|
|
||||||
|
tags = version.tags_json.map do |tag|
|
||||||
|
if tag.fetch('category') == 'nico'
|
||||||
|
{ 'external_tag_id' => tag.fetch('id') }
|
||||||
|
else
|
||||||
|
{ 'tag_id' => tag.fetch('id'),
|
||||||
|
'version_no' => tag.fetch('version_no'),
|
||||||
|
'name' => tag.fetch('name'),
|
||||||
|
'category' => tag.fetch('category'),
|
||||||
|
'sections' => tag.fetch('sections') }
|
||||||
|
end
|
||||||
|
end
|
||||||
|
|
||||||
|
version.update_columns(tags_json: tags)
|
||||||
|
count += 1
|
||||||
|
end
|
||||||
|
|
||||||
|
count
|
||||||
|
end
|
||||||
|
|
||||||
|
add_tags_json_constraint!
|
||||||
|
|
||||||
|
tag_name_ids = connection.select_values(<<~SQL)
|
||||||
|
SELECT
|
||||||
|
tag_name_id
|
||||||
|
FROM
|
||||||
|
tags
|
||||||
|
WHERE
|
||||||
|
category = 'nico'
|
||||||
|
SQL
|
||||||
|
|
||||||
|
connection.transaction do
|
||||||
|
execute <<~SQL
|
||||||
|
DELETE
|
||||||
|
pt
|
||||||
|
FROM
|
||||||
|
post_tags pt
|
||||||
|
INNER JOIN
|
||||||
|
tags t
|
||||||
|
ON
|
||||||
|
t.category = 'nico'
|
||||||
|
AND t.id = pt.tag_id
|
||||||
|
SQL
|
||||||
|
|
||||||
|
execute <<~SQL
|
||||||
|
DELETE
|
||||||
|
FROM
|
||||||
|
tags
|
||||||
|
WHERE
|
||||||
|
category = 'nico'
|
||||||
|
SQL
|
||||||
|
|
||||||
|
unless tag_name_ids.empty?
|
||||||
|
execute <<~SQL
|
||||||
|
DELETE
|
||||||
|
FROM
|
||||||
|
tag_names
|
||||||
|
WHERE
|
||||||
|
id IN (#{ tag_name_ids.join(', ') })
|
||||||
|
SQL
|
||||||
|
end
|
||||||
|
end
|
||||||
|
end
|
||||||
|
|
||||||
|
def down
|
||||||
|
raise ActiveRecord::IrreversibleMigration, '戻せません.'
|
||||||
|
end
|
||||||
|
|
||||||
|
private
|
||||||
|
|
||||||
|
def add_tags_json_constraint!
|
||||||
|
schema = { type: 'array',
|
||||||
|
items: { oneOf: [internal_tag_schema, external_tag_schema] } }
|
||||||
|
|
||||||
|
quoted_schema = connection.quote(JSON.generate(schema))
|
||||||
|
|
||||||
|
add_check_constraint :post_versions,
|
||||||
|
"JSON_SCHEMA_VALID(#{ quoted_schema }, tags_json)",
|
||||||
|
name: 'chk_post_versions_tags_json_schema'
|
||||||
|
end
|
||||||
|
|
||||||
|
def internal_tag_schema
|
||||||
|
{ type: 'object',
|
||||||
|
properties: { tag_id: { type: 'integer', minimum: 1 },
|
||||||
|
version_no: { type: 'integer', minimum: 1 },
|
||||||
|
name: { type: 'string', minLength: 1 },
|
||||||
|
category: { type: 'string',
|
||||||
|
enum: ['deerjikist',
|
||||||
|
'meme',
|
||||||
|
'character',
|
||||||
|
'general',
|
||||||
|
'material',
|
||||||
|
'meta'] },
|
||||||
|
sections: sections_schema },
|
||||||
|
required: ['tag_id', 'version_no', 'sections'],
|
||||||
|
additionalProperties: false }
|
||||||
|
end
|
||||||
|
|
||||||
|
def external_tag_schema
|
||||||
|
{ type: 'object',
|
||||||
|
properties: { external_tag_id: { type: 'integer', minimum: 1 } },
|
||||||
|
required: ['external_tag_id'],
|
||||||
|
additionalProperties: false }
|
||||||
|
end
|
||||||
|
|
||||||
|
def sections_schema
|
||||||
|
{ type: 'array',
|
||||||
|
items: { type: 'object',
|
||||||
|
properties: { begin_ms: { type: 'integer', minimum: 0 },
|
||||||
|
end_ms: { type: ['integer', 'null'], minimum: 0 } },
|
||||||
|
required: ['begin_ms', 'end_ms'],
|
||||||
|
additionalProperties: false } }
|
||||||
|
end
|
||||||
|
end
|
||||||
生成ファイル
+54
-29
@@ -10,7 +10,7 @@
|
|||||||
#
|
#
|
||||||
# It's strongly recommended that you check this file into your version control system.
|
# It's strongly recommended that you check this file into your version control system.
|
||||||
|
|
||||||
ActiveRecord::Schema[8.0].define(version: 2026_06_26_010000) do
|
ActiveRecord::Schema[8.0].define(version: 2026_09_21_230000) do
|
||||||
create_table "active_storage_attachments", charset: "utf8mb4", collation: "utf8mb4_0900_ai_ci", force: :cascade do |t|
|
create_table "active_storage_attachments", charset: "utf8mb4", collation: "utf8mb4_0900_ai_ci", force: :cascade do |t|
|
||||||
t.string "name", null: false
|
t.string "name", null: false
|
||||||
t.string "record_type", null: false
|
t.string "record_type", null: false
|
||||||
@@ -48,6 +48,14 @@ ActiveRecord::Schema[8.0].define(version: 2026_06_26_010000) do
|
|||||||
t.index ["tag_id"], name: "index_deerjikists_on_tag_id"
|
t.index ["tag_id"], name: "index_deerjikists_on_tag_id"
|
||||||
end
|
end
|
||||||
|
|
||||||
|
create_table "external_tags", charset: "utf8mb4", collation: "utf8mb4_0900_ai_ci", force: :cascade do |t|
|
||||||
|
t.string "platform", limit: 16, null: false
|
||||||
|
t.string "name", null: false
|
||||||
|
t.integer "post_count", default: 0, null: false
|
||||||
|
t.datetime "created_at", null: false
|
||||||
|
t.index ["platform", "name"], name: "index_external_tags_on_platform_and_name", unique: true
|
||||||
|
end
|
||||||
|
|
||||||
create_table "gekanator_ai_runs", charset: "utf8mb4", collation: "utf8mb4_0900_ai_ci", force: :cascade do |t|
|
create_table "gekanator_ai_runs", charset: "utf8mb4", collation: "utf8mb4_0900_ai_ci", force: :cascade do |t|
|
||||||
t.string "model", null: false
|
t.string "model", null: false
|
||||||
t.integer "input_tokens", default: 0, null: false
|
t.integer "input_tokens", default: 0, null: false
|
||||||
@@ -281,6 +289,13 @@ ActiveRecord::Schema[8.0].define(version: 2026_06_26_010000) do
|
|||||||
t.check_constraint "`version_no` > 0", name: "nico_tag_versions_version_no_positive"
|
t.check_constraint "`version_no` > 0", name: "nico_tag_versions_version_no_positive"
|
||||||
end
|
end
|
||||||
|
|
||||||
|
create_table "post_external_tags", primary_key: ["post_id", "external_tag_id"], charset: "utf8mb4", collation: "utf8mb4_0900_ai_ci", force: :cascade do |t|
|
||||||
|
t.bigint "post_id", null: false
|
||||||
|
t.bigint "external_tag_id", null: false
|
||||||
|
t.datetime "created_at", null: false
|
||||||
|
t.index ["external_tag_id"], name: "index_post_external_tags_on_external_tag_id"
|
||||||
|
end
|
||||||
|
|
||||||
create_table "post_implications", primary_key: ["post_id", "parent_post_id"], charset: "utf8mb4", collation: "utf8mb4_0900_ai_ci", force: :cascade do |t|
|
create_table "post_implications", primary_key: ["post_id", "parent_post_id"], charset: "utf8mb4", collation: "utf8mb4_0900_ai_ci", force: :cascade do |t|
|
||||||
t.bigint "post_id", null: false
|
t.bigint "post_id", null: false
|
||||||
t.bigint "parent_post_id", null: false
|
t.bigint "parent_post_id", null: false
|
||||||
@@ -311,24 +326,23 @@ ActiveRecord::Schema[8.0].define(version: 2026_06_26_010000) do
|
|||||||
t.check_constraint "`begin_ms` >= 0", name: "chk_post_tag_sections_begin_ms_natural"
|
t.check_constraint "`begin_ms` >= 0", name: "chk_post_tag_sections_begin_ms_natural"
|
||||||
end
|
end
|
||||||
|
|
||||||
create_table "post_tags", charset: "utf8mb4", collation: "utf8mb4_0900_ai_ci", force: :cascade do |t|
|
create_table "post_tags", primary_key: ["post_id", "tag_id"], charset: "utf8mb4", collation: "utf8mb4_0900_ai_ci", force: :cascade do |t|
|
||||||
t.bigint "post_id", null: false
|
t.bigint "post_id", null: false
|
||||||
t.bigint "tag_id", null: false
|
t.bigint "tag_id", null: false
|
||||||
t.bigint "created_user_id"
|
t.bigint "created_user_id"
|
||||||
t.bigint "deleted_user_id"
|
t.datetime "created_at", null: false
|
||||||
|
t.index ["created_user_id"], name: "index_post_tags_on_created_user_id"
|
||||||
|
t.index ["tag_id"], name: "index_post_tags_on_tag_id"
|
||||||
|
end
|
||||||
|
|
||||||
|
create_table "post_url_sanitisation_rules", primary_key: "priority", id: :integer, charset: "utf8mb4", collation: "utf8mb4_0900_ai_ci", force: :cascade do |t|
|
||||||
|
t.string "source_pattern", null: false
|
||||||
|
t.string "replacement", null: false
|
||||||
t.datetime "created_at", null: false
|
t.datetime "created_at", null: false
|
||||||
t.datetime "updated_at", null: false
|
t.datetime "updated_at", null: false
|
||||||
t.datetime "discarded_at"
|
t.datetime "discarded_at"
|
||||||
t.virtual "is_active", type: :boolean, as: "(`discarded_at` is null)", stored: true
|
t.index ["discarded_at"], name: "index_post_url_sanitisation_rules_on_discarded_at"
|
||||||
t.virtual "active_unique_key", type: :string, as: "(case when (`discarded_at` is null) then concat(`post_id`,_utf8mb4':',`tag_id`) else NULL end)", stored: true
|
t.index ["source_pattern"], name: "index_post_url_sanitisation_rules_on_source_pattern", unique: true
|
||||||
t.index ["active_unique_key"], name: "idx_post_tags_active_unique", unique: true
|
|
||||||
t.index ["created_user_id"], name: "index_post_tags_on_created_user_id"
|
|
||||||
t.index ["deleted_user_id"], name: "index_post_tags_on_deleted_user_id"
|
|
||||||
t.index ["discarded_at"], name: "index_post_tags_on_discarded_at"
|
|
||||||
t.index ["post_id", "discarded_at"], name: "index_post_tags_on_post_id_and_discarded_at"
|
|
||||||
t.index ["post_id"], name: "index_post_tags_on_post_id"
|
|
||||||
t.index ["tag_id", "discarded_at"], name: "index_post_tags_on_tag_id_and_discarded_at"
|
|
||||||
t.index ["tag_id"], name: "index_post_tags_on_tag_id"
|
|
||||||
end
|
end
|
||||||
|
|
||||||
create_table "post_versions", charset: "utf8mb4", collation: "utf8mb4_0900_ai_ci", force: :cascade do |t|
|
create_table "post_versions", charset: "utf8mb4", collation: "utf8mb4_0900_ai_ci", force: :cascade do |t|
|
||||||
@@ -339,6 +353,7 @@ ActiveRecord::Schema[8.0].define(version: 2026_06_26_010000) do
|
|||||||
t.string "url", limit: 768, null: false
|
t.string "url", limit: 768, null: false
|
||||||
t.string "thumbnail_base", limit: 2000
|
t.string "thumbnail_base", limit: 2000
|
||||||
t.text "tags", null: false
|
t.text "tags", null: false
|
||||||
|
t.json "tags_json", null: false
|
||||||
t.text "parent_post_ids", null: false
|
t.text "parent_post_ids", null: false
|
||||||
t.datetime "original_created_from"
|
t.datetime "original_created_from"
|
||||||
t.datetime "original_created_before"
|
t.datetime "original_created_before"
|
||||||
@@ -352,6 +367,7 @@ ActiveRecord::Schema[8.0].define(version: 2026_06_26_010000) do
|
|||||||
t.check_constraint "(`video_ms` is null) or (`video_ms` > 0)", name: "chk_post_versions_video_ms_positive"
|
t.check_constraint "(`video_ms` is null) or (`video_ms` > 0)", name: "chk_post_versions_video_ms_positive"
|
||||||
t.check_constraint "`event_type` in (_utf8mb4'create',_utf8mb4'update',_utf8mb4'discard',_utf8mb4'restore')", name: "post_versions_event_type_valid"
|
t.check_constraint "`event_type` in (_utf8mb4'create',_utf8mb4'update',_utf8mb4'discard',_utf8mb4'restore')", name: "post_versions_event_type_valid"
|
||||||
t.check_constraint "`version_no` > 0", name: "post_versions_version_no_positive"
|
t.check_constraint "`version_no` > 0", name: "post_versions_version_no_positive"
|
||||||
|
t.check_constraint "json_schema_valid(_utf8mb4'{\"type\":\"array\",\"items\":{\"oneOf\":[{\"type\":\"object\",\"properties\":{\"tag_id\":{\"type\":\"integer\",\"minimum\":1},\"version_no\":{\"type\":\"integer\",\"minimum\":1},\"name\":{\"type\":\"string\",\"minLength\":1},\"category\":{\"type\":\"string\",\"enum\":[\"deerjikist\",\"meme\",\"character\",\"general\",\"material\",\"meta\"]},\"sections\":{\"type\":\"array\",\"items\":{\"type\":\"object\",\"properties\":{\"begin_ms\":{\"type\":\"integer\",\"minimum\":0},\"end_ms\":{\"type\":[\"integer\",\"null\"],\"minimum\":0}},\"required\":[\"begin_ms\",\"end_ms\"],\"additionalProperties\":false}}},\"required\":[\"tag_id\",\"version_no\",\"sections\"],\"additionalProperties\":false},{\"type\":\"object\",\"properties\":{\"external_tag_id\":{\"type\":\"integer\",\"minimum\":1}},\"required\":[\"external_tag_id\"],\"additionalProperties\":false}]}}',`tags_json`)", name: "chk_post_versions_tags_json_schema"
|
||||||
end
|
end
|
||||||
|
|
||||||
create_table "posts", charset: "utf8mb4", collation: "utf8mb4_0900_ai_ci", force: :cascade do |t|
|
create_table "posts", charset: "utf8mb4", collation: "utf8mb4_0900_ai_ci", force: :cascade do |t|
|
||||||
@@ -374,11 +390,13 @@ ActiveRecord::Schema[8.0].define(version: 2026_06_26_010000) do
|
|||||||
|
|
||||||
create_table "settings", charset: "utf8mb4", collation: "utf8mb4_0900_ai_ci", force: :cascade do |t|
|
create_table "settings", charset: "utf8mb4", collation: "utf8mb4_0900_ai_ci", force: :cascade do |t|
|
||||||
t.bigint "user_id", null: false
|
t.bigint "user_id", null: false
|
||||||
t.string "key", null: false
|
|
||||||
t.json "value", null: false
|
|
||||||
t.datetime "created_at", null: false
|
t.datetime "created_at", null: false
|
||||||
t.datetime "updated_at", null: false
|
t.datetime "updated_at", null: false
|
||||||
t.index ["user_id"], name: "index_settings_on_user_id"
|
t.string "theme", default: "system", null: false
|
||||||
|
t.string "auto_fetch_title", default: "manual", null: false
|
||||||
|
t.string "auto_fetch_thumbnail", default: "manual", null: false
|
||||||
|
t.string "wiki_editor_mode", default: "split", null: false
|
||||||
|
t.index ["user_id"], name: "index_settings_on_user_id", unique: true
|
||||||
end
|
end
|
||||||
|
|
||||||
create_table "tag_implications", charset: "utf8mb4", collation: "utf8mb4_0900_ai_ci", force: :cascade do |t|
|
create_table "tag_implications", charset: "utf8mb4", collation: "utf8mb4_0900_ai_ci", force: :cascade do |t|
|
||||||
@@ -406,9 +424,7 @@ ActiveRecord::Schema[8.0].define(version: 2026_06_26_010000) do
|
|||||||
t.bigint "canonical_id"
|
t.bigint "canonical_id"
|
||||||
t.datetime "created_at", null: false
|
t.datetime "created_at", null: false
|
||||||
t.datetime "updated_at", null: false
|
t.datetime "updated_at", null: false
|
||||||
t.datetime "discarded_at"
|
|
||||||
t.index ["canonical_id"], name: "index_tag_names_on_canonical_id"
|
t.index ["canonical_id"], name: "index_tag_names_on_canonical_id"
|
||||||
t.index ["discarded_at"], name: "index_tag_names_on_discarded_at"
|
|
||||||
t.index ["name"], name: "index_tag_names_on_name", unique: true
|
t.index ["name"], name: "index_tag_names_on_name", unique: true
|
||||||
end
|
end
|
||||||
|
|
||||||
@@ -426,9 +442,9 @@ ActiveRecord::Schema[8.0].define(version: 2026_06_26_010000) do
|
|||||||
t.string "event_type", null: false
|
t.string "event_type", null: false
|
||||||
t.string "name", null: false
|
t.string "name", null: false
|
||||||
t.string "category", null: false
|
t.string "category", null: false
|
||||||
t.datetime "deprecated_at"
|
|
||||||
t.text "aliases", null: false
|
t.text "aliases", null: false
|
||||||
t.text "parent_tag_ids", null: false
|
t.text "parent_tag_ids", null: false
|
||||||
|
t.datetime "deprecated_at"
|
||||||
t.datetime "created_at", null: false
|
t.datetime "created_at", null: false
|
||||||
t.bigint "created_by_user_id"
|
t.bigint "created_by_user_id"
|
||||||
t.index ["created_at"], name: "index_tag_versions_on_created_at"
|
t.index ["created_at"], name: "index_tag_versions_on_created_at"
|
||||||
@@ -441,14 +457,12 @@ ActiveRecord::Schema[8.0].define(version: 2026_06_26_010000) do
|
|||||||
create_table "tags", charset: "utf8mb4", collation: "utf8mb4_0900_ai_ci", force: :cascade do |t|
|
create_table "tags", charset: "utf8mb4", collation: "utf8mb4_0900_ai_ci", force: :cascade do |t|
|
||||||
t.bigint "tag_name_id", null: false
|
t.bigint "tag_name_id", null: false
|
||||||
t.string "category", default: "general", null: false
|
t.string "category", default: "general", null: false
|
||||||
|
t.datetime "deprecated_at"
|
||||||
t.datetime "created_at", null: false
|
t.datetime "created_at", null: false
|
||||||
t.datetime "updated_at", null: false
|
t.datetime "updated_at", null: false
|
||||||
t.integer "post_count", default: 0, null: false
|
t.integer "post_count", default: 0, null: false
|
||||||
t.datetime "deprecated_at"
|
|
||||||
t.datetime "discarded_at"
|
|
||||||
t.integer "version_no", null: false
|
t.integer "version_no", null: false
|
||||||
t.index ["deprecated_at"], name: "index_tags_on_deprecated_at"
|
t.index ["deprecated_at"], name: "index_tags_on_deprecated_at"
|
||||||
t.index ["discarded_at"], name: "index_tags_on_discarded_at"
|
|
||||||
t.index ["tag_name_id"], name: "index_tags_on_tag_name_id", unique: true
|
t.index ["tag_name_id"], name: "index_tags_on_tag_name_id", unique: true
|
||||||
t.check_constraint "(`deprecated_at` is null) or (`category` <> _utf8mb4'nico')", name: "chk_tags_deprecated_at_not_nico"
|
t.check_constraint "(`deprecated_at` is null) or (`category` <> _utf8mb4'nico')", name: "chk_tags_deprecated_at_not_nico"
|
||||||
t.check_constraint "`version_no` > 0", name: "chk_tags_version_no_positive"
|
t.check_constraint "`version_no` > 0", name: "chk_tags_version_no_positive"
|
||||||
@@ -566,6 +580,19 @@ ActiveRecord::Schema[8.0].define(version: 2026_06_26_010000) do
|
|||||||
t.index ["post_id"], name: "index_user_post_views_on_post_id"
|
t.index ["post_id"], name: "index_user_post_views_on_post_id"
|
||||||
end
|
end
|
||||||
|
|
||||||
|
create_table "user_theme_slots", charset: "utf8mb4", collation: "utf8mb4_0900_ai_ci", force: :cascade do |t|
|
||||||
|
t.bigint "user_id", null: false
|
||||||
|
t.string "base_theme", null: false
|
||||||
|
t.integer "slot_no", null: false
|
||||||
|
t.json "tokens", null: false
|
||||||
|
t.datetime "created_at", null: false
|
||||||
|
t.datetime "updated_at", null: false
|
||||||
|
t.index ["user_id", "base_theme", "slot_no"], name: "index_user_theme_slots_on_user_theme_and_slot", unique: true
|
||||||
|
t.index ["user_id"], name: "index_user_theme_slots_on_user_id"
|
||||||
|
t.check_constraint "`base_theme` in (_utf8mb4'light',_utf8mb4'dark')", name: "user_theme_slots_base_theme_valid"
|
||||||
|
t.check_constraint "`slot_no` between 1 and 3", name: "user_theme_slots_slot_no_valid"
|
||||||
|
end
|
||||||
|
|
||||||
create_table "users", charset: "utf8mb4", collation: "utf8mb4_0900_ai_ci", force: :cascade do |t|
|
create_table "users", charset: "utf8mb4", collation: "utf8mb4_0900_ai_ci", force: :cascade do |t|
|
||||||
t.string "name"
|
t.string "name"
|
||||||
t.string "inheritance_code", limit: 64, null: false
|
t.string "inheritance_code", limit: 64, null: false
|
||||||
@@ -682,27 +709,25 @@ ActiveRecord::Schema[8.0].define(version: 2026_06_26_010000) do
|
|||||||
add_foreign_key "material_sync_suppressions", "users", column: "created_by_user_id"
|
add_foreign_key "material_sync_suppressions", "users", column: "created_by_user_id"
|
||||||
add_foreign_key "material_versions", "materials"
|
add_foreign_key "material_versions", "materials"
|
||||||
add_foreign_key "material_versions", "materials", column: "parent_id"
|
add_foreign_key "material_versions", "materials", column: "parent_id"
|
||||||
add_foreign_key "material_versions", "tags"
|
|
||||||
add_foreign_key "material_versions", "users", column: "created_by_user_id"
|
add_foreign_key "material_versions", "users", column: "created_by_user_id"
|
||||||
add_foreign_key "material_versions", "users", column: "updated_by_user_id"
|
add_foreign_key "material_versions", "users", column: "updated_by_user_id"
|
||||||
add_foreign_key "materials", "materials", column: "parent_id"
|
add_foreign_key "materials", "materials", column: "parent_id"
|
||||||
add_foreign_key "materials", "tags"
|
add_foreign_key "materials", "tags"
|
||||||
add_foreign_key "materials", "users", column: "created_by_user_id"
|
add_foreign_key "materials", "users", column: "created_by_user_id"
|
||||||
add_foreign_key "materials", "users", column: "updated_by_user_id"
|
add_foreign_key "materials", "users", column: "updated_by_user_id"
|
||||||
|
add_foreign_key "nico_tag_relations", "external_tags", column: "nico_tag_id"
|
||||||
add_foreign_key "nico_tag_relations", "tags"
|
add_foreign_key "nico_tag_relations", "tags"
|
||||||
add_foreign_key "nico_tag_relations", "tags", column: "nico_tag_id"
|
|
||||||
add_foreign_key "nico_tag_versions", "tags"
|
|
||||||
add_foreign_key "nico_tag_versions", "users", column: "created_by_user_id"
|
add_foreign_key "nico_tag_versions", "users", column: "created_by_user_id"
|
||||||
|
add_foreign_key "post_external_tags", "external_tags"
|
||||||
|
add_foreign_key "post_external_tags", "posts"
|
||||||
add_foreign_key "post_implications", "posts"
|
add_foreign_key "post_implications", "posts"
|
||||||
add_foreign_key "post_implications", "posts", column: "parent_post_id"
|
add_foreign_key "post_implications", "posts", column: "parent_post_id"
|
||||||
add_foreign_key "post_similarities", "posts"
|
add_foreign_key "post_similarities", "posts"
|
||||||
add_foreign_key "post_similarities", "posts", column: "target_post_id"
|
add_foreign_key "post_similarities", "posts", column: "target_post_id"
|
||||||
add_foreign_key "post_tag_sections", "posts"
|
add_foreign_key "post_tag_sections", "post_tags", column: ["post_id", "tag_id"], primary_key: ["post_id", "tag_id"], on_delete: :cascade
|
||||||
add_foreign_key "post_tag_sections", "tags"
|
|
||||||
add_foreign_key "post_tags", "posts"
|
add_foreign_key "post_tags", "posts"
|
||||||
add_foreign_key "post_tags", "tags"
|
add_foreign_key "post_tags", "tags"
|
||||||
add_foreign_key "post_tags", "users", column: "created_user_id"
|
add_foreign_key "post_tags", "users", column: "created_user_id"
|
||||||
add_foreign_key "post_tags", "users", column: "deleted_user_id"
|
|
||||||
add_foreign_key "post_versions", "posts"
|
add_foreign_key "post_versions", "posts"
|
||||||
add_foreign_key "post_versions", "users", column: "created_by_user_id"
|
add_foreign_key "post_versions", "users", column: "created_by_user_id"
|
||||||
add_foreign_key "posts", "users", column: "uploaded_user_id"
|
add_foreign_key "posts", "users", column: "uploaded_user_id"
|
||||||
@@ -712,7 +737,6 @@ ActiveRecord::Schema[8.0].define(version: 2026_06_26_010000) do
|
|||||||
add_foreign_key "tag_names", "tag_names", column: "canonical_id"
|
add_foreign_key "tag_names", "tag_names", column: "canonical_id"
|
||||||
add_foreign_key "tag_similarities", "tags"
|
add_foreign_key "tag_similarities", "tags"
|
||||||
add_foreign_key "tag_similarities", "tags", column: "target_tag_id"
|
add_foreign_key "tag_similarities", "tags", column: "target_tag_id"
|
||||||
add_foreign_key "tag_versions", "tags"
|
|
||||||
add_foreign_key "tag_versions", "users", column: "created_by_user_id"
|
add_foreign_key "tag_versions", "users", column: "created_by_user_id"
|
||||||
add_foreign_key "tags", "tag_names"
|
add_foreign_key "tags", "tag_names"
|
||||||
add_foreign_key "theatre_comments", "theatres"
|
add_foreign_key "theatre_comments", "theatres"
|
||||||
@@ -738,6 +762,7 @@ ActiveRecord::Schema[8.0].define(version: 2026_06_26_010000) do
|
|||||||
add_foreign_key "user_ips", "users"
|
add_foreign_key "user_ips", "users"
|
||||||
add_foreign_key "user_post_views", "posts"
|
add_foreign_key "user_post_views", "posts"
|
||||||
add_foreign_key "user_post_views", "users"
|
add_foreign_key "user_post_views", "users"
|
||||||
|
add_foreign_key "user_theme_slots", "users"
|
||||||
add_foreign_key "wiki_assets", "users", column: "created_by_user_id"
|
add_foreign_key "wiki_assets", "users", column: "created_by_user_id"
|
||||||
add_foreign_key "wiki_assets", "wiki_pages"
|
add_foreign_key "wiki_assets", "wiki_pages"
|
||||||
add_foreign_key "wiki_pages", "tag_names"
|
add_foreign_key "wiki_pages", "tag_names"
|
||||||
|
|||||||
@@ -8,6 +8,43 @@
|
|||||||
# MovieGenre.find_or_create_by!(name: genre_name)
|
# MovieGenre.find_or_create_by!(name: genre_name)
|
||||||
# end
|
# end
|
||||||
|
|
||||||
|
post_url_sanitisation_rules = [
|
||||||
|
{ priority: 10,
|
||||||
|
source_pattern: '\Ahttps?://youtu\.be/([^/?#]+)(?:[?#].*)?\z',
|
||||||
|
replacement: 'https://www.youtube.com/watch?v=\1' },
|
||||||
|
{ priority: 20,
|
||||||
|
source_pattern: '\Ahttps?://(?:www\.|m\.)?youtube\.com/live/([^/?#]+)(?:[?#].*)?\z',
|
||||||
|
replacement: 'https://www.youtube.com/watch?v=\1' },
|
||||||
|
{ priority: 30,
|
||||||
|
source_pattern: '\Ahttps?://(?:www\.|m\.)?youtube\.com/shorts/([^/?#]+)(?:[?#].*)?\z',
|
||||||
|
replacement: 'https://www.youtube.com/watch?v=\1' },
|
||||||
|
{ priority: 40,
|
||||||
|
source_pattern: '\Ahttps?://(?:www\.|m\.)?youtube\.com/embed/([^/?#]+)(?:[?#].*)?\z',
|
||||||
|
replacement: 'https://www.youtube.com/watch?v=\1' },
|
||||||
|
{ priority: 50,
|
||||||
|
source_pattern:
|
||||||
|
'\Ahttps?://(?:www\.|m\.)?youtube\.com/watch\?(?:[^#&]+&)*v=([^&#]+)(?:[&#].*)?\z',
|
||||||
|
replacement: 'https://www.youtube.com/watch?v=\1' },
|
||||||
|
{ priority: 60,
|
||||||
|
source_pattern: '\Ahttps?://nico\.ms/([^/?#]+)(?:[?#].*)?\z',
|
||||||
|
replacement: 'https://www.nicovideo.jp/watch/\1' },
|
||||||
|
{ priority: 70,
|
||||||
|
source_pattern: '\Ahttps?://(?:www\.)?nicovideo\.jp/watch/([^?#/]+)(?:[?#].*)?\z',
|
||||||
|
replacement: 'https://www.nicovideo.jp/watch/\1' }
|
||||||
|
]
|
||||||
|
|
||||||
|
post_url_sanitisation_rule_scope = PostUrlSanitisationRule.unscoped
|
||||||
|
|
||||||
|
post_url_sanitisation_rules.each do |attributes|
|
||||||
|
priority = attributes.fetch(:priority)
|
||||||
|
source_pattern = attributes.fetch(:source_pattern)
|
||||||
|
|
||||||
|
next if post_url_sanitisation_rule_scope.exists?(priority:)
|
||||||
|
next if post_url_sanitisation_rule_scope.exists?(source_pattern:)
|
||||||
|
|
||||||
|
post_url_sanitisation_rule_scope.create!(attributes)
|
||||||
|
end
|
||||||
|
|
||||||
material_sync_source_uri = ENV['MATERIAL_SYNC_SOURCE_URI']
|
material_sync_source_uri = ENV['MATERIAL_SYNC_SOURCE_URI']
|
||||||
material_sync_source_file_id = ENV['MATERIAL_SYNC_SOURCE_FILE_ID']
|
material_sync_source_file_id = ENV['MATERIAL_SYNC_SOURCE_FILE_ID']
|
||||||
|
|
||||||
|
|||||||
-1102
ファイル差分が大きすぎるため省略します
差分を読み込み
@@ -1,15 +0,0 @@
|
|||||||
{
|
|
||||||
"name": "lib",
|
|
||||||
"version": "1.0.0",
|
|
||||||
"main": "screenshot.js",
|
|
||||||
"scripts": {
|
|
||||||
"test": "echo \"Error: no test specified\" && exit 1"
|
|
||||||
},
|
|
||||||
"keywords": [],
|
|
||||||
"author": "",
|
|
||||||
"license": "ISC",
|
|
||||||
"description": "",
|
|
||||||
"dependencies": {
|
|
||||||
"puppeteer": "^24.10.0"
|
|
||||||
}
|
|
||||||
}
|
|
||||||
@@ -1,18 +0,0 @@
|
|||||||
const puppeteer = require ('puppeteer')
|
|
||||||
const fs = require ('fs')
|
|
||||||
|
|
||||||
|
|
||||||
void (async () => {
|
|
||||||
const url = process.argv[2]
|
|
||||||
const output = process.argv[3]
|
|
||||||
|
|
||||||
const browser = await puppeteer.launch ({
|
|
||||||
args: ['--no-sandbox', '--disable-setuid-sandbox'] })
|
|
||||||
|
|
||||||
const page = await browser.newPage ()
|
|
||||||
await page.setViewport ({ width: 960, height: 960 })
|
|
||||||
await page.goto (url, { waitUntil: 'networkidle2', timeout: 15000 })
|
|
||||||
|
|
||||||
await page.screenshot ({ path: output })
|
|
||||||
await browser.close ()
|
|
||||||
}) ()
|
|
||||||
@@ -16,7 +16,7 @@ namespace :nico do
|
|||||||
end
|
end
|
||||||
|
|
||||||
def sync_post_tags! post, desired_tag_ids, current_tag_ids: nil
|
def sync_post_tags! post, desired_tag_ids, current_tag_ids: nil
|
||||||
current_tag_ids ||= PostTag.kept.where(post_id: post.id).pluck(:tag_id).to_set
|
current_tag_ids ||= PostTag.where(post_id: post.id).pluck(:tag_id).to_set
|
||||||
desired_tag_ids = desired_tag_ids.compact.to_set
|
desired_tag_ids = desired_tag_ids.compact.to_set
|
||||||
|
|
||||||
to_add = desired_tag_ids - current_tag_ids
|
to_add = desired_tag_ids - current_tag_ids
|
||||||
@@ -30,9 +30,28 @@ namespace :nico do
|
|||||||
end
|
end
|
||||||
end
|
end
|
||||||
|
|
||||||
PostTag.where(post_id: post.id, tag_id: to_remove.to_a).kept.find_each do |pt|
|
PostTag.where(post_id: post.id, tag_id: to_remove.to_a).find_each(&:destroy!)
|
||||||
pt.discard_by!(nil)
|
end
|
||||||
|
|
||||||
|
def sync_post_external_tags! post, desired_external_tag_ids, current_external_tag_ids: nil
|
||||||
|
current_external_tag_ids ||=
|
||||||
|
PostExternalTag.where(post_id: post.id).pluck(:external_tag_id).to_set
|
||||||
|
desired_external_tag_ids = desired_external_tag_ids.compact.to_set
|
||||||
|
|
||||||
|
to_add = desired_external_tag_ids - current_external_tag_ids
|
||||||
|
to_remove = current_external_tag_ids - desired_external_tag_ids
|
||||||
|
|
||||||
|
ExternalTag.where(id: to_add.to_a).find_each do |external_tag|
|
||||||
|
begin
|
||||||
|
PostExternalTag.create!(post:, external_tag:)
|
||||||
|
rescue ActiveRecord::RecordNotUnique
|
||||||
|
;
|
||||||
|
end
|
||||||
end
|
end
|
||||||
|
|
||||||
|
PostExternalTag
|
||||||
|
.where(post_id: post.id, external_tag_id: to_remove.to_a)
|
||||||
|
.find_each(&:destroy!)
|
||||||
end
|
end
|
||||||
|
|
||||||
mysql_user = ENV['MYSQL_USER']
|
mysql_user = ENV['MYSQL_USER']
|
||||||
@@ -70,11 +89,17 @@ namespace :nico do
|
|||||||
unless post.thumbnail.attached?
|
unless post.thumbnail.attached?
|
||||||
thumbnail_base = fetch_thumbnail.(post.url) rescue nil
|
thumbnail_base = fetch_thumbnail.(post.url) rescue nil
|
||||||
if thumbnail_base.present?
|
if thumbnail_base.present?
|
||||||
post.thumbnail.attach(
|
|
||||||
io: URI.open(thumbnail_base),
|
|
||||||
filename: File.basename(URI.parse(thumbnail_base).path),
|
|
||||||
content_type: 'image/jpeg')
|
|
||||||
attrs[:thumbnail_base] = thumbnail_base
|
attrs[:thumbnail_base] = thumbnail_base
|
||||||
|
begin
|
||||||
|
post.attach_thumbnail_from_url!(thumbnail_base)
|
||||||
|
rescue Post::RemoteThumbnailFetchFailed => e
|
||||||
|
payload = {
|
||||||
|
post_id: post.id,
|
||||||
|
thumbnail_base:,
|
||||||
|
error: e.class.name,
|
||||||
|
message: e.message }
|
||||||
|
Rails.logger.info("nico_sync_thumbnail_fetch_failed #{ payload.to_json }")
|
||||||
|
end
|
||||||
end
|
end
|
||||||
end
|
end
|
||||||
|
|
||||||
@@ -82,7 +107,6 @@ namespace :nico do
|
|||||||
post_changed = post.changed?
|
post_changed = post.changed?
|
||||||
if post_changed
|
if post_changed
|
||||||
post.save!
|
post.save!
|
||||||
post.resized_thumbnail! if post.thumbnail.attached?
|
|
||||||
end
|
end
|
||||||
else
|
else
|
||||||
post_created = true
|
post_created = true
|
||||||
@@ -91,68 +115,79 @@ namespace :nico do
|
|||||||
post = Post.new(title:, url:, thumbnail_base:, uploaded_user: nil,
|
post = Post.new(title:, url:, thumbnail_base:, uploaded_user: nil,
|
||||||
original_created_from:, original_created_before:)
|
original_created_from:, original_created_before:)
|
||||||
if thumbnail_base.present?
|
if thumbnail_base.present?
|
||||||
post.thumbnail.attach(
|
begin
|
||||||
io: URI.open(thumbnail_base),
|
post.attach_thumbnail_from_url!(thumbnail_base)
|
||||||
filename: File.basename(URI.parse(thumbnail_base).path),
|
rescue Post::RemoteThumbnailFetchFailed => e
|
||||||
content_type: 'image/jpeg')
|
payload = {
|
||||||
|
post_id: nil,
|
||||||
|
thumbnail_base:,
|
||||||
|
error: e.class.name,
|
||||||
|
message: e.message }
|
||||||
|
Rails.logger.info("nico_sync_thumbnail_fetch_failed #{ payload.to_json }")
|
||||||
|
end
|
||||||
end
|
end
|
||||||
post.save!
|
post.save!
|
||||||
post.resized_thumbnail!
|
|
||||||
sync_post_tags!(post, [Tag.tagme.id, Tag.bot.id, Tag.niconico.id, Tag.video.id])
|
sync_post_tags!(post, [Tag.tagme.id, Tag.bot.id, Tag.niconico.id, Tag.video.id])
|
||||||
end
|
end
|
||||||
|
|
||||||
tags = post.tags
|
|
||||||
# 既存のタグ Id. 集合
|
# 既存のタグ Id. 集合
|
||||||
kept_tag_ids = tags.pluck(:id).to_set
|
kept_tag_ids = post.tags.pluck(:id).to_set
|
||||||
# うち内部タグ Id. 集合
|
|
||||||
kept_non_nico_tag_ids = tags.not_nico.pluck(:id).to_set
|
# 既存の外部タグ Id. 集合
|
||||||
|
kept_external_tag_ids = post.external_tags.nico.pluck(:id).to_set
|
||||||
|
|
||||||
|
# 記載すべき外部タグ Id. のリスト
|
||||||
|
desired_external_tag_ids = []
|
||||||
|
|
||||||
# 記載すべき外部タグ Id. および連携される内部タグ Id. のリスト
|
|
||||||
desired_nico_tag_based_ids = []
|
|
||||||
# 記載すべき内部タグ Id. のリスト
|
# 記載すべき内部タグ Id. のリスト
|
||||||
desired_non_nico_tag_ids = []
|
desired_tag_ids = kept_tag_ids.to_a
|
||||||
|
|
||||||
datum['tags'].each do |raw|
|
datum['tags'].each do |raw|
|
||||||
name = TagNameSanitisationRule.sanitise("nico:#{ raw }")
|
name = TagNameSanitisationRule.sanitise(raw)
|
||||||
tag = Tag.find_or_create_by_tag_name!(name, category: :nico)
|
tag = ExternalTag.find_or_create_by!(platform: :nico, name:)
|
||||||
|
|
||||||
event_type = tag.nico_tag_versions.exists? ? :update : :create
|
unless tag.nico_tag_versions.exists?
|
||||||
NicoTagVersionRecorder.record!(tag:, event_type:, created_by_user: nil)
|
NicoTagVersionRecorder.record!(external_tag: tag,
|
||||||
|
event_type: :create,
|
||||||
|
created_by_user: nil)
|
||||||
|
end
|
||||||
|
|
||||||
desired_nico_tag_based_ids << tag.id
|
desired_external_tag_ids << tag.id
|
||||||
|
|
||||||
# 新たに記載される外部タグと連携される内部タグを記載
|
# 新たに記載される外部タグと連携される内部タグを記載
|
||||||
unless tag.id.in?(kept_tag_ids)
|
# 連携タグは記載すれども消除せず.
|
||||||
linked_ids = tag.linked_tags.pluck(:id)
|
unless tag.id.in?(kept_external_tag_ids)
|
||||||
desired_non_nico_tag_ids.concat(linked_ids)
|
desired_tag_ids.concat(tag.linked_tags.pluck(:id))
|
||||||
desired_nico_tag_based_ids.concat(linked_ids)
|
|
||||||
end
|
end
|
||||||
end
|
end
|
||||||
|
|
||||||
deerjikist = Deerjikist.find_by(platform: :nico, code: datum['user'])
|
deerjikist = Deerjikist.find_by(platform: :nico, code: datum['user'])
|
||||||
if deerjikist
|
if deerjikist
|
||||||
desired_non_nico_tag_ids << deerjikist.tag_id
|
desired_tag_ids << deerjikist.tag_id
|
||||||
desired_nico_tag_based_ids << deerjikist.tag_id
|
elsif !(Tag.where(id: kept_tag_ids).where(category: :deerjikist).exists?)
|
||||||
elsif !(Tag.where(id: kept_non_nico_tag_ids).where(category: :deerjikist).exists?)
|
desired_tag_ids << Tag.no_deerjikist.id
|
||||||
desired_non_nico_tag_ids << Tag.no_deerjikist.id
|
|
||||||
desired_nico_tag_based_ids << Tag.no_deerjikist.id
|
|
||||||
end
|
end
|
||||||
|
|
||||||
desired_nico_tag_based_ids.uniq!
|
desired_external_tag_ids.uniq!
|
||||||
|
desired_tag_ids.uniq!
|
||||||
|
|
||||||
desired_all_tag_ids = kept_non_nico_tag_ids.to_a + desired_nico_tag_based_ids
|
# 外部タグの記載に際しては “bot 操作” タグを記載しなぃ.
|
||||||
desired_non_nico_tag_ids.concat(kept_non_nico_tag_ids.to_a)
|
if kept_tag_ids != desired_tag_ids.to_set
|
||||||
desired_non_nico_tag_ids.uniq!
|
desired_tag_ids << Tag.bot.id
|
||||||
if kept_non_nico_tag_ids != desired_non_nico_tag_ids.to_set
|
desired_tag_ids.uniq!
|
||||||
desired_all_tag_ids << Tag.bot.id
|
|
||||||
end
|
end
|
||||||
desired_all_tag_ids.uniq!
|
|
||||||
|
|
||||||
sync_post_tags!(post, desired_all_tag_ids, current_tag_ids: kept_tag_ids)
|
tags_changed =
|
||||||
|
kept_tag_ids != desired_tag_ids.to_set ||
|
||||||
|
kept_external_tag_ids != desired_external_tag_ids.to_set
|
||||||
|
|
||||||
|
sync_post_tags!(post, desired_tag_ids, current_tag_ids: kept_tag_ids)
|
||||||
|
sync_post_external_tags!(post, desired_external_tag_ids,
|
||||||
|
current_external_tag_ids: kept_external_tag_ids)
|
||||||
|
|
||||||
if post_created
|
if post_created
|
||||||
PostVersionRecorder.record!(post:, event_type: :create, created_by_user: nil)
|
PostVersionRecorder.record!(post:, event_type: :create, created_by_user: nil)
|
||||||
elsif post_changed || kept_tag_ids != desired_all_tag_ids.to_set
|
elsif post_changed || tags_changed
|
||||||
PostVersionRecorder.ensure_snapshot!(post, created_by_user: nil)
|
PostVersionRecorder.ensure_snapshot!(post, created_by_user: nil)
|
||||||
PostVersionRecorder.record!(post:, event_type: :update, created_by_user: nil)
|
PostVersionRecorder.record!(post:, event_type: :update, created_by_user: nil)
|
||||||
end
|
end
|
||||||
|
|||||||
@@ -0,0 +1,123 @@
|
|||||||
|
require 'rails_helper'
|
||||||
|
require_relative '../../db/migrate/20260921020000_delete_discarded_records_from_tags'
|
||||||
|
require_relative '../../db/migrate/20260921030000_delete_discarded_records_from_tag_names'
|
||||||
|
|
||||||
|
RSpec.describe 'discarded tag cleanup migrations' do
|
||||||
|
[DeleteDiscardedRecordsFromTags, DeleteDiscardedRecordsFromTagNames].each do |migration_class|
|
||||||
|
it "rejects rollback of #{ migration_class.name }" do
|
||||||
|
expect { migration_class.new.down }
|
||||||
|
.to raise_error(ActiveRecord::IrreversibleMigration)
|
||||||
|
end
|
||||||
|
end
|
||||||
|
|
||||||
|
context 'with legacy records' do
|
||||||
|
self.use_transactional_tests = false
|
||||||
|
|
||||||
|
before do
|
||||||
|
record_class = Class.new(ActiveRecord::Base) do
|
||||||
|
self.abstract_class = true
|
||||||
|
end
|
||||||
|
stub_const('TagCleanupMigrationRecord', record_class)
|
||||||
|
config = ActiveRecord::Base.connection_db_config.configuration_hash
|
||||||
|
@database = "btrc_hub_test_tag_cleanup_#{ Process.pid }_#{ SecureRandom.hex(4) }"
|
||||||
|
record_class.establish_connection(config.merge(database: nil))
|
||||||
|
@connection = record_class.lease_connection
|
||||||
|
@connection.create_database(@database)
|
||||||
|
@database_created = true
|
||||||
|
@connection.execute("USE #{ @connection.quote_table_name(@database) }")
|
||||||
|
end
|
||||||
|
|
||||||
|
after do
|
||||||
|
@connection.drop_database(@database) if @database_created
|
||||||
|
ensure
|
||||||
|
TagCleanupMigrationRecord.remove_connection
|
||||||
|
end
|
||||||
|
|
||||||
|
before do
|
||||||
|
@connection.create_table(:tag_names) do |t|
|
||||||
|
t.string :name, null: false, index: { unique: true }
|
||||||
|
t.bigint :canonical_id
|
||||||
|
t.datetime :discarded_at, index: true
|
||||||
|
end
|
||||||
|
@connection.add_foreign_key(:tag_names, :tag_names, column: :canonical_id)
|
||||||
|
@connection.create_table(:tags) do |t|
|
||||||
|
t.references :tag_name, null: false, foreign_key: true, index: { unique: true }
|
||||||
|
t.datetime :discarded_at, index: true
|
||||||
|
end
|
||||||
|
@connection.create_table(:nico_tag_relations) do |t|
|
||||||
|
t.references :tag, null: false, foreign_key: true
|
||||||
|
t.references :nico_tag, null: false, foreign_key: { to_table: :tags }
|
||||||
|
end
|
||||||
|
@connection.create_table(:tag_implications) do |t|
|
||||||
|
t.references :tag, null: false, foreign_key: true
|
||||||
|
t.references :parent_tag, null: false, foreign_key: { to_table: :tags }
|
||||||
|
end
|
||||||
|
[:tag_versions, :nico_tag_versions, :material_versions].each do |table|
|
||||||
|
@connection.create_table(table) do |t|
|
||||||
|
t.references :tag, null: false, foreign_key: true
|
||||||
|
end
|
||||||
|
end
|
||||||
|
|
||||||
|
@connection.execute(<<~SQL)
|
||||||
|
INSERT INTO tag_names (id, name, canonical_id, discarded_at) VALUES
|
||||||
|
(1, 'kept', NULL, NULL),
|
||||||
|
(2, 'merged_alias', 1, NULL),
|
||||||
|
(3, 'nico:deleted', NULL, '2026-09-20'),
|
||||||
|
(4, 'nico:kept', NULL, NULL),
|
||||||
|
(5, 'deleted_name', NULL, '2026-09-20')
|
||||||
|
SQL
|
||||||
|
@connection.execute(<<~SQL)
|
||||||
|
INSERT INTO tags (id, tag_name_id, discarded_at) VALUES
|
||||||
|
(1, 1, NULL), (2, 2, '2026-09-20'),
|
||||||
|
(3, 3, '2026-09-20'), (4, 4, NULL)
|
||||||
|
SQL
|
||||||
|
@connection.execute(<<~SQL)
|
||||||
|
INSERT INTO nico_tag_relations (id, tag_id, nico_tag_id) VALUES
|
||||||
|
(1, 1, 4), (2, 2, 4), (3, 1, 3), (4, 2, 3)
|
||||||
|
SQL
|
||||||
|
@connection.execute(<<~SQL)
|
||||||
|
INSERT INTO tag_implications (id, tag_id, parent_tag_id) VALUES
|
||||||
|
(1, 1, 4), (2, 2, 1), (3, 1, 2), (4, 2, 3)
|
||||||
|
SQL
|
||||||
|
@connection.execute('INSERT INTO tag_versions (tag_id) VALUES (1), (2)')
|
||||||
|
@connection.execute('INSERT INTO nico_tag_versions (tag_id) VALUES (3), (4)')
|
||||||
|
@connection.execute('INSERT INTO material_versions (tag_id) VALUES (1), (2)')
|
||||||
|
end
|
||||||
|
|
||||||
|
it 'removes discarded records and their links while retaining aliases and history' do
|
||||||
|
[DeleteDiscardedRecordsFromTags, DeleteDiscardedRecordsFromTagNames].each do |klass|
|
||||||
|
migration = klass.new
|
||||||
|
allow(migration).to receive(:connection).and_return(@connection)
|
||||||
|
migration.suppress_messages { migration.up }
|
||||||
|
end
|
||||||
|
|
||||||
|
expect(@connection.select_values('SELECT id FROM tags ORDER BY id')).to eq([1, 4])
|
||||||
|
expect(@connection.select_rows('SELECT id, canonical_id FROM tag_names ORDER BY id'))
|
||||||
|
.to eq([[1, nil], [2, 1], [4, nil]])
|
||||||
|
expect(@connection.select_values('SELECT id FROM nico_tag_relations')).to eq([1])
|
||||||
|
expect(@connection.select_values('SELECT id FROM tag_implications')).to eq([1])
|
||||||
|
expect(@connection.select_values('SELECT tag_id FROM tag_versions ORDER BY tag_id'))
|
||||||
|
.to eq([1, 2])
|
||||||
|
expect(@connection.select_values('SELECT tag_id FROM nico_tag_versions ORDER BY tag_id'))
|
||||||
|
.to eq([3, 4])
|
||||||
|
expect(@connection.select_values('SELECT tag_id FROM material_versions ORDER BY tag_id'))
|
||||||
|
.to eq([1, 2])
|
||||||
|
|
||||||
|
[:tags, :tag_names].each do |table|
|
||||||
|
expect(@connection.column_exists?(table, :discarded_at)).to be(false)
|
||||||
|
expect(@connection.index_exists?(table, :discarded_at)).to be(false)
|
||||||
|
end
|
||||||
|
[:tag_versions, :nico_tag_versions, :material_versions].each do |table|
|
||||||
|
expect(@connection.foreign_key_exists?(table, :tags, column: :tag_id)).to be(false)
|
||||||
|
end
|
||||||
|
expect(@connection.foreign_key_exists?(:tags, :tag_names)).to be(true)
|
||||||
|
expect(@connection.foreign_key_exists?(:tag_names, :tag_names, column: :canonical_id))
|
||||||
|
.to be(true)
|
||||||
|
expect(@connection.index_exists?(:tag_names, :name, unique: true)).to be(true)
|
||||||
|
expect(@connection.index_exists?(:tags, :tag_name_id, unique: true)).to be(true)
|
||||||
|
[:nico_tag_relations, :tag_implications].each do |table|
|
||||||
|
expect(@connection.foreign_keys(table).map(&:to_table)).to eq(['tags', 'tags'])
|
||||||
|
end
|
||||||
|
end
|
||||||
|
end
|
||||||
|
end
|
||||||
@@ -0,0 +1,191 @@
|
|||||||
|
require 'rails_helper'
|
||||||
|
require_relative '../../db/migrate/20260921040000_create_external_tags'
|
||||||
|
require_relative '../../db/migrate/20260921050000_create_post_external_tags'
|
||||||
|
require_relative '../../db/migrate/20260921060000_change_foreign_key_on_nico_tag_relations'
|
||||||
|
require_relative '../../db/migrate/20260921230000_migrate_external_tags'
|
||||||
|
|
||||||
|
RSpec.describe 'external tag migrations' do
|
||||||
|
self.use_transactional_tests = false
|
||||||
|
|
||||||
|
before do
|
||||||
|
record_class = Class.new(ActiveRecord::Base) do
|
||||||
|
self.abstract_class = true
|
||||||
|
end
|
||||||
|
stub_const('ExternalTagMigrationRecord', record_class)
|
||||||
|
config = ActiveRecord::Base.connection_db_config.configuration_hash
|
||||||
|
@database = "btrc_hub_test_external_tags_#{ Process.pid }_#{ SecureRandom.hex(4) }"
|
||||||
|
record_class.establish_connection(config.merge(database: nil))
|
||||||
|
@connection = record_class.lease_connection
|
||||||
|
@connection.create_database(@database)
|
||||||
|
@database_created = true
|
||||||
|
@connection.execute("USE #{ @connection.quote_table_name(@database) }")
|
||||||
|
version_class = Class.new(record_class) do
|
||||||
|
self.table_name = 'post_versions'
|
||||||
|
end
|
||||||
|
stub_const('MigrateExternalTags::MigrationPostVersion', version_class)
|
||||||
|
create_legacy_tables
|
||||||
|
seed_legacy_records
|
||||||
|
end
|
||||||
|
|
||||||
|
after do
|
||||||
|
@connection.drop_database(@database) if @database_created
|
||||||
|
ensure
|
||||||
|
ExternalTagMigrationRecord.remove_connection
|
||||||
|
end
|
||||||
|
|
||||||
|
def migrate klass
|
||||||
|
migration = klass.new
|
||||||
|
allow(migration).to receive(:connection).and_return(@connection)
|
||||||
|
migration.suppress_messages { migration.migrate(:up) }
|
||||||
|
end
|
||||||
|
|
||||||
|
def create_legacy_tables
|
||||||
|
@connection.create_table(:tag_names) { |t| t.string :name }
|
||||||
|
@connection.create_table(:tags) do |t|
|
||||||
|
t.references :tag_name, foreign_key: true
|
||||||
|
t.string :category
|
||||||
|
t.integer :post_count
|
||||||
|
t.datetime :created_at
|
||||||
|
end
|
||||||
|
@connection.create_table(:posts)
|
||||||
|
@connection.create_table(:post_tags) do |t|
|
||||||
|
t.references :post, foreign_key: true
|
||||||
|
t.references :tag, foreign_key: true
|
||||||
|
t.datetime :created_at
|
||||||
|
end
|
||||||
|
@connection.create_table(:nico_tag_versions) do |t|
|
||||||
|
t.bigint :tag_id
|
||||||
|
t.integer :version_no
|
||||||
|
t.string :name
|
||||||
|
t.datetime :created_at
|
||||||
|
end
|
||||||
|
@connection.create_table(:nico_tag_relations) do |t|
|
||||||
|
t.references :tag, foreign_key: true
|
||||||
|
t.references :nico_tag, foreign_key: { to_table: :tags }
|
||||||
|
end
|
||||||
|
[:post_tag_sections, :materials, :theatre_skip_event_tags].each do |table|
|
||||||
|
@connection.create_table(table) { |t| t.references :tag, foreign_key: true }
|
||||||
|
end
|
||||||
|
@connection.create_table(:tag_implications) do |t|
|
||||||
|
t.references :tag, foreign_key: true
|
||||||
|
t.references :parent_tag, foreign_key: { to_table: :tags }
|
||||||
|
end
|
||||||
|
@connection.create_table(:tag_similarities) do |t|
|
||||||
|
t.references :tag, foreign_key: true
|
||||||
|
t.references :target_tag, foreign_key: { to_table: :tags }
|
||||||
|
end
|
||||||
|
@connection.create_table(:post_versions) { |t| t.json :tags_json, null: false }
|
||||||
|
@connection.add_check_constraint(:post_versions, 'JSON_VALID(tags_json)',
|
||||||
|
name: 'chk_post_versions_tags_json_schema')
|
||||||
|
end
|
||||||
|
|
||||||
|
def seed_legacy_records
|
||||||
|
@connection.execute(<<~SQL)
|
||||||
|
INSERT INTO tag_names (id, name) VALUES (1, 'internal'), (2, 'nico:raw tag[]');
|
||||||
|
SQL
|
||||||
|
@connection.execute(<<~SQL)
|
||||||
|
INSERT INTO tags (id, tag_name_id, category, post_count, created_at) VALUES
|
||||||
|
(1, 1, 'general', 1, '2026-09-01'), (2, 2, 'nico', 1, '2026-09-02')
|
||||||
|
SQL
|
||||||
|
@connection.execute('INSERT INTO posts (id) VALUES (1)')
|
||||||
|
@connection.execute(<<~SQL)
|
||||||
|
INSERT INTO post_tags (post_id, tag_id, created_at) VALUES
|
||||||
|
(1, 1, '2026-09-03'), (1, 2, '2026-09-04')
|
||||||
|
SQL
|
||||||
|
@connection.execute(<<~SQL)
|
||||||
|
INSERT INTO nico_tag_versions (tag_id, version_no, name, created_at) VALUES
|
||||||
|
(2, 1, 'nico:raw tag[]', '2026-09-02'),
|
||||||
|
(3, 1, 'nico:deleted', '2026-09-01'),
|
||||||
|
(3, 2, 'nico:deleted_later', '2026-09-02')
|
||||||
|
SQL
|
||||||
|
@connection.execute('INSERT INTO nico_tag_relations (tag_id, nico_tag_id) VALUES (1, 2)')
|
||||||
|
@connection.execute(<<~SQL)
|
||||||
|
INSERT INTO tag_similarities (tag_id, target_tag_id) VALUES (1, 1), (1, 2), (2, 1)
|
||||||
|
SQL
|
||||||
|
@connection.execute('INSERT INTO theatre_skip_event_tags (tag_id) VALUES (1), (2)')
|
||||||
|
@legacy_internal = { 'id' => 1, 'version_no' => 2,
|
||||||
|
'name' => 'internal', 'category' => 'general',
|
||||||
|
'sections' => [{ 'begin_ms' => 1000, 'end_ms' => nil }] }
|
||||||
|
snapshots = [@legacy_internal,
|
||||||
|
{ 'id' => 2, 'category' => 'nico' },
|
||||||
|
{ 'id' => 3, 'category' => 'nico' }]
|
||||||
|
@connection.execute(<<~SQL)
|
||||||
|
INSERT INTO post_versions (tags_json) VALUES
|
||||||
|
(#{ @connection.quote(snapshots.to_json) }), ('[]')
|
||||||
|
SQL
|
||||||
|
end
|
||||||
|
|
||||||
|
def prepare_external_tables
|
||||||
|
migrate(CreateExternalTags)
|
||||||
|
migrate(CreatePostExternalTags)
|
||||||
|
migrate(ChangeForeignKeyOnNicoTagRelations)
|
||||||
|
end
|
||||||
|
|
||||||
|
it 'preserves ids, raw names, counts, timestamps, post links and historical-only tags' do
|
||||||
|
prepare_external_tables
|
||||||
|
|
||||||
|
rows = @connection.select_rows(<<~SQL)
|
||||||
|
SELECT id, platform, name, post_count, DATE_FORMAT(created_at, '%Y-%m-%d')
|
||||||
|
FROM external_tags ORDER BY id
|
||||||
|
SQL
|
||||||
|
expect(rows).to eq([
|
||||||
|
[2, 'nico', 'raw tag[]', 1, '2026-09-02'],
|
||||||
|
[3, 'nico', 'deleted', 0, '2026-09-01']])
|
||||||
|
expect(@connection.select_rows(<<~SQL)).to eq([[1, 2, '2026-09-04']])
|
||||||
|
SELECT post_id, external_tag_id, DATE_FORMAT(created_at, '%Y-%m-%d')
|
||||||
|
FROM post_external_tags
|
||||||
|
SQL
|
||||||
|
expect(@connection.foreign_key_exists?(:nico_tag_relations, :external_tags,
|
||||||
|
column: :nico_tag_id)).to be(true)
|
||||||
|
expect(@connection.foreign_key_exists?(:nico_tag_relations, :tags,
|
||||||
|
column: :tag_id)).to be(true)
|
||||||
|
end
|
||||||
|
|
||||||
|
it 'converts snapshots and removes only obsolete internal rows and derived links' do
|
||||||
|
prepare_external_tables
|
||||||
|
migrate(MigrateExternalTags)
|
||||||
|
|
||||||
|
expect(@connection.select_values('SELECT id FROM tags')).to eq([1])
|
||||||
|
expect(@connection.select_values('SELECT id FROM tag_names')).to eq([1])
|
||||||
|
expect(@connection.select_values('SELECT tag_id FROM post_tags')).to eq([1])
|
||||||
|
expect(@connection.select_rows('SELECT tag_id, nico_tag_id FROM nico_tag_relations'))
|
||||||
|
.to eq([[1, 2]])
|
||||||
|
expect(@connection.select_rows('SELECT tag_id, target_tag_id FROM tag_similarities'))
|
||||||
|
.to eq([[1, 1]])
|
||||||
|
expect(@connection.select_values('SELECT tag_id FROM theatre_skip_event_tags')).to eq([1])
|
||||||
|
expect(@connection.select_value('SELECT COUNT(*) FROM nico_tag_versions')).to eq(3)
|
||||||
|
versions = MigrateExternalTags::MigrationPostVersion.order(:id)
|
||||||
|
expect(versions.first.tags_json).to eq([
|
||||||
|
@legacy_internal.except('id').merge('tag_id' => 1),
|
||||||
|
{ 'external_tag_id' => 2 }, { 'external_tag_id' => 3 }])
|
||||||
|
expect(versions.last.tags_json).to eq([])
|
||||||
|
|
||||||
|
invalid_snapshots = [@legacy_internal,
|
||||||
|
{ 'external_tag_id' => 0 },
|
||||||
|
{ 'external_tag_id' => 2, 'tag_id' => 1 },
|
||||||
|
{ 'external_tag_id' => 2, 'name' => 'extra' },
|
||||||
|
{ 'tag_id' => 1, 'version_no' => 1 }]
|
||||||
|
invalid_snapshots.each do |snapshot|
|
||||||
|
expect {
|
||||||
|
MigrateExternalTags::MigrationPostVersion.create!(tags_json: [snapshot])
|
||||||
|
}.to raise_error(ActiveRecord::StatementInvalid, /check constraint/i)
|
||||||
|
end
|
||||||
|
expect { MigrateExternalTags.new.down }
|
||||||
|
.to raise_error(ActiveRecord::IrreversibleMigration)
|
||||||
|
end
|
||||||
|
|
||||||
|
{ post_tag_sections: '(tag_id) VALUES (2)',
|
||||||
|
materials: '(tag_id) VALUES (2)',
|
||||||
|
tag_implications: '(tag_id, parent_tag_id) VALUES (1, 2)' }.each do |table, values|
|
||||||
|
it "aborts before deleting data when #{ table } references a legacy nico tag" do
|
||||||
|
prepare_external_tables
|
||||||
|
@connection.execute("INSERT INTO #{ table } #{ values }")
|
||||||
|
|
||||||
|
expect { migrate(MigrateExternalTags) }.to raise_error(RuntimeError, /#{ table }/)
|
||||||
|
|
||||||
|
expect(@connection.select_values('SELECT id FROM tags ORDER BY id')).to eq([1, 2])
|
||||||
|
expect(MigrateExternalTags::MigrationPostVersion.first.tags_json.first)
|
||||||
|
.to eq(@legacy_internal)
|
||||||
|
end
|
||||||
|
end
|
||||||
|
end
|
||||||
@@ -0,0 +1,65 @@
|
|||||||
|
require 'rails_helper'
|
||||||
|
|
||||||
|
RSpec.describe 'database seeds' do
|
||||||
|
before do
|
||||||
|
PostUrlSanitisationRule.unscoped.delete_all
|
||||||
|
end
|
||||||
|
|
||||||
|
it 'registers the initial post URL sanitisation rules' do
|
||||||
|
load_seeds
|
||||||
|
|
||||||
|
urls = {
|
||||||
|
'https://youtu.be/abc123?si=share' => youtube_url('abc123'),
|
||||||
|
'https://www.youtube.com/live/abc123?t=10' => youtube_url('abc123'),
|
||||||
|
'https://youtube.com/shorts/abc123?feature=share' => youtube_url('abc123'),
|
||||||
|
'https://m.youtube.com/embed/abc123' => youtube_url('abc123'),
|
||||||
|
'https://youtube.com/watch?feature=share&v=abc123&t=10' => youtube_url('abc123'),
|
||||||
|
'https://nico.ms/sm123?from=share#fragment' => nico_url('sm123'),
|
||||||
|
'https://www.nicovideo.jp/watch/sm123?ref=share#fragment' => nico_url('sm123')
|
||||||
|
}
|
||||||
|
|
||||||
|
expect(PostUrlSanitisationRule.count).to eq(7)
|
||||||
|
urls.each do |url, canonical_url|
|
||||||
|
expect(PostUrlSanitisationRule.sanitise(url)).to eq(canonical_url)
|
||||||
|
end
|
||||||
|
end
|
||||||
|
|
||||||
|
it 'does not overwrite or restore an existing rule' do
|
||||||
|
rule = PostUrlSanitisationRule.create!(
|
||||||
|
priority: 10,
|
||||||
|
source_pattern: '\Ahttps://example\.com/custom\z',
|
||||||
|
replacement: 'https://example.com/replacement'
|
||||||
|
)
|
||||||
|
rule.discard!
|
||||||
|
original_attributes = rule.reload.attributes
|
||||||
|
|
||||||
|
2.times { load_seeds }
|
||||||
|
|
||||||
|
persisted_rule = PostUrlSanitisationRule.unscoped.find(10)
|
||||||
|
expect(persisted_rule.attributes).to eq(original_attributes)
|
||||||
|
expect(PostUrlSanitisationRule.unscoped.count).to eq(7)
|
||||||
|
end
|
||||||
|
|
||||||
|
it 'does not duplicate a rule moved to another priority' do
|
||||||
|
source_pattern = '\Ahttps?://youtu\.be/([^/?#]+)(?:[?#].*)?\z'
|
||||||
|
PostUrlSanitisationRule.create!(
|
||||||
|
priority: 80,
|
||||||
|
source_pattern:,
|
||||||
|
replacement: 'https://example.com/custom/\1'
|
||||||
|
)
|
||||||
|
|
||||||
|
load_seeds
|
||||||
|
|
||||||
|
rules = PostUrlSanitisationRule.unscoped
|
||||||
|
expect(rules.where(source_pattern:).count).to eq(1)
|
||||||
|
expect(rules.find(80).replacement).to eq('https://example.com/custom/\1')
|
||||||
|
end
|
||||||
|
|
||||||
|
def load_seeds
|
||||||
|
load Rails.root.join('db/seeds.rb')
|
||||||
|
end
|
||||||
|
|
||||||
|
def youtube_url(video_id) = "https://www.youtube.com/watch?v=#{ video_id }"
|
||||||
|
|
||||||
|
def nico_url(video_id) = "https://www.nicovideo.jp/watch/#{ video_id }"
|
||||||
|
end
|
||||||
@@ -0,0 +1,6 @@
|
|||||||
|
FactoryBot.define do
|
||||||
|
factory :external_tag do
|
||||||
|
platform { :nico }
|
||||||
|
sequence(:name) { |n| "external_tag_#{ n }" }
|
||||||
|
end
|
||||||
|
end
|
||||||
@@ -11,12 +11,5 @@ FactoryBot.define do
|
|||||||
after(:build) do |tag, evaluator|
|
after(:build) do |tag, evaluator|
|
||||||
tag.name = evaluator.name if evaluator.name.present?
|
tag.name = evaluator.name if evaluator.name.present?
|
||||||
end
|
end
|
||||||
|
|
||||||
trait :nico do
|
|
||||||
category { :nico }
|
|
||||||
transient do
|
|
||||||
name { "nico:#{ SecureRandom.hex(4) }" }
|
|
||||||
end
|
|
||||||
end
|
|
||||||
end
|
end
|
||||||
end
|
end
|
||||||
|
|||||||
@@ -0,0 +1,38 @@
|
|||||||
|
require 'rails_helper'
|
||||||
|
|
||||||
|
RSpec.describe ExternalTag, type: :model do
|
||||||
|
it 'preserves names without internal tag sanitisation or TagName records' do
|
||||||
|
external = nil
|
||||||
|
|
||||||
|
expect {
|
||||||
|
external = described_class.create!(platform: :nico, name: 'raw tag[]')
|
||||||
|
}.not_to change(TagName, :count)
|
||||||
|
|
||||||
|
expect(external.reload.name).to eq('raw tag[]')
|
||||||
|
end
|
||||||
|
|
||||||
|
it 'deletes post associations without deleting posts or version history' do
|
||||||
|
external = create(:external_tag)
|
||||||
|
post = create(:post)
|
||||||
|
PostExternalTag.create!(post:, external_tag: external)
|
||||||
|
version = NicoTagVersionRecorder.record!(
|
||||||
|
external_tag: external, event_type: :create, created_by_user: nil)
|
||||||
|
|
||||||
|
external.destroy!
|
||||||
|
|
||||||
|
expect(PostExternalTag.where(external_tag_id: external.id)).to be_empty
|
||||||
|
expect(post.reload.external_tags).to be_empty
|
||||||
|
expect(version.reload.tag_id).to eq(external.id)
|
||||||
|
end
|
||||||
|
|
||||||
|
it 'deletes external links without deleting linked internal tags' do
|
||||||
|
external = create(:external_tag)
|
||||||
|
tag = create(:tag)
|
||||||
|
NicoTagRelation.create!(nico_tag: external, tag:)
|
||||||
|
|
||||||
|
external.destroy!
|
||||||
|
|
||||||
|
expect(NicoTagRelation.where(nico_tag_id: external.id)).to be_empty
|
||||||
|
expect(Tag.exists?(tag.id)).to be(true)
|
||||||
|
end
|
||||||
|
end
|
||||||
@@ -0,0 +1,27 @@
|
|||||||
|
require 'rails_helper'
|
||||||
|
|
||||||
|
RSpec.describe NicoTagRelation, type: :model do
|
||||||
|
it 'does not constrain the ExternalTag association to the nico platform' do
|
||||||
|
external_tag = create(:external_tag)
|
||||||
|
tag = create(:tag)
|
||||||
|
allow(external_tag).to receive(:platform).and_return('registered_external')
|
||||||
|
allow(external_tag).to receive(:nico?).and_return(false)
|
||||||
|
|
||||||
|
expect {
|
||||||
|
described_class.create!(nico_tag: external_tag, tag:)
|
||||||
|
}.to change(described_class, :count).by(1)
|
||||||
|
expect(tag.linked_nico_tags).to contain_exactly(external_tag)
|
||||||
|
end
|
||||||
|
|
||||||
|
it 'rejects an internal Tag through the external association type' do
|
||||||
|
expect {
|
||||||
|
described_class.new(nico_tag: create(:tag), tag: create(:tag))
|
||||||
|
}.to raise_error(ActiveRecord::AssociationTypeMismatch)
|
||||||
|
end
|
||||||
|
|
||||||
|
it 'rejects an ExternalTag through the internal association type' do
|
||||||
|
expect {
|
||||||
|
described_class.new(nico_tag: create(:external_tag), tag: create(:external_tag))
|
||||||
|
}.to raise_error(ActiveRecord::AssociationTypeMismatch)
|
||||||
|
end
|
||||||
|
end
|
||||||
@@ -0,0 +1,25 @@
|
|||||||
|
require 'rails_helper'
|
||||||
|
|
||||||
|
RSpec.describe PostExternalTag, type: :model do
|
||||||
|
it 'exposes both sides and enforces a unique post and external tag pair' do
|
||||||
|
post = create(:post)
|
||||||
|
external_tag = create(:external_tag)
|
||||||
|
described_class.create!(post:, external_tag:)
|
||||||
|
|
||||||
|
expect(post.external_tags).to contain_exactly(external_tag)
|
||||||
|
expect(external_tag.posts).to contain_exactly(post)
|
||||||
|
expect { described_class.create!(post:, external_tag:) }
|
||||||
|
.to raise_error(ActiveRecord::RecordNotUnique)
|
||||||
|
end
|
||||||
|
|
||||||
|
it 'removes associations when the post is deleted while retaining the external tag' do
|
||||||
|
post = create(:post)
|
||||||
|
external_tag = create(:external_tag)
|
||||||
|
described_class.create!(post:, external_tag:)
|
||||||
|
|
||||||
|
post.destroy!
|
||||||
|
|
||||||
|
expect(described_class.where(post_id: post.id)).to be_empty
|
||||||
|
expect(external_tag.reload.posts).to be_empty
|
||||||
|
end
|
||||||
|
end
|
||||||
@@ -0,0 +1,380 @@
|
|||||||
|
require 'rails_helper'
|
||||||
|
require 'tempfile'
|
||||||
|
|
||||||
|
RSpec.describe Post, type: :model do
|
||||||
|
before do
|
||||||
|
PostUrlSanitisationRule.unscoped.delete_all
|
||||||
|
end
|
||||||
|
|
||||||
|
describe '#snapshot_tags_json' do
|
||||||
|
it 'keeps internal snapshots and external identifiers distinct, even with the same id' do
|
||||||
|
post = create(:post)
|
||||||
|
tag = create(:tag)
|
||||||
|
external = create(:external_tag, id: tag.id)
|
||||||
|
create(:post_tag, post:, tag:)
|
||||||
|
create(:post_tag_section, post:, tag:, begin_ms: 2000, end_ms: nil)
|
||||||
|
PostExternalTag.create!(post:, external_tag: external)
|
||||||
|
|
||||||
|
expect(post.snapshot_tags_json).to eq([
|
||||||
|
{ 'tag_id' => tag.id,
|
||||||
|
'version_no' => tag.version_no,
|
||||||
|
'name' => tag.name,
|
||||||
|
'category' => tag.category,
|
||||||
|
'sections' => [{ 'begin_ms' => 2000, 'end_ms' => nil }] },
|
||||||
|
{ 'external_tag_id' => external.id }])
|
||||||
|
end
|
||||||
|
end
|
||||||
|
|
||||||
|
describe 'URL normalisation' do
|
||||||
|
it 'normalises the HTTP URL before applying sanitisation rules' do
|
||||||
|
PostUrlSanitisationRule.create!(
|
||||||
|
priority: 10,
|
||||||
|
source_pattern: '\\Ahttps://example\\.com/videos/([^/]+)\\z',
|
||||||
|
replacement: 'https://example.com/watch/\\1'
|
||||||
|
)
|
||||||
|
|
||||||
|
post = described_class.create!(
|
||||||
|
title: 'normalised URL',
|
||||||
|
url: ' https://EXAMPLE.com/videos/123/ '
|
||||||
|
)
|
||||||
|
|
||||||
|
expect(post.url).to eq('https://example.com/watch/123')
|
||||||
|
end
|
||||||
|
|
||||||
|
it 'does not normalise an unchanged URL when another attribute changes' do
|
||||||
|
post = create(:post)
|
||||||
|
post.update_column(:url, 'https://EXAMPLE.com/unchanged/')
|
||||||
|
|
||||||
|
post.update!(title: 'updated title')
|
||||||
|
|
||||||
|
expect(post.reload.url).to eq('https://EXAMPLE.com/unchanged/')
|
||||||
|
end
|
||||||
|
|
||||||
|
it 'validates the sanitised URL length' do
|
||||||
|
path = 'a' * 375
|
||||||
|
|
||||||
|
PostUrlSanitisationRule.create!(
|
||||||
|
priority: 10,
|
||||||
|
source_pattern: '\\Ahttps://example\\.com/(a+)\\z',
|
||||||
|
replacement: 'https://example.com/\\1\\1'
|
||||||
|
)
|
||||||
|
|
||||||
|
post = described_class.new(
|
||||||
|
title: 'long URL',
|
||||||
|
url: "https://example.com/#{ path }"
|
||||||
|
)
|
||||||
|
|
||||||
|
expect(post).to be_invalid
|
||||||
|
expect(post.errors.details.fetch(:url)).to include(
|
||||||
|
error: :too_long,
|
||||||
|
count: 768
|
||||||
|
)
|
||||||
|
end
|
||||||
|
|
||||||
|
it 'validates uniqueness after sanitisation' do
|
||||||
|
PostUrlSanitisationRule.create!(
|
||||||
|
priority: 10,
|
||||||
|
source_pattern: '\\Ahttps://example\\.com/alias\\z',
|
||||||
|
replacement: 'https://example.com/canonical'
|
||||||
|
)
|
||||||
|
create(:post, url: 'https://example.com/canonical')
|
||||||
|
|
||||||
|
post = described_class.new(title: 'duplicate URL', url: 'https://example.com/alias')
|
||||||
|
|
||||||
|
expect(post).to be_invalid
|
||||||
|
expect(post.errors.details.fetch(:url)).to include(error: :taken, value: post.url)
|
||||||
|
end
|
||||||
|
end
|
||||||
|
|
||||||
|
describe 'thumbnail processing' do
|
||||||
|
def image_blob(width:, height:, background:, draw: nil)
|
||||||
|
Tempfile.create(['post-thumbnail', '.png']) do |file|
|
||||||
|
MiniMagick::Tool::Convert.new do |convert|
|
||||||
|
convert.size "#{ width }x#{ height }"
|
||||||
|
convert.xc background
|
||||||
|
draw&.call(convert)
|
||||||
|
convert << file.path
|
||||||
|
end
|
||||||
|
File.binread(file.path)
|
||||||
|
end
|
||||||
|
end
|
||||||
|
|
||||||
|
def upload_for(blob)
|
||||||
|
StringIO.new(blob).tap(&:rewind)
|
||||||
|
end
|
||||||
|
|
||||||
|
def read_image(attachment)
|
||||||
|
blob =
|
||||||
|
attachment.is_a?(Hash) ? attachment.fetch(:io).read : attachment.download
|
||||||
|
MiniMagick::Image.read(blob)
|
||||||
|
end
|
||||||
|
|
||||||
|
def colour_at(image, x, y)
|
||||||
|
image.get_pixels.fetch(y).fetch(x)
|
||||||
|
end
|
||||||
|
|
||||||
|
def expect_green(pixel)
|
||||||
|
expect(pixel[1]).to be > pixel[0] + 40
|
||||||
|
expect(pixel[1]).to be > pixel[2] + 40
|
||||||
|
end
|
||||||
|
|
||||||
|
def expect_red(pixel)
|
||||||
|
expect(pixel[0]).to be > pixel[1] + 40
|
||||||
|
expect(pixel[0]).to be > pixel[2] + 40
|
||||||
|
end
|
||||||
|
|
||||||
|
def expect_blue(pixel)
|
||||||
|
expect(pixel[2]).to be > pixel[0] + 40
|
||||||
|
expect(pixel[2]).to be > pixel[1] + 40
|
||||||
|
end
|
||||||
|
|
||||||
|
describe '.resized_thumbnail_attachment' do
|
||||||
|
it 'fits a wide image within 180x180 without distorting it' do
|
||||||
|
blob = image_blob(
|
||||||
|
width: 360,
|
||||||
|
height: 180,
|
||||||
|
background: 'red',
|
||||||
|
draw: -> convert {
|
||||||
|
convert.fill 'green'
|
||||||
|
convert.draw 'rectangle 90,0 269,179'
|
||||||
|
})
|
||||||
|
|
||||||
|
resized = described_class.resized_thumbnail_attachment(upload_for(blob))
|
||||||
|
image = read_image(resized)
|
||||||
|
|
||||||
|
expect(image.dimensions).to eq([180, 90])
|
||||||
|
expect_red(colour_at(image, 0, 45))
|
||||||
|
expect_green(colour_at(image, 90, 45))
|
||||||
|
expect_red(colour_at(image, 179, 45))
|
||||||
|
end
|
||||||
|
|
||||||
|
it 'fits a tall image within 180x180 without distorting it' do
|
||||||
|
blob = image_blob(
|
||||||
|
width: 180,
|
||||||
|
height: 360,
|
||||||
|
background: 'red',
|
||||||
|
draw: -> convert {
|
||||||
|
convert.fill 'green'
|
||||||
|
convert.draw 'rectangle 0,90 179,269'
|
||||||
|
})
|
||||||
|
|
||||||
|
resized = described_class.resized_thumbnail_attachment(upload_for(blob))
|
||||||
|
image = read_image(resized)
|
||||||
|
|
||||||
|
expect(image.dimensions).to eq([90, 180])
|
||||||
|
expect_red(colour_at(image, 45, 0))
|
||||||
|
expect_green(colour_at(image, 45, 90))
|
||||||
|
expect_red(colour_at(image, 45, 179))
|
||||||
|
end
|
||||||
|
|
||||||
|
it 'keeps a square image square without distortion' do
|
||||||
|
blob = image_blob(
|
||||||
|
width: 180,
|
||||||
|
height: 180,
|
||||||
|
background: 'red',
|
||||||
|
draw: -> convert {
|
||||||
|
convert.fill 'blue'
|
||||||
|
convert.draw 'rectangle 90,0 179,179'
|
||||||
|
})
|
||||||
|
|
||||||
|
resized = described_class.resized_thumbnail_attachment(upload_for(blob))
|
||||||
|
image = read_image(resized)
|
||||||
|
|
||||||
|
expect(image.dimensions).to eq([180, 180])
|
||||||
|
expect_red(colour_at(image, 20, 90))
|
||||||
|
expect_blue(colour_at(image, 160, 90))
|
||||||
|
end
|
||||||
|
end
|
||||||
|
|
||||||
|
describe '#attach_thumbnail_from_url!' do
|
||||||
|
it 'attaches a fetched remote image through the common resize path' do
|
||||||
|
blob = image_blob(
|
||||||
|
width: 240,
|
||||||
|
height: 180,
|
||||||
|
background: 'red',
|
||||||
|
draw: -> convert {
|
||||||
|
convert.fill 'green'
|
||||||
|
convert.draw 'rectangle 30,0 209,179'
|
||||||
|
})
|
||||||
|
response = Preview::HttpFetcher::Response.new(
|
||||||
|
blob,
|
||||||
|
'image/png',
|
||||||
|
'https://example.com/thumb.png')
|
||||||
|
allow(Preview::ThumbnailFetcher).to receive(:fetch_image_response)
|
||||||
|
.with('https://example.com/thumb.png')
|
||||||
|
.and_return(response)
|
||||||
|
|
||||||
|
post = described_class.create!(title: 'title', url: 'https://example.com/post')
|
||||||
|
|
||||||
|
expect(post.thumbnail).to receive(:attach).once.and_call_original
|
||||||
|
post.attach_thumbnail_from_url!('https://example.com/thumb.png')
|
||||||
|
|
||||||
|
expect(post.thumbnail).to be_attached
|
||||||
|
image = read_image(post.thumbnail)
|
||||||
|
expect(image.dimensions).to eq([180, 135])
|
||||||
|
end
|
||||||
|
|
||||||
|
it 'does not attach anything when thumbnail conversion fails' do
|
||||||
|
response = Preview::HttpFetcher::Response.new(
|
||||||
|
'not-an-image',
|
||||||
|
'image/png',
|
||||||
|
'https://example.com/thumb.png')
|
||||||
|
allow(Preview::ThumbnailFetcher).to receive(:fetch_image_response)
|
||||||
|
.with('https://example.com/thumb.png')
|
||||||
|
.and_return(response)
|
||||||
|
allow(described_class).to receive(:resized_thumbnail_attachment)
|
||||||
|
.and_raise(MiniMagick::Error, 'convert failed')
|
||||||
|
|
||||||
|
post = described_class.create!(title: 'title', url: 'https://example.com/post')
|
||||||
|
|
||||||
|
expect {
|
||||||
|
post.attach_thumbnail_from_url!('https://example.com/thumb.png')
|
||||||
|
}.to raise_error(Post::RemoteThumbnailFetchFailed, 'convert failed')
|
||||||
|
|
||||||
|
expect(post.thumbnail).not_to be_attached
|
||||||
|
end
|
||||||
|
|
||||||
|
it 'keeps an existing thumbnail when remote conversion fails' do
|
||||||
|
existing = described_class.create!(title: 'title', url: 'https://example.com/post')
|
||||||
|
existing.thumbnail.attach(
|
||||||
|
io: StringIO.new('existing'),
|
||||||
|
filename: 'existing.jpg',
|
||||||
|
content_type: 'image/jpeg')
|
||||||
|
blob_id = existing.thumbnail.blob.id
|
||||||
|
response = Preview::HttpFetcher::Response.new(
|
||||||
|
'not-an-image',
|
||||||
|
'image/png',
|
||||||
|
'https://example.com/thumb.png')
|
||||||
|
allow(Preview::ThumbnailFetcher).to receive(:fetch_image_response)
|
||||||
|
.with('https://example.com/thumb.png')
|
||||||
|
.and_return(response)
|
||||||
|
allow(described_class).to receive(:resized_thumbnail_attachment)
|
||||||
|
.and_raise(MiniMagick::Error, 'convert failed')
|
||||||
|
|
||||||
|
expect {
|
||||||
|
existing.attach_thumbnail_from_url!('https://example.com/thumb.png')
|
||||||
|
}.to raise_error(Post::RemoteThumbnailFetchFailed, 'convert failed')
|
||||||
|
|
||||||
|
existing.reload
|
||||||
|
expect(existing.thumbnail).to be_attached
|
||||||
|
expect(existing.thumbnail.blob.id).to eq(blob_id)
|
||||||
|
end
|
||||||
|
end
|
||||||
|
end
|
||||||
|
|
||||||
|
describe 'original created datetime validation' do
|
||||||
|
it 'allows unrelated updates on persisted posts with second-bearing datetimes' do
|
||||||
|
post = described_class.create!(title: 'title', url: 'https://example.com/post')
|
||||||
|
post.update_columns(
|
||||||
|
original_created_from: Time.zone.parse('2024-01-01T12:34:30Z'),
|
||||||
|
original_created_before: Time.zone.parse('2024-01-01T12:35:30Z')
|
||||||
|
)
|
||||||
|
|
||||||
|
post.title = 'updated title'
|
||||||
|
|
||||||
|
expect(post).to be_valid
|
||||||
|
expect { post.save! }.not_to raise_error
|
||||||
|
end
|
||||||
|
|
||||||
|
it 'rejects second-bearing updates when the datetime field changes' do
|
||||||
|
post = described_class.create!(title: 'title', url: 'https://example.com/post')
|
||||||
|
|
||||||
|
post.original_created_from = '2024-01-01T12:34:30Z'
|
||||||
|
|
||||||
|
expect(post).to be_invalid
|
||||||
|
expect(post.errors[:original_created_from]).to eq(
|
||||||
|
[described_class::ORIGINAL_CREATED_MINUTE_PRECISION_MESSAGE]
|
||||||
|
)
|
||||||
|
end
|
||||||
|
|
||||||
|
it 'accepts fixing persisted datetimes to minute precision' do
|
||||||
|
post = described_class.create!(title: 'title', url: 'https://example.com/post')
|
||||||
|
post.update_columns(
|
||||||
|
original_created_from: Time.zone.parse('2024-01-01T12:34:30Z'),
|
||||||
|
original_created_before: Time.zone.parse('2024-01-01T12:35:30Z')
|
||||||
|
)
|
||||||
|
|
||||||
|
post.original_created_from = '2024-01-01T12:34Z'
|
||||||
|
post.original_created_before = '2024-01-01T12:35Z'
|
||||||
|
|
||||||
|
expect(post).to be_valid
|
||||||
|
expect { post.save! }.not_to raise_error
|
||||||
|
end
|
||||||
|
|
||||||
|
it 'adds only the minute-precision error for second precision values' do
|
||||||
|
post = described_class.new(
|
||||||
|
title: 'title',
|
||||||
|
url: 'https://example.com/post',
|
||||||
|
original_created_from: '2024-01-01T12:34:30',
|
||||||
|
original_created_before: '2024-01-01T12:35'
|
||||||
|
)
|
||||||
|
|
||||||
|
expect(post).to be_invalid
|
||||||
|
expect(post.errors[:original_created_from]).to eq(
|
||||||
|
[described_class::ORIGINAL_CREATED_MINUTE_PRECISION_MESSAGE]
|
||||||
|
)
|
||||||
|
expect(post.errors[:original_created_at]).to be_empty
|
||||||
|
end
|
||||||
|
|
||||||
|
it 'adds only the minute-precision error for fractional-second values' do
|
||||||
|
post = described_class.new(
|
||||||
|
title: 'title',
|
||||||
|
url: 'https://example.com/post',
|
||||||
|
original_created_from: '2024-01-01T12:34:00.123',
|
||||||
|
original_created_before: '2024-01-01T12:35'
|
||||||
|
)
|
||||||
|
|
||||||
|
expect(post).to be_invalid
|
||||||
|
expect(post.errors[:original_created_from]).to eq(
|
||||||
|
[described_class::ORIGINAL_CREATED_MINUTE_PRECISION_MESSAGE]
|
||||||
|
)
|
||||||
|
expect(post.errors[:original_created_at]).to be_empty
|
||||||
|
end
|
||||||
|
|
||||||
|
it 'checks range rules only for valid minute-precision endpoints' do
|
||||||
|
post = described_class.new(
|
||||||
|
title: 'title',
|
||||||
|
url: 'https://example.com/post',
|
||||||
|
original_created_from: '2024-01-01T12:34',
|
||||||
|
original_created_before: '2024-01-01T12:34'
|
||||||
|
)
|
||||||
|
|
||||||
|
expect(post).to be_invalid
|
||||||
|
expect(post.errors[:original_created_at]).to eq(
|
||||||
|
[described_class::ORIGINAL_CREATED_ORDER_MESSAGE]
|
||||||
|
)
|
||||||
|
end
|
||||||
|
|
||||||
|
it 'accepts a one-minute range at minute precision' do
|
||||||
|
post = described_class.new(
|
||||||
|
title: 'title',
|
||||||
|
url: 'https://example.com/post',
|
||||||
|
original_created_from: '2024-01-01T12:34',
|
||||||
|
original_created_before: '2024-01-01T12:35'
|
||||||
|
)
|
||||||
|
|
||||||
|
expect(post).to be_valid
|
||||||
|
end
|
||||||
|
|
||||||
|
it 'rejects invalid calendar dates and invalid hours' do
|
||||||
|
invalid_dates = [
|
||||||
|
'2024-02-31T12:00',
|
||||||
|
'2023-02-29T12:00',
|
||||||
|
'2024-02-29T24:00'
|
||||||
|
]
|
||||||
|
|
||||||
|
invalid_dates.each do |value|
|
||||||
|
post = described_class.new(
|
||||||
|
title: 'title',
|
||||||
|
url: 'https://example.com/post',
|
||||||
|
original_created_from: value
|
||||||
|
)
|
||||||
|
|
||||||
|
expect(post).to be_invalid
|
||||||
|
expect(post.errors[:original_created_from]).to eq(
|
||||||
|
[described_class::ORIGINAL_CREATED_INVALID_MESSAGE]
|
||||||
|
)
|
||||||
|
end
|
||||||
|
end
|
||||||
|
end
|
||||||
|
end
|
||||||
@@ -1,5 +1,73 @@
|
|||||||
|
require 'rails_helper'
|
||||||
|
|
||||||
RSpec.describe PostTag, type: :model do
|
RSpec.describe PostTag, type: :model do
|
||||||
|
describe 'uniqueness' do
|
||||||
|
it 'rejects duplicate post and tag pairs but allows either to be reused' do
|
||||||
|
post_tag = create(:post_tag)
|
||||||
|
duplicate = build(:post_tag, post: post_tag.post, tag: post_tag.tag)
|
||||||
|
|
||||||
|
expect(duplicate).not_to be_valid
|
||||||
|
expect(duplicate.errors.of_kind?(:post_id, :taken)).to be(true)
|
||||||
|
expect(build(:post_tag, post: post_tag.post, tag: create(:tag))).to be_valid
|
||||||
|
expect(build(:post_tag, post: create(:post), tag: post_tag.tag)).to be_valid
|
||||||
|
end
|
||||||
|
|
||||||
|
it 'enforces uniqueness in the database when validation is bypassed' do
|
||||||
|
post_tag = create(:post_tag)
|
||||||
|
duplicate = build(:post_tag, post: post_tag.post, tag: post_tag.tag)
|
||||||
|
|
||||||
|
expect { duplicate.save!(validate: false) }
|
||||||
|
.to raise_error(ActiveRecord::RecordNotUnique)
|
||||||
|
end
|
||||||
|
end
|
||||||
|
|
||||||
|
describe '#destroy!' do
|
||||||
|
it 'deletes only the selected pair and its sections and updates the counter' do
|
||||||
|
post_tag = create(:post_tag)
|
||||||
|
same_post = create(:post_tag, post: post_tag.post)
|
||||||
|
same_tag = create(:post_tag, tag: post_tag.tag)
|
||||||
|
sections = [post_tag, same_post, same_tag].map do |link|
|
||||||
|
create(:post_tag_section, post: link.post, tag: link.tag,
|
||||||
|
begin_ms: 1000, end_ms: 2000)
|
||||||
|
end
|
||||||
|
|
||||||
|
expect { post_tag.destroy! }.to change(described_class, :count).by(-1)
|
||||||
|
.and change(PostTagSection, :count).by(-1)
|
||||||
|
.and change { post_tag.tag.reload.post_count }.from(2).to(1)
|
||||||
|
|
||||||
|
expect(described_class.exists?(post: post_tag.post, tag: post_tag.tag)).to be(false)
|
||||||
|
expect(same_post.reload).to be_persisted
|
||||||
|
expect(same_tag.reload).to be_persisted
|
||||||
|
expect(PostTagSection.all).to contain_exactly(*sections.drop(1))
|
||||||
|
expect(post_tag.post.reload.tags).to contain_exactly(same_post.tag)
|
||||||
|
expect(post_tag.tag.reload.posts).to contain_exactly(same_tag.post)
|
||||||
|
end
|
||||||
|
|
||||||
|
it 'allows a removed tag to be added again without restoring old sections' do
|
||||||
|
post_tag = create(:post_tag)
|
||||||
|
create(:post_tag_section, post: post_tag.post, tag: post_tag.tag,
|
||||||
|
begin_ms: 1000, end_ms: 2000)
|
||||||
|
post_tag.destroy!
|
||||||
|
|
||||||
|
replacement = create(:post_tag, post: post_tag.post, tag: post_tag.tag)
|
||||||
|
|
||||||
|
expect(replacement.reload.sections).to be_empty
|
||||||
|
expect(replacement.tag.reload.post_count).to eq(1)
|
||||||
|
end
|
||||||
|
end
|
||||||
|
|
||||||
describe '#sections' do
|
describe '#sections' do
|
||||||
|
it 'loads the owning post_tag from a section using both keys' do
|
||||||
|
post_tag = create(:post_tag)
|
||||||
|
create(:post_tag, post: post_tag.post)
|
||||||
|
create(:post_tag, tag: post_tag.tag)
|
||||||
|
section = create(:post_tag_section, post: post_tag.post,
|
||||||
|
tag: post_tag.tag,
|
||||||
|
begin_ms: 1000, end_ms: 2000)
|
||||||
|
|
||||||
|
expect(section.reload.post_tag).to eq(post_tag)
|
||||||
|
end
|
||||||
|
|
||||||
it 'loads sections by post_id and tag_id' do
|
it 'loads sections by post_id and tag_id' do
|
||||||
post_tag = create(:post_tag)
|
post_tag = create(:post_tag)
|
||||||
section = create(:post_tag_section,
|
section = create(:post_tag_section,
|
||||||
@@ -12,18 +80,25 @@ RSpec.describe PostTag, type: :model do
|
|||||||
end
|
end
|
||||||
|
|
||||||
it 'does not load sections for another tag on the same post' do
|
it 'does not load sections for another tag on the same post' do
|
||||||
post = create(:post)
|
post_tag = create(:post_tag)
|
||||||
tag = create(:tag)
|
post = post_tag.post
|
||||||
other_tag = create(:tag)
|
other_tag = create(:tag)
|
||||||
|
|
||||||
post_tag = create(:post_tag, post:, tag:)
|
own_section = create(:post_tag_section,
|
||||||
|
post:,
|
||||||
|
tag: post_tag.tag,
|
||||||
|
begin_ms: 1000,
|
||||||
|
end_ms: 2000)
|
||||||
|
|
||||||
|
create(:post_tag, post:, tag: other_tag)
|
||||||
|
|
||||||
create(:post_tag_section,
|
create(:post_tag_section,
|
||||||
post:,
|
post:,
|
||||||
tag: other_tag,
|
tag: other_tag,
|
||||||
begin_ms: 1000,
|
begin_ms: 1000,
|
||||||
end_ms: 2000)
|
end_ms: 2000)
|
||||||
|
|
||||||
expect(post_tag.sections).to be_empty
|
expect(post_tag.reload.sections).to contain_exactly(own_section)
|
||||||
end
|
end
|
||||||
|
|
||||||
it 'allows open-ended sections' do
|
it 'allows open-ended sections' do
|
||||||
|
|||||||
@@ -0,0 +1,265 @@
|
|||||||
|
require 'rails_helper'
|
||||||
|
|
||||||
|
RSpec.describe PostUrlSanitisationRule, type: :model do
|
||||||
|
before do
|
||||||
|
described_class.unscoped.delete_all
|
||||||
|
end
|
||||||
|
|
||||||
|
describe 'validations' do
|
||||||
|
it 'requires a source pattern' do
|
||||||
|
rule = described_class.new(priority: 10, source_pattern: nil, replacement: '')
|
||||||
|
|
||||||
|
expect(rule).to be_invalid
|
||||||
|
expect(rule.errors.details.fetch(:source_pattern)).to eq([{ error: :blank }])
|
||||||
|
end
|
||||||
|
|
||||||
|
it 'requires a unique source pattern' do
|
||||||
|
described_class.create!(priority: 10, source_pattern: 'source', replacement: 'first')
|
||||||
|
rule = described_class.new(
|
||||||
|
priority: 20,
|
||||||
|
source_pattern: 'source',
|
||||||
|
replacement: 'second'
|
||||||
|
)
|
||||||
|
|
||||||
|
expect(rule).to be_invalid
|
||||||
|
expect(rule.errors.details.fetch(:source_pattern)).to include(
|
||||||
|
error: :taken,
|
||||||
|
value: 'source'
|
||||||
|
)
|
||||||
|
end
|
||||||
|
|
||||||
|
it 'rejects an invalid regexp' do
|
||||||
|
rule = described_class.new(priority: 10, source_pattern: '[', replacement: '')
|
||||||
|
|
||||||
|
expect(rule).to be_invalid
|
||||||
|
expect(rule.errors[:source_pattern]).to include('変な正規表現だね〜(笑)')
|
||||||
|
end
|
||||||
|
end
|
||||||
|
|
||||||
|
describe '.sanitise' do
|
||||||
|
it 'applies each active rule once in priority order' do
|
||||||
|
described_class.create!(priority: 30, source_pattern: 'c', replacement: 'd')
|
||||||
|
described_class.create!(priority: 10, source_pattern: 'a', replacement: 'aa')
|
||||||
|
described_class.create!(priority: 20, source_pattern: 'aa', replacement: 'c')
|
||||||
|
discarded = described_class.create!(
|
||||||
|
priority: 5,
|
||||||
|
source_pattern: '.',
|
||||||
|
replacement: 'discarded'
|
||||||
|
)
|
||||||
|
discarded.discard!
|
||||||
|
|
||||||
|
expect(described_class.sanitise('a')).to eq('d')
|
||||||
|
end
|
||||||
|
|
||||||
|
it 'canonicalises the initial YouTube and Nico URL forms' do
|
||||||
|
create_initial_rules
|
||||||
|
|
||||||
|
urls = {
|
||||||
|
'https://youtu.be/abc123?si=share' => youtube_url('abc123'),
|
||||||
|
'https://www.youtube.com/live/abc123?t=10' => youtube_url('abc123'),
|
||||||
|
'https://youtube.com/shorts/abc123?feature=share' => youtube_url('abc123'),
|
||||||
|
'https://m.youtube.com/embed/abc123' => youtube_url('abc123'),
|
||||||
|
'https://youtube.com/watch?feature=share&v=abc123&t=10' => youtube_url('abc123'),
|
||||||
|
'https://nico.ms/sm123?from=share#fragment' => nico_url('sm123'),
|
||||||
|
'https://www.nicovideo.jp/watch/sm123?ref=share#fragment' => nico_url('sm123')
|
||||||
|
}
|
||||||
|
|
||||||
|
urls.each do |url, canonical_url|
|
||||||
|
expect(described_class.sanitise(url)).to eq(canonical_url)
|
||||||
|
end
|
||||||
|
end
|
||||||
|
end
|
||||||
|
|
||||||
|
describe '.apply!' do
|
||||||
|
it 'locks posts in ID order and updates through temporary URLs' do
|
||||||
|
first = create(:post, url: 'https://example.com/source/1')
|
||||||
|
second = create(:post, url: 'https://example.com/source/2')
|
||||||
|
create_source_rule
|
||||||
|
sql = capture_sql { described_class.apply! }
|
||||||
|
|
||||||
|
lock_sql = sql.find { _1.match?(/SELECT .*posts.*FOR UPDATE/i) }
|
||||||
|
expect(lock_sql).to match(/ORDER BY .*posts.*id.* ASC/i)
|
||||||
|
expect(sql.grep(/post-url-sanitising\.invalid/).size).to eq(2)
|
||||||
|
expect(first.reload.url).to eq('https://example.com/canonical/1')
|
||||||
|
expect(second.reload.url).to eq('https://example.com/canonical/2')
|
||||||
|
end
|
||||||
|
|
||||||
|
it 'loads and compiles rules only once' do
|
||||||
|
create(:post, url: 'https://example.com/source/1')
|
||||||
|
create(:post, url: 'https://example.com/source/2')
|
||||||
|
create_source_rule
|
||||||
|
|
||||||
|
expect(described_class).to receive(:rules).once.and_call_original
|
||||||
|
|
||||||
|
described_class.apply!
|
||||||
|
end
|
||||||
|
|
||||||
|
it 'does not change post versions, version_no, or updated_at' do
|
||||||
|
post = create(:post, url: 'https://example.com/source/1')
|
||||||
|
post.update_columns(version_no: 7, updated_at: 1.day.ago)
|
||||||
|
original_updated_at = post.reload.updated_at
|
||||||
|
create_source_rule
|
||||||
|
|
||||||
|
expect { described_class.apply! }.not_to change(PostVersion, :count)
|
||||||
|
|
||||||
|
post.reload
|
||||||
|
expect(post.url).to eq('https://example.com/canonical/1')
|
||||||
|
expect(post.version_no).to eq(7)
|
||||||
|
expect(post.updated_at).to eq(original_updated_at)
|
||||||
|
end
|
||||||
|
|
||||||
|
invalid_urls = {
|
||||||
|
'a blank URL' => '',
|
||||||
|
'an unparseable URL' => 'https://[',
|
||||||
|
'a non-HTTP URL' => 'ftp://example.com/file',
|
||||||
|
'an HTTP URL without a host' => 'https:/path'
|
||||||
|
}
|
||||||
|
|
||||||
|
invalid_urls.each do |description, sanitised_url|
|
||||||
|
it "rolls back every update when sanitisation produces #{ description }" do
|
||||||
|
valid_post = create(:post, url: 'https://example.com/source/1')
|
||||||
|
invalid_post = create(:post, url: 'https://example.com/invalid')
|
||||||
|
create_source_rule
|
||||||
|
described_class.create!(
|
||||||
|
priority: 20,
|
||||||
|
source_pattern: '\\Ahttps://example\\.com/invalid\\z',
|
||||||
|
replacement: sanitised_url
|
||||||
|
)
|
||||||
|
|
||||||
|
expect { described_class.apply! }
|
||||||
|
.to raise_error(described_class::InvalidUrlError) { |error|
|
||||||
|
expect(error.invalid_rows).to eq([
|
||||||
|
{ post_id: invalid_post.id,
|
||||||
|
original_url: 'https://example.com/invalid',
|
||||||
|
sanitised_url: }
|
||||||
|
])
|
||||||
|
}
|
||||||
|
expect(valid_post.reload.url).to eq('https://example.com/source/1')
|
||||||
|
expect(invalid_post.reload.url).to eq('https://example.com/invalid')
|
||||||
|
end
|
||||||
|
end
|
||||||
|
|
||||||
|
it 'rolls back every update when sanitisation produces a URL longer than 768 characters' do
|
||||||
|
path = 'a' * 375
|
||||||
|
original_url = "https://example.com/#{ path }"
|
||||||
|
sanitised_url = "https://example.com/#{ path }#{ path }"
|
||||||
|
valid_post = create(:post, url: 'https://example.com/source/1')
|
||||||
|
invalid_post = create(:post, url: original_url)
|
||||||
|
create_source_rule
|
||||||
|
described_class.create!(
|
||||||
|
priority: 20,
|
||||||
|
source_pattern: '\\Ahttps://example\\.com/(a+)\\z',
|
||||||
|
replacement: 'https://example.com/\\1\\1'
|
||||||
|
)
|
||||||
|
|
||||||
|
expect { described_class.apply! }
|
||||||
|
.to raise_error(described_class::InvalidUrlError) { |error|
|
||||||
|
expect(error.invalid_rows).to eq([
|
||||||
|
{ post_id: invalid_post.id,
|
||||||
|
original_url:,
|
||||||
|
sanitised_url: }
|
||||||
|
])
|
||||||
|
}
|
||||||
|
expect(valid_post.reload.url).to eq('https://example.com/source/1')
|
||||||
|
expect(invalid_post.reload.url).to eq(original_url)
|
||||||
|
end
|
||||||
|
|
||||||
|
it 'rejects sanitised URL collisions case-insensitively without changing posts' do
|
||||||
|
source = create(:post, url: 'https://example.com/source/Foo')
|
||||||
|
canonical = create(:post, url: 'https://example.com/canonical/foo')
|
||||||
|
create_source_rule
|
||||||
|
|
||||||
|
expect { described_class.apply! }
|
||||||
|
.to raise_error(described_class::UrlConflictError) { |error|
|
||||||
|
expect(error.conflicts).to contain_exactly(
|
||||||
|
{ url: 'https://example.com/canonical/Foo',
|
||||||
|
post_id: source.id,
|
||||||
|
original_url: 'https://example.com/source/Foo' },
|
||||||
|
{ url: 'https://example.com/canonical/foo',
|
||||||
|
post_id: canonical.id,
|
||||||
|
original_url: 'https://example.com/canonical/foo' }
|
||||||
|
)
|
||||||
|
}
|
||||||
|
expect(source.reload.url).to eq('https://example.com/source/Foo')
|
||||||
|
expect(canonical.reload.url).to eq('https://example.com/canonical/foo')
|
||||||
|
expect(Post.count).to eq(2)
|
||||||
|
end
|
||||||
|
|
||||||
|
it 'converts a persisted URL constraint race into UrlConflictError' do
|
||||||
|
post = create(:post, url: 'https://example.com/source/1')
|
||||||
|
create_source_rule
|
||||||
|
conflict = { url: 'https://example.com/canonical/1',
|
||||||
|
post_id: post.id,
|
||||||
|
original_url: post.url }
|
||||||
|
database_error = ActiveRecord::RecordNotUnique.new('duplicate URL')
|
||||||
|
allow_any_instance_of(ActiveRecord::Relation)
|
||||||
|
.to receive(:update_all).and_raise(database_error)
|
||||||
|
allow(described_class).to receive(:build_persisted_conflicts).and_return([conflict])
|
||||||
|
|
||||||
|
expect { described_class.apply! }
|
||||||
|
.to raise_error(described_class::UrlConflictError) { |error|
|
||||||
|
expect(error.conflicts).to eq([conflict])
|
||||||
|
expect(error.cause).to equal(database_error)
|
||||||
|
}
|
||||||
|
expect(post.reload.url).to eq('https://example.com/source/1')
|
||||||
|
end
|
||||||
|
|
||||||
|
it 're-raises an unidentified RecordNotUnique error' do
|
||||||
|
post = create(:post, url: 'https://example.com/source/1')
|
||||||
|
create_source_rule
|
||||||
|
database_error = ActiveRecord::RecordNotUnique.new('another unique constraint')
|
||||||
|
allow_any_instance_of(ActiveRecord::Relation)
|
||||||
|
.to receive(:update_all).and_raise(database_error)
|
||||||
|
allow(described_class).to receive(:build_persisted_conflicts).and_return([])
|
||||||
|
|
||||||
|
expect { described_class.apply! }.to raise_error(database_error)
|
||||||
|
expect(post.reload.url).to eq('https://example.com/source/1')
|
||||||
|
end
|
||||||
|
end
|
||||||
|
|
||||||
|
def capture_sql
|
||||||
|
statements = []
|
||||||
|
subscriber = lambda do |_name, _start, _finish, _id, payload|
|
||||||
|
binds = payload.fetch(:binds, []).map { _1.value_for_database.to_s }
|
||||||
|
statements << ([payload.fetch(:sql)] + binds).join(' ')
|
||||||
|
end
|
||||||
|
ActiveSupport::Notifications.subscribed(subscriber, 'sql.active_record') { yield }
|
||||||
|
statements
|
||||||
|
end
|
||||||
|
|
||||||
|
def create_source_rule
|
||||||
|
described_class.create!(
|
||||||
|
priority: 10,
|
||||||
|
source_pattern: '\\Ahttps://example\\.com/source/(.+)\\z',
|
||||||
|
replacement: 'https://example.com/canonical/\\1'
|
||||||
|
)
|
||||||
|
end
|
||||||
|
|
||||||
|
def create_initial_rules
|
||||||
|
rules = [
|
||||||
|
['\\Ahttps?://youtu\\.be/([^/?#]+)(?:[?#].*)?\\z', youtube_url('\\1')],
|
||||||
|
['\\Ahttps?://(?:www\\.|m\\.)?youtube\\.com/live/([^/?#]+)(?:[?#].*)?\\z',
|
||||||
|
youtube_url('\\1')],
|
||||||
|
['\\Ahttps?://(?:www\\.|m\\.)?youtube\\.com/shorts/([^/?#]+)(?:[?#].*)?\\z',
|
||||||
|
youtube_url('\\1')],
|
||||||
|
['\\Ahttps?://(?:www\\.|m\\.)?youtube\\.com/embed/([^/?#]+)(?:[?#].*)?\\z',
|
||||||
|
youtube_url('\\1')],
|
||||||
|
['\\Ahttps?://(?:www\\.|m\\.)?youtube\\.com/watch\\?(?:[^#&]+&)*' \
|
||||||
|
'v=([^&#]+)(?:[&#].*)?\\z', youtube_url('\\1')],
|
||||||
|
['\\Ahttps?://nico\\.ms/([^/?#]+)(?:[?#].*)?\\z', nico_url('\\1')],
|
||||||
|
['\\Ahttps?://(?:www\\.)?nicovideo\\.jp/watch/([^?#/]+)(?:[?#].*)?\\z',
|
||||||
|
nico_url('\\1')]
|
||||||
|
]
|
||||||
|
rules.each_with_index do |(source_pattern, replacement), index|
|
||||||
|
described_class.create!(
|
||||||
|
priority: (index + 1) * 10,
|
||||||
|
source_pattern:,
|
||||||
|
replacement:
|
||||||
|
)
|
||||||
|
end
|
||||||
|
end
|
||||||
|
|
||||||
|
def youtube_url(video_id) = "https://www.youtube.com/watch?v=#{ video_id }"
|
||||||
|
|
||||||
|
def nico_url(video_id) = "https://www.nicovideo.jp/watch/#{ video_id }"
|
||||||
|
end
|
||||||
@@ -19,6 +19,7 @@ RSpec.describe PostVersion, type: :model do
|
|||||||
url: post_record.url,
|
url: post_record.url,
|
||||||
thumbnail_base: post_record.thumbnail_base,
|
thumbnail_base: post_record.thumbnail_base,
|
||||||
tags: post_record.snapshot_tag_names.join(' '),
|
tags: post_record.snapshot_tag_names.join(' '),
|
||||||
|
tags_json: post_record.snapshot_tags_json,
|
||||||
parent_post_ids: post_record.snapshot_parent_post_ids.join(' '),
|
parent_post_ids: post_record.snapshot_parent_post_ids.join(' '),
|
||||||
original_created_from: post_record.original_created_from,
|
original_created_from: post_record.original_created_from,
|
||||||
original_created_before: post_record.original_created_before,
|
original_created_before: post_record.original_created_before,
|
||||||
|
|||||||
@@ -0,0 +1,34 @@
|
|||||||
|
require 'rails_helper'
|
||||||
|
|
||||||
|
RSpec.describe Setting, type: :model do
|
||||||
|
it 'accepts the default typed user settings' do
|
||||||
|
setting = described_class.new({ user: create(:user) }.merge(described_class.defaults))
|
||||||
|
|
||||||
|
expect(setting).to be_valid
|
||||||
|
end
|
||||||
|
|
||||||
|
it 'requires one settings row per user' do
|
||||||
|
user = create(:user)
|
||||||
|
described_class.create!({ user: }.merge(described_class.defaults))
|
||||||
|
|
||||||
|
duplicate = described_class.new({ user: }.merge(described_class.defaults))
|
||||||
|
|
||||||
|
expect(duplicate).not_to be_valid
|
||||||
|
expect(duplicate.errors[:user_id]).to be_present
|
||||||
|
end
|
||||||
|
|
||||||
|
it 'validates enum-like settings columns' do
|
||||||
|
setting = described_class.new(
|
||||||
|
{ user: create(:user),
|
||||||
|
theme: 'neon',
|
||||||
|
auto_fetch_title: 'sometimes',
|
||||||
|
auto_fetch_thumbnail: 'sometimes',
|
||||||
|
wiki_editor_mode: 'sideways' })
|
||||||
|
|
||||||
|
expect(setting).not_to be_valid
|
||||||
|
expect(setting.errors[:theme]).to be_present
|
||||||
|
expect(setting.errors[:auto_fetch_title]).to be_present
|
||||||
|
expect(setting.errors[:auto_fetch_thumbnail]).to be_present
|
||||||
|
expect(setting.errors[:wiki_editor_mode]).to be_present
|
||||||
|
end
|
||||||
|
end
|
||||||
@@ -57,7 +57,7 @@ RSpec.describe TagNameSanitisationRule, type: :model do
|
|||||||
|
|
||||||
it 'deletes the source tag_name' do
|
it 'deletes the source tag_name' do
|
||||||
described_class.apply!
|
described_class.apply!
|
||||||
expect(TagName.exists?(source.id)).to be(false)
|
expect(TagName.unscoped.exists?(source.id)).to be(false)
|
||||||
expect(existing.reload.name).to eq('foobar')
|
expect(existing.reload.name).to eq('foobar')
|
||||||
end
|
end
|
||||||
end
|
end
|
||||||
@@ -75,7 +75,27 @@ RSpec.describe TagNameSanitisationRule, type: :model do
|
|||||||
described_class.apply!
|
described_class.apply!
|
||||||
expected_tag_name_id = existing.canonical_id || existing.id
|
expected_tag_name_id = existing.canonical_id || existing.id
|
||||||
expect(source_tag.reload.tag_name_id).to eq(expected_tag_name_id)
|
expect(source_tag.reload.tag_name_id).to eq(expected_tag_name_id)
|
||||||
expect(TagName.exists?(source_tag_name_id)).to be(false)
|
expect(TagName.unscoped.exists?(source_tag_name_id)).to be(false)
|
||||||
|
end
|
||||||
|
end
|
||||||
|
|
||||||
|
context 'when the sanitised name is an alias of an existing tag' do
|
||||||
|
let!(:existing_tag) { create(:tag) }
|
||||||
|
let!(:alias_name) do
|
||||||
|
TagName.create!(name: 'foobar', canonical: existing_tag.tag_name)
|
||||||
|
end
|
||||||
|
let!(:source) do
|
||||||
|
TagName.create!(name: 'tmp').tap do |tn|
|
||||||
|
tn.update_columns(name: 'foo_bar', updated_at: Time.current)
|
||||||
|
end
|
||||||
|
end
|
||||||
|
|
||||||
|
it 'deletes only the source and preserves the alias and its canonical tag' do
|
||||||
|
described_class.apply!
|
||||||
|
|
||||||
|
expect(TagName.unscoped.exists?(source.id)).to be(false)
|
||||||
|
expect(alias_name.reload.canonical).to eq(existing_tag.tag_name)
|
||||||
|
expect(Tag.find(existing_tag.id)).to eq(existing_tag)
|
||||||
end
|
end
|
||||||
end
|
end
|
||||||
|
|
||||||
@@ -92,13 +112,15 @@ RSpec.describe TagNameSanitisationRule, type: :model do
|
|||||||
end
|
end
|
||||||
|
|
||||||
it 'merges the source tag into the existing tag and deletes the source tag_name' do
|
it 'merges the source tag into the existing tag and deletes the source tag_name' do
|
||||||
expect(TagName.find_by(name: 'foobar')&.tag&.id).to eq(existing_tag.id)
|
post = create(:post)
|
||||||
expect(TagName.find_by(name: 'foo_bar')&.tag&.id).to eq(source_tag.id)
|
PostTag.create!(post:, tag: source_tag)
|
||||||
|
|
||||||
described_class.apply!
|
described_class.apply!
|
||||||
|
|
||||||
expect(Tag.exists?(source_tag.id)).to be(false)
|
expect(Tag.unscoped.exists?(source_tag.id)).to be(false)
|
||||||
expect(TagName.exists?(source_tag.tag_name_id)).to be(false)
|
expect(TagName.unscoped.exists?(source_tag_name_id)).to be(false)
|
||||||
|
expect(post.reload.tags).to contain_exactly(existing_tag)
|
||||||
|
expect(existing_tag.reload.name).to eq('foobar')
|
||||||
end
|
end
|
||||||
end
|
end
|
||||||
end
|
end
|
||||||
|
|||||||
+172
-67
@@ -1,6 +1,29 @@
|
|||||||
require 'rails_helper'
|
require 'rails_helper'
|
||||||
|
|
||||||
RSpec.describe Tag, type: :model do
|
RSpec.describe Tag, type: :model do
|
||||||
|
describe 'external tag separation' do
|
||||||
|
['nico:reserved', 'NiCo:reserved'].each do |name|
|
||||||
|
it "rejects the reserved prefix #{ name } for internal tags" do
|
||||||
|
tag = build(:tag, name:)
|
||||||
|
|
||||||
|
expect(tag).to be_invalid
|
||||||
|
expect(tag.errors[:name]).to be_present
|
||||||
|
expect {
|
||||||
|
described_class.normalise_tags!([name])
|
||||||
|
}.to raise_error(Tag::NicoTagNormalisationError)
|
||||||
|
end
|
||||||
|
end
|
||||||
|
|
||||||
|
it 'finds external links by the internal tag id even when ids differ' do
|
||||||
|
tag = create(:tag)
|
||||||
|
external = create(:external_tag, id: tag.id + 10_000)
|
||||||
|
# The migration retains existing links without running model validation.
|
||||||
|
NicoTagRelation.insert_all!([{ tag_id: tag.id, nico_tag_id: external.id }])
|
||||||
|
|
||||||
|
expect(tag.linked_nico_tags).to contain_exactly(external)
|
||||||
|
end
|
||||||
|
end
|
||||||
|
|
||||||
describe '.normalise_tags!' do
|
describe '.normalise_tags!' do
|
||||||
it 'rejects deprecated tags when deny_deprecated is enabled' do
|
it 'rejects deprecated tags when deny_deprecated is enabled' do
|
||||||
tag_name = TagName.create!(name: 'normalise deprecated tag')
|
tag_name = TagName.create!(name: 'normalise deprecated tag')
|
||||||
@@ -159,17 +182,44 @@ RSpec.describe Tag, type: :model do
|
|||||||
end
|
end
|
||||||
end
|
end
|
||||||
|
|
||||||
describe 'deprecated validation' do
|
describe '.find_or_create_by_tag_name!' do
|
||||||
it 'rejects deprecated nico tags' do
|
it 'creates a tag and name with the requested category after stripping whitespace' do
|
||||||
tag = build(
|
tag = nil
|
||||||
:tag,
|
|
||||||
name: 'nico:deprecated_validation',
|
|
||||||
category: :nico,
|
|
||||||
deprecated_at: Time.current
|
|
||||||
)
|
|
||||||
|
|
||||||
expect(tag).not_to be_valid
|
expect {
|
||||||
expect(tag.errors[:deprecated_at]).to include('ニコタグは廃止できません.')
|
tag = described_class.find_or_create_by_tag_name!(
|
||||||
|
' lookup_new ', category: :character)
|
||||||
|
}.to change(Tag, :count).by(1).and change(TagName, :count).by(1)
|
||||||
|
|
||||||
|
expect(tag.name).to eq('lookup_new')
|
||||||
|
expect(tag.category).to eq('character')
|
||||||
|
end
|
||||||
|
|
||||||
|
it 'reuses the canonical tag for an alias without changing its category' do
|
||||||
|
tag = create(:tag, category: :character)
|
||||||
|
alias_name = TagName.create!(name: 'lookup_alias', canonical: tag.tag_name)
|
||||||
|
found = nil
|
||||||
|
|
||||||
|
expect {
|
||||||
|
found = described_class.find_or_create_by_tag_name!(
|
||||||
|
alias_name.name, category: :general)
|
||||||
|
}.to change(Tag, :count).by(0).and change(TagName, :count).by(0)
|
||||||
|
|
||||||
|
expect(found).to eq(tag)
|
||||||
|
expect(found.category).to eq('character')
|
||||||
|
end
|
||||||
|
|
||||||
|
it 'creates a tag for an existing canonical name reached through an alias' do
|
||||||
|
canonical = create(:tag_name)
|
||||||
|
alias_name = TagName.create!(name: 'lookup_alias', canonical:)
|
||||||
|
tag = nil
|
||||||
|
|
||||||
|
expect {
|
||||||
|
tag = described_class.find_or_create_by_tag_name!(
|
||||||
|
alias_name.name, category: :general)
|
||||||
|
}.to change(Tag, :count).by(1).and change(TagName, :count).by(0)
|
||||||
|
|
||||||
|
expect(tag.tag_name).to eq(canonical)
|
||||||
end
|
end
|
||||||
end
|
end
|
||||||
|
|
||||||
@@ -185,18 +235,14 @@ RSpec.describe Tag, type: :model do
|
|||||||
context 'when merging a simple source tag' do
|
context 'when merging a simple source tag' do
|
||||||
let!(:source_post_tag) { PostTag.create!(post: post_record, tag: source_tag) }
|
let!(:source_post_tag) { PostTag.create!(post: post_record, tag: source_tag) }
|
||||||
|
|
||||||
it 'discards the source post_tag, creates an active target post_tag, discards the source tag, and aliases the source tag_name' do
|
it 'deletes the source tag, moves its post link, and keeps its name as an alias' do
|
||||||
described_class.merge_tags!(target_tag, [source_tag])
|
described_class.merge_tags!(target_tag, [source_tag])
|
||||||
|
|
||||||
source_pt = PostTag.with_discarded.find(source_post_tag.id)
|
target_link = PostTag.find_by(post: post_record, tag: target_tag)
|
||||||
active_target = PostTag.kept.find_by(post_id: post_record.id, tag_id: target_tag.id)
|
|
||||||
|
|
||||||
expect(source_pt.discarded_at).to be_present
|
expect(PostTag.exists?(post: post_record, tag: source_tag)).to be(false)
|
||||||
expect(source_pt.tag_id).to eq(source_tag.id)
|
expect(target_link).to be_present
|
||||||
expect(active_target).to be_present
|
expect(Tag.unscoped.exists?(source_tag.id)).to be(false)
|
||||||
|
|
||||||
expect(Tag.with_discarded.find(source_tag.id)).to be_discarded
|
|
||||||
expect(TagName.with_discarded.find(source_tag_name.id)).not_to be_discarded
|
|
||||||
expect(source_tag_name.reload.canonical_id).to eq(target_tag.tag_name_id)
|
expect(source_tag_name.reload.canonical_id).to eq(target_tag.tag_name_id)
|
||||||
expect(target_tag.reload.post_count).to eq(1)
|
expect(target_tag.reload.post_count).to eq(1)
|
||||||
end
|
end
|
||||||
@@ -206,38 +252,101 @@ RSpec.describe Tag, type: :model do
|
|||||||
let!(:target_post_tag) { PostTag.create!(post: post_record, tag: target_tag) }
|
let!(:target_post_tag) { PostTag.create!(post: post_record, tag: target_tag) }
|
||||||
let!(:source_post_tag) { PostTag.create!(post: post_record, tag: source_tag) }
|
let!(:source_post_tag) { PostTag.create!(post: post_record, tag: source_tag) }
|
||||||
|
|
||||||
it 'discards the source post_tag, keeps one active target post_tag, discards the source tag, and aliases the source tag_name' do
|
it 'deletes the source link and preserves the existing target link' do
|
||||||
|
create(:post_tag_section, post: post_record, tag: source_tag,
|
||||||
|
begin_ms: 1000, end_ms: 2000)
|
||||||
|
target_section = create(:post_tag_section, post: post_record,
|
||||||
|
tag: target_tag,
|
||||||
|
begin_ms: 3000, end_ms: nil)
|
||||||
|
|
||||||
described_class.merge_tags!(target_tag, [source_tag])
|
described_class.merge_tags!(target_tag, [source_tag])
|
||||||
|
|
||||||
source_pt = PostTag.with_discarded.find(source_post_tag.id)
|
target_links = PostTag.where(post: post_record, tag: target_tag)
|
||||||
active = PostTag.kept.where(post_id: post_record.id, tag_id: target_tag.id)
|
|
||||||
|
|
||||||
expect(source_pt.discarded_at).to be_present
|
expect(PostTag.exists?(post: post_record, tag: source_tag)).to be(false)
|
||||||
expect(source_pt.tag_id).to eq(source_tag.id)
|
expect(target_links).to contain_exactly(target_post_tag)
|
||||||
expect(active.count).to eq(1)
|
expect(PostTagSection.where(post: post_record, tag: source_tag)).to be_empty
|
||||||
expect(active.first.id).to eq(target_post_tag.id)
|
expect(target_post_tag.reload.sections).to contain_exactly(target_section)
|
||||||
|
|
||||||
expect(Tag.with_discarded.find(source_tag.id)).to be_discarded
|
expect(Tag.unscoped.exists?(source_tag.id)).to be(false)
|
||||||
expect(TagName.with_discarded.find(source_tag_name.id)).not_to be_discarded
|
|
||||||
expect(source_tag_name.reload.canonical_id).to eq(target_tag.tag_name_id)
|
expect(source_tag_name.reload.canonical_id).to eq(target_tag.tag_name_id)
|
||||||
expect(target_tag.reload.post_count).to eq(1)
|
expect(target_tag.reload.post_count).to eq(1)
|
||||||
end
|
end
|
||||||
end
|
end
|
||||||
|
|
||||||
|
it 'keeps source history and records the new target alias after deleting the source' do
|
||||||
|
user = create_member_user!
|
||||||
|
source_name = source_tag.name
|
||||||
|
source_alias = TagName.create!(name: 'merge_alias', canonical: source_tag_name)
|
||||||
|
TagVersioning.ensure_snapshot!(source_tag, created_by_user: user)
|
||||||
|
original_version = source_tag.tag_versions.first
|
||||||
|
|
||||||
|
described_class.merge_tags!(target_tag, [source_tag], created_by_user: user)
|
||||||
|
|
||||||
|
versions = TagVersion.where(tag_id: source_tag.id).order(:version_no)
|
||||||
|
expect(versions.pluck(:version_no, :event_type))
|
||||||
|
.to eq([[1, 'create'], [2, 'discard']])
|
||||||
|
expect(versions.first).to eq(original_version)
|
||||||
|
expect(versions.last).to have_attributes(
|
||||||
|
name: source_name, aliases: source_alias.name, created_by_user: user)
|
||||||
|
expect(Tag.unscoped.exists?(source_tag.id)).to be(false)
|
||||||
|
|
||||||
|
target_versions = target_tag.tag_versions.order(:version_no)
|
||||||
|
expect(target_versions.pluck(:event_type)).to eq(['create', 'update'])
|
||||||
|
expect(target_versions.last.aliases.split).to eq([source_name])
|
||||||
|
end
|
||||||
|
|
||||||
|
it 'deletes source relationships while preserving unrelated relationships' do
|
||||||
|
parent = create(:tag)
|
||||||
|
child = create(:tag)
|
||||||
|
nico_tag = create(:external_tag)
|
||||||
|
TagImplication.create!(tag: source_tag, parent_tag: parent)
|
||||||
|
TagImplication.create!(tag: child, parent_tag: source_tag)
|
||||||
|
kept_implication = TagImplication.create!(tag: target_tag, parent_tag: parent)
|
||||||
|
NicoTagRelation.create!(tag: source_tag, nico_tag:)
|
||||||
|
kept_relation = NicoTagRelation.create!(tag: target_tag, nico_tag:)
|
||||||
|
TagSimilarity.create!(tag: source_tag, target_tag:, cos: 0.5)
|
||||||
|
TagSimilarity.create!(tag: target_tag, target_tag: source_tag, cos: 0.5)
|
||||||
|
kept_similarity = TagSimilarity.create!(tag: target_tag, target_tag: parent,
|
||||||
|
cos: 0.5)
|
||||||
|
|
||||||
|
described_class.merge_tags!(target_tag, [source_tag])
|
||||||
|
|
||||||
|
expect(TagImplication.all).to contain_exactly(kept_implication)
|
||||||
|
expect(NicoTagRelation.all).to contain_exactly(kept_relation)
|
||||||
|
expect(TagSimilarity.all).to contain_exactly(kept_similarity)
|
||||||
|
expect(TagVersion.where(tag_id: source_tag.id).order(:version_no).last.parent_tag_ids)
|
||||||
|
.to eq(parent.id.to_s)
|
||||||
|
end
|
||||||
|
|
||||||
|
it 'preserves material history referencing the deleted source tag' do
|
||||||
|
source_tag.update!(category: :material)
|
||||||
|
target_tag.update!(category: :material)
|
||||||
|
material = Material.create!(tag: source_tag, url: 'https://example.com/material')
|
||||||
|
version = MaterialVersionRecorder.record!(
|
||||||
|
material:, event_type: :create, created_by_user: nil)
|
||||||
|
material.update!(tag: target_tag)
|
||||||
|
|
||||||
|
described_class.merge_tags!(target_tag, [source_tag])
|
||||||
|
|
||||||
|
expect(version.reload).to have_attributes(
|
||||||
|
tag_id: source_tag.id, tag_name: source_tag_name.name, tag_category: 'material')
|
||||||
|
expect(Tag.unscoped.exists?(source_tag.id)).to be(false)
|
||||||
|
expect(material.reload.tag).to eq(target_tag)
|
||||||
|
end
|
||||||
|
|
||||||
context 'when source_tags includes the target itself' do
|
context 'when source_tags includes the target itself' do
|
||||||
let!(:source_post_tag) { PostTag.create!(post: post_record, tag: source_tag) }
|
let!(:source_post_tag) { PostTag.create!(post: post_record, tag: source_tag) }
|
||||||
|
|
||||||
it 'ignores the target in source_tags while still merging the source tag' do
|
it 'ignores the target in source_tags while still merging the source tag' do
|
||||||
described_class.merge_tags!(target_tag, [source_tag, target_tag])
|
described_class.merge_tags!(target_tag, [source_tag, target_tag])
|
||||||
|
|
||||||
source_pt = PostTag.with_discarded.find(source_post_tag.id)
|
target_link = PostTag.find_by(post: post_record, tag: target_tag)
|
||||||
active_target = PostTag.kept.find_by(post_id: post_record.id, tag_id: target_tag.id)
|
|
||||||
|
|
||||||
expect(Tag.find(target_tag.id)).to be_present
|
expect(Tag.find(target_tag.id)).to be_present
|
||||||
expect(Tag.with_discarded.find(source_tag.id)).to be_discarded
|
expect(Tag.unscoped.exists?(source_tag.id)).to be(false)
|
||||||
expect(source_pt.discarded_at).to be_present
|
expect(PostTag.exists?(post: post_record, tag: source_tag)).to be(false)
|
||||||
expect(source_pt.tag_id).to eq(source_tag.id)
|
expect(target_link).to be_present
|
||||||
expect(active_target).to be_present
|
|
||||||
expect(source_tag_name.reload.canonical_id).to eq(target_tag.tag_name_id)
|
expect(source_tag_name.reload.canonical_id).to eq(target_tag.tag_name_id)
|
||||||
expect(target_tag.reload.post_count).to eq(1)
|
expect(target_tag.reload.post_count).to eq(1)
|
||||||
end
|
end
|
||||||
@@ -260,18 +369,16 @@ RSpec.describe Tag, type: :model do
|
|||||||
)
|
)
|
||||||
end
|
end
|
||||||
|
|
||||||
it 'still merges, but discards the source tag_name instead of aliasing it' do
|
it 'still merges and keeps the source name as an alias without validating it' do
|
||||||
described_class.merge_tags!(target_tag, [source_tag])
|
described_class.merge_tags!(target_tag, [source_tag])
|
||||||
|
|
||||||
source_pt = PostTag.with_discarded.find(source_post_tag.id)
|
target_link = PostTag.find_by(post: post_record, tag: target_tag)
|
||||||
active_target = PostTag.kept.find_by(post_id: post_record.id, tag_id: target_tag.id)
|
|
||||||
discarded_source_tag_name = TagName.with_discarded.find(source_tag_name.id)
|
|
||||||
|
|
||||||
expect(source_pt.discarded_at).to be_present
|
expect(PostTag.exists?(post: post_record, tag: source_tag)).to be(false)
|
||||||
expect(source_pt.tag_id).to eq(source_tag.id)
|
expect(target_link).to be_present
|
||||||
expect(active_target).to be_present
|
|
||||||
|
|
||||||
expect(Tag.with_discarded.find(source_tag.id)).to be_discarded
|
expect(Tag.unscoped.exists?(source_tag.id)).to be(false)
|
||||||
|
expect(source_tag_name.reload.canonical_id).to eq(target_tag.tag_name_id)
|
||||||
expect(target_tag.reload.post_count).to eq(1)
|
expect(target_tag.reload.post_count).to eq(1)
|
||||||
end
|
end
|
||||||
end
|
end
|
||||||
@@ -288,38 +395,32 @@ RSpec.describe Tag, type: :model do
|
|||||||
message: 'init')
|
message: 'init')
|
||||||
end
|
end
|
||||||
|
|
||||||
it 'rolls back the transaction' do
|
it 'rolls back earlier deletions, links, and history when a later source has a wiki' do
|
||||||
|
earlier_source = create(:tag)
|
||||||
|
earlier_name = earlier_source.tag_name
|
||||||
|
source_section = create(:post_tag_section, post: post_record,
|
||||||
|
tag: source_tag,
|
||||||
|
begin_ms: 1000, end_ms: 2000)
|
||||||
|
|
||||||
expect {
|
expect {
|
||||||
described_class.merge_tags!(target_tag, [source_tag])
|
described_class.merge_tags!(target_tag, [earlier_source, source_tag])
|
||||||
}.to raise_error(ActiveRecord::RecordInvalid)
|
}.to raise_error(ActiveRecord::RecordInvalid)
|
||||||
|
|
||||||
expect(Tag.with_discarded.find(source_tag.id)).not_to be_discarded
|
expect(Tag.unscoped.exists?(earlier_source.id)).to be(true)
|
||||||
expect(TagName.with_discarded.find(source_tag_name.id)).not_to be_discarded
|
expect(earlier_name.reload.canonical_id).to be_nil
|
||||||
expect(PostTag.kept.find(source_post_tag.id).tag_id).to eq(source_tag.id)
|
expect(TagVersion.where(tag_id: [earlier_source.id, source_tag.id, target_tag.id]))
|
||||||
expect(PostTag.kept.find_by(post_id: post_record.id, tag_id: target_tag.id)).to be_nil
|
.to be_empty
|
||||||
|
expect(Tag.unscoped.exists?(source_tag.id)).to be(true)
|
||||||
|
expect(TagName.unscoped.exists?(source_tag_name.id)).to be(true)
|
||||||
|
expect(source_post_tag.reload.tag_id).to eq(source_tag.id)
|
||||||
|
expect(source_post_tag.sections).to contain_exactly(source_section)
|
||||||
|
expect(PostTag.find_by(post: post_record, tag: target_tag)).to be_nil
|
||||||
|
expect(source_tag.reload.post_count).to eq(1)
|
||||||
expect(source_tag_name.reload.canonical_id).to be_nil
|
expect(source_tag_name.reload.canonical_id).to be_nil
|
||||||
expect(target_tag.reload.post_count).to eq(0)
|
expect(target_tag.reload.post_count).to eq(0)
|
||||||
end
|
end
|
||||||
end
|
end
|
||||||
|
|
||||||
context 'when merging a nico source tag' do
|
|
||||||
let!(:target_tag) { create(:tag, category: :nico, name: 'nico:foo') }
|
|
||||||
let!(:source_tag) { create(:tag, category: :nico, name: 'nico:bar') }
|
|
||||||
let!(:source_tag_name_id) { source_tag.tag_name_id }
|
|
||||||
|
|
||||||
it 'discards the source tag_name instead of aliasing it' do
|
|
||||||
described_class.merge_tags!(target_tag, [source_tag])
|
|
||||||
|
|
||||||
discarded_source_tag = Tag.with_discarded.find(source_tag.id)
|
|
||||||
discarded_source_tag_name = TagName.with_discarded.find(source_tag_name_id)
|
|
||||||
|
|
||||||
expect(discarded_source_tag).to be_discarded
|
|
||||||
expect(discarded_source_tag_name).to be_discarded
|
|
||||||
expect(discarded_source_tag_name.canonical_id).to be_nil
|
|
||||||
expect(target_tag.reload.post_count).to eq(0)
|
|
||||||
end
|
|
||||||
end
|
|
||||||
|
|
||||||
def snapshot_tags(post)
|
def snapshot_tags(post)
|
||||||
post.snapshot_tag_names.join(' ')
|
post.snapshot_tag_names.join(' ')
|
||||||
end
|
end
|
||||||
@@ -333,6 +434,7 @@ RSpec.describe Tag, type: :model do
|
|||||||
url: post.url,
|
url: post.url,
|
||||||
thumbnail_base: post.thumbnail_base,
|
thumbnail_base: post.thumbnail_base,
|
||||||
tags: snapshot_tags(post),
|
tags: snapshot_tags(post),
|
||||||
|
tags_json: post.snapshot_tags_json,
|
||||||
parent_post_ids: post.snapshot_parent_post_ids.join(' '),
|
parent_post_ids: post.snapshot_parent_post_ids.join(' '),
|
||||||
original_created_from: post.original_created_from,
|
original_created_from: post.original_created_from,
|
||||||
original_created_before: post.original_created_before,
|
original_created_before: post.original_created_before,
|
||||||
@@ -364,12 +466,15 @@ RSpec.describe Tag, type: :model do
|
|||||||
expect(latest.event_type).to eq('update')
|
expect(latest.event_type).to eq('update')
|
||||||
expect(latest.created_by_user).to be_nil
|
expect(latest.created_by_user).to be_nil
|
||||||
expect(latest.tags).to eq(snapshot_tags(post_record.reload))
|
expect(latest.tags).to eq(snapshot_tags(post_record.reload))
|
||||||
|
expect(latest.tags_json.map { |item| item.fetch('tag_id') }).to eq([target_tag.id])
|
||||||
|
expect(affected_versions.first.tags_json.map { |item| item.fetch('tag_id') })
|
||||||
|
.to eq([source_tag.id])
|
||||||
|
|
||||||
expect(unaffected_post.reload.post_versions.count).to eq(1)
|
expect(unaffected_post.reload.post_versions.count).to eq(1)
|
||||||
end
|
end
|
||||||
end
|
end
|
||||||
|
|
||||||
context 'when the source tag has no active post_tags' do
|
context 'when the source tag has no post_tags' do
|
||||||
let!(:another_post) do
|
let!(:another_post) do
|
||||||
Post.create!(url: 'https://example.com/posts/3', title: 'another post')
|
Post.create!(url: 'https://example.com/posts/3', title: 'another post')
|
||||||
end
|
end
|
||||||
|
|||||||
@@ -0,0 +1,40 @@
|
|||||||
|
require 'rails_helper'
|
||||||
|
|
||||||
|
RSpec.describe UserThemeSlot, type: :model do
|
||||||
|
it 'accepts a theme slot for one user, base theme, and slot number' do
|
||||||
|
slot = described_class.new(user: create(:user),
|
||||||
|
base_theme: 'light',
|
||||||
|
slot_no: 1,
|
||||||
|
tokens: { 'background' => '0 0% 100%' })
|
||||||
|
|
||||||
|
expect(slot).to be_valid
|
||||||
|
end
|
||||||
|
|
||||||
|
it 'requires unique slot numbers per user and base theme' do
|
||||||
|
user = create(:user)
|
||||||
|
described_class.create!(user:,
|
||||||
|
base_theme: 'dark',
|
||||||
|
slot_no: 2,
|
||||||
|
tokens: { 'background' => '222.2 84% 4.9%' })
|
||||||
|
|
||||||
|
duplicate = described_class.new(user:,
|
||||||
|
base_theme: 'dark',
|
||||||
|
slot_no: 2,
|
||||||
|
tokens: { 'background' => '0 0% 100%' })
|
||||||
|
|
||||||
|
expect(duplicate).not_to be_valid
|
||||||
|
expect(duplicate.errors[:user_id]).to be_present
|
||||||
|
end
|
||||||
|
|
||||||
|
it 'validates base theme, slot number, and token object shape' do
|
||||||
|
slot = described_class.new(user: create(:user),
|
||||||
|
base_theme: 'system',
|
||||||
|
slot_no: 4,
|
||||||
|
tokens: 'not-object')
|
||||||
|
|
||||||
|
expect(slot).not_to be_valid
|
||||||
|
expect(slot.errors[:base_theme]).to be_present
|
||||||
|
expect(slot.errors[:slot_no]).to be_present
|
||||||
|
expect(slot.errors[:tokens]).to be_present
|
||||||
|
end
|
||||||
|
end
|
||||||
@@ -2,7 +2,7 @@ require 'rails_helper'
|
|||||||
|
|
||||||
RSpec.describe VersionRecord, type: :model do
|
RSpec.describe VersionRecord, type: :model do
|
||||||
let!(:tag) { create(:tag, name: 'version_record_tag') }
|
let!(:tag) { create(:tag, name: 'version_record_tag') }
|
||||||
let!(:nico_tag) { create(:tag, :nico, name: 'nico:version_record_tag') }
|
let!(:nico_tag) { create(:external_tag, name: 'version_record_tag') }
|
||||||
|
|
||||||
it 'makes TagVersion read only after create' do
|
it 'makes TagVersion read only after create' do
|
||||||
version = TagVersion.create!(
|
version = TagVersion.create!(
|
||||||
@@ -42,10 +42,10 @@ RSpec.describe VersionRecord, type: :model do
|
|||||||
|
|
||||||
it 'makes NicoTagVersion read only after create' do
|
it 'makes NicoTagVersion read only after create' do
|
||||||
version = NicoTagVersion.create!(
|
version = NicoTagVersion.create!(
|
||||||
tag: nico_tag,
|
external_tag: nico_tag,
|
||||||
version_no: 1,
|
version_no: 1,
|
||||||
event_type: 'create',
|
event_type: 'create',
|
||||||
name: nico_tag.name,
|
name: "nico:#{ nico_tag.name }",
|
||||||
linked_tags: '',
|
linked_tags: '',
|
||||||
created_at: Time.current,
|
created_at: Time.current,
|
||||||
created_by_user: nil
|
created_by_user: nil
|
||||||
@@ -58,10 +58,10 @@ RSpec.describe VersionRecord, type: :model do
|
|||||||
|
|
||||||
it 'prevents NicoTagVersion destroy' do
|
it 'prevents NicoTagVersion destroy' do
|
||||||
version = NicoTagVersion.create!(
|
version = NicoTagVersion.create!(
|
||||||
tag: nico_tag,
|
external_tag: nico_tag,
|
||||||
version_no: 1,
|
version_no: 1,
|
||||||
event_type: 'create',
|
event_type: 'create',
|
||||||
name: nico_tag.name,
|
name: "nico:#{ nico_tag.name }",
|
||||||
linked_tags: '',
|
linked_tags: '',
|
||||||
created_at: Time.current,
|
created_at: Time.current,
|
||||||
created_by_user: nil
|
created_by_user: nil
|
||||||
|
|||||||
@@ -287,6 +287,25 @@ RSpec.describe 'Materials API', type: :request do
|
|||||||
expect(json.dig('export_paths', 'legacy_drive')).to eq('伊地知ニジカ/created.png')
|
expect(json.dig('export_paths', 'legacy_drive')).to eq('伊地知ニジカ/created.png')
|
||||||
end
|
end
|
||||||
|
|
||||||
|
it 'creates a create tag_version for a newly created material tag' do
|
||||||
|
expect do
|
||||||
|
post '/materials', params: {
|
||||||
|
tag: 'material_create_versioned_tag',
|
||||||
|
file: dummy_upload(filename: 'created.png')
|
||||||
|
}
|
||||||
|
end.to change(TagVersion, :count).by(1)
|
||||||
|
|
||||||
|
expect(response).to have_http_status(:created)
|
||||||
|
|
||||||
|
tag = Tag.joins(:tag_name).find_by!(tag_names: { name: 'material_create_versioned_tag' })
|
||||||
|
version = tag.tag_versions.order(:version_no).last
|
||||||
|
|
||||||
|
expect(version.event_type).to eq('create')
|
||||||
|
expect(version.name).to eq('material_create_versioned_tag')
|
||||||
|
expect(version.category).to eq('material')
|
||||||
|
expect(version.created_by_user).to eq(member_user)
|
||||||
|
end
|
||||||
|
|
||||||
it 'snapshots attached file metadata and sha256' do
|
it 'snapshots attached file metadata and sha256' do
|
||||||
post '/materials', params: {
|
post '/materials', params: {
|
||||||
tag: 'material_create_file_version',
|
tag: 'material_create_file_version',
|
||||||
@@ -466,6 +485,73 @@ RSpec.describe 'Materials API', type: :request do
|
|||||||
expect(json.dig('tag', 'name')).to eq('material_update_new')
|
expect(json.dig('tag', 'name')).to eq('material_update_new')
|
||||||
end
|
end
|
||||||
|
|
||||||
|
it 'creates a create tag_version when update creates a new tag' do
|
||||||
|
expect do
|
||||||
|
put "/materials/#{ material.id }", params: {
|
||||||
|
tag: 'material_update_versioned_tag',
|
||||||
|
file: dummy_upload(filename: 'updated.png')
|
||||||
|
}
|
||||||
|
end.to change(Tag, :count).by(1)
|
||||||
|
.and change(TagName, :count).by(1)
|
||||||
|
.and change(TagVersion, :count).by(1)
|
||||||
|
|
||||||
|
expect(response).to have_http_status(:ok)
|
||||||
|
|
||||||
|
tag = Tag.joins(:tag_name).find_by!(tag_names: { name: 'material_update_versioned_tag' })
|
||||||
|
version = tag.tag_versions.order(:version_no).last
|
||||||
|
|
||||||
|
expect(version.event_type).to eq('create')
|
||||||
|
expect(version.name).to eq('material_update_versioned_tag')
|
||||||
|
expect(version.category).to eq('material')
|
||||||
|
expect(version.created_by_user).to eq(member_user)
|
||||||
|
end
|
||||||
|
|
||||||
|
it 'backfills a create tag_version for an existing material tag without history' do
|
||||||
|
existing_tag =
|
||||||
|
Tag.create!(tag_name: TagName.create!(name: 'material_update_existing_no_history'),
|
||||||
|
category: :material)
|
||||||
|
|
||||||
|
expect(existing_tag.tag_versions).to be_empty
|
||||||
|
|
||||||
|
expect do
|
||||||
|
put "/materials/#{ material.id }", params: {
|
||||||
|
tag: 'material_update_existing_no_history',
|
||||||
|
file: dummy_upload(filename: 'updated.png')
|
||||||
|
}
|
||||||
|
end.to change(TagVersion, :count).by(1)
|
||||||
|
|
||||||
|
expect(response).to have_http_status(:ok)
|
||||||
|
|
||||||
|
version = existing_tag.reload.tag_versions.order(:version_no).last
|
||||||
|
expect(version.event_type).to eq('create')
|
||||||
|
expect(version.name).to eq('material_update_existing_no_history')
|
||||||
|
expect(version.category).to eq('material')
|
||||||
|
expect(version.created_by_user).to eq(member_user)
|
||||||
|
end
|
||||||
|
|
||||||
|
it 'backfills a create tag_version for an existing character tag without history' do
|
||||||
|
existing_tag =
|
||||||
|
Tag.create!(tag_name: TagName.create!(name: 'material_update_character_no_history'),
|
||||||
|
category: :character)
|
||||||
|
|
||||||
|
expect(existing_tag.tag_versions).to be_empty
|
||||||
|
|
||||||
|
expect do
|
||||||
|
put "/materials/#{ material.id }", params: {
|
||||||
|
tag: 'material_update_character_no_history',
|
||||||
|
file: dummy_upload(filename: 'updated.png')
|
||||||
|
}
|
||||||
|
end.to change(TagVersion, :count).by(1)
|
||||||
|
|
||||||
|
expect(response).to have_http_status(:ok)
|
||||||
|
|
||||||
|
version = existing_tag.reload.tag_versions.order(:version_no).last
|
||||||
|
expect(version.event_type).to eq('create')
|
||||||
|
expect(version.name).to eq('material_update_character_no_history')
|
||||||
|
expect(version.category).to eq('character')
|
||||||
|
expect(version.created_by_user).to eq(member_user)
|
||||||
|
end
|
||||||
|
|
||||||
it 'detaches the existing file without purging blob when url replaces file' do
|
it 'detaches the existing file without purging blob when url replaces file' do
|
||||||
old_blob_id = material.file.blob.id
|
old_blob_id = material.file.blob.id
|
||||||
|
|
||||||
@@ -494,6 +580,7 @@ RSpec.describe 'Materials API', type: :request do
|
|||||||
it 'does not increase version for the same snapshot update' do
|
it 'does not increase version for the same snapshot update' do
|
||||||
MaterialVersionRecorder.record!(material:, event_type: :create,
|
MaterialVersionRecorder.record!(material:, event_type: :create,
|
||||||
created_by_user: member_user)
|
created_by_user: member_user)
|
||||||
|
TagVersioning.ensure_snapshot!(tag, created_by_user: member_user)
|
||||||
|
|
||||||
expect do
|
expect do
|
||||||
put "/materials/#{ material.id }", params: {
|
put "/materials/#{ material.id }", params: {
|
||||||
@@ -501,6 +588,8 @@ RSpec.describe 'Materials API', type: :request do
|
|||||||
}
|
}
|
||||||
end.not_to change(MaterialVersion, :count)
|
end.not_to change(MaterialVersion, :count)
|
||||||
|
|
||||||
|
expect(tag.reload.tag_versions.count).to eq(1)
|
||||||
|
|
||||||
expect(response).to have_http_status(:ok)
|
expect(response).to have_http_status(:ok)
|
||||||
expect(material.reload.version_no).to eq(1)
|
expect(material.reload.version_no).to eq(1)
|
||||||
end
|
end
|
||||||
|
|||||||
@@ -3,10 +3,34 @@ require 'rails_helper'
|
|||||||
|
|
||||||
RSpec.describe 'NicoTags', type: :request do
|
RSpec.describe 'NicoTags', type: :request do
|
||||||
describe 'GET /tags/nico' do
|
describe 'GET /tags/nico' do
|
||||||
it 'returns paginated tags and total count' do
|
it 'returns the legacy Tag-compatible external fields' do
|
||||||
create_list(:tag, 3, :nico)
|
external = create(:external_tag, name: 'legacy_external', post_count: 3)
|
||||||
|
|
||||||
get '/tags/nico', params: { page: 2, limit: 2 }
|
get '/tags/nico', params: { name: 'legacy_external' }
|
||||||
|
|
||||||
|
expect(response).to have_http_status(:ok)
|
||||||
|
expect(json.fetch('count')).to eq(1)
|
||||||
|
expect(json.fetch('tags')).to contain_exactly(
|
||||||
|
a_hash_including(
|
||||||
|
'id' => external.id,
|
||||||
|
'name' => 'nico:legacy_external',
|
||||||
|
'category' => 'nico',
|
||||||
|
'post_count' => 3,
|
||||||
|
'created_at' => external.created_at.as_json,
|
||||||
|
'updated_at' => external.created_at.as_json,
|
||||||
|
'deprecated_at' => nil,
|
||||||
|
'aliases' => [],
|
||||||
|
'parents' => [],
|
||||||
|
'has_wiki' => false,
|
||||||
|
'material_id' => nil,
|
||||||
|
'has_deerjikists' => false,
|
||||||
|
'linked_tags' => []))
|
||||||
|
end
|
||||||
|
|
||||||
|
it 'returns paginated tags and total count' do
|
||||||
|
3.times { |i| create(:external_tag, name: "pagination_#{ i }") }
|
||||||
|
|
||||||
|
get '/tags/nico', params: { page: 2, limit: 2, name: 'pagination_' }
|
||||||
|
|
||||||
expect(response).to have_http_status(:ok)
|
expect(response).to have_http_status(:ok)
|
||||||
expect(json['tags'].size).to eq(1)
|
expect(json['tags'].size).to eq(1)
|
||||||
@@ -14,12 +38,12 @@ RSpec.describe 'NicoTags', type: :request do
|
|||||||
end
|
end
|
||||||
|
|
||||||
it 'filters by nico tag name, linked tag name, and link status' do
|
it 'filters by nico tag name, linked tag name, and link status' do
|
||||||
linked = create(:tag, :nico)
|
linked = create(:external_tag)
|
||||||
linked.tag_name.update!(name: 'nico:search_linked')
|
linked.update!(name: 'search_linked')
|
||||||
unlinked = create(:tag, :nico)
|
unlinked = create(:external_tag)
|
||||||
unlinked.tag_name.update!(name: 'nico:search_unlinked')
|
unlinked.update!(name: 'search_unlinked')
|
||||||
other = create(:tag, :nico)
|
other = create(:external_tag)
|
||||||
other.tag_name.update!(name: 'nico:other')
|
other.update!(name: 'other')
|
||||||
destination = create(:tag, :general)
|
destination = create(:tag, :general)
|
||||||
destination.tag_name.update!(name: 'destination_search')
|
destination.tag_name.update!(name: 'destination_search')
|
||||||
NicoTagRelation.create!(nico_tag: linked, tag: destination)
|
NicoTagRelation.create!(nico_tag: linked, tag: destination)
|
||||||
@@ -41,27 +65,41 @@ RSpec.describe 'NicoTags', type: :request do
|
|||||||
expect(json.fetch('tags').map { |tag| tag['id'] }).to eq([unlinked.id])
|
expect(json.fetch('tags').map { |tag| tag['id'] }).to eq([unlinked.id])
|
||||||
end
|
end
|
||||||
|
|
||||||
|
it 'filters by the qualified legacy name as well as the raw name' do
|
||||||
|
external = create(:external_tag, name: 'qualified_filter')
|
||||||
|
create(:external_tag, name: 'unrelated_filter')
|
||||||
|
|
||||||
|
['qualified_filter', 'nico:qualified_filter'].each do |name|
|
||||||
|
get '/tags/nico', params: { name: }
|
||||||
|
|
||||||
|
expect(response).to have_http_status(:ok)
|
||||||
|
expect(json.fetch('count')).to eq(1)
|
||||||
|
expect(json.fetch('tags')).to contain_exactly(
|
||||||
|
a_hash_including('id' => external.id, 'name' => 'nico:qualified_filter'))
|
||||||
|
end
|
||||||
|
end
|
||||||
|
|
||||||
it 'sorts by name and timestamps' do
|
it 'sorts by name and timestamps' do
|
||||||
older = create(:tag, :nico)
|
older = create(:external_tag)
|
||||||
older.tag_name.update!(name: 'nico:a')
|
older.update!(name: 'ordered_a')
|
||||||
older.update_columns(created_at: 2.days.ago)
|
older.update_columns(created_at: 2.days.ago)
|
||||||
newer = create(:tag, :nico)
|
newer = create(:external_tag)
|
||||||
newer.tag_name.update!(name: 'nico:b')
|
newer.update!(name: 'ordered_b')
|
||||||
newer.update_columns(created_at: 1.day.ago)
|
newer.update_columns(created_at: 1.day.ago)
|
||||||
older_post_tag =
|
older_post_tag =
|
||||||
PostTag.create!(post: Post.create!(url: 'https://example.com/nico-older'), tag: older)
|
PostExternalTag.create!(post: create(:post), external_tag: older)
|
||||||
older_post_tag.update_columns(created_at: 1.hour.ago)
|
older_post_tag.update_columns(created_at: 1.hour.ago)
|
||||||
newer_post_tag =
|
newer_post_tag =
|
||||||
PostTag.create!(post: Post.create!(url: 'https://example.com/nico-newer'), tag: newer)
|
PostExternalTag.create!(post: create(:post), external_tag: newer)
|
||||||
newer_post_tag.update_columns(created_at: 2.hours.ago)
|
newer_post_tag.update_columns(created_at: 2.hours.ago)
|
||||||
|
|
||||||
get '/tags/nico', params: { order: 'name:desc' }
|
get '/tags/nico', params: { order: 'name:desc', name: 'ordered_' }
|
||||||
expect(json.fetch('tags').map { |tag| tag['id'] }).to eq([newer.id, older.id])
|
expect(json.fetch('tags').map { |tag| tag['id'] }).to eq([newer.id, older.id])
|
||||||
|
|
||||||
get '/tags/nico', params: { order: 'created_at:asc' }
|
get '/tags/nico', params: { order: 'created_at:asc', name: 'ordered_' }
|
||||||
expect(json.fetch('tags').map { |tag| tag['id'] }).to eq([older.id, newer.id])
|
expect(json.fetch('tags').map { |tag| tag['id'] }).to eq([older.id, newer.id])
|
||||||
|
|
||||||
get '/tags/nico', params: { order: 'updated_at:desc' }
|
get '/tags/nico', params: { order: 'updated_at:desc', name: 'ordered_' }
|
||||||
expect(json.fetch('tags').map { |tag| tag['id'] }).to eq([older.id, newer.id])
|
expect(json.fetch('tags').map { |tag| tag['id'] }).to eq([older.id, newer.id])
|
||||||
expect(Time.zone.parse(json.fetch('tags').first.fetch('recent_post_tag_created_at')))
|
expect(Time.zone.parse(json.fetch('tags').first.fetch('recent_post_tag_created_at')))
|
||||||
.to be_within(1.second).of(older_post_tag.created_at)
|
.to be_within(1.second).of(older_post_tag.created_at)
|
||||||
@@ -71,7 +109,7 @@ RSpec.describe 'NicoTags', type: :request do
|
|||||||
describe 'PATCH /tags/nico/:id' do
|
describe 'PATCH /tags/nico/:id' do
|
||||||
let(:member) { create(:user, :member) }
|
let(:member) { create(:user, :member) }
|
||||||
let(:admin) { create(:user, :admin) }
|
let(:admin) { create(:user, :admin) }
|
||||||
let(:nico_tag) { create(:tag, :nico) }
|
let(:nico_tag) { create(:external_tag) }
|
||||||
|
|
||||||
it '401 when not logged in' do
|
it '401 when not logged in' do
|
||||||
sign_out
|
sign_out
|
||||||
@@ -85,18 +123,18 @@ RSpec.describe 'NicoTags', type: :request do
|
|||||||
expect(response).to have_http_status(:forbidden)
|
expect(response).to have_http_status(:forbidden)
|
||||||
end
|
end
|
||||||
|
|
||||||
it '400 when target is not nico category' do
|
it '404 when only an internal tag exists for the target id' do
|
||||||
sign_in_as(member)
|
sign_in_as(member)
|
||||||
non_nico = create(:tag, :general)
|
non_nico = create(:tag, :general)
|
||||||
|
expect(ExternalTag.exists?(non_nico.id)).to be(false)
|
||||||
patch "/tags/nico/#{non_nico.id}", params: { tags: 'a b' }
|
patch "/tags/nico/#{non_nico.id}", params: { tags: 'a b' }
|
||||||
expect(response).to have_http_status(:bad_request)
|
expect(response).to have_http_status(:not_found)
|
||||||
end
|
end
|
||||||
|
|
||||||
it '200 and updates linked tags while recording tag versions' do
|
it '200 and updates linked tags while recording tag versions' do
|
||||||
sign_in_as(admin)
|
sign_in_as(admin)
|
||||||
|
|
||||||
nico_tag_name = TagName.create!(name: 'nico:nico_tags_spec_source')
|
nico_tag = create(:external_tag, name: 'nico_tags_spec_source')
|
||||||
nico_tag = Tag.create!(tag_name: nico_tag_name, category: :nico)
|
|
||||||
|
|
||||||
linked_a_name = TagName.create!(name: 'nico_linked_a')
|
linked_a_name = TagName.create!(name: 'nico_linked_a')
|
||||||
linked_a = Tag.create!(tag_name: linked_a_name, category: :general)
|
linked_a = Tag.create!(tag_name: linked_a_name, category: :general)
|
||||||
@@ -104,7 +142,8 @@ RSpec.describe 'NicoTags', type: :request do
|
|||||||
linked_b_name = TagName.create!(name: 'nico_linked_b')
|
linked_b_name = TagName.create!(name: 'nico_linked_b')
|
||||||
linked_b = Tag.create!(tag_name: linked_b_name, category: :general)
|
linked_b = Tag.create!(tag_name: linked_b_name, category: :general)
|
||||||
|
|
||||||
TagVersioning.ensure_snapshot!(nico_tag, created_by_user: admin)
|
NicoTagVersionRecorder.record!(external_tag: nico_tag,
|
||||||
|
event_type: :create, created_by_user: admin)
|
||||||
|
|
||||||
expect {
|
expect {
|
||||||
patch "/tags/nico/#{nico_tag.id}", params: {
|
patch "/tags/nico/#{nico_tag.id}", params: {
|
||||||
@@ -126,26 +165,26 @@ RSpec.describe 'NicoTags', type: :request do
|
|||||||
expect(versions.map(&:event_type)).to eq(['create', 'update'])
|
expect(versions.map(&:event_type)).to eq(['create', 'update'])
|
||||||
expect(versions.last.linked_tags.split).to match_array([
|
expect(versions.last.linked_tags.split).to match_array([
|
||||||
'nico_linked_a',
|
'nico_linked_a',
|
||||||
'nico_linked_b'
|
'nico_linked_b'])
|
||||||
])
|
|
||||||
expect(versions.last.created_by_user_id).to eq(admin.id)
|
expect(versions.last.created_by_user_id).to eq(admin.id)
|
||||||
end
|
end
|
||||||
|
|
||||||
it 'returns 422 when linked tag normalises to nico tag' do
|
it 'clears existing links and records the empty mapping for a member' do
|
||||||
sign_in_as(member)
|
sign_in_as(member)
|
||||||
|
linked = create(:tag)
|
||||||
other_nico = create(:tag, :nico, name: 'nico:linked_ng')
|
NicoTagRelation.insert_all!([{ nico_tag_id: nico_tag.id, tag_id: linked.id }])
|
||||||
TagName.create!(name: 'linked_ng_alias', canonical: other_nico.tag_name)
|
NicoTagVersionRecorder.record!(external_tag: nico_tag,
|
||||||
|
event_type: :create, created_by_user: member)
|
||||||
TagVersioning.ensure_snapshot!(nico_tag, created_by_user: member)
|
|
||||||
|
|
||||||
expect {
|
expect {
|
||||||
patch "/tags/nico/#{nico_tag.id}", params: { tags: 'linked_ng_alias' }
|
patch "/tags/nico/#{ nico_tag.id }", params: { tags: '' }
|
||||||
}.not_to change(NicoTagVersion, :count)
|
}.to change(NicoTagVersion, :count).by(1)
|
||||||
|
|
||||||
expect(response).to have_http_status(:unprocessable_entity)
|
expect(response).to have_http_status(:ok)
|
||||||
expect(json.fetch('errors')).to include(
|
expect(json).to eq([])
|
||||||
'tags' => ['ニコニコ・タグ同士は連携できません.'])
|
expect(nico_tag.reload.linked_tags).to be_empty
|
||||||
|
expect(nico_tag.nico_tag_versions.order(:version_no).last)
|
||||||
|
.to have_attributes(linked_tags: '', created_by_user: member)
|
||||||
end
|
end
|
||||||
|
|
||||||
it 'returns the tags field error when a nico tag is specified directly' do
|
it 'returns the tags field error when a nico tag is specified directly' do
|
||||||
|
|||||||
+651
-152
ファイル差分が大きすぎるため省略します
差分を読み込み
@@ -1,28 +1,46 @@
|
|||||||
require "rails_helper"
|
require 'rails_helper'
|
||||||
|
|
||||||
|
|
||||||
RSpec.describe "Preview", type: :request do
|
RSpec.describe 'Preview', type: :request do
|
||||||
describe "GET /preview/title" do
|
describe 'GET /preview/title' do
|
||||||
it "401 unless logged in" do
|
it '401 unless logged in' do
|
||||||
sign_out
|
sign_out
|
||||||
get "/preview/title", params: { url: "example.com" }
|
get '/preview/title', params: { url: 'example.com' }
|
||||||
expect(response).to have_http_status(:unauthorized)
|
expect(response).to have_http_status(:unauthorized)
|
||||||
end
|
end
|
||||||
|
|
||||||
it "400 when url blank" do
|
it '403 when logged in as guest' do
|
||||||
sign_in_as(create(:user))
|
sign_in_as(create(:user, :guest))
|
||||||
get "/preview/title", params: { url: "" }
|
get '/preview/title', params: { url: 'example.com' }
|
||||||
|
expect(response).to have_http_status(:forbidden)
|
||||||
|
end
|
||||||
|
|
||||||
|
it '400 when url blank' do
|
||||||
|
sign_in_as(create(:user, :member))
|
||||||
|
get '/preview/title', params: { url: '' }
|
||||||
expect(response).to have_http_status(:bad_request)
|
expect(response).to have_http_status(:bad_request)
|
||||||
end
|
end
|
||||||
|
|
||||||
it "returns parsed title (stubbing URI.open)" do
|
it 'returns parsed title' do
|
||||||
sign_in_as(create(:user))
|
sign_in_as(create(:user, :member))
|
||||||
fake_html = "<html><head><title> Hello </title></head></html>"
|
allow(Preview::ThumbnailFetcher)
|
||||||
allow(URI).to receive(:open).and_return(StringIO.new(fake_html))
|
.to receive(:title)
|
||||||
|
.with('example.com')
|
||||||
|
.and_return('Hello')
|
||||||
|
|
||||||
get "/preview/title", params: { url: "example.com" }
|
get '/preview/title', params: { url: 'example.com' }
|
||||||
expect(response).to have_http_status(:ok)
|
expect(response).to have_http_status(:ok)
|
||||||
expect(json["title"]).to eq("Hello")
|
expect(json['title']).to eq('Hello')
|
||||||
|
end
|
||||||
|
|
||||||
|
it '413 when fetched response is too large' do
|
||||||
|
sign_in_as(create(:user, :member))
|
||||||
|
allow(Preview::ThumbnailFetcher)
|
||||||
|
.to receive(:title)
|
||||||
|
.and_raise(Preview::HttpFetcher::ResponseTooLarge, '外部データが大きすぎます.')
|
||||||
|
|
||||||
|
get '/preview/title', params: { url: 'example.com' }
|
||||||
|
expect(response).to have_http_status(:payload_too_large)
|
||||||
end
|
end
|
||||||
end
|
end
|
||||||
end
|
end
|
||||||
|
|||||||
@@ -80,38 +80,6 @@ RSpec.describe "TagChildren", type: :request do
|
|||||||
expect(response).to have_http_status(:not_found)
|
expect(response).to have_http_status(:not_found)
|
||||||
end
|
end
|
||||||
end
|
end
|
||||||
|
|
||||||
context 'when parent is nico' do
|
|
||||||
before { stub_current_user(admin) }
|
|
||||||
|
|
||||||
let!(:parent) { create(:tag, :nico, name: 'nico:parent_ng') }
|
|
||||||
let(:parent_id) { parent.id }
|
|
||||||
let(:child_id) { child.id }
|
|
||||||
|
|
||||||
it 'returns 400 and does not create relation' do
|
|
||||||
expect {
|
|
||||||
do_request
|
|
||||||
}.not_to change(TagImplication, :count)
|
|
||||||
|
|
||||||
expect(response).to have_http_status(:bad_request)
|
|
||||||
end
|
|
||||||
end
|
|
||||||
|
|
||||||
context 'when child is nico' do
|
|
||||||
before { stub_current_user(admin) }
|
|
||||||
|
|
||||||
let!(:child) { create(:tag, :nico, name: 'nico:child_ng') }
|
|
||||||
let(:parent_id) { parent.id }
|
|
||||||
let(:child_id) { child.id }
|
|
||||||
|
|
||||||
it 'returns 400 and does not create relation' do
|
|
||||||
expect {
|
|
||||||
do_request
|
|
||||||
}.not_to change(TagImplication, :count)
|
|
||||||
|
|
||||||
expect(response).to have_http_status(:bad_request)
|
|
||||||
end
|
|
||||||
end
|
|
||||||
end
|
end
|
||||||
|
|
||||||
describe "DELETE /tag_children" do
|
describe "DELETE /tag_children" do
|
||||||
@@ -186,31 +154,5 @@ RSpec.describe "TagChildren", type: :request do
|
|||||||
expect(response).to have_http_status(:not_found)
|
expect(response).to have_http_status(:not_found)
|
||||||
end
|
end
|
||||||
end
|
end
|
||||||
|
|
||||||
context 'when parent is nico' do
|
|
||||||
before { stub_current_user(admin) }
|
|
||||||
|
|
||||||
let!(:parent) { create(:tag, :nico, name: 'nico:parent_ng_delete') }
|
|
||||||
let(:parent_id) { parent.id }
|
|
||||||
let(:child_id) { child.id }
|
|
||||||
|
|
||||||
it 'returns 400' do
|
|
||||||
do_request
|
|
||||||
expect(response).to have_http_status(:bad_request)
|
|
||||||
end
|
|
||||||
end
|
|
||||||
|
|
||||||
context 'when child is nico' do
|
|
||||||
before { stub_current_user(admin) }
|
|
||||||
|
|
||||||
let!(:child) { create(:tag, :nico, name: 'nico:child_ng_delete') }
|
|
||||||
let(:parent_id) { parent.id }
|
|
||||||
let(:child_id) { child.id }
|
|
||||||
|
|
||||||
it 'returns 400' do
|
|
||||||
do_request
|
|
||||||
expect(response).to have_http_status(:bad_request)
|
|
||||||
end
|
|
||||||
end
|
|
||||||
end
|
end
|
||||||
end
|
end
|
||||||
|
|||||||
@@ -226,6 +226,14 @@ RSpec.describe 'Tags deerjikists API', type: :request do
|
|||||||
[platform2, code2],
|
[platform2, code2],
|
||||||
)
|
)
|
||||||
end
|
end
|
||||||
|
|
||||||
|
it 'locks the tag before replacing the complete list' do
|
||||||
|
expect_any_instance_of(Tag).to receive(:lock!).and_call_original
|
||||||
|
|
||||||
|
do_request
|
||||||
|
|
||||||
|
expect(response).to have_http_status(:ok)
|
||||||
|
end
|
||||||
end
|
end
|
||||||
|
|
||||||
context 'when tag already has deerjikists' do
|
context 'when tag already has deerjikists' do
|
||||||
@@ -299,6 +307,81 @@ RSpec.describe 'Tags deerjikists API', type: :request do
|
|||||||
end
|
end
|
||||||
end
|
end
|
||||||
|
|
||||||
|
context 'when platform is outside the enum' do
|
||||||
|
let(:payload) do
|
||||||
|
[
|
||||||
|
{ platform: 'invalid', code: code1 },
|
||||||
|
]
|
||||||
|
end
|
||||||
|
|
||||||
|
it 'returns 422 with an indexed platform error without changing the list' do
|
||||||
|
Deerjikist.create!(platform: platform1, code: code1, tag: tag)
|
||||||
|
|
||||||
|
expect {
|
||||||
|
do_request
|
||||||
|
}.not_to change { Deerjikist.where(tag: tag).map { |d| [d.platform, d.code] } }
|
||||||
|
|
||||||
|
expect(response).to have_http_status(:unprocessable_entity)
|
||||||
|
expect(json.fetch('errors')).to include(
|
||||||
|
'deerjikists.0.platform' => [be_present],
|
||||||
|
)
|
||||||
|
end
|
||||||
|
end
|
||||||
|
|
||||||
|
context 'when a requested deerjikist belongs to another tag' do
|
||||||
|
let!(:other_tag) { create(:tag, category: :deerjikist) }
|
||||||
|
let!(:owned_deerjikist) do
|
||||||
|
Deerjikist.create!(platform: platform1, code: code1, tag: tag)
|
||||||
|
end
|
||||||
|
let!(:conflicting_deerjikist) do
|
||||||
|
Deerjikist.create!(platform: platform2, code: code2, tag: other_tag)
|
||||||
|
end
|
||||||
|
let(:payload) do
|
||||||
|
[
|
||||||
|
{ platform: 'nico', code: 'new-code' },
|
||||||
|
{ platform: platform2, code: code2 },
|
||||||
|
]
|
||||||
|
end
|
||||||
|
|
||||||
|
before do
|
||||||
|
other_tag.tag_name.update!(name: 'existing-deerjikist')
|
||||||
|
end
|
||||||
|
|
||||||
|
it 'returns an indexed 422 error and rolls back the complete replacement' do
|
||||||
|
expect {
|
||||||
|
do_request
|
||||||
|
}.not_to change { Deerjikist.order(:platform, :code).pluck(:platform, :code, :tag_id) }
|
||||||
|
|
||||||
|
expect(response).to have_http_status(:unprocessable_entity)
|
||||||
|
expect(json.fetch('errors')).to include(
|
||||||
|
'deerjikists.1.code' => [include('existing-deerjikist')],
|
||||||
|
)
|
||||||
|
expect(owned_deerjikist.reload.tag_id).to eq(tag.id)
|
||||||
|
expect(conflicting_deerjikist.reload.tag_id).to eq(other_tag.id)
|
||||||
|
end
|
||||||
|
end
|
||||||
|
|
||||||
|
context 'when a requested deerjikist already belongs to the same tag' do
|
||||||
|
let!(:existing_deerjikist) do
|
||||||
|
Deerjikist.create!(platform: platform1, code: code1, tag: tag)
|
||||||
|
end
|
||||||
|
let(:payload) do
|
||||||
|
[
|
||||||
|
{ platform: platform1, code: code1 },
|
||||||
|
]
|
||||||
|
end
|
||||||
|
|
||||||
|
it 'keeps the existing row' do
|
||||||
|
expect {
|
||||||
|
do_request
|
||||||
|
}.not_to change { existing_deerjikist.reload.created_at }
|
||||||
|
|
||||||
|
expect(response).to have_http_status(:ok)
|
||||||
|
expect(Deerjikist.where(tag: tag).pluck(:platform, :code))
|
||||||
|
.to eq([[platform1, code1]])
|
||||||
|
end
|
||||||
|
end
|
||||||
|
|
||||||
context 'when youtube code is handle' do
|
context 'when youtube code is handle' do
|
||||||
let(:channel_id) { 'UCabcdefghijklmnopqrstuv' }
|
let(:channel_id) { 'UCabcdefghijklmnopqrstuv' }
|
||||||
let(:payload) do
|
let(:payload) do
|
||||||
|
|||||||
@@ -30,6 +30,30 @@ RSpec.describe 'Tags API', type: :request do
|
|||||||
end
|
end
|
||||||
|
|
||||||
describe 'GET /tags' do
|
describe 'GET /tags' do
|
||||||
|
it 'includes legacy external JSON alongside an internal tag with the same id' do
|
||||||
|
external = create(:external_tag, id: tag.id, name: 'spec_external', post_count: 3)
|
||||||
|
|
||||||
|
get '/tags', params: { name: 'spec_' }
|
||||||
|
|
||||||
|
expect(response).to have_http_status(:ok)
|
||||||
|
expect(json.fetch('count')).to eq(2)
|
||||||
|
expect(response_tags).to contain_exactly(
|
||||||
|
a_hash_including('id' => tag.id, 'name' => tag.name, 'category' => 'general'),
|
||||||
|
a_hash_including(
|
||||||
|
'id' => external.id,
|
||||||
|
'name' => 'nico:spec_external',
|
||||||
|
'category' => 'nico',
|
||||||
|
'post_count' => 3,
|
||||||
|
'created_at' => external.created_at.as_json,
|
||||||
|
'updated_at' => external.created_at.as_json,
|
||||||
|
'deprecated_at' => nil,
|
||||||
|
'aliases' => [],
|
||||||
|
'parents' => [],
|
||||||
|
'has_wiki' => false,
|
||||||
|
'material_id' => nil,
|
||||||
|
'has_deerjikists' => false))
|
||||||
|
end
|
||||||
|
|
||||||
it 'returns tags with count and metadata' do
|
it 'returns tags with count and metadata' do
|
||||||
get '/tags'
|
get '/tags'
|
||||||
|
|
||||||
@@ -164,20 +188,79 @@ RSpec.describe 'Tags API', type: :request do
|
|||||||
Tag.create!(tag_name: TagName.create!(name: 'cat_general'), category: :general)
|
Tag.create!(tag_name: TagName.create!(name: 'cat_general'), category: :general)
|
||||||
Tag.create!(tag_name: TagName.create!(name: 'cat_material'), category: :material)
|
Tag.create!(tag_name: TagName.create!(name: 'cat_material'), category: :material)
|
||||||
Tag.create!(tag_name: TagName.create!(name: 'cat_meta'), category: :meta)
|
Tag.create!(tag_name: TagName.create!(name: 'cat_meta'), category: :meta)
|
||||||
Tag.create!(tag_name: TagName.create!(name: 'nico:cat_nico'), category: :nico)
|
create(:external_tag, name: 'cat_nico')
|
||||||
|
|
||||||
get '/tags', params: { name: 'cat_', order: 'category:asc', limit: 20 }
|
get '/tags', params: { name: 'cat_', order: 'category:asc', limit: 20 }
|
||||||
|
|
||||||
expect(response).to have_http_status(:ok)
|
expect(response).to have_http_status(:ok)
|
||||||
expect(response_names).to eq(%w[
|
expect(response_names).to eq([
|
||||||
cat_deerjikist
|
'cat_deerjikist', 'cat_meme', 'cat_character',
|
||||||
cat_meme
|
'cat_general', 'cat_material', 'cat_meta', 'nico:cat_nico'])
|
||||||
cat_character
|
expect(json.fetch('count')).to eq(7)
|
||||||
cat_general
|
end
|
||||||
cat_material
|
|
||||||
cat_meta
|
context 'with mixed legacy pagination' do
|
||||||
nico:cat_nico
|
let!(:first_tag) do
|
||||||
])
|
create(:tag,
|
||||||
|
tag_name: create(:tag_name, name: 'a_mixed_page'),
|
||||||
|
category: :meme)
|
||||||
|
end
|
||||||
|
|
||||||
|
let!(:middle_tag) do
|
||||||
|
create(:tag,
|
||||||
|
tag_name: create(:tag_name, name: 'm_mixed_page'),
|
||||||
|
category: :meta)
|
||||||
|
end
|
||||||
|
|
||||||
|
let!(:last_tag) do
|
||||||
|
create(:tag,
|
||||||
|
tag_name: create(:tag_name, name: 'z_mixed_page'),
|
||||||
|
category: :general)
|
||||||
|
end
|
||||||
|
|
||||||
|
let!(:first_external) do
|
||||||
|
create(:external_tag, id: first_tag.id, name: 'a_mixed_page')
|
||||||
|
end
|
||||||
|
let!(:last_external) do
|
||||||
|
create(:external_tag, id: last_tag.id, name: 'z_mixed_page')
|
||||||
|
end
|
||||||
|
|
||||||
|
let(:name_order) do
|
||||||
|
[
|
||||||
|
[first_tag.id, 'a_mixed_page'], [middle_tag.id, 'm_mixed_page'],
|
||||||
|
[first_external.id, 'nico:a_mixed_page'], [last_external.id, 'nico:z_mixed_page'],
|
||||||
|
[last_tag.id, 'z_mixed_page']]
|
||||||
|
end
|
||||||
|
let(:category_order) do
|
||||||
|
[
|
||||||
|
[first_tag.id, 'a_mixed_page'], [last_tag.id, 'z_mixed_page'],
|
||||||
|
[middle_tag.id, 'm_mixed_page'], [first_external.id, 'nico:a_mixed_page'],
|
||||||
|
[last_external.id, 'nico:z_mixed_page']]
|
||||||
|
end
|
||||||
|
|
||||||
|
['name', 'category'].each do |order|
|
||||||
|
['asc', 'desc'].each do |direction|
|
||||||
|
it "orders the combined records by #{ order }:#{ direction } before paging" do
|
||||||
|
ascending = order == 'name' ? name_order : category_order
|
||||||
|
expected = direction == 'asc' ? ascending : ascending.reverse
|
||||||
|
|
||||||
|
[2, 3].each do |limit|
|
||||||
|
pages = expected.each_slice(limit).to_a + [[]]
|
||||||
|
pages.each.with_index(1) do |expected_page, page|
|
||||||
|
get '/tags', params: {
|
||||||
|
name: 'mixed_page', order: "#{ order }:#{ direction }", page:, limit: }
|
||||||
|
|
||||||
|
expect(response).to have_http_status(:ok)
|
||||||
|
expect(json.fetch('count')).to eq(5)
|
||||||
|
expect(response_tags.size).to eq(expected_page.size)
|
||||||
|
expect(response_tags.size).to be <= limit
|
||||||
|
expect(response_tags.map { [_1.fetch('id'), _1.fetch('name')] })
|
||||||
|
.to eq(expected_page)
|
||||||
|
end
|
||||||
|
end
|
||||||
|
end
|
||||||
|
end
|
||||||
|
end
|
||||||
end
|
end
|
||||||
|
|
||||||
it 'paginates and keeps total count' do
|
it 'paginates and keeps total count' do
|
||||||
@@ -299,6 +382,89 @@ RSpec.describe 'Tags API', type: :request do
|
|||||||
end
|
end
|
||||||
|
|
||||||
describe 'GET /tags/autocomplete' do
|
describe 'GET /tags/autocomplete' do
|
||||||
|
it 'combines internal and external matches without conflating equal ids' do
|
||||||
|
internal = Tag.create!(category: :general, name: 'mixed_internal', post_count: 2)
|
||||||
|
external = create(:external_tag, id: internal.id,
|
||||||
|
name: 'mixed_external', post_count: 3)
|
||||||
|
|
||||||
|
get '/tags/autocomplete', params: { q: 'not:mixed' }
|
||||||
|
|
||||||
|
expect(response).to have_http_status(:ok)
|
||||||
|
expect(json.map { |row| row.fetch('name') })
|
||||||
|
.to eq(['nico:mixed_external', 'mixed_internal'])
|
||||||
|
expect(json.first).to include(
|
||||||
|
'id' => external.id, 'category' => 'nico', 'post_count' => 3,
|
||||||
|
'created_at' => external.created_at.as_json,
|
||||||
|
'updated_at' => external.created_at.as_json,
|
||||||
|
'deprecated_at' => nil, 'matched_alias' => nil,
|
||||||
|
'aliases' => [], 'parents' => [], 'has_wiki' => false,
|
||||||
|
'material_id' => nil, 'has_deerjikists' => false)
|
||||||
|
end
|
||||||
|
|
||||||
|
it 'matches an external tag by its platform prefix' do
|
||||||
|
create(:external_tag, name: 'prefix_match', post_count: 1)
|
||||||
|
|
||||||
|
get '/tags/autocomplete', params: { q: 'nico:prefix' }
|
||||||
|
|
||||||
|
expect(json.map { |row| row.fetch('name') }).to eq(['nico:prefix_match'])
|
||||||
|
end
|
||||||
|
|
||||||
|
it 'excludes unused external tags unless present is false' do
|
||||||
|
create(:external_tag, name: 'unused_external')
|
||||||
|
|
||||||
|
get '/tags/autocomplete', params: { q: 'unused' }
|
||||||
|
expect(json).to be_empty
|
||||||
|
|
||||||
|
get '/tags/autocomplete', params: { q: 'unused', present: '0' }
|
||||||
|
expect(json.map { |row| row.fetch('name') }).to eq(['nico:unused_external'])
|
||||||
|
end
|
||||||
|
|
||||||
|
it 'limits the combined results to 20 and sorts ties by displayed name' do
|
||||||
|
11.times do |i|
|
||||||
|
name = "combined_#{ i.to_s.rjust(2, '0') }"
|
||||||
|
Tag.create!(category: :general, name:, post_count: 1)
|
||||||
|
create(:external_tag, name:, post_count: 1)
|
||||||
|
end
|
||||||
|
|
||||||
|
get '/tags/autocomplete', params: { q: 'combined' }
|
||||||
|
|
||||||
|
expected = 11.times.map { |i| "combined_#{ i.to_s.rjust(2, '0') }" }
|
||||||
|
expected += expected.map { |name| "nico:#{ name }" }
|
||||||
|
expect(json.map { |row| row.fetch('name') }).to eq(expected.first(20))
|
||||||
|
end
|
||||||
|
|
||||||
|
it 'excludes external tags but preserves internal alias matches when nico is false' do
|
||||||
|
internal = Tag.create!(category: :general, name: 'switch_internal', post_count: 1)
|
||||||
|
alias_target = Tag.create!(category: :general, name: 'alias_target', post_count: 1)
|
||||||
|
TagName.create!(name: 'switch_alias', canonical: alias_target.tag_name)
|
||||||
|
create(:external_tag, name: 'switch_external', post_count: 1)
|
||||||
|
|
||||||
|
get '/tags/autocomplete', params: { q: 'switch', nico: '0' }
|
||||||
|
|
||||||
|
expect(response).to have_http_status(:ok)
|
||||||
|
expect(json).to contain_exactly(
|
||||||
|
a_hash_including('id' => internal.id, 'name' => internal.name),
|
||||||
|
a_hash_including('id' => alias_target.id, 'name' => alias_target.name,
|
||||||
|
'matched_alias' => 'switch_alias'))
|
||||||
|
end
|
||||||
|
|
||||||
|
['%', '_'].each do |wildcard|
|
||||||
|
it "treats #{ wildcard } literally for canonical, alias, and external names" do
|
||||||
|
literal = "literal#{ wildcard }match"
|
||||||
|
Tag.create!(category: :general, name: literal, post_count: 1)
|
||||||
|
alias_target = Tag.create!(category: :general, name: 'literal_alias_target', post_count: 1)
|
||||||
|
TagName.create!(name: "#{ literal }_alias", canonical: alias_target.tag_name)
|
||||||
|
create(:external_tag, name: literal, post_count: 1)
|
||||||
|
Tag.create!(category: :general, name: 'literalXmatch', post_count: 2)
|
||||||
|
create(:external_tag, name: 'literalXmatch', post_count: 2)
|
||||||
|
|
||||||
|
get '/tags/autocomplete', params: { q: "literal#{ wildcard }" }
|
||||||
|
|
||||||
|
expect(json.map { |row| row.fetch('name') })
|
||||||
|
.to contain_exactly(literal, alias_target.name, "nico:#{ literal }")
|
||||||
|
end
|
||||||
|
end
|
||||||
|
|
||||||
it 'returns matching tags by q' do
|
it 'returns matching tags by q' do
|
||||||
get '/tags/autocomplete', params: { q: 'spec' }
|
get '/tags/autocomplete', params: { q: 'spec' }
|
||||||
|
|
||||||
@@ -340,6 +506,27 @@ RSpec.describe 'Tags API', type: :request do
|
|||||||
end
|
end
|
||||||
|
|
||||||
describe 'GET /tags/name/:name' do
|
describe 'GET /tags/name/:name' do
|
||||||
|
it 'preserves qualified external name lookup used by wiki pages' do
|
||||||
|
external = create(:external_tag, name: 'detail_external', post_count: 3)
|
||||||
|
|
||||||
|
get "/tags/name/#{ CGI.escape('nico:detail_external') }"
|
||||||
|
|
||||||
|
expect(response).to have_http_status(:ok)
|
||||||
|
expect(json).to include(
|
||||||
|
'id' => external.id,
|
||||||
|
'name' => 'nico:detail_external',
|
||||||
|
'category' => 'nico',
|
||||||
|
'post_count' => 3,
|
||||||
|
'created_at' => external.created_at.as_json,
|
||||||
|
'updated_at' => external.created_at.as_json,
|
||||||
|
'deprecated_at' => nil,
|
||||||
|
'aliases' => [],
|
||||||
|
'parents' => [],
|
||||||
|
'has_wiki' => false,
|
||||||
|
'material_id' => nil,
|
||||||
|
'has_deerjikists' => false)
|
||||||
|
end
|
||||||
|
|
||||||
it 'returns tag by name' do
|
it 'returns tag by name' do
|
||||||
get "/tags/name/#{ CGI.escape('spec_tag') }"
|
get "/tags/name/#{ CGI.escape('spec_tag') }"
|
||||||
|
|
||||||
@@ -487,18 +674,6 @@ RSpec.describe 'Tags API', type: :request do
|
|||||||
expect(json.fetch('deprecated_at')).to be_present
|
expect(json.fetch('deprecated_at')).to be_present
|
||||||
end
|
end
|
||||||
|
|
||||||
it 'rejects deprecating a nico tag' do
|
|
||||||
nico_tag = Tag.create!(name: 'nico:deprecated_update', category: :nico)
|
|
||||||
|
|
||||||
patch "/tags/#{ nico_tag.id }", params: { deprecated: '1' }
|
|
||||||
|
|
||||||
expect(response).to have_http_status(:unprocessable_entity)
|
|
||||||
expect(nico_tag.reload.deprecated_at).to be_nil
|
|
||||||
expect(json.fetch('errors')).to include(
|
|
||||||
'deprecated' => ['ニコタグは廃止できません.']
|
|
||||||
)
|
|
||||||
end
|
|
||||||
|
|
||||||
it 'returns 422 when changing normal tag category to nico' do
|
it 'returns 422 when changing normal tag category to nico' do
|
||||||
expect {
|
expect {
|
||||||
patch "/tags/#{tag.id}", params: { category: 'nico' }
|
patch "/tags/#{tag.id}", params: { category: 'nico' }
|
||||||
@@ -508,32 +683,6 @@ RSpec.describe 'Tags API', type: :request do
|
|||||||
expect(tag.reload.category).to eq('general')
|
expect(tag.reload.category).to eq('general')
|
||||||
end
|
end
|
||||||
|
|
||||||
it 'returns 422 when updating nico tag name' do
|
|
||||||
nico_tag_name = TagName.create!(name: 'nico:tags_spec_source')
|
|
||||||
nico_tag = Tag.create!(tag_name: nico_tag_name, category: :nico)
|
|
||||||
|
|
||||||
expect {
|
|
||||||
patch "/tags/#{ nico_tag.id }", params: { name: 'nico:tags_spec_renamed' }
|
|
||||||
}.not_to change(NicoTagVersion, :count)
|
|
||||||
|
|
||||||
expect(response).to have_http_status(:unprocessable_entity)
|
|
||||||
|
|
||||||
expect(nico_tag.reload.name).to eq('nico:tags_spec_source')
|
|
||||||
expect(nico_tag.category).to eq('nico')
|
|
||||||
end
|
|
||||||
|
|
||||||
it 'returns 422 when changing nico tag category to normal category' do
|
|
||||||
nico_tag_name = TagName.create!(name: 'nico:category_change_ng')
|
|
||||||
nico_tag = Tag.create!(tag_name: nico_tag_name, category: :nico)
|
|
||||||
|
|
||||||
expect {
|
|
||||||
patch "/tags/#{nico_tag.id}", params: { category: 'general' }
|
|
||||||
}.not_to change(NicoTagVersion, :count)
|
|
||||||
|
|
||||||
expect(response).to have_http_status(:unprocessable_entity)
|
|
||||||
expect(nico_tag.reload.category).to eq('nico')
|
|
||||||
end
|
|
||||||
|
|
||||||
it 'PATCH で tag の name を変更すると対応する wiki version を作成する' do
|
it 'PATCH で tag の name を変更すると対応する wiki version を作成する' do
|
||||||
wiki_page =
|
wiki_page =
|
||||||
Wiki::Commit.create_content!(
|
Wiki::Commit.create_content!(
|
||||||
@@ -1169,28 +1318,6 @@ RSpec.describe 'Tags API', type: :request do
|
|||||||
expect(tag.category).to eq('general')
|
expect(tag.category).to eq('general')
|
||||||
end
|
end
|
||||||
|
|
||||||
it 'nico tag は更新できない' do
|
|
||||||
nico_tag = Tag.create!(
|
|
||||||
tag_name: TagName.create!(name: 'nico:put_update_all_ng'),
|
|
||||||
category: :nico
|
|
||||||
)
|
|
||||||
|
|
||||||
expect {
|
|
||||||
put "/tags/#{ nico_tag.id }", params: {
|
|
||||||
name: 'nico:put_update_all_renamed',
|
|
||||||
category: 'nico',
|
|
||||||
aliases: '',
|
|
||||||
parent_tags: '',
|
|
||||||
deprecated: '0',
|
|
||||||
}
|
|
||||||
}.not_to change(NicoTagVersion, :count)
|
|
||||||
|
|
||||||
expect(response).to have_http_status(:unprocessable_entity)
|
|
||||||
|
|
||||||
expect(nico_tag.reload.name).to eq('nico:put_update_all_ng')
|
|
||||||
expect(nico_tag.category).to eq('nico')
|
|
||||||
end
|
|
||||||
|
|
||||||
it 'system tag の name は変更できない' do
|
it 'system tag の name は変更できない' do
|
||||||
system_tag = Tag.tagme
|
system_tag = Tag.tagme
|
||||||
old_name = system_tag.name
|
old_name = system_tag.name
|
||||||
|
|||||||
@@ -0,0 +1,98 @@
|
|||||||
|
require 'rails_helper'
|
||||||
|
|
||||||
|
RSpec.describe 'user settings', type: :request do
|
||||||
|
describe 'GET /users/settings' do
|
||||||
|
it 'requires a current user' do
|
||||||
|
sign_out
|
||||||
|
|
||||||
|
get '/users/settings'
|
||||||
|
|
||||||
|
expect(response).to have_http_status(:unauthorized)
|
||||||
|
end
|
||||||
|
|
||||||
|
it 'returns defaults and creates a missing settings row' do
|
||||||
|
user = create(:user)
|
||||||
|
sign_in_as(user)
|
||||||
|
|
||||||
|
expect {
|
||||||
|
get '/users/settings'
|
||||||
|
}.to change(Setting, :count).by(1)
|
||||||
|
|
||||||
|
expect(response).to have_http_status(:ok)
|
||||||
|
expect(json).to eq(
|
||||||
|
'theme' => 'system',
|
||||||
|
'auto_fetch_title' => 'manual',
|
||||||
|
'auto_fetch_thumbnail' => 'manual',
|
||||||
|
'wiki_editor_mode' => 'split')
|
||||||
|
expect(user.reload.setting).to be_present
|
||||||
|
end
|
||||||
|
end
|
||||||
|
|
||||||
|
describe 'PATCH /users/settings' do
|
||||||
|
it 'requires a current user' do
|
||||||
|
sign_out
|
||||||
|
|
||||||
|
patch '/users/settings', params: { theme: 'dark' }, as: :json
|
||||||
|
|
||||||
|
expect(response).to have_http_status(:unauthorized)
|
||||||
|
end
|
||||||
|
|
||||||
|
it 'updates multiple typed settings at once' do
|
||||||
|
user = create(:user)
|
||||||
|
sign_in_as(user)
|
||||||
|
|
||||||
|
patch '/users/settings',
|
||||||
|
params: {
|
||||||
|
theme: 'dark',
|
||||||
|
auto_fetch_title: 'auto',
|
||||||
|
auto_fetch_thumbnail: 'off',
|
||||||
|
wiki_editor_mode: 'preview' },
|
||||||
|
as: :json
|
||||||
|
|
||||||
|
expect(response).to have_http_status(:ok)
|
||||||
|
expect(json).to include(
|
||||||
|
'theme' => 'dark',
|
||||||
|
'auto_fetch_title' => 'auto',
|
||||||
|
'auto_fetch_thumbnail' => 'off',
|
||||||
|
'wiki_editor_mode' => 'preview')
|
||||||
|
expect(user.reload.setting).to have_attributes(
|
||||||
|
theme: 'dark',
|
||||||
|
auto_fetch_title: 'auto',
|
||||||
|
auto_fetch_thumbnail: 'off',
|
||||||
|
wiki_editor_mode: 'preview')
|
||||||
|
end
|
||||||
|
|
||||||
|
it 'does not treat Rails parameter wrapping as an editable setting' do
|
||||||
|
user = create(:user)
|
||||||
|
sign_in_as(user)
|
||||||
|
|
||||||
|
patch '/users/settings',
|
||||||
|
params: { theme: 'light', user_setting: { theme: 'dark' } },
|
||||||
|
as: :json
|
||||||
|
|
||||||
|
expect(response).to have_http_status(:ok)
|
||||||
|
expect(user.reload.setting.theme).to eq('light')
|
||||||
|
end
|
||||||
|
|
||||||
|
it 'returns validation_error for type mismatches' do
|
||||||
|
sign_in_as(create(:user))
|
||||||
|
|
||||||
|
patch '/users/settings', params: { theme: 1 }, as: :json
|
||||||
|
|
||||||
|
expect(response).to have_http_status(:unprocessable_entity)
|
||||||
|
expect(json).to include('type' => 'validation_error')
|
||||||
|
expect(json.fetch('errors')).to include(
|
||||||
|
'theme' => ['値の型が不正です.'])
|
||||||
|
end
|
||||||
|
|
||||||
|
it 'returns validation_error for invalid enum values' do
|
||||||
|
sign_in_as(create(:user))
|
||||||
|
|
||||||
|
patch '/users/settings', params: { theme: 'neon' }, as: :json
|
||||||
|
|
||||||
|
expect(response).to have_http_status(:unprocessable_entity)
|
||||||
|
expect(json).to include('type' => 'validation_error')
|
||||||
|
expect(json.fetch('errors').fetch('theme')).to be_present
|
||||||
|
end
|
||||||
|
end
|
||||||
|
end
|
||||||
@@ -0,0 +1,104 @@
|
|||||||
|
require 'rails_helper'
|
||||||
|
|
||||||
|
RSpec.describe 'user theme slots', type: :request do
|
||||||
|
describe 'GET /users/theme_slots' do
|
||||||
|
it 'requires a current user' do
|
||||||
|
sign_out
|
||||||
|
|
||||||
|
get '/users/theme_slots'
|
||||||
|
|
||||||
|
expect(response).to have_http_status(:unauthorized)
|
||||||
|
end
|
||||||
|
|
||||||
|
it 'returns only the current user slots in stable order' do
|
||||||
|
user = create(:user)
|
||||||
|
other_user = create(:user)
|
||||||
|
sign_in_as(user)
|
||||||
|
UserThemeSlot.create!(user:, base_theme: 'light', slot_no: 2, tokens: { 'a' => 1 })
|
||||||
|
UserThemeSlot.create!(user:, base_theme: 'dark', slot_no: 1, tokens: { 'b' => 2 })
|
||||||
|
UserThemeSlot.create!(user: other_user,
|
||||||
|
base_theme: 'dark',
|
||||||
|
slot_no: 3,
|
||||||
|
tokens: { 'c' => 3 })
|
||||||
|
|
||||||
|
get '/users/theme_slots'
|
||||||
|
|
||||||
|
expect(response).to have_http_status(:ok)
|
||||||
|
expect(json.map { |slot| [slot.fetch('base_theme'), slot.fetch('slot_no')] }).to eq(
|
||||||
|
[['dark', 1], ['light', 2]])
|
||||||
|
expect(json.map { |slot| slot.fetch('tokens') }).to eq(
|
||||||
|
[{ 'b' => 2 }, { 'a' => 1 }])
|
||||||
|
end
|
||||||
|
end
|
||||||
|
|
||||||
|
describe 'PUT /users/theme_slots/:base_theme/:slot_no' do
|
||||||
|
it 'requires a current user' do
|
||||||
|
sign_out
|
||||||
|
|
||||||
|
put '/users/theme_slots/light/1',
|
||||||
|
params: { tokens: { background: '0 0% 100%' } },
|
||||||
|
as: :json
|
||||||
|
|
||||||
|
expect(response).to have_http_status(:unauthorized)
|
||||||
|
end
|
||||||
|
|
||||||
|
it 'creates a theme slot' do
|
||||||
|
user = create(:user)
|
||||||
|
sign_in_as(user)
|
||||||
|
|
||||||
|
expect {
|
||||||
|
put '/users/theme_slots/light/1',
|
||||||
|
params: { tokens: { background: '0 0% 100%', tagColours: { nico: '#ffffff' } } },
|
||||||
|
as: :json
|
||||||
|
}.to change(UserThemeSlot, :count).by(1)
|
||||||
|
|
||||||
|
expect(response).to have_http_status(:ok)
|
||||||
|
expect(json).to include('base_theme' => 'light', 'slot_no' => 1)
|
||||||
|
expect(json.fetch('tokens')).to include(
|
||||||
|
'background' => '0 0% 100%',
|
||||||
|
'tagColours' => { 'nico' => '#ffffff' })
|
||||||
|
expect(user.theme_slots.find_by!(base_theme: 'light', slot_no: 1).tokens).to include(
|
||||||
|
'background' => '0 0% 100%')
|
||||||
|
end
|
||||||
|
|
||||||
|
it 'updates an existing theme slot instead of creating a duplicate' do
|
||||||
|
user = create(:user)
|
||||||
|
sign_in_as(user)
|
||||||
|
UserThemeSlot.create!(user:, base_theme: 'dark', slot_no: 2, tokens: { 'old' => true })
|
||||||
|
|
||||||
|
expect {
|
||||||
|
put '/users/theme_slots/dark/2',
|
||||||
|
params: { tokens: { background: '222.2 84% 4.9%' } },
|
||||||
|
as: :json
|
||||||
|
}.not_to change(UserThemeSlot, :count)
|
||||||
|
|
||||||
|
expect(response).to have_http_status(:ok)
|
||||||
|
expect(user.theme_slots.find_by!(base_theme: 'dark', slot_no: 2).tokens).to eq(
|
||||||
|
'background' => '222.2 84% 4.9%')
|
||||||
|
end
|
||||||
|
|
||||||
|
it 'rejects invalid path parameters and token shapes' do
|
||||||
|
sign_in_as(create(:user))
|
||||||
|
|
||||||
|
put '/users/theme_slots/system/4',
|
||||||
|
params: { tokens: 'not-object' },
|
||||||
|
as: :json
|
||||||
|
|
||||||
|
expect(response).to have_http_status(:unprocessable_entity)
|
||||||
|
expect(json.fetch('errors')).to include(
|
||||||
|
'base_theme' => ['値が不正です.'])
|
||||||
|
end
|
||||||
|
|
||||||
|
it 'rejects non-object tokens for valid slots' do
|
||||||
|
sign_in_as(create(:user))
|
||||||
|
|
||||||
|
put '/users/theme_slots/light/1',
|
||||||
|
params: { tokens: 'not-object' },
|
||||||
|
as: :json
|
||||||
|
|
||||||
|
expect(response).to have_http_status(:unprocessable_entity)
|
||||||
|
expect(json.fetch('errors')).to include(
|
||||||
|
'tokens' => ['JSON object で指定してください.'])
|
||||||
|
end
|
||||||
|
end
|
||||||
|
end
|
||||||
@@ -1,27 +0,0 @@
|
|||||||
require 'rails_helper'
|
|
||||||
|
|
||||||
RSpec.describe 'Wiki body search', type: :request do
|
|
||||||
let!(:user) { create_member_user! }
|
|
||||||
|
|
||||||
it 'searches wiki pages by body text' do
|
|
||||||
pending '#336 で対応予定'
|
|
||||||
|
|
||||||
Wiki::Commit.create_content!(
|
|
||||||
tag_name: TagName.create!(name: 'wiki_body_search_hit'),
|
|
||||||
body: 'unique body keyword for wiki search',
|
|
||||||
created_by_user: user,
|
|
||||||
message: 'init')
|
|
||||||
|
|
||||||
Wiki::Commit.create_content!(
|
|
||||||
tag_name: TagName.create!(name: 'wiki_body_search_miss'),
|
|
||||||
body: 'ordinary body',
|
|
||||||
created_by_user: user,
|
|
||||||
message: 'init')
|
|
||||||
|
|
||||||
get '/wiki/search', params: { body: 'unique body keyword' }
|
|
||||||
|
|
||||||
expect(response).to have_http_status(:ok)
|
|
||||||
expect(json.map { |page| page['title'] }).to include('wiki_body_search_hit')
|
|
||||||
expect(json.map { |page| page['title'] }).not_to include('wiki_body_search_miss')
|
|
||||||
end
|
|
||||||
end
|
|
||||||
@@ -1,37 +0,0 @@
|
|||||||
require 'rails_helper'
|
|
||||||
|
|
||||||
RSpec.describe 'Wiki restore', type: :request do
|
|
||||||
let!(:user) { create_member_user! }
|
|
||||||
|
|
||||||
def auth_headers user
|
|
||||||
{ 'X-Transfer-Code' => user.inheritance_code }
|
|
||||||
end
|
|
||||||
|
|
||||||
it 'restores wiki page to previous version' do
|
|
||||||
pending '#337 で対応予定'
|
|
||||||
|
|
||||||
page =
|
|
||||||
Wiki::Commit.create_content!(
|
|
||||||
tag_name: TagName.create!(name: 'wiki_restore_page'),
|
|
||||||
body: 'v1',
|
|
||||||
created_by_user: user,
|
|
||||||
message: 'init')
|
|
||||||
|
|
||||||
v1 = page.wiki_versions.order(:version_no).last
|
|
||||||
|
|
||||||
Wiki::Commit.content!(
|
|
||||||
page:,
|
|
||||||
body: 'v2',
|
|
||||||
created_user: user,
|
|
||||||
message: 'edit',
|
|
||||||
base_revision_id: page.current_revision.id)
|
|
||||||
|
|
||||||
post "/wiki/#{ page.id }/restore",
|
|
||||||
params: { version_no: v1.version_no },
|
|
||||||
headers: auth_headers(user)
|
|
||||||
|
|
||||||
expect(response).to have_http_status(:ok)
|
|
||||||
expect(page.reload.body).to eq('v1')
|
|
||||||
expect(page.wiki_versions.order(:version_no).last.event_type).to eq('restore')
|
|
||||||
end
|
|
||||||
end
|
|
||||||
@@ -0,0 +1,76 @@
|
|||||||
|
require 'rails_helper'
|
||||||
|
|
||||||
|
RSpec.describe NicoTagVersionRecorder do
|
||||||
|
let(:external_tag) { create(:external_tag, name: 'raw tag[]') }
|
||||||
|
let(:member) { create(:user, :member) }
|
||||||
|
|
||||||
|
def record event_type
|
||||||
|
described_class.record!(external_tag:, event_type:, created_by_user: member)
|
||||||
|
end
|
||||||
|
|
||||||
|
it 'records the external association and platform-qualified name' do
|
||||||
|
version = record(:create)
|
||||||
|
|
||||||
|
expect(version).to have_attributes(
|
||||||
|
external_tag:, version_no: 1, event_type: 'create',
|
||||||
|
name: 'nico:raw tag[]', linked_tags: '', created_by_user: member)
|
||||||
|
expect(external_tag.reload.nico_tag_versions).to contain_exactly(version)
|
||||||
|
end
|
||||||
|
|
||||||
|
it 'uses the latest history number when the record has no version_no column' do
|
||||||
|
first = record(:create)
|
||||||
|
external_tag.update!(name: 'changed')
|
||||||
|
second = record(:update)
|
||||||
|
|
||||||
|
expect(second).to have_attributes(version_no: 2, name: 'nico:changed')
|
||||||
|
expect(first.reload.name).to eq('nico:raw tag[]')
|
||||||
|
expect(external_tag.reload.has_attribute?(:version_no)).to be(false)
|
||||||
|
end
|
||||||
|
|
||||||
|
it 'builds the qualified name from the registered platform value' do
|
||||||
|
external = create(:external_tag, name: 'foo')
|
||||||
|
locked_scope = instance_double(ActiveRecord::Relation)
|
||||||
|
allow(ExternalTag).to receive(:unscoped).and_return(locked_scope)
|
||||||
|
allow(locked_scope).to receive(:lock).and_return(locked_scope)
|
||||||
|
allow(locked_scope).to receive(:find).with(external.id).and_return(external)
|
||||||
|
allow(external).to receive(:platform).and_return('registered_external')
|
||||||
|
|
||||||
|
version = described_class.record!(
|
||||||
|
external_tag: external, event_type: :create, created_by_user: member)
|
||||||
|
|
||||||
|
expect(version).to have_attributes(
|
||||||
|
external_tag: external, name: 'registered_external:foo', version_no: 1,
|
||||||
|
event_type: 'create', linked_tags: '', created_by_user: member)
|
||||||
|
end
|
||||||
|
|
||||||
|
it 'returns the latest version without appending an unchanged snapshot' do
|
||||||
|
first = record(:create)
|
||||||
|
|
||||||
|
expect { expect(record(:update)).to eq(first) }
|
||||||
|
.not_to change(NicoTagVersion, :count)
|
||||||
|
end
|
||||||
|
|
||||||
|
it 'still requires a create event before any update' do
|
||||||
|
expect { record(:update) }
|
||||||
|
.to raise_error(RuntimeError, 'NicoTagVersion first event must be create')
|
||||||
|
expect(external_tag.nico_tag_versions).to be_empty
|
||||||
|
end
|
||||||
|
|
||||||
|
it 'still rejects a second create event' do
|
||||||
|
first = record(:create)
|
||||||
|
|
||||||
|
expect { record(:create) }
|
||||||
|
.to raise_error(RuntimeError, 'NicoTagVersion create event already exists')
|
||||||
|
expect(external_tag.nico_tag_versions).to contain_exactly(first)
|
||||||
|
end
|
||||||
|
|
||||||
|
it 'records sorted linked internal names and later link removal' do
|
||||||
|
tags = ['z_link', 'a_link'].map { |name| Tag.create!(name:, category: :general) }
|
||||||
|
tags.each { |tag| NicoTagRelation.create!(nico_tag: external_tag, tag:) }
|
||||||
|
|
||||||
|
expect(record(:create).linked_tags).to eq('a_link z_link')
|
||||||
|
external_tag.linked_tags = [tags.first]
|
||||||
|
|
||||||
|
expect(record(:update)).to have_attributes(version_no: 2, linked_tags: 'z_link')
|
||||||
|
end
|
||||||
|
end
|
||||||
@@ -0,0 +1,59 @@
|
|||||||
|
require 'rails_helper'
|
||||||
|
|
||||||
|
RSpec.describe PostBulkCreator do
|
||||||
|
it 'limits workers to two and keeps failures in their request slots' do
|
||||||
|
actor = instance_double(User, id: 123)
|
||||||
|
allow(User).to receive(:find).with(123) {
|
||||||
|
instance_double(User, id: 123)
|
||||||
|
}
|
||||||
|
mutex = Mutex.new
|
||||||
|
active = 0
|
||||||
|
maximum_active = 0
|
||||||
|
|
||||||
|
allow(PostCreatePreflight).to receive(:new) do |attributes:, **|
|
||||||
|
preflight = instance_double(PostCreatePreflight)
|
||||||
|
allow(preflight).to receive(:run) do
|
||||||
|
mutex.synchronize do
|
||||||
|
active += 1
|
||||||
|
maximum_active = [maximum_active, active].max
|
||||||
|
end
|
||||||
|
sleep 0.02
|
||||||
|
mutex.synchronize { active -= 1 }
|
||||||
|
attributes.symbolize_keys.merge(
|
||||||
|
existing_post_id: nil,
|
||||||
|
field_warnings: { },
|
||||||
|
base_warnings: [])
|
||||||
|
end
|
||||||
|
preflight
|
||||||
|
end
|
||||||
|
allow(PostCreator).to receive(:new) do |attributes:, **|
|
||||||
|
creator = instance_double(PostCreator)
|
||||||
|
if attributes[:title] == 'broken'
|
||||||
|
allow(creator).to receive(:create!).and_raise(StandardError, 'broken')
|
||||||
|
else
|
||||||
|
post = instance_double(Post, id: attributes[:title].delete_prefix('post ').to_i)
|
||||||
|
allow(creator).to receive(:create!).and_return(post)
|
||||||
|
end
|
||||||
|
creator
|
||||||
|
end
|
||||||
|
posts = [
|
||||||
|
{ 'title' => 'post 1', 'url' => 'https://example.com/1' },
|
||||||
|
{ 'title' => 'broken', 'url' => 'https://example.com/2' },
|
||||||
|
{ 'title' => 'post 3', 'url' => 'https://example.com/3' },
|
||||||
|
{ 'title' => 'post 4', 'url' => 'https://example.com/4' }]
|
||||||
|
|
||||||
|
results = described_class.new(
|
||||||
|
actor:,
|
||||||
|
posts:,
|
||||||
|
thumbnails: { }).run.fetch(:results)
|
||||||
|
|
||||||
|
expect(maximum_active).to eq(2)
|
||||||
|
expect(results.length).to eq(posts.length)
|
||||||
|
expect(results.map { _1[:status] }).to eq(
|
||||||
|
['created', 'failed', 'created', 'created'])
|
||||||
|
expect(results[0].dig(:post, :id)).to eq(1)
|
||||||
|
expect(results[1]).to include(status: 'failed', recoverable: false)
|
||||||
|
expect(results[2].dig(:post, :id)).to eq(3)
|
||||||
|
expect(results[3].dig(:post, :id)).to eq(4)
|
||||||
|
end
|
||||||
|
end
|
||||||
@@ -0,0 +1,95 @@
|
|||||||
|
require 'rails_helper'
|
||||||
|
|
||||||
|
RSpec.describe PostCreatePlan do
|
||||||
|
def create_tag! name, category
|
||||||
|
Tag.create!(name:, category:)
|
||||||
|
end
|
||||||
|
|
||||||
|
before do
|
||||||
|
create_tag!('タグ希望', :meta)
|
||||||
|
create_tag!('ニジラー情報不詳', :meta)
|
||||||
|
end
|
||||||
|
|
||||||
|
it 'plans direct and existing default tags without persisting records' do
|
||||||
|
counts = [TagName.count, Tag.count]
|
||||||
|
|
||||||
|
plan = described_class.new(
|
||||||
|
attributes: {
|
||||||
|
url: 'https://example.com/post',
|
||||||
|
title: 'title',
|
||||||
|
tags: 'character:new_character',
|
||||||
|
parent_post_ids: '' }).build!
|
||||||
|
|
||||||
|
expect(plan[:tags]).to eq('new_character')
|
||||||
|
expect(plan[:direct_tag_specs]).to eq(
|
||||||
|
[{ name: 'new_character', category: :character }])
|
||||||
|
expect(plan[:default_tag_specs]).to include(
|
||||||
|
{ name: 'タグ希望', category: :meta },
|
||||||
|
{ name: 'ニジラー情報不詳', category: :meta })
|
||||||
|
expect([TagName.count, Tag.count]).to eq(counts)
|
||||||
|
end
|
||||||
|
|
||||||
|
it 'resolves aliases and keeps tag sections separate from canonical names' do
|
||||||
|
canonical = create_tag!('虹夏', :character)
|
||||||
|
TagName.create!(name: 'にじか', canonical: canonical.tag_name)
|
||||||
|
create_tag!('動画', :meta)
|
||||||
|
|
||||||
|
plan = described_class.new(
|
||||||
|
attributes: {
|
||||||
|
url: 'https://example.com/video',
|
||||||
|
title: 'video',
|
||||||
|
tags: '動画 にじか[0:10-0:20]',
|
||||||
|
duration: '1:00',
|
||||||
|
parent_post_ids: '' }).build!
|
||||||
|
|
||||||
|
expect(plan[:tags].split).to include('動画', '虹夏[0:10-0:20]')
|
||||||
|
expect(plan[:display_tags]).to include(
|
||||||
|
{ name: '虹夏',
|
||||||
|
category: 'character',
|
||||||
|
section_literals: ['[0:10-0:20]'] })
|
||||||
|
expect(plan[:video_ms]).to eq(60_000)
|
||||||
|
end
|
||||||
|
|
||||||
|
it 'rejects direct external tag input without creating internal records' do
|
||||||
|
create(:external_tag, name: 'reserved')
|
||||||
|
counts = [Tag.count, TagName.count, ExternalTag.count]
|
||||||
|
|
||||||
|
expect {
|
||||||
|
described_class.new(attributes: { tags: 'NiCo:reserved' }).build!
|
||||||
|
}.to raise_error(ActiveRecord::RecordInvalid) { |error|
|
||||||
|
expect(error.record.errors[:tags]).to be_present
|
||||||
|
}
|
||||||
|
|
||||||
|
expect([Tag.count, TagName.count, ExternalTag.count]).to eq(counts)
|
||||||
|
end
|
||||||
|
|
||||||
|
it 'validates a new tag name without persisting it' do
|
||||||
|
long_name = 'a' * 256
|
||||||
|
counts = [TagName.count, Tag.count]
|
||||||
|
|
||||||
|
expect {
|
||||||
|
described_class.new(
|
||||||
|
attributes: {
|
||||||
|
url: 'https://example.com/post',
|
||||||
|
title: 'title',
|
||||||
|
tags: long_name,
|
||||||
|
parent_post_ids: '' }).build!
|
||||||
|
}.to raise_error(ActiveRecord::RecordInvalid) { |error|
|
||||||
|
expect(error.record.errors[:tags]).not_to be_empty
|
||||||
|
}
|
||||||
|
expect([TagName.count, Tag.count]).to eq(counts)
|
||||||
|
end
|
||||||
|
|
||||||
|
it 'ignores duration when the planned tags do not include video' do
|
||||||
|
plan = described_class.new(
|
||||||
|
attributes: {
|
||||||
|
url: 'https://example.com/post',
|
||||||
|
title: 'title',
|
||||||
|
tags: 'ordinary_tag',
|
||||||
|
duration: 'invalid',
|
||||||
|
parent_post_ids: '' }).build!
|
||||||
|
|
||||||
|
expect(plan[:duration]).to eq('invalid')
|
||||||
|
expect(plan[:video_ms]).to be_nil
|
||||||
|
end
|
||||||
|
end
|
||||||
変更されたファイルが多すぎるため、一部のファイルは表示されません さらに表示
新しいイシューから参照
ユーザーをブロックする