コミットを比較
7 コミット
| 作成者 | SHA1 | 日付 | |
|---|---|---|---|
| 130d8b256a | |||
| eeefb09d0c | |||
| d2e69b72da | |||
| b8126eeea9 | |||
| 2a40dd999b | |||
| 2cd9c60368 | |||
| 8eb8fb355b |
-186
@@ -67,192 +67,6 @@ pass or the remaining failure is clearly blocked.
|
|||||||
Before changing behavior, inspect the matching route, controller, model,
|
Before changing behavior, inspect the matching route, controller, model,
|
||||||
service, representation, and spec.
|
service, representation, and spec.
|
||||||
|
|
||||||
## Shared backend systems
|
|
||||||
|
|
||||||
Before adding backend behaviour, search the existing backend first. At minimum,
|
|
||||||
check these locations:
|
|
||||||
|
|
||||||
- `app/controllers`
|
|
||||||
- `app/controllers/concerns`
|
|
||||||
- `app/models`
|
|
||||||
- `app/models/concerns`
|
|
||||||
- `app/representations`
|
|
||||||
- `app/services`
|
|
||||||
- `app/services/*`
|
|
||||||
- `app/jobs`
|
|
||||||
- `lib`
|
|
||||||
- `lib/tasks`
|
|
||||||
- `config/initializers`
|
|
||||||
|
|
||||||
Do not infer commonality from directory names alone. Read the actual
|
|
||||||
responsibility and representative usage sites.
|
|
||||||
|
|
||||||
### Controller reuse
|
|
||||||
|
|
||||||
Before adding logic to a controller, inspect:
|
|
||||||
|
|
||||||
- `ApplicationController` authentication, authorization, BAN, and IP BAN
|
|
||||||
- existing render and validation-error helpers
|
|
||||||
- existing param parsing
|
|
||||||
- controller concerns
|
|
||||||
- the controller for the same resource
|
|
||||||
- existing services
|
|
||||||
- existing representations
|
|
||||||
|
|
||||||
Keep controllers focused on:
|
|
||||||
|
|
||||||
- authentication and authorization
|
|
||||||
- parameter intake
|
|
||||||
- service and model invocation
|
|
||||||
- HTTP status selection
|
|
||||||
- representation selection
|
|
||||||
|
|
||||||
Do not reimplement these per controller when an existing path already owns
|
|
||||||
them:
|
|
||||||
|
|
||||||
- authentication and role checks
|
|
||||||
- validation error JSON
|
|
||||||
- URL normalisation
|
|
||||||
- tag normalisation
|
|
||||||
- thumbnail handling
|
|
||||||
- version recording
|
|
||||||
- complex transactions
|
|
||||||
- external HTTP fetching
|
|
||||||
- response representation assembly
|
|
||||||
|
|
||||||
### Authentication, authorization, and BAN
|
|
||||||
|
|
||||||
Treat these as the canonical backend entrypoints:
|
|
||||||
|
|
||||||
- `ApplicationController#authenticate_user`
|
|
||||||
- `current_user`
|
|
||||||
- `X-Transfer-Code`
|
|
||||||
- `reject_banned_ip_address!`
|
|
||||||
- `reject_banned_user!`
|
|
||||||
- `gte_member?`
|
|
||||||
- `admin?`
|
|
||||||
|
|
||||||
Do not create feature-local permission services, role comparisons, or header
|
|
||||||
parsing when the existing authentication boundary already owns the behaviour.
|
|
||||||
If the current boundary is insufficient, extend it minimally instead of adding
|
|
||||||
another permission path.
|
|
||||||
|
|
||||||
### Representations
|
|
||||||
|
|
||||||
If an endpoint for the same resource already uses `app/representations`, do not
|
|
||||||
assemble a separate JSON shape directly inside the controller without first
|
|
||||||
checking the existing representation contract.
|
|
||||||
|
|
||||||
Inspect at least:
|
|
||||||
|
|
||||||
- `PostRepr`
|
|
||||||
- `TagRepr`
|
|
||||||
- `MaterialRepr`
|
|
||||||
- `TheatreRepr`
|
|
||||||
- `UserRepr`
|
|
||||||
- `WikiPageRepr`
|
|
||||||
- `DeerjikistRepr`
|
|
||||||
|
|
||||||
When a lightweight response is genuinely different in purpose, keep it
|
|
||||||
deliberate and compatible with the surrounding contracts. Do not force every
|
|
||||||
identifier list into a large representation, but do not fork the same resource
|
|
||||||
shape casually either.
|
|
||||||
|
|
||||||
### Domain services
|
|
||||||
|
|
||||||
When work touches multiple models, transactions, external APIs, file handling,
|
|
||||||
history creation, or multi-step workflow, search `app/services` first.
|
|
||||||
|
|
||||||
At minimum, search for existing services in these responsibility areas:
|
|
||||||
|
|
||||||
- version recorder and versioning
|
|
||||||
- wiki commit
|
|
||||||
- YouTube or Google Drive API client
|
|
||||||
- material sync or ZIP export
|
|
||||||
- similarity calculation
|
|
||||||
- theatre selection or skip finalisation
|
|
||||||
- metadata, thumbnail, or file processing
|
|
||||||
- URL normaliser or sanitisation
|
|
||||||
- import or export
|
|
||||||
- preview safety or HTTP fetch
|
|
||||||
|
|
||||||
Do not create a same-responsibility service under another namespace or another
|
|
||||||
name. If an existing service is close, extend that API minimally instead of
|
|
||||||
wrapping it in a feature-local service.
|
|
||||||
|
|
||||||
### Versioning
|
|
||||||
|
|
||||||
When a feature writes history, snapshots, or restore roots, search the existing
|
|
||||||
versioning path first. At minimum, inspect:
|
|
||||||
|
|
||||||
- `VersionRecorder`
|
|
||||||
- `PostVersionRecorder`
|
|
||||||
- `TagVersionRecorder`
|
|
||||||
- `TagVersioning`
|
|
||||||
- `MaterialVersionRecorder`
|
|
||||||
- `NicoTagVersionRecorder`
|
|
||||||
- `WikiVersionRecorder`
|
|
||||||
|
|
||||||
Do not implement history writes in controllers, callbacks, or ad hoc feature
|
|
||||||
services when the recorder layer already owns the transaction boundary and
|
|
||||||
meaning.
|
|
||||||
|
|
||||||
### Normalisation, sanitisation, and parsing
|
|
||||||
|
|
||||||
For URLs, tag names, times, video durations, identifiers, and paths, search the
|
|
||||||
existing normaliser, sanitisation rule, parser, and model-callback path first.
|
|
||||||
|
|
||||||
Do not let frontend, controller, service, and model each invent different rules
|
|
||||||
for the same value. Use one canonical normalisation path and keep input
|
|
||||||
validation distinct from pre-persistence normalisation.
|
|
||||||
|
|
||||||
### External HTTP and URL safety
|
|
||||||
|
|
||||||
When fetching external URLs, reuse the existing preview-safety stack. Search at
|
|
||||||
least for:
|
|
||||||
|
|
||||||
- URL safety
|
|
||||||
- redirect validation
|
|
||||||
- response size limits
|
|
||||||
- timeouts
|
|
||||||
- network failure mapping
|
|
||||||
- HTML metadata extraction
|
|
||||||
- known-site extraction
|
|
||||||
- thumbnail fetching
|
|
||||||
|
|
||||||
Do not add direct `Net::HTTP`, `Faraday`, or equivalent feature-local HTTP code
|
|
||||||
that reimplements SSRF checks, redirect restrictions, size limits, or timeouts.
|
|
||||||
If the current fetcher is insufficient, extend its existing safety contract.
|
|
||||||
|
|
||||||
### Storage, files, and Active Storage
|
|
||||||
|
|
||||||
When handling files, thumbnails, ZIP output, object storage, or Active Storage
|
|
||||||
blobs, inspect existing storage helpers, exporters, thumbnail generators, and
|
|
||||||
checksum helpers first. Do not reimplement the same attach, export path,
|
|
||||||
download, resize, or checksum flow in a controller or one-off service.
|
|
||||||
|
|
||||||
### Concerns
|
|
||||||
|
|
||||||
Do not create controller or model concerns merely because some code is shared.
|
|
||||||
Use a concern only when multiple classes share the same lifecycle, macro,
|
|
||||||
callback, or tightly cohesive behaviour. Utility collections belong in explicit
|
|
||||||
objects or services, not in `CommonConcern`, `SharedMethods`, or `Utils`.
|
|
||||||
|
|
||||||
### Model boundaries
|
|
||||||
|
|
||||||
Model-specific invariants, associations, validations, and normalisation may
|
|
||||||
live in the model. Multi-model workflow, external access, complex transaction
|
|
||||||
flow, and feature orchestration belong in services. Do not hide feature
|
|
||||||
workflow in model callbacks.
|
|
||||||
|
|
||||||
### Transactions, locking, and race handling
|
|
||||||
|
|
||||||
If transactions, locking, idempotency, or race recovery already exist in a
|
|
||||||
service or model method, do not add a second implementation in a controller or
|
|
||||||
new service. Inspect the existing transaction boundary first, avoid wrapping
|
|
||||||
the same operation in needless nested transactions, and handle unique-constraint
|
|
||||||
races according to the target constraint's business meaning.
|
|
||||||
|
|
||||||
## Ruby style
|
## Ruby style
|
||||||
|
|
||||||
- Prefer precise, minimal changes.
|
- Prefer precise, minimal changes.
|
||||||
|
|||||||
@@ -1,49 +0,0 @@
|
|||||||
class MaterialSyncSuppressionsController < ApplicationController
|
|
||||||
def index
|
|
||||||
return head :unauthorized unless current_user
|
|
||||||
return head :forbidden unless current_user.gte_member?
|
|
||||||
|
|
||||||
suppressions =
|
|
||||||
MaterialSyncSuppression
|
|
||||||
.includes(:created_by_user)
|
|
||||||
.order(created_at: :desc, id: :desc)
|
|
||||||
|
|
||||||
render json: { suppressions: suppressions.map { |s| repr(s) } }
|
|
||||||
end
|
|
||||||
|
|
||||||
def create
|
|
||||||
return head :unauthorized unless current_user
|
|
||||||
return head :forbidden unless current_user.gte_member?
|
|
||||||
|
|
||||||
suppression =
|
|
||||||
MaterialSyncSuppressionRegistrar.create!(suppression_params,
|
|
||||||
created_by_user: current_user)
|
|
||||||
|
|
||||||
render json: repr(suppression), status: :created
|
|
||||||
rescue ActiveRecord::RecordInvalid => e
|
|
||||||
render_validation_error e.record
|
|
||||||
end
|
|
||||||
|
|
||||||
private
|
|
||||||
|
|
||||||
def suppression_params
|
|
||||||
params.permit(:source_kind,
|
|
||||||
:source_uri,
|
|
||||||
:drive_path,
|
|
||||||
:drive_file_id,
|
|
||||||
:reason)
|
|
||||||
end
|
|
||||||
|
|
||||||
def repr suppression
|
|
||||||
{ id: suppression.id,
|
|
||||||
source_kind: suppression.source_kind,
|
|
||||||
source_uri: suppression.source_uri,
|
|
||||||
drive_path: suppression.drive_path,
|
|
||||||
drive_file_id: suppression.drive_file_id,
|
|
||||||
normalized_source_key: suppression.normalized_source_key,
|
|
||||||
reason: suppression.reason,
|
|
||||||
created_by_user: suppression.created_by_user&.as_json(UserRepr::BASE),
|
|
||||||
created_at: suppression.created_at,
|
|
||||||
updated_at: suppression.updated_at }
|
|
||||||
end
|
|
||||||
end
|
|
||||||
@@ -1,54 +0,0 @@
|
|||||||
class MaterialVersionsController < ApplicationController
|
|
||||||
def index
|
|
||||||
page = (params[:page].presence || 1).to_i
|
|
||||||
limit = (params[:limit].presence || 20).to_i
|
|
||||||
|
|
||||||
page = 1 if page < 1
|
|
||||||
limit = 1 if limit < 1
|
|
||||||
|
|
||||||
offset = (page - 1) * limit
|
|
||||||
|
|
||||||
q = MaterialVersion.includes(:created_by_user)
|
|
||||||
q = q.where(material_id: params[:material_id]) if params[:material_id].present?
|
|
||||||
q = q.where(tag_name: params[:tag].to_s.strip) if params[:tag].present?
|
|
||||||
q = q.where(event_type: params[:event_type]) if params[:event_type].present?
|
|
||||||
|
|
||||||
count = q.except(:order, :limit, :offset).count
|
|
||||||
versions =
|
|
||||||
q
|
|
||||||
.order(Arel.sql('material_versions.created_at DESC, material_versions.id DESC'))
|
|
||||||
.limit(limit)
|
|
||||||
.offset(offset)
|
|
||||||
|
|
||||||
render json: { versions: serialise_versions(versions), count: }
|
|
||||||
end
|
|
||||||
|
|
||||||
private
|
|
||||||
|
|
||||||
def serialise_versions rows
|
|
||||||
rows.map do |row|
|
|
||||||
{ id: row.id,
|
|
||||||
material_id: row.material_id,
|
|
||||||
version_no: row.version_no,
|
|
||||||
event_type: row.event_type,
|
|
||||||
tag_id: row.tag_id,
|
|
||||||
tag_name: row.tag_name,
|
|
||||||
tag_category: row.tag_category,
|
|
||||||
url: row.url,
|
|
||||||
source_kind: row.source_kind,
|
|
||||||
source_uri: row.source_uri,
|
|
||||||
source_path: row.source_path,
|
|
||||||
source_file_id: row.source_file_id,
|
|
||||||
normalized_source_key: row.normalized_source_key,
|
|
||||||
file_blob_id: row.file_blob_id,
|
|
||||||
file_filename: row.file_filename,
|
|
||||||
file_content_type: row.file_content_type,
|
|
||||||
file_byte_size: row.file_byte_size,
|
|
||||||
file_sha256: row.file_sha256,
|
|
||||||
export_paths_json: row.export_paths_hash,
|
|
||||||
discarded_at: row.discarded_at,
|
|
||||||
created_by_user: row.created_by_user&.as_json(UserRepr::BASE),
|
|
||||||
created_at: row.created_at }
|
|
||||||
end
|
|
||||||
end
|
|
||||||
end
|
|
||||||
@@ -1,11 +1,4 @@
|
|||||||
require 'set'
|
|
||||||
|
|
||||||
|
|
||||||
class MaterialsController < ApplicationController
|
class MaterialsController < ApplicationController
|
||||||
rescue_from MaterialZipExporter::EmptyExportError, with: :render_zip_empty
|
|
||||||
rescue_from MaterialZipExporter::DuplicatePathError, with: :render_zip_duplicate_path
|
|
||||||
rescue_from MaterialZipExporter::MissingFileError, with: :render_zip_missing_file
|
|
||||||
|
|
||||||
def index
|
def index
|
||||||
page = (params[:page].presence || 1).to_i
|
page = (params[:page].presence || 1).to_i
|
||||||
limit = (params[:limit].presence || 20).to_i
|
limit = (params[:limit].presence || 20).to_i
|
||||||
@@ -15,98 +8,54 @@ class MaterialsController < ApplicationController
|
|||||||
|
|
||||||
offset = (page - 1) * limit
|
offset = (page - 1) * limit
|
||||||
|
|
||||||
filters = material_index_filters
|
tag_id = params[:tag_id].presence
|
||||||
tag_graph = material_index_tag_graph(filters)
|
parent_id = params[:parent_id].presence
|
||||||
q = Material.includes(:material_export_items,
|
|
||||||
thumbnail_attachment: :blob,
|
|
||||||
file_attachment: :blob,
|
|
||||||
tag: :tag_name)
|
|
||||||
q = q.where(tag_id: nil) if filters[:tag_state] == 'untagged'
|
|
||||||
q = q.where.not(tag_id: nil) if filters[:tag_state] == 'tagged'
|
|
||||||
q = apply_material_tag_filter(q, filters, tag_graph)
|
|
||||||
q = q.where('materials.created_at >= ?', filters[:created_from]) if filters[:created_from]
|
|
||||||
q = q.where('materials.created_at <= ?', filters[:created_to]) if filters[:created_to]
|
|
||||||
q = q.where('materials.updated_at >= ?', filters[:updated_from]) if filters[:updated_from]
|
|
||||||
q = q.where('materials.updated_at <= ?', filters[:updated_to]) if filters[:updated_to]
|
|
||||||
|
|
||||||
q = material_index_join_tag_name(q) if material_index_needs_tag_name?(filters)
|
q = Material.includes(:tag, :created_by_user).with_attached_file
|
||||||
q = material_index_join_file_blob(q) if material_index_needs_file_blob?(filters)
|
q = q.where(tag_id:) if tag_id
|
||||||
q = apply_material_query(q, filters[:q]) if filters[:q].present?
|
q = q.where(parent_id:) if parent_id
|
||||||
q = apply_material_media_kind(q, filters[:media_kind])
|
|
||||||
|
|
||||||
count = q.distinct.count(:id)
|
count = q.count
|
||||||
materials =
|
materials = q.order(created_at: :desc, id: :desc).limit(limit).offset(offset)
|
||||||
q
|
|
||||||
.order(Arel.sql(material_index_order_sql(filters)))
|
|
||||||
.limit(limit)
|
|
||||||
.offset(offset)
|
|
||||||
.to_a
|
|
||||||
|
|
||||||
response = { materials: MaterialRepr.list_many(materials, host: request.base_url),
|
render json: { materials: MaterialRepr.many(materials, host: request.base_url), count: count }
|
||||||
count: }
|
|
||||||
if filters[:tag_id]
|
|
||||||
response[:tag_scope] = material_index_tag_scope(filters, tag_graph)
|
|
||||||
end
|
|
||||||
if filters[:group_by] == 'parent_tag'
|
|
||||||
response[:groups] = material_index_groups(materials, tag_graph)
|
|
||||||
end
|
|
||||||
|
|
||||||
render json: response
|
|
||||||
end
|
end
|
||||||
|
|
||||||
def show
|
def show
|
||||||
material =
|
material =
|
||||||
Material
|
Material
|
||||||
.includes(:tag, :material_export_items)
|
.includes(:tag)
|
||||||
.with_attached_thumbnail
|
|
||||||
.with_attached_file
|
.with_attached_file
|
||||||
.find_by(id: params[:id])
|
.find_by(id: params[:id])
|
||||||
return head :not_found unless material
|
return head :not_found unless material
|
||||||
|
|
||||||
wiki_page_body = material.tag&.tag_name&.wiki_page&.current_revision&.body
|
wiki_page_body = material.tag.tag_name.wiki_page&.current_revision&.body
|
||||||
|
|
||||||
render json: MaterialRepr.base(material, host: request.base_url).merge(wiki_page_body:)
|
render json: MaterialRepr.base(material, host: request.base_url).merge(wiki_page_body:)
|
||||||
end
|
end
|
||||||
|
|
||||||
def create
|
def create
|
||||||
return head :unauthorized unless current_user
|
return head :unauthorized unless current_user
|
||||||
return head :forbidden unless current_user.gte_member?
|
|
||||||
|
|
||||||
tag_name_raw = params[:tag].to_s.strip
|
tag_name_raw = params[:tag].to_s.strip
|
||||||
file = params[:file]
|
file = params[:file]
|
||||||
file_sha256 = MaterialFileSha256.from_upload(file)
|
|
||||||
url = params[:url].to_s.strip.presence
|
url = params[:url].to_s.strip.presence
|
||||||
return render_unprocessable_entity('タグは必須です.', field: :tag) if tag_name_raw.blank?
|
return render_unprocessable_entity('タグは必須です.', field: :tag) if tag_name_raw.blank?
|
||||||
if file.blank? && url.blank?
|
if file.blank? && url.blank?
|
||||||
return render_validation_error fields: { file: ['ファイルまたは URL は必須です.'],
|
return render_validation_error fields: { file: ['ファイルまたは URL は必須です.'],
|
||||||
url: ['ファイルまたは URL は必須です.'] }
|
url: ['ファイルまたは URL は必須です.'] }
|
||||||
end
|
end
|
||||||
block = MaterialImportBlockMatcher.match_for_sha256(file_sha256)
|
|
||||||
return render_material_import_block(block) if block
|
|
||||||
|
|
||||||
uploaded_blob = build_uploaded_material_blob!(file, file_sha256)
|
tag_name = TagName.find_undiscard_or_create_by!(name: tag_name_raw)
|
||||||
material = nil
|
tag = tag_name.tag
|
||||||
|
tag = Tag.create!(tag_name:, category: :material) unless tag
|
||||||
|
|
||||||
begin
|
material = Material.new(tag:, url:,
|
||||||
Material.transaction do
|
created_by_user: current_user,
|
||||||
tag = resolve_material_tag!(tag_name_raw)
|
updated_by_user: current_user)
|
||||||
material = Material.new(tag:, url:,
|
material.file.attach(file)
|
||||||
created_by_user: current_user,
|
|
||||||
updated_by_user: current_user)
|
|
||||||
material.file.attach(uploaded_blob) if uploaded_blob
|
|
||||||
material.save!
|
|
||||||
TagVersioning.record_tag_snapshot!(tag, created_by_user: current_user)
|
|
||||||
upsert_export_paths!(material)
|
|
||||||
MaterialVersionRecorder.record!(material:, event_type: :create,
|
|
||||||
created_by_user: current_user)
|
|
||||||
end
|
|
||||||
rescue StandardError
|
|
||||||
uploaded_blob&.purge_later
|
|
||||||
raise
|
|
||||||
end
|
|
||||||
|
|
||||||
if material
|
if material.save
|
||||||
log_thumbnail_generation(material, MaterialThumbnailGenerator.generate!(material))
|
|
||||||
render json: MaterialRepr.base(material, host: request.base_url), status: :created
|
render json: MaterialRepr.base(material, host: request.base_url), status: :created
|
||||||
else
|
else
|
||||||
render_validation_error material
|
render_validation_error material
|
||||||
@@ -122,42 +71,29 @@ class MaterialsController < ApplicationController
|
|||||||
|
|
||||||
tag_name_raw = params[:tag].to_s.strip
|
tag_name_raw = params[:tag].to_s.strip
|
||||||
file = params[:file]
|
file = params[:file]
|
||||||
file_sha256 = MaterialFileSha256.from_upload(file)
|
url = params[:url].to_s.strip.presence
|
||||||
url = params.key?(:url) ? params[:url].to_s.strip.presence : material.url
|
|
||||||
return render_unprocessable_entity('タグは必須です.', field: :tag) if tag_name_raw.blank?
|
return render_unprocessable_entity('タグは必須です.', field: :tag) if tag_name_raw.blank?
|
||||||
if file.blank? && url.blank? && !material.file.attached?
|
if file.blank? && url.blank?
|
||||||
return render_validation_error fields: { file: ['ファイルまたは URL は必須です.'],
|
return render_validation_error fields: { file: ['ファイルまたは URL は必須です.'],
|
||||||
url: ['ファイルまたは URL は必須です.'] }
|
url: ['ファイルまたは URL は必須です.'] }
|
||||||
end
|
end
|
||||||
block = MaterialImportBlockMatcher.match_for_sha256(file_sha256)
|
|
||||||
return render_material_import_block(block) if block
|
|
||||||
|
|
||||||
uploaded_blob = build_uploaded_material_blob!(file, file_sha256)
|
tag_name = TagName.find_undiscard_or_create_by!(name: tag_name_raw)
|
||||||
|
tag = tag_name.tag
|
||||||
|
tag = Tag.create!(tag_name:, category: :material) unless tag
|
||||||
|
|
||||||
begin
|
material.update!(tag:, url:, updated_by_user: current_user)
|
||||||
Material.transaction do
|
if file
|
||||||
MaterialVersionRecorder.ensure_snapshot!(material, created_by_user: current_user)
|
material.file.attach(file)
|
||||||
tag = resolve_material_tag!(tag_name_raw)
|
else
|
||||||
material.assign_attributes(tag:, url:, updated_by_user: current_user)
|
material.file.purge
|
||||||
if uploaded_blob
|
|
||||||
material.file.attach(uploaded_blob)
|
|
||||||
elsif params.key?(:url) && url.present? && file.blank?
|
|
||||||
material.file.detach
|
|
||||||
end
|
|
||||||
material.save!
|
|
||||||
TagVersioning.record_tag_snapshot!(tag, created_by_user: current_user)
|
|
||||||
upsert_export_paths!(material)
|
|
||||||
MaterialVersionRecorder.record!(material:, event_type: :update,
|
|
||||||
created_by_user: current_user)
|
|
||||||
end
|
|
||||||
rescue StandardError
|
|
||||||
uploaded_blob&.purge_later
|
|
||||||
raise
|
|
||||||
end
|
end
|
||||||
|
|
||||||
log_thumbnail_generation(material, MaterialThumbnailGenerator.generate!(material))
|
if material.save
|
||||||
|
render json: MaterialRepr.base(material, host: request.base_url)
|
||||||
render json: MaterialRepr.base(material, host: request.base_url)
|
else
|
||||||
|
render_validation_error material
|
||||||
|
end
|
||||||
end
|
end
|
||||||
|
|
||||||
def destroy
|
def destroy
|
||||||
@@ -167,381 +103,7 @@ class MaterialsController < ApplicationController
|
|||||||
material = Material.find_by(id: params[:id])
|
material = Material.find_by(id: params[:id])
|
||||||
return head :not_found unless material
|
return head :not_found unless material
|
||||||
|
|
||||||
Material.transaction do
|
material.discard
|
||||||
MaterialVersionRecorder.ensure_snapshot!(material, created_by_user: current_user)
|
|
||||||
material.discard!
|
|
||||||
MaterialVersionRecorder.record!(material:, event_type: :discard,
|
|
||||||
created_by_user: current_user)
|
|
||||||
end
|
|
||||||
head :no_content
|
head :no_content
|
||||||
end
|
end
|
||||||
|
|
||||||
def download
|
|
||||||
zip = MaterialZipExporter.new(profile: params[:profile],
|
|
||||||
tag_id: params[:tag_id]).export
|
|
||||||
profile = params[:profile].presence || 'legacy_drive'
|
|
||||||
send_data zip,
|
|
||||||
type: 'application/zip',
|
|
||||||
disposition: 'attachment',
|
|
||||||
filename: "btrc-materials-#{ profile }.zip"
|
|
||||||
end
|
|
||||||
|
|
||||||
private
|
|
||||||
|
|
||||||
def material_index_filters
|
|
||||||
tag_state = params[:tag_state].to_s.presence
|
|
||||||
tag_state = 'untagged' if bool?(:unclassified)
|
|
||||||
tag_state = 'all' unless ['all', 'tagged', 'untagged'].include?(tag_state)
|
|
||||||
|
|
||||||
media_kind = params[:media_kind].to_s.presence
|
|
||||||
unless ['all', 'image', 'video', 'audio', 'file_other', 'url_only'].include?(media_kind)
|
|
||||||
media_kind = 'all'
|
|
||||||
end
|
|
||||||
|
|
||||||
sort = params[:sort].to_s.presence
|
|
||||||
unless ['created_at', 'updated_at', 'tag_name', 'media_kind', 'file_byte_size',
|
|
||||||
'version_no', 'id'].include?(sort)
|
|
||||||
sort = 'created_at'
|
|
||||||
end
|
|
||||||
|
|
||||||
direction = params[:direction].to_s.downcase
|
|
||||||
direction = 'desc' unless ['asc', 'desc'].include?(direction)
|
|
||||||
|
|
||||||
group_by = params[:group_by].to_s.presence
|
|
||||||
group_by = 'none' unless ['none', 'parent_tag'].include?(group_by)
|
|
||||||
|
|
||||||
tag_id = params[:tag_id].to_i
|
|
||||||
tag_id = nil if tag_id <= 0
|
|
||||||
|
|
||||||
{ q: params[:q].to_s.strip.presence,
|
|
||||||
tag_state:,
|
|
||||||
media_kind:,
|
|
||||||
tag_id:,
|
|
||||||
include_descendants: bool?(:include_descendants),
|
|
||||||
group_by:,
|
|
||||||
created_from: parse_time_param(:created_from),
|
|
||||||
created_to: parse_time_param(:created_to),
|
|
||||||
updated_from: parse_time_param(:updated_from),
|
|
||||||
updated_to: parse_time_param(:updated_to),
|
|
||||||
sort:,
|
|
||||||
direction: }
|
|
||||||
end
|
|
||||||
|
|
||||||
def parse_time_param name
|
|
||||||
value = params[name].to_s.strip
|
|
||||||
return nil if value.blank?
|
|
||||||
|
|
||||||
Time.zone.parse(value)
|
|
||||||
rescue ArgumentError
|
|
||||||
nil
|
|
||||||
end
|
|
||||||
|
|
||||||
def resolve_material_tag! tag_name_raw
|
|
||||||
tag_name = TagName.find_undiscard_or_create_by!(name: tag_name_raw)
|
|
||||||
tag = tag_name.tag
|
|
||||||
tag || Tag.create!(tag_name:, category: :material)
|
|
||||||
end
|
|
||||||
|
|
||||||
def material_index_needs_tag_name? filters
|
|
||||||
filters[:q].present? || filters[:sort] == 'tag_name'
|
|
||||||
end
|
|
||||||
|
|
||||||
def material_index_needs_file_blob? filters
|
|
||||||
filters[:q].present? ||
|
|
||||||
filters[:media_kind] != 'all' ||
|
|
||||||
['media_kind', 'file_byte_size'].include?(filters[:sort])
|
|
||||||
end
|
|
||||||
|
|
||||||
def material_index_join_tag_name q
|
|
||||||
q.left_joins(tag: :tag_name)
|
|
||||||
end
|
|
||||||
|
|
||||||
def material_index_join_file_blob q
|
|
||||||
q.joins(<<~SQL.squish)
|
|
||||||
LEFT JOIN active_storage_attachments material_file_attachments
|
|
||||||
ON material_file_attachments.record_type = 'Material'
|
|
||||||
AND material_file_attachments.record_id = materials.id
|
|
||||||
AND material_file_attachments.name = 'file'
|
|
||||||
LEFT JOIN active_storage_blobs material_file_blobs
|
|
||||||
ON material_file_blobs.id = material_file_attachments.blob_id
|
|
||||||
SQL
|
|
||||||
end
|
|
||||||
|
|
||||||
def apply_material_query q, term
|
|
||||||
like = "%#{ ActiveRecord::Base.sanitize_sql_like(term) }%"
|
|
||||||
q.where('tag_names.name LIKE :q OR materials.url LIKE :q OR ' \
|
|
||||||
'material_file_blobs.filename LIKE :q',
|
|
||||||
q: like)
|
|
||||||
end
|
|
||||||
|
|
||||||
def apply_material_media_kind q, media_kind
|
|
||||||
case media_kind
|
|
||||||
when 'image'
|
|
||||||
q.where('material_file_blobs.content_type LIKE ?', 'image/%')
|
|
||||||
when 'video'
|
|
||||||
q.where('material_file_blobs.content_type LIKE ?', 'video/%')
|
|
||||||
when 'audio'
|
|
||||||
q.where('material_file_blobs.content_type LIKE ?', 'audio/%')
|
|
||||||
when 'file_other'
|
|
||||||
q.where('material_file_attachments.id IS NOT NULL')
|
|
||||||
.where.not('material_file_blobs.content_type LIKE ?', 'image/%')
|
|
||||||
.where.not('material_file_blobs.content_type LIKE ?', 'video/%')
|
|
||||||
.where.not('material_file_blobs.content_type LIKE ?', 'audio/%')
|
|
||||||
when 'url_only'
|
|
||||||
q.where('material_file_attachments.id IS NULL').where.not(url: [nil, ''])
|
|
||||||
else
|
|
||||||
q
|
|
||||||
end
|
|
||||||
end
|
|
||||||
|
|
||||||
def apply_material_tag_filter q, filters, tag_graph
|
|
||||||
return q unless filters[:tag_id]
|
|
||||||
|
|
||||||
tag_ids =
|
|
||||||
if tag_graph
|
|
||||||
tag_graph[:scope_tag_ids]
|
|
||||||
else
|
|
||||||
[filters[:tag_id]]
|
|
||||||
end
|
|
||||||
|
|
||||||
q.where(tag_id: tag_ids)
|
|
||||||
end
|
|
||||||
|
|
||||||
def material_index_tag_graph filters
|
|
||||||
return nil unless filters[:tag_id]
|
|
||||||
|
|
||||||
children_by_parent_id = Hash.new { |h, k| h[k] = [] }
|
|
||||||
TagImplication.pluck(:parent_tag_id, :tag_id).each do |parent_id, child_id|
|
|
||||||
children_by_parent_id[parent_id] << child_id
|
|
||||||
end
|
|
||||||
|
|
||||||
scope_tag_ids =
|
|
||||||
if filters[:include_descendants]
|
|
||||||
collect_material_descendant_ids(filters[:tag_id], children_by_parent_id)
|
|
||||||
else
|
|
||||||
Set.new([filters[:tag_id]])
|
|
||||||
end
|
|
||||||
tags_by_id =
|
|
||||||
Tag
|
|
||||||
.joins(:tag_name)
|
|
||||||
.where(id: scope_tag_ids)
|
|
||||||
.pluck('tags.id', 'tag_names.name', 'tags.category', 'tags.deprecated_at')
|
|
||||||
.each_with_object({ }) do |(id, name, category, deprecated_at), hash|
|
|
||||||
hash[id] = { id:, name:, category:, deprecated: deprecated_at.present? }
|
|
||||||
end
|
|
||||||
|
|
||||||
selected_tag = tags_by_id[filters[:tag_id]]
|
|
||||||
return nil unless selected_tag
|
|
||||||
|
|
||||||
group_tag_id_by_tag_id = { }
|
|
||||||
children_by_parent_id[filters[:tag_id]].each do |child_tag_id|
|
|
||||||
assign_material_group_branch(child_tag_id,
|
|
||||||
children_by_parent_id:,
|
|
||||||
tags_by_id:,
|
|
||||||
group_tag_id_by_tag_id:,
|
|
||||||
current_group_tag_id: nil,
|
|
||||||
seen: Set.new([filters[:tag_id]]))
|
|
||||||
end
|
|
||||||
|
|
||||||
{ selected_tag_id: filters[:tag_id],
|
|
||||||
selected_tag:,
|
|
||||||
scope_tag_ids: scope_tag_ids.to_a,
|
|
||||||
tags_by_id:,
|
|
||||||
group_tag_id_by_tag_id: }
|
|
||||||
end
|
|
||||||
|
|
||||||
def collect_material_descendant_ids selected_tag_id, children_by_parent_id
|
|
||||||
ids = Set.new([selected_tag_id])
|
|
||||||
stack = [selected_tag_id]
|
|
||||||
|
|
||||||
until stack.empty?
|
|
||||||
tag_id = stack.pop
|
|
||||||
children_by_parent_id[tag_id].each do |child_tag_id|
|
|
||||||
next if ids.include?(child_tag_id)
|
|
||||||
|
|
||||||
ids << child_tag_id
|
|
||||||
stack << child_tag_id
|
|
||||||
end
|
|
||||||
end
|
|
||||||
|
|
||||||
ids
|
|
||||||
end
|
|
||||||
|
|
||||||
def assign_material_group_branch tag_id, children_by_parent_id:, tags_by_id:,
|
|
||||||
group_tag_id_by_tag_id:, current_group_tag_id:, seen:
|
|
||||||
return if seen.include?(tag_id)
|
|
||||||
|
|
||||||
tag = tags_by_id[tag_id]
|
|
||||||
return unless tag
|
|
||||||
|
|
||||||
seen = seen.dup << tag_id
|
|
||||||
next_group_tag_id = current_group_tag_id
|
|
||||||
next_group_tag_id = tag_id if next_group_tag_id.nil? && !tag[:deprecated]
|
|
||||||
group_tag_id_by_tag_id[tag_id] = next_group_tag_id if next_group_tag_id
|
|
||||||
|
|
||||||
children_by_parent_id[tag_id].each do |child_tag_id|
|
|
||||||
assign_material_group_branch(child_tag_id,
|
|
||||||
children_by_parent_id:,
|
|
||||||
tags_by_id:,
|
|
||||||
group_tag_id_by_tag_id:,
|
|
||||||
current_group_tag_id: next_group_tag_id,
|
|
||||||
seen:)
|
|
||||||
end
|
|
||||||
end
|
|
||||||
|
|
||||||
def material_index_groups materials, tag_graph
|
|
||||||
return [] unless tag_graph
|
|
||||||
|
|
||||||
groups_by_key = { }
|
|
||||||
|
|
||||||
materials.each do |material|
|
|
||||||
next unless material.tag_id
|
|
||||||
|
|
||||||
group_tag_id =
|
|
||||||
if material.tag_id == tag_graph[:selected_tag_id]
|
|
||||||
tag_graph[:selected_tag_id]
|
|
||||||
else
|
|
||||||
tag_graph[:group_tag_id_by_tag_id][material.tag_id] || tag_graph[:selected_tag_id]
|
|
||||||
end
|
|
||||||
|
|
||||||
group_tag = tag_graph[:tags_by_id][group_tag_id]
|
|
||||||
next unless group_tag
|
|
||||||
|
|
||||||
key = "tag:#{ group_tag_id }"
|
|
||||||
group =
|
|
||||||
groups_by_key[key] ||= { key:,
|
|
||||||
tag: group_tag_repr(group_tag),
|
|
||||||
material_ids: [],
|
|
||||||
count: 0 }
|
|
||||||
group[:material_ids] << material.id
|
|
||||||
group[:count] += 1
|
|
||||||
end
|
|
||||||
|
|
||||||
groups_by_key.values.sort_by { |group| group[:tag][:name] }
|
|
||||||
end
|
|
||||||
|
|
||||||
def material_index_tag_scope filters, tag_graph
|
|
||||||
return nil unless tag_graph
|
|
||||||
|
|
||||||
{
|
|
||||||
tag: group_tag_repr(tag_graph[:selected_tag]),
|
|
||||||
include_descendants: filters[:include_descendants]
|
|
||||||
}
|
|
||||||
end
|
|
||||||
|
|
||||||
def group_tag_repr group_tag
|
|
||||||
{ id: group_tag[:id],
|
|
||||||
name: group_tag[:name],
|
|
||||||
category: group_tag[:category] }
|
|
||||||
end
|
|
||||||
|
|
||||||
def material_index_order_sql filters
|
|
||||||
direction = filters[:direction] == 'asc' ? 'ASC' : 'DESC'
|
|
||||||
sort_sql =
|
|
||||||
case filters[:sort]
|
|
||||||
when 'tag_name'
|
|
||||||
'tag_names.name'
|
|
||||||
when 'media_kind'
|
|
||||||
material_media_kind_sql
|
|
||||||
when 'file_byte_size'
|
|
||||||
'material_file_blobs.byte_size'
|
|
||||||
when 'updated_at'
|
|
||||||
'materials.updated_at'
|
|
||||||
when 'version_no'
|
|
||||||
'materials.version_no'
|
|
||||||
when 'id'
|
|
||||||
'materials.id'
|
|
||||||
else
|
|
||||||
'materials.created_at'
|
|
||||||
end
|
|
||||||
|
|
||||||
"#{ sort_sql } #{ direction }, materials.id #{ direction }"
|
|
||||||
end
|
|
||||||
|
|
||||||
def material_media_kind_sql
|
|
||||||
"CASE " \
|
|
||||||
"WHEN material_file_attachments.id IS NULL AND materials.url IS NOT NULL THEN 5 " \
|
|
||||||
"WHEN material_file_blobs.content_type LIKE 'image/%' THEN 1 " \
|
|
||||||
"WHEN material_file_blobs.content_type LIKE 'video/%' THEN 2 " \
|
|
||||||
"WHEN material_file_blobs.content_type LIKE 'audio/%' THEN 3 " \
|
|
||||||
"WHEN material_file_attachments.id IS NOT NULL THEN 4 " \
|
|
||||||
"ELSE 6 END"
|
|
||||||
end
|
|
||||||
|
|
||||||
def upsert_export_paths! material
|
|
||||||
raw = params[:export_paths]
|
|
||||||
return if raw.blank?
|
|
||||||
|
|
||||||
export_paths = raw.respond_to?(:to_unsafe_h) ? raw.to_unsafe_h : raw.to_h
|
|
||||||
export_paths.each do |profile, export_path|
|
|
||||||
profile = profile.to_s
|
|
||||||
export_path = export_path.to_s.strip
|
|
||||||
item = material.material_export_items.find_or_initialize_by(profile:)
|
|
||||||
|
|
||||||
if export_path.blank?
|
|
||||||
item.destroy! if item.persisted?
|
|
||||||
next
|
|
||||||
end
|
|
||||||
|
|
||||||
item.export_path = export_path
|
|
||||||
item.enabled = true
|
|
||||||
item.created_by_user ||= current_user
|
|
||||||
item.save!
|
|
||||||
end
|
|
||||||
end
|
|
||||||
|
|
||||||
def render_zip_empty
|
|
||||||
render_unprocessable_entity('ZIP export 対象の素材がありません.')
|
|
||||||
end
|
|
||||||
|
|
||||||
def render_zip_duplicate_path error
|
|
||||||
render_unprocessable_entity("ZIP export path が重複してゐます: #{ error.message }")
|
|
||||||
end
|
|
||||||
|
|
||||||
def render_zip_missing_file error
|
|
||||||
missing_files = error.missing_files.map do |missing_file|
|
|
||||||
{ material_id: missing_file.material_id,
|
|
||||||
export_path: missing_file.export_path,
|
|
||||||
blob_id: missing_file.blob_id,
|
|
||||||
filename: missing_file.filename }
|
|
||||||
end
|
|
||||||
|
|
||||||
render json: { type: 'validation_error',
|
|
||||||
message: 'ZIP export に必要な素材ファイルが欠損しています.',
|
|
||||||
errors: { },
|
|
||||||
base_errors: ['ZIP export に必要な素材ファイルが欠損しています.'],
|
|
||||||
missing_files: },
|
|
||||||
status: :unprocessable_entity
|
|
||||||
end
|
|
||||||
|
|
||||||
def build_uploaded_material_blob! file, file_sha256
|
|
||||||
return nil unless file
|
|
||||||
|
|
||||||
file.tempfile.rewind
|
|
||||||
|
|
||||||
blob = ActiveStorage::Blob.create_and_upload!(
|
|
||||||
io: file.tempfile,
|
|
||||||
filename: file.original_filename,
|
|
||||||
content_type: file.content_type,
|
|
||||||
)
|
|
||||||
|
|
||||||
MaterialFileSha256.assign_metadata_sha256!(blob, file_sha256)
|
|
||||||
|
|
||||||
blob
|
|
||||||
ensure
|
|
||||||
file.tempfile.rewind if file&.tempfile
|
|
||||||
end
|
|
||||||
|
|
||||||
def render_material_import_block block
|
|
||||||
render_validation_error fields: { file: ["抑止された素材です: #{ block.reason }"] }
|
|
||||||
end
|
|
||||||
|
|
||||||
def log_thumbnail_generation material, result
|
|
||||||
material.reload
|
|
||||||
Rails.logger.info(
|
|
||||||
"Material thumbnail generation: material_id=#{ material.id } " \
|
|
||||||
"result=#{ result } " \
|
|
||||||
"thumbnail_attached=#{ material.thumbnail.attached? } " \
|
|
||||||
"content_type=#{ material.content_type }")
|
|
||||||
end
|
|
||||||
end
|
end
|
||||||
|
|||||||
@@ -1,44 +0,0 @@
|
|||||||
class PostImportsController < ApplicationController
|
|
||||||
before_action :require_member!
|
|
||||||
|
|
||||||
def preview
|
|
||||||
rows = PostImportPreviewer.new.preview_rows(
|
|
||||||
rows: PostImportUrlListParser.parse(params[:source]))
|
|
||||||
render json: { rows: }
|
|
||||||
rescue ArgumentError => e
|
|
||||||
render_bad_request e.message
|
|
||||||
end
|
|
||||||
|
|
||||||
def validate
|
|
||||||
rows = normalised_import_rows allow_warning_fields: true
|
|
||||||
changed_row = Integer(params[:changed_row], exception: false)
|
|
||||||
result =
|
|
||||||
PostImportPreviewer.new.preview_rows(rows:,
|
|
||||||
fetch_metadata: changed_row,
|
|
||||||
metadata_cache: { })
|
|
||||||
render json: { rows: result }
|
|
||||||
rescue ArgumentError => e
|
|
||||||
render_bad_request e.message
|
|
||||||
end
|
|
||||||
|
|
||||||
def create
|
|
||||||
result = PostImportRunner.new(actor: current_user,
|
|
||||||
rows: normalised_import_rows).run
|
|
||||||
render json: result, status: result[:created].positive? ? :created : :ok
|
|
||||||
rescue ArgumentError => e
|
|
||||||
render_bad_request e.message
|
|
||||||
end
|
|
||||||
|
|
||||||
private
|
|
||||||
|
|
||||||
def require_member!
|
|
||||||
return head :unauthorized unless current_user
|
|
||||||
return if current_user.gte_member?
|
|
||||||
|
|
||||||
head :forbidden
|
|
||||||
end
|
|
||||||
|
|
||||||
def normalised_import_rows allow_warning_fields: false
|
|
||||||
PostImportRowNormaliser.normalise!(params[:rows], allow_warning_fields:)
|
|
||||||
end
|
|
||||||
end
|
|
||||||
@@ -25,7 +25,6 @@ class PostVersionsController < ApplicationController
|
|||||||
SQL
|
SQL
|
||||||
.select('post_versions.*', 'prev.title AS prev_title', 'prev.url AS prev_url',
|
.select('post_versions.*', 'prev.title AS prev_title', 'prev.url AS prev_url',
|
||||||
'prev.thumbnail_base AS prev_thumbnail_base', 'prev.tags AS prev_tags',
|
'prev.thumbnail_base AS prev_thumbnail_base', 'prev.tags AS prev_tags',
|
||||||
'prev.video_ms AS prev_video_ms',
|
|
||||||
'prev.original_created_from AS prev_original_created_from',
|
'prev.original_created_from AS prev_original_created_from',
|
||||||
'prev.original_created_before AS prev_original_created_before')
|
'prev.original_created_before AS prev_original_created_before')
|
||||||
q = q.where('post_versions.post_id = ?', post_id) if post_id
|
q = q.where('post_versions.post_id = ?', post_id) if post_id
|
||||||
@@ -75,10 +74,6 @@ class PostVersionsController < ApplicationController
|
|||||||
current: row.thumbnail_base,
|
current: row.thumbnail_base,
|
||||||
prev: row.attributes['prev_thumbnail_base']
|
prev: row.attributes['prev_thumbnail_base']
|
||||||
},
|
},
|
||||||
video_ms: {
|
|
||||||
current: row.video_ms,
|
|
||||||
prev: row.attributes['prev_video_ms']
|
|
||||||
},
|
|
||||||
tags: build_version_tags(cur_tags, prev_tags),
|
tags: build_version_tags(cur_tags, prev_tags),
|
||||||
original_created_from: {
|
original_created_from: {
|
||||||
current: row.original_created_from&.iso8601,
|
current: row.original_created_from&.iso8601,
|
||||||
|
|||||||
@@ -1,10 +1,6 @@
|
|||||||
class PostsController < ApplicationController
|
class PostsController < ApplicationController
|
||||||
Event = Struct.new(:post, :tag, :user, :change_type, :timestamp, keyword_init: true)
|
Event = Struct.new(:post, :tag, :user, :change_type, :timestamp, keyword_init: true)
|
||||||
|
|
||||||
class VideoMsParseError < ArgumentError
|
|
||||||
;
|
|
||||||
end
|
|
||||||
|
|
||||||
def index
|
def index
|
||||||
url = params[:url].presence
|
url = params[:url].presence
|
||||||
title = params[:title].presence
|
title = params[:title].presence
|
||||||
@@ -49,9 +45,7 @@ class PostsController < ApplicationController
|
|||||||
.joins("LEFT JOIN (#{ pt_max_sql }) pt_max ON pt_max.post_id = posts.id")
|
.joins("LEFT JOIN (#{ pt_max_sql }) pt_max ON pt_max.post_id = posts.id")
|
||||||
.reselect('posts.*', Arel.sql("#{ updated_at_all_sql } AS updated_at_all"))
|
.reselect('posts.*', Arel.sql("#{ updated_at_all_sql } AS updated_at_all"))
|
||||||
.preload(:uploaded_user, :parents, :children,
|
.preload(:uploaded_user, :parents, :children,
|
||||||
active_post_tags: [:sections,
|
tags: [:deerjikists, :materials, { tag_name: :wiki_page }])
|
||||||
{ tag: [:deerjikists, :materials,
|
|
||||||
{ tag_name: :wiki_page }] }])
|
|
||||||
.with_attached_thumbnail
|
.with_attached_thumbnail
|
||||||
|
|
||||||
q = q.where('posts.url LIKE ?', "%#{ url }%") if url
|
q = q.where('posts.url LIKE ?', "%#{ url }%") if url
|
||||||
@@ -103,9 +97,7 @@ class PostsController < ApplicationController
|
|||||||
|
|
||||||
def random
|
def random
|
||||||
post = filtered_posts.preload(:uploaded_user, :parents, :children,
|
post = filtered_posts.preload(:uploaded_user, :parents, :children,
|
||||||
active_post_tags: [:sections,
|
tags: [:deerjikists, :materials, { tag_name: :wiki_page }])
|
||||||
{ tag: [:deerjikists, :materials,
|
|
||||||
{ tag_name: :wiki_page }] }])
|
|
||||||
.with_attached_thumbnail
|
.with_attached_thumbnail
|
||||||
.order('RAND()')
|
.order('RAND()')
|
||||||
.first
|
.first
|
||||||
@@ -118,9 +110,7 @@ class PostsController < ApplicationController
|
|||||||
post =
|
post =
|
||||||
Post
|
Post
|
||||||
.includes(:uploaded_user, :parents, :children,
|
.includes(:uploaded_user, :parents, :children,
|
||||||
active_post_tags: [:sections,
|
tags: [:deerjikists, :materials, { tag_name: :wiki_page }])
|
||||||
{ tag: [:deerjikists, :materials,
|
|
||||||
{ tag_name: :wiki_page }] }])
|
|
||||||
.with_attached_thumbnail
|
.with_attached_thumbnail
|
||||||
.find_by(id: params[:id])
|
.find_by(id: params[:id])
|
||||||
return head :not_found unless post
|
return head :not_found unless post
|
||||||
@@ -135,21 +125,41 @@ class PostsController < ApplicationController
|
|||||||
child_posts:,
|
child_posts:,
|
||||||
sibling_posts:,
|
sibling_posts:,
|
||||||
related:)
|
related:)
|
||||||
.merge(tags: build_tag_tree_for(post))
|
.merge(tags: build_tag_tree_for(post.tags))
|
||||||
end
|
end
|
||||||
|
|
||||||
def create
|
def create
|
||||||
return head :unauthorized unless current_user
|
return head :unauthorized unless current_user
|
||||||
return head :forbidden unless current_user.gte_member?
|
return head :forbidden unless current_user.gte_member?
|
||||||
|
|
||||||
post = PostCreator.new(actor: current_user,
|
# TODO: サイトに応じて thumbnail_base 設定
|
||||||
attributes: { title: params[:title], url: params[:url],
|
title = params[:title].presence
|
||||||
thumbnail: params[:thumbnail], tags: params[:tags],
|
url = params[:url]
|
||||||
original_created_from: params[:original_created_from],
|
thumbnail = params[:thumbnail]
|
||||||
original_created_before: params[:original_created_before],
|
tag_names = params[:tags].to_s.split
|
||||||
parent_post_ids: parse_parent_post_ids,
|
original_created_from = params[:original_created_from]
|
||||||
video_ms: params[:video_ms],
|
original_created_before = params[:original_created_before]
|
||||||
duration: params[:duration] }).create!
|
parent_post_ids = parse_parent_post_ids
|
||||||
|
|
||||||
|
post = Post.new(title:, url:, thumbnail_base: nil, uploaded_user: current_user,
|
||||||
|
original_created_from:, original_created_before:)
|
||||||
|
post.thumbnail.attach(thumbnail) if thumbnail.present?
|
||||||
|
|
||||||
|
ApplicationRecord.transaction do
|
||||||
|
post.save!
|
||||||
|
|
||||||
|
tags = Tag.normalise_tags!(tag_names, deny_deprecated: true)
|
||||||
|
TagVersioning.record_tag_snapshots!(tags, created_by_user: current_user)
|
||||||
|
|
||||||
|
tags = Tag.expand_parent_tags(tags).reject(&:deprecated?)
|
||||||
|
sync_post_tags!(post, tags)
|
||||||
|
|
||||||
|
sync_parent_posts!(post, parent_post_ids)
|
||||||
|
|
||||||
|
post.resized_thumbnail!
|
||||||
|
|
||||||
|
PostVersionRecorder.record!(post:, event_type: :create, created_by_user: current_user)
|
||||||
|
end
|
||||||
|
|
||||||
post.reload
|
post.reload
|
||||||
render json: PostRepr.base(post), status: :created
|
render json: PostRepr.base(post), status: :created
|
||||||
@@ -157,12 +167,6 @@ class PostsController < ApplicationController
|
|||||||
render_validation_error fields: { tags: 'ニコニコ・タグは直接指定できません.' }
|
render_validation_error fields: { tags: 'ニコニコ・タグは直接指定できません.' }
|
||||||
rescue Tag::DeprecatedTagNormalisationError
|
rescue Tag::DeprecatedTagNormalisationError
|
||||||
render_unprocessable_entity '廃止済みタグは付与できません.', field: :tags
|
render_unprocessable_entity '廃止済みタグは付与できません.', field: :tags
|
||||||
rescue Tag::SectionLiteralParseError
|
|
||||||
render_validation_error fields: { tags: ['タグ区間の記法が不正です.'] }
|
|
||||||
rescue PostCreator::VideoMsParseError
|
|
||||||
render_validation_error fields: { video_ms: ['動画時間の記法が不正です.'] }
|
|
||||||
rescue MiniMagick::Error
|
|
||||||
render_validation_error fields: { thumbnail: ['サムネイル画像の変換に失敗しました.'] }
|
|
||||||
rescue ArgumentError => e
|
rescue ArgumentError => e
|
||||||
render_validation_error fields: { parent_post_ids: [e.message] }
|
render_validation_error fields: { parent_post_ids: [e.message] }
|
||||||
rescue ActiveRecord::RecordInvalid => e
|
rescue ActiveRecord::RecordInvalid => e
|
||||||
@@ -219,8 +223,6 @@ class PostsController < ApplicationController
|
|||||||
original_created_from:,
|
original_created_from:,
|
||||||
original_created_before:,
|
original_created_before:,
|
||||||
tag_names:,
|
tag_names:,
|
||||||
video_ms_param: params[:video_ms],
|
|
||||||
duration_param: params[:duration],
|
|
||||||
parent_post_ids:)
|
parent_post_ids:)
|
||||||
|
|
||||||
snapshot_to_apply =
|
snapshot_to_apply =
|
||||||
@@ -251,16 +253,12 @@ class PostsController < ApplicationController
|
|||||||
|
|
||||||
post.reload
|
post.reload
|
||||||
json = PostRepr.base(post, current_user)
|
json = PostRepr.base(post, current_user)
|
||||||
json['tags'] = build_tag_tree_for(post)
|
json['tags'] = build_tag_tree_for(post.tags)
|
||||||
render json:, status: :ok
|
render json:, status: :ok
|
||||||
rescue Tag::NicoTagNormalisationError
|
rescue Tag::NicoTagNormalisationError
|
||||||
render_validation_error fields: { tags: ['ニコニコ・タグは直接指定できません.'] }
|
render_validation_error fields: { tags: ['ニコニコ・タグは直接指定できません.'] }
|
||||||
rescue Tag::DeprecatedTagNormalisationError
|
rescue Tag::DeprecatedTagNormalisationError
|
||||||
render_unprocessable_entity '廃止済みタグは付与できません.', field: :tags
|
render_unprocessable_entity '廃止済みタグは付与できません.', field: :tags
|
||||||
rescue Tag::SectionLiteralParseError
|
|
||||||
render_validation_error fields: { tags: ['タグ区間の記法が不正です.'] }
|
|
||||||
rescue VideoMsParseError
|
|
||||||
render_validation_error fields: { video_ms: ['動画時間の記法が不正です.'] }
|
|
||||||
rescue ArgumentError => e
|
rescue ArgumentError => e
|
||||||
render_validation_error fields: { parent_post_ids: [e.message] }
|
render_validation_error fields: { parent_post_ids: [e.message] }
|
||||||
rescue ActiveRecord::RecordInvalid => e
|
rescue ActiveRecord::RecordInvalid => e
|
||||||
@@ -359,7 +357,7 @@ class PostsController < ApplicationController
|
|||||||
def tagged_post_ids_for(name) =
|
def tagged_post_ids_for(name) =
|
||||||
Post.joins(tags: :tag_name).where(tag_names: { name: }).select(:id)
|
Post.joins(tags: :tag_name).where(tag_names: { name: }).select(:id)
|
||||||
|
|
||||||
def sync_post_tags! post, desired_tags, sections
|
def sync_post_tags! post, desired_tags
|
||||||
desired_tags.each do |t|
|
desired_tags.each do |t|
|
||||||
t.save! if t.new_record?
|
t.save! if t.new_record?
|
||||||
end
|
end
|
||||||
@@ -378,21 +376,13 @@ class PostsController < ApplicationController
|
|||||||
end
|
end
|
||||||
end
|
end
|
||||||
|
|
||||||
PostTagSection.where(post_id: post.id).destroy_all
|
|
||||||
sections.each do |tag_id, ranges|
|
|
||||||
ranges.each do |begin_ms, end_ms|
|
|
||||||
PostTagSection.create!(post_id: post.id, tag_id:, begin_ms:, end_ms:)
|
|
||||||
end
|
|
||||||
end
|
|
||||||
|
|
||||||
PostTag.where(post_id: post.id, tag_id: to_remove.to_a).kept.find_each do |pt|
|
PostTag.where(post_id: post.id, tag_id: to_remove.to_a).kept.find_each do |pt|
|
||||||
pt.discard_by!(current_user)
|
pt.discard_by!(current_user)
|
||||||
end
|
end
|
||||||
end
|
end
|
||||||
|
|
||||||
def build_tag_tree_for post
|
def build_tag_tree_for tags
|
||||||
post_tags = post.active_post_tags.reject { |post_tag| post_tag.tag.deprecated? }
|
tags = tags.reject(&:deprecated?).to_a
|
||||||
tags = post_tags.map(&:tag)
|
|
||||||
tag_ids = tags.map(&:id)
|
tag_ids = tags.map(&:id)
|
||||||
|
|
||||||
implications = TagImplication.where(parent_tag_id: tag_ids, tag_id: tag_ids)
|
implications = TagImplication.where(parent_tag_id: tag_ids, tag_id: tag_ids)
|
||||||
@@ -407,9 +397,6 @@ class PostsController < ApplicationController
|
|||||||
root_ids = tag_ids - child_ids
|
root_ids = tag_ids - child_ids
|
||||||
|
|
||||||
tags_by_id = tags.index_by(&:id)
|
tags_by_id = tags.index_by(&:id)
|
||||||
sections_by_tag_id = post_tags.to_h { |post_tag|
|
|
||||||
[post_tag.tag_id, post_tag.sections.as_json(only: [:begin_ms, :end_ms])]
|
|
||||||
}
|
|
||||||
|
|
||||||
memo = { }
|
memo = { }
|
||||||
|
|
||||||
@@ -417,10 +404,8 @@ class PostsController < ApplicationController
|
|||||||
tag = tags_by_id[tag_id]
|
tag = tags_by_id[tag_id]
|
||||||
return nil unless tag
|
return nil unless tag
|
||||||
|
|
||||||
sections = sections_by_tag_id.fetch(tag_id, [])
|
|
||||||
|
|
||||||
if path.include?(tag_id)
|
if path.include?(tag_id)
|
||||||
return TagRepr.inline(tag).merge(children: [], sections:)
|
return TagRepr.inline(tag).merge(children: [])
|
||||||
end
|
end
|
||||||
|
|
||||||
if memo.key?(tag_id)
|
if memo.key?(tag_id)
|
||||||
@@ -432,7 +417,7 @@ class PostsController < ApplicationController
|
|||||||
|
|
||||||
children = child_ids.filter_map { |cid| build_node.(cid, new_path) }
|
children = child_ids.filter_map { |cid| build_node.(cid, new_path) }
|
||||||
|
|
||||||
memo[tag_id] = TagRepr.inline(tag).merge(children:, sections:)
|
memo[tag_id] = TagRepr.inline(tag).merge(children:)
|
||||||
end
|
end
|
||||||
|
|
||||||
root_ids.filter_map { |id| build_node.call(id, []) }
|
root_ids.filter_map { |id| build_node.call(id, []) }
|
||||||
@@ -501,7 +486,6 @@ class PostsController < ApplicationController
|
|||||||
|
|
||||||
def post_snapshot_from_version version
|
def post_snapshot_from_version version
|
||||||
{ title: version.title,
|
{ title: version.title,
|
||||||
video_ms: version.respond_to?(:video_ms) ? version.video_ms : nil,
|
|
||||||
original_created_from: snapshot_time(version.original_created_from),
|
original_created_from: snapshot_time(version.original_created_from),
|
||||||
original_created_before: snapshot_time(version.original_created_before),
|
original_created_before: snapshot_time(version.original_created_before),
|
||||||
tag_names: editable_tag_names_from_version(version),
|
tag_names: editable_tag_names_from_version(version),
|
||||||
@@ -514,7 +498,6 @@ class PostsController < ApplicationController
|
|||||||
|
|
||||||
def post_snapshot_from_record post
|
def post_snapshot_from_record post
|
||||||
{ title: post.title,
|
{ title: post.title,
|
||||||
video_ms: post.video_ms,
|
|
||||||
original_created_from: snapshot_time(post.original_created_from),
|
original_created_from: snapshot_time(post.original_created_from),
|
||||||
original_created_before: snapshot_time(post.original_created_before),
|
original_created_before: snapshot_time(post.original_created_before),
|
||||||
tag_names: editable_tag_names_from_post(post),
|
tag_names: editable_tag_names_from_post(post),
|
||||||
@@ -522,41 +505,16 @@ class PostsController < ApplicationController
|
|||||||
end
|
end
|
||||||
|
|
||||||
def editable_tag_names_from_post post
|
def editable_tag_names_from_post post
|
||||||
post
|
post.tags.not_nico.where(deprecated_at: nil)
|
||||||
.post_tags
|
.joins(:tag_name).order('tag_names.name').pluck('tag_names.name')
|
||||||
.kept
|
|
||||||
.joins(tag: :tag_name)
|
|
||||||
.merge(Tag.not_nico)
|
|
||||||
.merge(Tag.where(deprecated_at: nil))
|
|
||||||
.includes(:sections, tag: :tag_name)
|
|
||||||
.order('tag_names.name')
|
|
||||||
.map do |post_tag|
|
|
||||||
name = post_tag.tag.tag_name.name
|
|
||||||
sections = post_tag.sections.sort_by(&:begin_ms)
|
|
||||||
|
|
||||||
next name if sections.empty?
|
|
||||||
|
|
||||||
"#{ name }#{ sections.map { Post.section_literal(_1) }.join }"
|
|
||||||
end
|
|
||||||
end
|
end
|
||||||
|
|
||||||
def post_incoming_snapshot title:, original_created_from:, original_created_before:,
|
def post_incoming_snapshot title:, original_created_from:, original_created_before:,
|
||||||
tag_names:, video_ms_param:, duration_param:, parent_post_ids:
|
tag_names:, parent_post_ids:
|
||||||
Tag.normalise_tags!(tag_names, with_tagme: false, deny_deprecated: true,
|
|
||||||
with_sections: true) =>
|
|
||||||
{ tags:, sections: }
|
|
||||||
|
|
||||||
tags = Tag.expand_parent_tags(tags).reject(&:deprecated?)
|
|
||||||
video_ms = normalise_video_ms(tags, video_ms_param:, duration_param:)
|
|
||||||
validate_video_sections!(video_ms, sections)
|
|
||||||
|
|
||||||
{ title:,
|
{ title:,
|
||||||
video_ms:,
|
|
||||||
original_created_from: snapshot_time(original_created_from),
|
original_created_from: snapshot_time(original_created_from),
|
||||||
original_created_before: snapshot_time(original_created_before),
|
original_created_before: snapshot_time(original_created_before),
|
||||||
tag_names: tags.uniq(&:id).map { |tag|
|
tag_names: incoming_tag_names_for_snapshot(tag_names),
|
||||||
"#{ tag.name }#{ sections[tag.id].to_a.map { section_literal(_1) }.join }"
|
|
||||||
}.sort,
|
|
||||||
parent_post_ids: parent_post_ids.sort }
|
parent_post_ids: parent_post_ids.sort }
|
||||||
end
|
end
|
||||||
|
|
||||||
@@ -579,8 +537,11 @@ class PostsController < ApplicationController
|
|||||||
value.to_s
|
value.to_s
|
||||||
end
|
end
|
||||||
|
|
||||||
def section_literal section
|
def incoming_tag_names_for_snapshot raw_tag_names
|
||||||
"[#{ Post.ms_to_time(section[0]) }-#{ section[1] ? Post.ms_to_time(section[1]) : '' }]"
|
tags = Tag.normalise_tags!(raw_tag_names, with_tagme: false,
|
||||||
|
deny_deprecated: true)
|
||||||
|
|
||||||
|
Tag.expand_parent_tags(tags).reject(&:deprecated?).map(&:name).uniq.sort
|
||||||
end
|
end
|
||||||
|
|
||||||
def post_conflict_json post:, base_version_no:, base_snapshot:,
|
def post_conflict_json post:, base_version_no:, base_snapshot:,
|
||||||
@@ -601,8 +562,6 @@ class PostsController < ApplicationController
|
|||||||
def post_snapshot_changes base_snapshot, current_snapshot, incoming_snapshot
|
def post_snapshot_changes base_snapshot, current_snapshot, incoming_snapshot
|
||||||
[scalar_snapshot_change(:title, 'タイトル',
|
[scalar_snapshot_change(:title, 'タイトル',
|
||||||
base_snapshot, current_snapshot, incoming_snapshot),
|
base_snapshot, current_snapshot, incoming_snapshot),
|
||||||
scalar_snapshot_change(:video_ms, '動画時間',
|
|
||||||
base_snapshot, current_snapshot, incoming_snapshot),
|
|
||||||
scalar_snapshot_change(:original_created_from, 'オリジナルの作成日時(以降)',
|
scalar_snapshot_change(:original_created_from, 'オリジナルの作成日時(以降)',
|
||||||
base_snapshot, current_snapshot, incoming_snapshot),
|
base_snapshot, current_snapshot, incoming_snapshot),
|
||||||
scalar_snapshot_change(:original_created_before, 'オリジナルの作成日時(より前)',
|
scalar_snapshot_change(:original_created_before, 'オリジナルの作成日時(より前)',
|
||||||
@@ -666,14 +625,11 @@ class PostsController < ApplicationController
|
|||||||
PostVersionRecorder.ensure_snapshot!(post, created_by_user: current_user)
|
PostVersionRecorder.ensure_snapshot!(post, created_by_user: current_user)
|
||||||
|
|
||||||
post.update!(title: snapshot[:title],
|
post.update!(title: snapshot[:title],
|
||||||
video_ms: snapshot[:video_ms],
|
|
||||||
original_created_from: snapshot[:original_created_from],
|
original_created_from: snapshot[:original_created_from],
|
||||||
original_created_before: snapshot[:original_created_before])
|
original_created_before: snapshot[:original_created_before])
|
||||||
|
|
||||||
Tag.normalise_tags!(snapshot[:tag_names], with_tagme: false,
|
editable_tags = Tag.normalise_tags!(snapshot[:tag_names], with_tagme: false,
|
||||||
deny_deprecated: true,
|
deny_deprecated: true)
|
||||||
with_sections: true) =>
|
|
||||||
{ tags: editable_tags, sections: }
|
|
||||||
TagVersioning.record_tag_snapshots!(editable_tags, created_by_user: current_user)
|
TagVersioning.record_tag_snapshots!(editable_tags, created_by_user: current_user)
|
||||||
|
|
||||||
readonly_tags = post.tags.nico.to_a
|
readonly_tags = post.tags.nico.to_a
|
||||||
@@ -681,17 +637,14 @@ class PostsController < ApplicationController
|
|||||||
tags = readonly_tags + editable_tags
|
tags = readonly_tags + editable_tags
|
||||||
tags = Tag.expand_parent_tags(tags).reject(&:deprecated?)
|
tags = Tag.expand_parent_tags(tags).reject(&:deprecated?)
|
||||||
|
|
||||||
post.video_ms = tags.any? { _1.id == Tag.video.id } ? snapshot[:video_ms] : nil
|
sync_post_tags!(post, tags)
|
||||||
validate_video_sections!(post.video_ms, sections)
|
|
||||||
post.save!
|
|
||||||
sync_post_tags!(post, tags, sections)
|
|
||||||
sync_parent_posts!(post, snapshot[:parent_post_ids])
|
sync_parent_posts!(post, snapshot[:parent_post_ids])
|
||||||
|
|
||||||
PostVersionRecorder.record!(post:, event_type: :update, created_by_user: current_user)
|
PostVersionRecorder.record!(post:, event_type: :update, created_by_user: current_user)
|
||||||
end
|
end
|
||||||
|
|
||||||
def merge_post_snapshots base_snapshot, current_snapshot, incoming_snapshot
|
def merge_post_snapshots base_snapshot, current_snapshot, incoming_snapshot
|
||||||
[:title, :video_ms, :original_created_from, :original_created_before].map {
|
[:title, :original_created_from, :original_created_before].map {
|
||||||
[_1, merge_scalar_snapshot_value(base_snapshot[_1],
|
[_1, merge_scalar_snapshot_value(base_snapshot[_1],
|
||||||
current_snapshot[_1],
|
current_snapshot[_1],
|
||||||
incoming_snapshot[_1])]
|
incoming_snapshot[_1])]
|
||||||
@@ -735,44 +688,4 @@ class PostsController < ApplicationController
|
|||||||
render_validation_error record
|
render_validation_error record
|
||||||
end
|
end
|
||||||
end
|
end
|
||||||
|
|
||||||
def normalise_video_ms tags, video_ms_param: params[:video_ms], duration_param: params[:duration]
|
|
||||||
return nil unless tags.any? { _1.id == Tag.video.id }
|
|
||||||
|
|
||||||
if video_ms_param.present?
|
|
||||||
video_ms = Integer(video_ms_param, exception: false)
|
|
||||||
raise VideoMsParseError unless video_ms&.positive?
|
|
||||||
|
|
||||||
return video_ms
|
|
||||||
end
|
|
||||||
|
|
||||||
return nil if duration_param.blank?
|
|
||||||
|
|
||||||
video_ms = Tag.time_to_ms!(duration_param.to_s, tag_name: '動画時間')
|
|
||||||
raise VideoMsParseError unless video_ms.positive?
|
|
||||||
|
|
||||||
video_ms
|
|
||||||
rescue Tag::SectionLiteralParseError
|
|
||||||
raise VideoMsParseError
|
|
||||||
end
|
|
||||||
|
|
||||||
def validate_video_sections! video_ms, sections
|
|
||||||
return unless video_ms
|
|
||||||
|
|
||||||
sections.each_value do |ranges|
|
|
||||||
ranges.each do |begin_ms, end_ms|
|
|
||||||
if begin_ms >= video_ms
|
|
||||||
post = Post.new
|
|
||||||
post.errors.add :video_ms, 'タグ区間の開始が動画時間以上です.'
|
|
||||||
raise ActiveRecord::RecordInvalid, post
|
|
||||||
end
|
|
||||||
|
|
||||||
if end_ms && end_ms > video_ms
|
|
||||||
post = Post.new
|
|
||||||
post.errors.add :video_ms, 'タグ区間の終端が動画時間を超えてゐます.'
|
|
||||||
raise ActiveRecord::RecordInvalid, post
|
|
||||||
end
|
|
||||||
end
|
|
||||||
end
|
|
||||||
end
|
|
||||||
end
|
end
|
||||||
|
|||||||
@@ -1,57 +1,50 @@
|
|||||||
class PreviewController < ApplicationController
|
class PreviewController < ApplicationController
|
||||||
before_action :require_member!
|
|
||||||
|
|
||||||
def title
|
def title
|
||||||
return render_bad_request('URL は必須です.') if params[:url].blank?
|
# TODO: # 既知サイトなら決まったフォーマットで title 取得するやぅに.
|
||||||
|
return head :unauthorized unless current_user
|
||||||
|
|
||||||
render json: { title: Preview::ThumbnailFetcher.title(params[:url]) }
|
url = params[:url]
|
||||||
rescue Preview::UrlSafety::UnsafeUrl => e
|
return render_bad_request('URL は必須です.') unless url.present?
|
||||||
|
|
||||||
|
unless url.start_with?(/http(s)?:\/\//)
|
||||||
|
url = 'http://' + url
|
||||||
|
end
|
||||||
|
|
||||||
|
html = URI.open(url, open_timeout: 5, read_timeout: 5).read
|
||||||
|
doc = Nokogiri::HTML.parse(html)
|
||||||
|
title = doc.at('title')&.text&.strip
|
||||||
|
|
||||||
|
render json: { title: title }
|
||||||
|
rescue => e
|
||||||
render_bad_request(e.message)
|
render_bad_request(e.message)
|
||||||
rescue Preview::HttpFetcher::FetchTimeout => e
|
|
||||||
render_preview_error(e.message, :gateway_timeout)
|
|
||||||
rescue Preview::HttpFetcher::ResponseTooLarge => e
|
|
||||||
render_preview_error(e.message, :payload_too_large)
|
|
||||||
rescue Preview::HttpFetcher::FetchFailed => e
|
|
||||||
render_preview_error(e.message, :bad_gateway)
|
|
||||||
end
|
end
|
||||||
|
|
||||||
def thumbnail
|
def thumbnail
|
||||||
return render_bad_request('URL は必須です.') if params[:url].blank?
|
# TODO: 既知ドメインであれば指定のアドレスからサムネールを取得するやぅにする.
|
||||||
|
|
||||||
image = MiniMagick::Image.read(Preview::ThumbnailFetcher.fetch(params[:url]))
|
|
||||||
image.auto_orient
|
|
||||||
image.resize '180x180>'
|
|
||||||
image.format 'png'
|
|
||||||
width, height = image.dimensions
|
|
||||||
raise Preview::ThumbnailFetcher::GenerationFailed, 'サムネール画像の変換に失敗しました.' if width > 180 || height > 180
|
|
||||||
|
|
||||||
send_data image.to_blob, type: 'image/png', disposition: 'inline'
|
|
||||||
rescue Preview::UrlSafety::UnsafeUrl => e
|
|
||||||
render_bad_request(e.message)
|
|
||||||
rescue Preview::HttpFetcher::FetchTimeout => e
|
|
||||||
render_preview_error(e.message, :gateway_timeout)
|
|
||||||
rescue Preview::HttpFetcher::ResponseTooLarge => e
|
|
||||||
render_preview_error(e.message, :payload_too_large)
|
|
||||||
rescue Preview::HttpFetcher::FetchFailed => e
|
|
||||||
render_preview_error(e.message, :bad_gateway)
|
|
||||||
rescue Preview::ThumbnailFetcher::GenerationFailed, MiniMagick::Error => e
|
|
||||||
render_unprocessable_entity(e.message)
|
|
||||||
end
|
|
||||||
|
|
||||||
private
|
|
||||||
|
|
||||||
def require_member!
|
|
||||||
return head :unauthorized unless current_user
|
return head :unauthorized unless current_user
|
||||||
return if current_user.gte_member?
|
|
||||||
|
|
||||||
head :forbidden
|
url = params[:url]
|
||||||
end
|
return render_bad_request('URL は必須です.') if url.blank?
|
||||||
|
|
||||||
def render_preview_error(message, status)
|
unless url.start_with?(/http(s)?:\/\//)
|
||||||
render json: { type: status.to_s,
|
url = 'http://' + url
|
||||||
message:,
|
end
|
||||||
errors: { },
|
|
||||||
base_errors: [message] },
|
path = Rails.root.join('tmp', "thumb_#{ SecureRandom.hex }.png")
|
||||||
status:
|
system("node #{ Rails.root }/lib/screenshot.js #{ Shellwords.escape(url) } #{ path }")
|
||||||
|
|
||||||
|
if File.exist?(path)
|
||||||
|
image = MiniMagick::Image.open(path)
|
||||||
|
image.resize '180x180'
|
||||||
|
File.delete(path) rescue nil
|
||||||
|
send_file image.path, type: 'image/png', disposition: 'inline'
|
||||||
|
else
|
||||||
|
render json: { type: 'internal_server_error',
|
||||||
|
message: 'サムネールを生成できませんでした.',
|
||||||
|
errors: { },
|
||||||
|
base_errors: ['サムネールを生成できませんでした.'] },
|
||||||
|
status: :internal_server_error
|
||||||
|
end
|
||||||
end
|
end
|
||||||
end
|
end
|
||||||
|
|||||||
@@ -82,9 +82,8 @@ class TagsController < ApplicationController
|
|||||||
def with_depth
|
def with_depth
|
||||||
parent_tag_id = params[:parent].to_i
|
parent_tag_id = params[:parent].to_i
|
||||||
parent_tag_id = nil if parent_tag_id <= 0
|
parent_tag_id = nil if parent_tag_id <= 0
|
||||||
material_filter = material_filter_param(default: 'any')
|
|
||||||
|
|
||||||
graph = build_with_depth_graph(material_filter)
|
graph = build_with_depth_graph
|
||||||
|
|
||||||
tag_ids =
|
tag_ids =
|
||||||
if parent_tag_id
|
if parent_tag_id
|
||||||
@@ -93,9 +92,19 @@ class TagsController < ApplicationController
|
|||||||
visible_root_tag_ids(graph)
|
visible_root_tag_ids(graph)
|
||||||
end
|
end
|
||||||
|
|
||||||
render json: tag_ids
|
tags =
|
||||||
.sort_by { |tag_id| graph[:tags_by_id][tag_id][:name] }
|
Tag
|
||||||
.map { |tag_id| with_depth_lightweight_repr(tag_id, graph) }
|
.joins(:tag_name)
|
||||||
|
.includes(:tag_name, :materials, tag_name: :wiki_page)
|
||||||
|
.where(id: tag_ids)
|
||||||
|
.order('tag_names.name')
|
||||||
|
.distinct
|
||||||
|
.to_a
|
||||||
|
|
||||||
|
render json: tags.map { |tag|
|
||||||
|
TagRepr.base(tag).merge(has_children: visible_child_tag_ids(tag.id, graph).present?,
|
||||||
|
children: [])
|
||||||
|
}
|
||||||
end
|
end
|
||||||
|
|
||||||
def autocomplete
|
def autocomplete
|
||||||
@@ -220,16 +229,13 @@ class TagsController < ApplicationController
|
|||||||
def materials_by_name
|
def materials_by_name
|
||||||
name = params[:name].to_s.strip
|
name = params[:name].to_s.strip
|
||||||
return render_bad_request('name は必須です.') if name.blank?
|
return render_bad_request('name は必須です.') if name.blank?
|
||||||
material_filter = material_filter_param(default: 'any')
|
|
||||||
|
|
||||||
tag = Tag.joins(:tag_name)
|
tag = Tag.joins(:tag_name)
|
||||||
.includes(:tag_name, :materials, tag_name: :wiki_page)
|
.includes(:tag_name, :materials, tag_name: :wiki_page)
|
||||||
.find_by(tag_names: { name: })
|
.find_by(tag_names: { name: })
|
||||||
return head :not_found unless tag
|
return head :not_found unless tag
|
||||||
|
|
||||||
graph = build_with_depth_graph(material_filter)
|
render json: build_tag_children(tag)
|
||||||
|
|
||||||
render json: build_tag_children(tag, graph:)
|
|
||||||
end
|
end
|
||||||
|
|
||||||
def update_all
|
def update_all
|
||||||
@@ -245,7 +251,10 @@ class TagsController < ApplicationController
|
|||||||
return render_unprocessable_entity('カテゴリは必須です.', field: :category) if category.blank?
|
return render_unprocessable_entity('カテゴリは必須です.', field: :category) if category.blank?
|
||||||
return render_unprocessable_entity '廃止状態は必須です.', field: :deprecated unless params.key?(:deprecated)
|
return render_unprocessable_entity '廃止状態は必須です.', field: :deprecated unless params.key?(:deprecated)
|
||||||
|
|
||||||
return unless validate_tag_rename(tag, name)
|
if (name != tag.name &&
|
||||||
|
tag.in?([Tag.tagme, Tag.bot, Tag.no_deerjikist, Tag.video, Tag.niconico]))
|
||||||
|
return render_unprocessable_entity 'システム・タグの名称は変更できません.', field: :name
|
||||||
|
end
|
||||||
|
|
||||||
alias_names = params[:aliases].to_s.split.uniq
|
alias_names = params[:aliases].to_s.split.uniq
|
||||||
parent_names = params[:parent_tags].to_s.split.uniq
|
parent_names = params[:parent_tags].to_s.split.uniq
|
||||||
@@ -271,7 +280,7 @@ class TagsController < ApplicationController
|
|||||||
else
|
else
|
||||||
tag.update!(category:, deprecated_at: deprecated ? Time.current : nil)
|
tag.update!(category:, deprecated_at: deprecated ? Time.current : nil)
|
||||||
end
|
end
|
||||||
rename_tag_name!(tag, name) if name_changed
|
tag.tag_name.update!(name:)
|
||||||
|
|
||||||
alias_names << old_name if name_changed
|
alias_names << old_name if name_changed
|
||||||
alias_names.delete(name)
|
alias_names.delete(name)
|
||||||
@@ -307,8 +316,6 @@ class TagsController < ApplicationController
|
|||||||
return render_unprocessable_entity 'ニコタグは廃止できません.', field: :deprecated
|
return render_unprocessable_entity 'ニコタグは廃止できません.', field: :deprecated
|
||||||
end
|
end
|
||||||
|
|
||||||
return unless validate_tag_rename(tag, name)
|
|
||||||
|
|
||||||
if tag.nico? || (category.present? && category == 'nico')
|
if tag.nico? || (category.present? && category == 'nico')
|
||||||
return render_unprocessable_entity 'ニコタグは変更できません.', field: :category
|
return render_unprocessable_entity 'ニコタグは変更できません.', field: :category
|
||||||
end
|
end
|
||||||
@@ -320,7 +327,7 @@ class TagsController < ApplicationController
|
|||||||
name_changed = name.present? && name != old_name
|
name_changed = name.present? && name != old_name
|
||||||
wiki_page = tag.tag_name.wiki_page if name_changed
|
wiki_page = tag.tag_name.wiki_page if name_changed
|
||||||
|
|
||||||
rename_tag_name!(tag, name) if name_changed
|
tag.tag_name.update!(name:) if name.present?
|
||||||
tag.update!(category:) if category.present?
|
tag.update!(category:) if category.present?
|
||||||
if deprecated_given && tag.deprecated? != deprecated
|
if deprecated_given && tag.deprecated? != deprecated
|
||||||
tag.update!(deprecated_at: deprecated ? Time.current : nil)
|
tag.update!(deprecated_at: deprecated ? Time.current : nil)
|
||||||
@@ -341,14 +348,7 @@ class TagsController < ApplicationController
|
|||||||
|
|
||||||
private
|
private
|
||||||
|
|
||||||
def material_filter_param default:
|
def build_with_depth_graph
|
||||||
value = params[:material_filter].to_s.presence
|
|
||||||
return default unless ['present', 'missing', 'any'].include?(value)
|
|
||||||
|
|
||||||
value
|
|
||||||
end
|
|
||||||
|
|
||||||
def build_with_depth_graph material_filter
|
|
||||||
children_by_parent_id = Hash.new { |h, k| h[k] = [] }
|
children_by_parent_id = Hash.new { |h, k| h[k] = [] }
|
||||||
parent_ids_by_child_id = Hash.new { |h, k| h[k] = [] }
|
parent_ids_by_child_id = Hash.new { |h, k| h[k] = [] }
|
||||||
|
|
||||||
@@ -361,37 +361,23 @@ class TagsController < ApplicationController
|
|||||||
parent_ids_by_child_id.keys +
|
parent_ids_by_child_id.keys +
|
||||||
Tag.where(category: ['meme', 'character', 'material']).pluck(:id)).uniq
|
Tag.where(category: ['meme', 'character', 'material']).pluck(:id)).uniq
|
||||||
|
|
||||||
material_tag_ids = Material.unscoped.kept.where.not(tag_id: nil).distinct.pluck(:tag_id).to_set
|
tags_by_id = Tag.where(id: tag_ids)
|
||||||
|
.pluck(:id, :category, :deprecated_at)
|
||||||
tags_by_id = Tag.joins(:tag_name)
|
.each_with_object({ }) do |(id, category, deprecated_at), h|
|
||||||
.where(id: tag_ids)
|
h[id] = { category:, deprecated: deprecated_at.present? }
|
||||||
.pluck('tags.id', 'tag_names.name', 'tags.category', 'tags.deprecated_at')
|
|
||||||
.each_with_object({ }) do |(id, name, category, deprecated_at), h|
|
|
||||||
h[id] = { name:, category:, deprecated: deprecated_at.present?,
|
|
||||||
has_material: material_tag_ids.include?(id) }
|
|
||||||
end
|
end
|
||||||
|
|
||||||
{ children_by_parent_id:, parent_ids_by_child_id:, tags_by_id:,
|
{ children_by_parent_id:, parent_ids_by_child_id:, tags_by_id:,
|
||||||
visible_child_tag_ids_by_parent_id: { },
|
visible_child_tag_ids_by_parent_id: { } }
|
||||||
visible_subtree_by_tag_id: { },
|
|
||||||
material_filter: }
|
|
||||||
end
|
end
|
||||||
|
|
||||||
def visible_root_tag_ids graph
|
def visible_root_tag_ids graph
|
||||||
root_ids = Set.new
|
graph[:tags_by_id].filter_map do |tag_id, attrs|
|
||||||
|
next unless with_depth_visible_tag?(attrs)
|
||||||
graph[:tags_by_id].each do |tag_id, attrs|
|
|
||||||
next unless visible_root_tag?(tag_id, graph)
|
next unless visible_root_tag?(tag_id, graph)
|
||||||
next unless visible_subtree?(tag_id, graph)
|
|
||||||
|
|
||||||
if attrs[:deprecated]
|
tag_id
|
||||||
collect_visible_child_tag_ids(tag_id, graph, root_ids, Set.new)
|
|
||||||
else
|
|
||||||
root_ids << tag_id
|
|
||||||
end
|
|
||||||
end
|
end
|
||||||
|
|
||||||
root_ids.to_a
|
|
||||||
end
|
end
|
||||||
|
|
||||||
def visible_root_tag? tag_id, graph
|
def visible_root_tag? tag_id, graph
|
||||||
@@ -442,77 +428,25 @@ class TagsController < ApplicationController
|
|||||||
return
|
return
|
||||||
end
|
end
|
||||||
|
|
||||||
if with_depth_visible_tag?(tag, graph[:material_filter])
|
visible_ids << tag_id if with_depth_visible_tag?(tag)
|
||||||
visible_ids << tag_id
|
|
||||||
return
|
|
||||||
end
|
|
||||||
|
|
||||||
visible_ids << tag_id if graph[:children_by_parent_id][tag_id].any? { |child_tag_id|
|
|
||||||
visible_subtree?(child_tag_id, graph)
|
|
||||||
}
|
|
||||||
end
|
end
|
||||||
|
|
||||||
def with_depth_visible_tag? tag, material_filter
|
def with_depth_visible_tag? tag
|
||||||
return false unless tag[:category].in?(['meme', 'character', 'material']) && !tag[:deprecated]
|
tag[:category].in?(['meme', 'character', 'material']) && !tag[:deprecated]
|
||||||
|
|
||||||
case material_filter
|
|
||||||
when 'present'
|
|
||||||
tag[:has_material]
|
|
||||||
when 'missing'
|
|
||||||
!tag[:has_material]
|
|
||||||
else
|
|
||||||
true
|
|
||||||
end
|
|
||||||
end
|
end
|
||||||
|
|
||||||
def visible_subtree? tag_id, graph
|
def build_tag_children tag
|
||||||
cache = graph[:visible_subtree_by_tag_id]
|
|
||||||
return cache[tag_id] if cache.key?(tag_id)
|
|
||||||
|
|
||||||
tag = graph[:tags_by_id][tag_id]
|
|
||||||
return cache[tag_id] = false unless tag
|
|
||||||
|
|
||||||
if tag[:deprecated]
|
|
||||||
return cache[tag_id] =
|
|
||||||
graph[:children_by_parent_id][tag_id].any? { |child_tag_id|
|
|
||||||
visible_subtree?(child_tag_id, graph)
|
|
||||||
}
|
|
||||||
end
|
|
||||||
|
|
||||||
cache[tag_id] =
|
|
||||||
with_depth_visible_tag?(tag, graph[:material_filter]) ||
|
|
||||||
graph[:children_by_parent_id][tag_id].any? { |child_tag_id|
|
|
||||||
visible_subtree?(child_tag_id, graph)
|
|
||||||
}
|
|
||||||
end
|
|
||||||
|
|
||||||
def with_depth_lightweight_repr tag_id, graph
|
|
||||||
tag = graph[:tags_by_id][tag_id]
|
|
||||||
|
|
||||||
{ id: tag_id,
|
|
||||||
name: tag[:name],
|
|
||||||
category: tag[:category],
|
|
||||||
deprecated: tag[:deprecated],
|
|
||||||
has_material: tag[:has_material],
|
|
||||||
has_children: visible_child_tag_ids(tag_id, graph).present?,
|
|
||||||
children: [] }
|
|
||||||
end
|
|
||||||
|
|
||||||
def build_tag_children tag, graph: nil
|
|
||||||
material = tag.materials.first
|
material = tag.materials.first
|
||||||
tag_graph = graph && graph[:tags_by_id][tag.id]
|
file = nil
|
||||||
material =
|
content_type = nil
|
||||||
nil if tag_graph && !with_depth_visible_tag?(tag_graph, graph[:material_filter])
|
if material&.file&.attached?
|
||||||
|
file = rails_storage_proxy_url(material.file, only_path: false)
|
||||||
children = tag.children.sort_by(&:name)
|
content_type = material.file.blob.content_type
|
||||||
if graph
|
|
||||||
children = children.filter { |child_tag| visible_subtree?(child_tag.id, graph) }
|
|
||||||
end
|
end
|
||||||
|
|
||||||
TagRepr.base(tag).merge(
|
TagRepr.base(tag).merge(
|
||||||
children: children.map { build_tag_children(_1, graph:) },
|
children: tag.children.sort_by { _1.name }.map { build_tag_children(_1) },
|
||||||
has_material: tag_graph ? tag_graph[:has_material] : material.present?,
|
material: material.as_json&.merge(file:, content_type:))
|
||||||
material: material && MaterialRepr.base(material, host: request.base_url))
|
|
||||||
end
|
end
|
||||||
|
|
||||||
def record_tag_version! tag, event_type:, created_by_user:, name_changed: false, wiki_page: nil
|
def record_tag_version! tag, event_type:, created_by_user:, name_changed: false, wiki_page: nil
|
||||||
@@ -534,67 +468,6 @@ class TagsController < ApplicationController
|
|||||||
created_by_user:)
|
created_by_user:)
|
||||||
end
|
end
|
||||||
|
|
||||||
def validate_tag_rename tag, name
|
|
||||||
return true if name.blank? || name == tag.name
|
|
||||||
|
|
||||||
if tag.in?([Tag.tagme, Tag.bot, Tag.no_deerjikist, Tag.video, Tag.niconico])
|
|
||||||
render_unprocessable_entity 'システム・タグの名称は変更できません.', field: :name
|
|
||||||
return false
|
|
||||||
end
|
|
||||||
|
|
||||||
target_tag_name = TagName.with_discarded.find_by(name:)
|
|
||||||
return true if target_tag_name.nil?
|
|
||||||
return true if target_tag_name.canonical_id?
|
|
||||||
|
|
||||||
render_unprocessable_entity 'その名前は既に使はれてゐます.', field: :name
|
|
||||||
false
|
|
||||||
end
|
|
||||||
|
|
||||||
def rename_tag_name! tag, name
|
|
||||||
return if name == tag.name
|
|
||||||
|
|
||||||
current_tag_name = tag.tag_name
|
|
||||||
target_tag_name = TagName.with_discarded.find_by(name:)
|
|
||||||
|
|
||||||
if target_tag_name.nil?
|
|
||||||
current_tag_name.update!(name:)
|
|
||||||
return
|
|
||||||
end
|
|
||||||
|
|
||||||
promote_tag_alias!(
|
|
||||||
tag,
|
|
||||||
current_tag_name:,
|
|
||||||
promoted_tag_name: target_tag_name)
|
|
||||||
end
|
|
||||||
|
|
||||||
def promote_tag_alias! tag, current_tag_name:, promoted_tag_name:
|
|
||||||
old_owner_tag = promoted_tag_name.canonical&.tag
|
|
||||||
|
|
||||||
if old_owner_tag && old_owner_tag != tag
|
|
||||||
TagVersioning.ensure_snapshot!(old_owner_tag, created_by_user: current_user)
|
|
||||||
end
|
|
||||||
|
|
||||||
promoted_tag_name.undiscard! if promoted_tag_name.discarded?
|
|
||||||
promoted_tag_name.update!(canonical: nil)
|
|
||||||
|
|
||||||
TagName.with_discarded
|
|
||||||
.where(canonical_id: current_tag_name.id)
|
|
||||||
.where.not(id: promoted_tag_name.id)
|
|
||||||
.find_each do |alias_tag_name|
|
|
||||||
alias_tag_name.update!(canonical: promoted_tag_name)
|
|
||||||
end
|
|
||||||
|
|
||||||
current_tag_name.wiki_page&.update!(tag_name: promoted_tag_name)
|
|
||||||
tag.update!(tag_name: promoted_tag_name)
|
|
||||||
current_tag_name.association(:wiki_page).reset
|
|
||||||
current_tag_name.association(:tag).reset
|
|
||||||
current_tag_name.reload.update!(canonical: promoted_tag_name)
|
|
||||||
|
|
||||||
return unless old_owner_tag && old_owner_tag != tag
|
|
||||||
|
|
||||||
record_tag_version!(old_owner_tag.reload, event_type: :update, created_by_user: current_user)
|
|
||||||
end
|
|
||||||
|
|
||||||
def update_aliases! tag, alias_names
|
def update_aliases! tag, alias_names
|
||||||
alias_names = alias_names.uniq
|
alias_names = alias_names.uniq
|
||||||
|
|
||||||
|
|||||||
@@ -1,61 +0,0 @@
|
|||||||
class UserSettingsController < ApplicationController
|
|
||||||
wrap_parameters false
|
|
||||||
|
|
||||||
def show
|
|
||||||
return head :unauthorized unless current_user
|
|
||||||
|
|
||||||
render json: current_setting.serializable_hash
|
|
||||||
end
|
|
||||||
|
|
||||||
def update
|
|
||||||
return head :unauthorized unless current_user
|
|
||||||
|
|
||||||
raw_attributes = editable_raw_attributes
|
|
||||||
field_errors = validate_raw_attributes(raw_attributes)
|
|
||||||
return render_validation_error fields: field_errors if field_errors.present?
|
|
||||||
|
|
||||||
setting = current_setting
|
|
||||||
setting.assign_attributes(raw_attributes)
|
|
||||||
|
|
||||||
if setting.save
|
|
||||||
render json: setting.serializable_hash, status: :ok
|
|
||||||
else
|
|
||||||
render_validation_error setting
|
|
||||||
end
|
|
||||||
end
|
|
||||||
|
|
||||||
private
|
|
||||||
|
|
||||||
def current_setting
|
|
||||||
Setting.find_or_create_by!(user: current_user) do |setting|
|
|
||||||
setting.assign_attributes(Setting.defaults)
|
|
||||||
end
|
|
||||||
rescue ActiveRecord::RecordNotUnique
|
|
||||||
Setting.find_by!(user: current_user)
|
|
||||||
end
|
|
||||||
|
|
||||||
def editable_raw_attributes
|
|
||||||
request.request_parameters.slice(*Setting::EDITABLE_ATTRIBUTES)
|
|
||||||
end
|
|
||||||
|
|
||||||
def validate_raw_attributes raw_attributes
|
|
||||||
raw_attributes.each_with_object({ }) do |(key, value), errors|
|
|
||||||
next if value_matches_type?(key, value)
|
|
||||||
|
|
||||||
errors[key.to_sym] = ['値の型が不正です.']
|
|
||||||
end
|
|
||||||
end
|
|
||||||
|
|
||||||
def value_matches_type? key, value
|
|
||||||
case Setting::TYPE_BY_ATTRIBUTE.fetch(key)
|
|
||||||
when :string
|
|
||||||
value.is_a?(String)
|
|
||||||
when :integer
|
|
||||||
value.is_a?(Integer)
|
|
||||||
when :boolean
|
|
||||||
value == true || value == false
|
|
||||||
else
|
|
||||||
false
|
|
||||||
end
|
|
||||||
end
|
|
||||||
end
|
|
||||||
@@ -1,38 +0,0 @@
|
|||||||
class UserThemeSlotsController < ApplicationController
|
|
||||||
wrap_parameters false
|
|
||||||
|
|
||||||
def index
|
|
||||||
return head :unauthorized unless current_user
|
|
||||||
|
|
||||||
render json: current_user
|
|
||||||
.theme_slots
|
|
||||||
.order(:base_theme, :slot_no)
|
|
||||||
.map { |slot| slot.serializable_hash }
|
|
||||||
end
|
|
||||||
|
|
||||||
def update
|
|
||||||
return head :unauthorized unless current_user
|
|
||||||
|
|
||||||
base_theme = params[:base_theme].to_s
|
|
||||||
slot_no = params[:slot_no].to_i
|
|
||||||
tokens = params[:tokens]
|
|
||||||
|
|
||||||
unless UserThemeSlot::BASE_THEMES.include?(base_theme)
|
|
||||||
return render_validation_error fields: { base_theme: ['値が不正です.'] }
|
|
||||||
end
|
|
||||||
unless UserThemeSlot::SLOT_NOS.include?(slot_no)
|
|
||||||
return render_validation_error fields: { slot_no: ['値が不正です.'] }
|
|
||||||
end
|
|
||||||
unless tokens.is_a?(ActionController::Parameters) || tokens.is_a?(Hash)
|
|
||||||
return render_validation_error fields: { tokens: ['JSON object で指定してください.'] }
|
|
||||||
end
|
|
||||||
|
|
||||||
slot = UserThemeSlot.find_or_initialize_by(user: current_user,
|
|
||||||
base_theme:,
|
|
||||||
slot_no:)
|
|
||||||
slot.tokens = tokens.is_a?(ActionController::Parameters) ? tokens.to_unsafe_h : tokens
|
|
||||||
slot.save!
|
|
||||||
|
|
||||||
render json: slot.serializable_hash, status: :ok
|
|
||||||
end
|
|
||||||
end
|
|
||||||
@@ -1,9 +1,6 @@
|
|||||||
class Material < ApplicationRecord
|
class Material < ApplicationRecord
|
||||||
include MyDiscard
|
include MyDiscard
|
||||||
|
|
||||||
SOURCE_KINDS = ['uri', 'google_drive_path', 'google_drive_file',
|
|
||||||
'legacy_drive_path'].freeze
|
|
||||||
|
|
||||||
default_scope -> { kept }
|
default_scope -> { kept }
|
||||||
|
|
||||||
belongs_to :parent, class_name: 'Material', optional: true
|
belongs_to :parent, class_name: 'Material', optional: true
|
||||||
@@ -13,18 +10,9 @@ class Material < ApplicationRecord
|
|||||||
belongs_to :created_by_user, class_name: 'User', optional: true
|
belongs_to :created_by_user, class_name: 'User', optional: true
|
||||||
belongs_to :updated_by_user, class_name: 'User', optional: true
|
belongs_to :updated_by_user, class_name: 'User', optional: true
|
||||||
|
|
||||||
has_many :material_versions, dependent: :destroy
|
|
||||||
has_many :material_export_items, dependent: :destroy
|
|
||||||
|
|
||||||
has_one_attached :file, dependent: :purge
|
has_one_attached :file, dependent: :purge
|
||||||
has_one_attached :thumbnail, dependent: :purge
|
|
||||||
|
|
||||||
before_validation :assign_normalized_source_key
|
validates :tag_id, presence: true, uniqueness: true
|
||||||
|
|
||||||
validates :tag_id, uniqueness: true, allow_nil: true
|
|
||||||
validates :source_kind,
|
|
||||||
inclusion: { in: SOURCE_KINDS },
|
|
||||||
allow_blank: true
|
|
||||||
|
|
||||||
validate :file_must_be_attached
|
validate :file_must_be_attached
|
||||||
validate :tag_must_be_material_category
|
validate :tag_must_be_material_category
|
||||||
@@ -35,37 +23,8 @@ class Material < ApplicationRecord
|
|||||||
file.blob.content_type
|
file.blob.content_type
|
||||||
end
|
end
|
||||||
|
|
||||||
def file_byte_size
|
|
||||||
return nil unless file&.attached?
|
|
||||||
|
|
||||||
file.blob.byte_size
|
|
||||||
end
|
|
||||||
|
|
||||||
def file_filename
|
|
||||||
return nil unless file&.attached?
|
|
||||||
|
|
||||||
file.blob.filename.to_s
|
|
||||||
end
|
|
||||||
|
|
||||||
def snapshot_export_paths
|
|
||||||
material_export_items.order(:profile).pluck(:profile, :export_path).to_h
|
|
||||||
end
|
|
||||||
|
|
||||||
private
|
private
|
||||||
|
|
||||||
def assign_normalized_source_key
|
|
||||||
if source_kind.blank?
|
|
||||||
self.normalized_source_key = nil
|
|
||||||
return
|
|
||||||
end
|
|
||||||
|
|
||||||
self.normalized_source_key =
|
|
||||||
MaterialSyncSuppression.normalize_source_key(source_kind:,
|
|
||||||
source_uri:,
|
|
||||||
drive_path: source_path,
|
|
||||||
drive_file_id: source_file_id)
|
|
||||||
end
|
|
||||||
|
|
||||||
def file_must_be_attached
|
def file_must_be_attached
|
||||||
return if url.present? || file.attached?
|
return if url.present? || file.attached?
|
||||||
|
|
||||||
|
|||||||
@@ -1,48 +0,0 @@
|
|||||||
class MaterialExportItem < ApplicationRecord
|
|
||||||
VALID_PROFILES = ['legacy_drive'].freeze
|
|
||||||
|
|
||||||
belongs_to :material
|
|
||||||
belongs_to :created_by_user, class_name: 'User', optional: true
|
|
||||||
|
|
||||||
validates :profile, presence: true, inclusion: { in: VALID_PROFILES }
|
|
||||||
validates :export_path, presence: true, uniqueness: { scope: :profile }
|
|
||||||
validates :material_id, uniqueness: { scope: :profile }
|
|
||||||
validate :export_path_must_be_relative_safe_path
|
|
||||||
|
|
||||||
scope :enabled, -> { where(enabled: true) }
|
|
||||||
|
|
||||||
private
|
|
||||||
|
|
||||||
def export_path_must_be_relative_safe_path
|
|
||||||
return if export_path.blank?
|
|
||||||
|
|
||||||
if export_path.start_with?('/')
|
|
||||||
errors.add(:export_path, '絶対パスは使へません.')
|
|
||||||
end
|
|
||||||
|
|
||||||
if export_path.match?(/\A[A-Za-z]:\//)
|
|
||||||
errors.add(:export_path, '絶対パスは使へません.')
|
|
||||||
end
|
|
||||||
|
|
||||||
if export_path.include?('\\')
|
|
||||||
errors.add(:export_path, '/ 区切りで指定してください.')
|
|
||||||
end
|
|
||||||
|
|
||||||
if export_path.include?("\0")
|
|
||||||
errors.add(:export_path, 'NUL は使へません.')
|
|
||||||
end
|
|
||||||
|
|
||||||
parts = export_path.split('/')
|
|
||||||
if export_path.include?('//')
|
|
||||||
errors.add(:export_path, '空の path segment は使へません.')
|
|
||||||
end
|
|
||||||
|
|
||||||
if parts.any? { |part| part.in?(['.', '..']) }
|
|
||||||
errors.add(:export_path, '.. は使へません.')
|
|
||||||
end
|
|
||||||
|
|
||||||
if export_path.end_with?('/')
|
|
||||||
errors.add(:export_path, 'directory path は使へません.')
|
|
||||||
end
|
|
||||||
end
|
|
||||||
end
|
|
||||||
@@ -1,29 +0,0 @@
|
|||||||
class MaterialImportBlock < ApplicationRecord
|
|
||||||
MATCH_KINDS = ['sha256', 'exact_path', 'path_prefix', 'manual'].freeze
|
|
||||||
REASONS = [
|
|
||||||
'copyright_high_risk',
|
|
||||||
'copyright_takedown',
|
|
||||||
'adult_or_sensitive',
|
|
||||||
'personal_information',
|
|
||||||
'malware_or_dangerous_file',
|
|
||||||
'duplicate_or_low_quality',
|
|
||||||
'source_owner_request',
|
|
||||||
'other'
|
|
||||||
].freeze
|
|
||||||
|
|
||||||
belongs_to :created_by_user, class_name: 'User', optional: true
|
|
||||||
|
|
||||||
validates :match_kind, presence: true, inclusion: { in: MATCH_KINDS }
|
|
||||||
validates :reason, presence: true, inclusion: { in: REASONS }
|
|
||||||
validates :sha256, length: { is: 64 }, allow_blank: true
|
|
||||||
validate :match_value_must_be_present
|
|
||||||
|
|
||||||
private
|
|
||||||
|
|
||||||
def match_value_must_be_present
|
|
||||||
return if match_kind == 'manual'
|
|
||||||
return if sha256.present? || external_path_pattern.present?
|
|
||||||
|
|
||||||
errors.add(:base, 'sha256 または external_path_pattern は必須です.')
|
|
||||||
end
|
|
||||||
end
|
|
||||||
@@ -1,45 +0,0 @@
|
|||||||
class MaterialSyncSource < ApplicationRecord
|
|
||||||
SOURCE_KINDS = ['uri', 'google_drive_path', 'google_drive_file', 'legacy_drive_path'].freeze
|
|
||||||
|
|
||||||
belongs_to :created_by_user, class_name: 'User', optional: true
|
|
||||||
belongs_to :updated_by_user, class_name: 'User', optional: true
|
|
||||||
|
|
||||||
validates :name, presence: true
|
|
||||||
validates :source_kind,
|
|
||||||
presence: true,
|
|
||||||
inclusion: { in: SOURCE_KINDS }
|
|
||||||
validates :profile, presence: true, inclusion: { in: MaterialExportItem::VALID_PROFILES }
|
|
||||||
validate :source_value_must_be_present
|
|
||||||
|
|
||||||
scope :enabled, -> { where(enabled: true) }
|
|
||||||
|
|
||||||
def normalized_source_key
|
|
||||||
MaterialSyncSuppression.normalize_source_key(source_kind:,
|
|
||||||
source_uri:,
|
|
||||||
source_path:,
|
|
||||||
source_file_id:)
|
|
||||||
end
|
|
||||||
|
|
||||||
private
|
|
||||||
|
|
||||||
def source_value_must_be_present
|
|
||||||
return if source_value_present?
|
|
||||||
|
|
||||||
errors.add(:base, '同期元を指定してください.')
|
|
||||||
end
|
|
||||||
|
|
||||||
def source_value_present?
|
|
||||||
case source_kind
|
|
||||||
when 'uri'
|
|
||||||
source_uri.present?
|
|
||||||
when 'google_drive_file'
|
|
||||||
source_file_id.present? || source_uri.present?
|
|
||||||
when 'google_drive_path'
|
|
||||||
source_file_id.present? || source_uri.present?
|
|
||||||
when 'legacy_drive_path'
|
|
||||||
source_path.present?
|
|
||||||
else
|
|
||||||
normalized_source_key.present?
|
|
||||||
end
|
|
||||||
end
|
|
||||||
end
|
|
||||||
@@ -1,105 +0,0 @@
|
|||||||
class MaterialSyncSuppression < ApplicationRecord
|
|
||||||
SOURCE_KINDS = [
|
|
||||||
'uri',
|
|
||||||
'google_drive_path',
|
|
||||||
'google_drive_path_prefix',
|
|
||||||
'google_drive_file',
|
|
||||||
'legacy_drive_path',
|
|
||||||
'legacy_drive_path_prefix'
|
|
||||||
].freeze
|
|
||||||
|
|
||||||
REASONS = [
|
|
||||||
'copyright_high_risk',
|
|
||||||
'copyright_takedown',
|
|
||||||
'adult_or_sensitive',
|
|
||||||
'personal_information',
|
|
||||||
'malware_or_dangerous_file',
|
|
||||||
'duplicate_or_low_quality',
|
|
||||||
'source_owner_request',
|
|
||||||
'other'
|
|
||||||
].freeze
|
|
||||||
|
|
||||||
belongs_to :created_by_user, class_name: 'User', optional: true
|
|
||||||
|
|
||||||
before_validation :assign_normalized_source_key
|
|
||||||
|
|
||||||
validates :source_kind, presence: true, inclusion: { in: SOURCE_KINDS }
|
|
||||||
validates :reason, presence: true, inclusion: { in: REASONS }
|
|
||||||
validates :normalized_source_key, presence: true, uniqueness: true
|
|
||||||
validate :source_value_must_be_present
|
|
||||||
validate :drive_path_must_be_safe_relative_path
|
|
||||||
|
|
||||||
def self.normalize_source_key source_kind:,
|
|
||||||
source_uri: nil,
|
|
||||||
drive_path: nil,
|
|
||||||
drive_file_id: nil,
|
|
||||||
source_path: nil,
|
|
||||||
source_file_id: nil
|
|
||||||
kind = source_kind.to_s
|
|
||||||
value =
|
|
||||||
case kind
|
|
||||||
when 'uri'
|
|
||||||
source_uri.to_s.strip
|
|
||||||
when 'google_drive_path', 'google_drive_path_prefix',
|
|
||||||
'legacy_drive_path', 'legacy_drive_path_prefix'
|
|
||||||
(source_path || drive_path).to_s.strip.gsub(%r{/+}, '/')
|
|
||||||
when 'google_drive_file'
|
|
||||||
(source_file_id || drive_file_id).to_s.strip
|
|
||||||
else
|
|
||||||
''
|
|
||||||
end
|
|
||||||
|
|
||||||
return nil if kind.blank? || value.blank?
|
|
||||||
|
|
||||||
"#{ kind }:#{ value }"
|
|
||||||
end
|
|
||||||
|
|
||||||
private
|
|
||||||
|
|
||||||
def assign_normalized_source_key
|
|
||||||
normalize_path_fields!
|
|
||||||
self.normalized_source_key =
|
|
||||||
self.class.normalize_source_key(source_kind:,
|
|
||||||
source_uri:,
|
|
||||||
drive_path:,
|
|
||||||
drive_file_id:)
|
|
||||||
end
|
|
||||||
|
|
||||||
def source_value_must_be_present
|
|
||||||
return if normalized_source_key.present?
|
|
||||||
|
|
||||||
errors.add(:base, '同期元を指定してください.')
|
|
||||||
end
|
|
||||||
|
|
||||||
def drive_path_must_be_safe_relative_path
|
|
||||||
return unless source_kind.in?(
|
|
||||||
['google_drive_path', 'google_drive_path_prefix',
|
|
||||||
'legacy_drive_path', 'legacy_drive_path_prefix'])
|
|
||||||
|
|
||||||
value = drive_path.to_s
|
|
||||||
return if value.blank?
|
|
||||||
|
|
||||||
if value.start_with?('/') || value.match?(/\A[A-Za-z]:\//)
|
|
||||||
errors.add(:drive_path, '相対 path を指定してください.')
|
|
||||||
end
|
|
||||||
if value.start_with?('My Drive/', 'マイドライブ/')
|
|
||||||
errors.add(:drive_path, '同期元フォルダからの相対 path を指定してください.')
|
|
||||||
end
|
|
||||||
errors.add(:drive_path, 'NUL は使へません.') if value.include?("\0")
|
|
||||||
errors.add(:drive_path, '連続スラッシュは使へません.') if value.include?('//')
|
|
||||||
errors.add(:drive_path, '末尾スラッシュは使へません.') if value.end_with?('/')
|
|
||||||
|
|
||||||
parts = value.split('/')
|
|
||||||
if parts.any? { |part| part.in?(['.', '..']) }
|
|
||||||
errors.add(:drive_path, '. や .. は使へません.')
|
|
||||||
end
|
|
||||||
end
|
|
||||||
|
|
||||||
def normalize_path_fields!
|
|
||||||
return unless source_kind.in?(
|
|
||||||
['google_drive_path', 'google_drive_path_prefix',
|
|
||||||
'legacy_drive_path', 'legacy_drive_path_prefix'])
|
|
||||||
|
|
||||||
self.drive_path = drive_path.to_s.tr('\\', '/').strip.presence
|
|
||||||
end
|
|
||||||
end
|
|
||||||
@@ -1,17 +0,0 @@
|
|||||||
class MaterialVersion < ApplicationRecord
|
|
||||||
EVENT_TYPE_MAP = { create: 'create',
|
|
||||||
update: 'update',
|
|
||||||
discard: 'discard',
|
|
||||||
restore: 'restore' }.freeze
|
|
||||||
|
|
||||||
include VersionRecord
|
|
||||||
|
|
||||||
belongs_to :material
|
|
||||||
belongs_to :tag, optional: true
|
|
||||||
belongs_to :parent, class_name: 'Material', optional: true
|
|
||||||
belongs_to :updated_by_user, class_name: 'User', optional: true
|
|
||||||
|
|
||||||
def export_paths_hash
|
|
||||||
export_paths_json || {}
|
|
||||||
end
|
|
||||||
end
|
|
||||||
+19
-55
@@ -1,19 +1,5 @@
|
|||||||
class Post < ApplicationRecord
|
class Post < ApplicationRecord
|
||||||
require 'mini_magick'
|
require 'mini_magick'
|
||||||
require 'stringio'
|
|
||||||
|
|
||||||
def self.resized_thumbnail_attachment(upload)
|
|
||||||
upload.rewind
|
|
||||||
image = MiniMagick::Image.read(upload.read)
|
|
||||||
image.resize '180x180'
|
|
||||||
image.format 'jpg'
|
|
||||||
|
|
||||||
{ io: StringIO.new(image.to_blob),
|
|
||||||
filename: 'resized_thumbnail.jpg',
|
|
||||||
content_type: 'image/jpeg' }
|
|
||||||
ensure
|
|
||||||
upload.rewind
|
|
||||||
end
|
|
||||||
|
|
||||||
belongs_to :uploaded_user, class_name: 'User', optional: true
|
belongs_to :uploaded_user, class_name: 'User', optional: true
|
||||||
|
|
||||||
@@ -57,10 +43,9 @@ class Post < ApplicationRecord
|
|||||||
|
|
||||||
attribute :version_no, :integer, default: 1
|
attribute :version_no, :integer, default: 1
|
||||||
|
|
||||||
before_validation :normalise_url, if: :will_save_change_to_url?
|
before_validation :normalise_url
|
||||||
|
|
||||||
validates :url, presence: true, uniqueness: true, length: { maximum: 768 }
|
validates :url, presence: true, uniqueness: true
|
||||||
validates :video_ms, numericality: { only_integer: true, greater_than: 0 }, allow_nil: true
|
|
||||||
|
|
||||||
validate :validate_original_created_range
|
validate :validate_original_created_range
|
||||||
validate :url_must_be_http_url
|
validate :url_must_be_http_url
|
||||||
@@ -84,42 +69,7 @@ class Post < ApplicationRecord
|
|||||||
super(options).merge(thumbnail: nil)
|
super(options).merge(thumbnail: nil)
|
||||||
end
|
end
|
||||||
|
|
||||||
def snapshot_tag_names
|
def snapshot_tag_names = tags.joins(:tag_name).order('tag_names.name').pluck('tag_names.name')
|
||||||
post_tags
|
|
||||||
.kept
|
|
||||||
.joins(tag: :tag_name)
|
|
||||||
.includes(:sections, tag: :tag_name)
|
|
||||||
.order('tag_names.name')
|
|
||||||
.map do |post_tag|
|
|
||||||
name = post_tag.tag.tag_name.name
|
|
||||||
sections = post_tag.sections.sort_by(&:begin_ms)
|
|
||||||
|
|
||||||
next name if sections.empty?
|
|
||||||
|
|
||||||
"#{ name }#{ sections.map { Post.section_literal(_1) }.join }"
|
|
||||||
end
|
|
||||||
end
|
|
||||||
|
|
||||||
def self.section_literal section
|
|
||||||
"[#{ Post.ms_to_time(section.begin_ms) }-#{ section.end_ms ? Post.ms_to_time(section.end_ms) : '' }]"
|
|
||||||
end
|
|
||||||
|
|
||||||
def self.ms_to_time ms
|
|
||||||
total_s = ms / 1_000
|
|
||||||
s = total_s % 60
|
|
||||||
min = (total_s / 60) % 60
|
|
||||||
h = total_s / 3_600
|
|
||||||
remainder_ms = ms % 1_000
|
|
||||||
|
|
||||||
base =
|
|
||||||
if h.positive?
|
|
||||||
'%d:%02d:%02d' % [h, min, s]
|
|
||||||
else
|
|
||||||
'%d:%02d' % [min, s]
|
|
||||||
end
|
|
||||||
|
|
||||||
remainder_ms.positive? ? "#{ base }.#{ remainder_ms.to_s.rjust(3, '0') }" : base
|
|
||||||
end
|
|
||||||
|
|
||||||
def snapshot_parent_post_ids = parents.order(:id).pluck(:id)
|
def snapshot_parent_post_ids = parents.order(:id).pluck(:id)
|
||||||
|
|
||||||
@@ -137,7 +87,12 @@ class Post < ApplicationRecord
|
|||||||
def resized_thumbnail!
|
def resized_thumbnail!
|
||||||
return unless thumbnail.attached?
|
return unless thumbnail.attached?
|
||||||
|
|
||||||
thumbnail.attach(self.class.resized_thumbnail_attachment(StringIO.new(thumbnail.download)))
|
image = MiniMagick::Image.read(thumbnail.download)
|
||||||
|
image.resize '180x180'
|
||||||
|
thumbnail.purge
|
||||||
|
thumbnail.attach(io: File.open(image.path),
|
||||||
|
filename: 'resized_thumbnail.jpg',
|
||||||
|
content_type: 'image/jpeg')
|
||||||
end
|
end
|
||||||
|
|
||||||
private
|
private
|
||||||
@@ -173,6 +128,15 @@ class Post < ApplicationRecord
|
|||||||
def normalise_url
|
def normalise_url
|
||||||
return if url.blank?
|
return if url.blank?
|
||||||
|
|
||||||
self.url = PostUrlNormaliser.normalise(url) || url.strip
|
self.url = url.strip
|
||||||
|
|
||||||
|
u = URI.parse(url)
|
||||||
|
return unless u in URI::HTTP
|
||||||
|
|
||||||
|
u.host = u.host.downcase if u.host
|
||||||
|
u.path = u.path.sub(/\/\Z/, '') if u.path.present?
|
||||||
|
self.url = u.to_s
|
||||||
|
rescue URI::InvalidURIError
|
||||||
|
;
|
||||||
end
|
end
|
||||||
end
|
end
|
||||||
|
|||||||
@@ -10,12 +10,6 @@ class PostTag < ApplicationRecord
|
|||||||
belongs_to :created_user, class_name: 'User', optional: true
|
belongs_to :created_user, class_name: 'User', optional: true
|
||||||
belongs_to :deleted_user, class_name: 'User', optional: true
|
belongs_to :deleted_user, class_name: 'User', optional: true
|
||||||
|
|
||||||
has_many :sections, -> { order(:begin_ms) }, class_name: 'PostTagSection',
|
|
||||||
foreign_key: [:post_id, :tag_id],
|
|
||||||
primary_key: [:post_id, :tag_id],
|
|
||||||
dependent: :delete_all,
|
|
||||||
inverse_of: :post_tag
|
|
||||||
|
|
||||||
validates :post_id, presence: true
|
validates :post_id, presence: true
|
||||||
validates :tag_id, presence: true
|
validates :tag_id, presence: true
|
||||||
validates :post_id, uniqueness: {
|
validates :post_id, uniqueness: {
|
||||||
|
|||||||
@@ -1,20 +0,0 @@
|
|||||||
class PostTagSection < ApplicationRecord
|
|
||||||
self.primary_key = :post_id, :tag_id, :begin_ms
|
|
||||||
|
|
||||||
belongs_to :post
|
|
||||||
belongs_to :tag
|
|
||||||
|
|
||||||
belongs_to :post_tag, -> { kept }, foreign_key: [:post_id, :tag_id],
|
|
||||||
primary_key: [:post_id, :tag_id],
|
|
||||||
inverse_of: :sections,
|
|
||||||
optional: true
|
|
||||||
|
|
||||||
validates :post_id, presence: true
|
|
||||||
validates :tag_id, presence: true
|
|
||||||
|
|
||||||
validates :begin_ms, presence: true,
|
|
||||||
numericality: { only_integer: true, greater_than_or_equal_to: 0 }
|
|
||||||
|
|
||||||
validates :end_ms, numericality: { only_integer: true, greater_than: :begin_ms },
|
|
||||||
allow_nil: true
|
|
||||||
end
|
|
||||||
@@ -1,163 +0,0 @@
|
|||||||
class PostUrlSanitisationRule < ApplicationRecord
|
|
||||||
include Discard::Model
|
|
||||||
|
|
||||||
class InvalidUrlError < StandardError
|
|
||||||
attr_reader :invalid_rows
|
|
||||||
|
|
||||||
def initialize(invalid_rows)
|
|
||||||
@invalid_rows = invalid_rows
|
|
||||||
ids = invalid_rows.map { _1.fetch(:post_id) }.join(', ')
|
|
||||||
super("post URL sanitisation produced invalid URLs for posts #{ ids }")
|
|
||||||
end
|
|
||||||
end
|
|
||||||
|
|
||||||
class UrlConflictError < StandardError
|
|
||||||
attr_reader :conflicts
|
|
||||||
|
|
||||||
def initialize(conflicts)
|
|
||||||
@conflicts = conflicts
|
|
||||||
urls = conflicts.map { _1.fetch(:url) }.uniq.join(', ')
|
|
||||||
super("post URL sanitisation conflicts detected for #{ urls }")
|
|
||||||
end
|
|
||||||
end
|
|
||||||
|
|
||||||
self.primary_key = :priority
|
|
||||||
|
|
||||||
default_scope -> { kept }
|
|
||||||
|
|
||||||
validates :source_pattern, presence: true, uniqueness: true
|
|
||||||
|
|
||||||
validate :source_pattern_must_be_regexp
|
|
||||||
|
|
||||||
class << self
|
|
||||||
def sanitise(url) = sanitise_with_rules(url, rules)
|
|
||||||
|
|
||||||
def apply!
|
|
||||||
rewrites = nil
|
|
||||||
|
|
||||||
Post.transaction do
|
|
||||||
compiled_rules = rules
|
|
||||||
|
|
||||||
rewrites = Post.order(:id)
|
|
||||||
.lock('FOR UPDATE')
|
|
||||||
.pluck(:id, :url)
|
|
||||||
.map do |post_id, original_url|
|
|
||||||
{ post_id:,
|
|
||||||
original_url:,
|
|
||||||
sanitised_url: sanitise_with_rules(original_url, compiled_rules) }
|
|
||||||
end
|
|
||||||
|
|
||||||
invalid_rows = rewrites.filter { invalid_sanitised_url?(_1.fetch(:sanitised_url)) }
|
|
||||||
.map { { post_id: _1.fetch(:post_id),
|
|
||||||
original_url: _1.fetch(:original_url),
|
|
||||||
sanitised_url: _1.fetch(:sanitised_url) } }
|
|
||||||
raise InvalidUrlError.new(invalid_rows) if invalid_rows.present?
|
|
||||||
|
|
||||||
conflicts = build_conflicts(rewrites)
|
|
||||||
raise UrlConflictError.new(conflicts) if conflicts.present?
|
|
||||||
|
|
||||||
changed = rewrites.filter { _1.fetch(:original_url) != _1.fetch(:sanitised_url) }
|
|
||||||
return if changed.empty?
|
|
||||||
|
|
||||||
token = SecureRandom.hex(6)
|
|
||||||
|
|
||||||
changed.each do |row|
|
|
||||||
Post.where(id: row.fetch(:post_id))
|
|
||||||
.update_all(url: temporary_url_for(row.fetch(:post_id), token))
|
|
||||||
end
|
|
||||||
|
|
||||||
changed.each do |row|
|
|
||||||
Post.where(id: row.fetch(:post_id))
|
|
||||||
.update_all(url: row.fetch(:sanitised_url))
|
|
||||||
end
|
|
||||||
end
|
|
||||||
rescue ActiveRecord::RecordNotUnique => error
|
|
||||||
conflicts = build_persisted_conflicts(rewrites)
|
|
||||||
raise error if conflicts.empty?
|
|
||||||
|
|
||||||
raise UrlConflictError.new(conflicts), cause: error
|
|
||||||
end
|
|
||||||
|
|
||||||
private
|
|
||||||
|
|
||||||
def rules = kept.order(:priority).map { |r| [Regexp.new(r.source_pattern), r.replacement] }
|
|
||||||
|
|
||||||
def sanitise_with_rules(url, compiled_rules)
|
|
||||||
compiled_rules.reduce(url.dup) do |value, (pattern, replacement)|
|
|
||||||
value.sub(pattern, replacement)
|
|
||||||
end
|
|
||||||
end
|
|
||||||
|
|
||||||
def temporary_url_for(post_id, token) =
|
|
||||||
"https://post-url-sanitising.invalid/#{ token }/#{ post_id }"
|
|
||||||
|
|
||||||
def invalid_sanitised_url?(url)
|
|
||||||
return true if url.blank?
|
|
||||||
return true if url.length > 768
|
|
||||||
|
|
||||||
parsed = URI.parse(url)
|
|
||||||
return true if !(parsed in URI::HTTP)
|
|
||||||
return true if parsed.host.blank?
|
|
||||||
|
|
||||||
false
|
|
||||||
rescue URI::InvalidURIError
|
|
||||||
true
|
|
||||||
end
|
|
||||||
|
|
||||||
def build_conflicts(rewrites)
|
|
||||||
rewrites
|
|
||||||
.group_by { _1.fetch(:sanitised_url).downcase }
|
|
||||||
.values
|
|
||||||
.filter { _1.size > 1 }
|
|
||||||
.flatten
|
|
||||||
.map { { url: _1.fetch(:sanitised_url),
|
|
||||||
post_id: _1.fetch(:post_id),
|
|
||||||
original_url: _1.fetch(:original_url) } }
|
|
||||||
end
|
|
||||||
|
|
||||||
def build_persisted_conflicts(rewrites)
|
|
||||||
return [] if rewrites.blank?
|
|
||||||
|
|
||||||
target_rows = rewrites.filter { _1.fetch(:original_url) != _1.fetch(:sanitised_url) }
|
|
||||||
target_keys = target_rows.map { _1.fetch(:sanitised_url).downcase }.uniq
|
|
||||||
return [] if target_keys.empty?
|
|
||||||
|
|
||||||
target_pairs = target_rows.to_h do |row|
|
|
||||||
[row.fetch(:post_id), row.fetch(:sanitised_url).downcase]
|
|
||||||
end
|
|
||||||
|
|
||||||
persisted_rows = Post.order(:id)
|
|
||||||
.where('LOWER(url) IN (?)', target_keys)
|
|
||||||
.pluck(:id, :url)
|
|
||||||
.reject { |post_id, original_url| target_pairs[post_id] == original_url.downcase }
|
|
||||||
.map { |post_id, original_url|
|
|
||||||
{ url: original_url,
|
|
||||||
post_id:,
|
|
||||||
original_url:,
|
|
||||||
conflict_key: original_url.downcase }
|
|
||||||
}
|
|
||||||
|
|
||||||
target_conflicts = target_rows.map { { url: _1.fetch(:sanitised_url),
|
|
||||||
post_id: _1.fetch(:post_id),
|
|
||||||
original_url: _1.fetch(:original_url),
|
|
||||||
conflict_key: _1.fetch(:sanitised_url).downcase } }
|
|
||||||
|
|
||||||
(persisted_rows + target_conflicts)
|
|
||||||
.group_by { _1.fetch(:conflict_key) }
|
|
||||||
.values
|
|
||||||
.filter { _1.size > 1 }
|
|
||||||
.flatten
|
|
||||||
.map { _1.except(:conflict_key) }
|
|
||||||
end
|
|
||||||
end
|
|
||||||
|
|
||||||
private
|
|
||||||
|
|
||||||
def source_pattern_must_be_regexp
|
|
||||||
return if source_pattern.blank?
|
|
||||||
|
|
||||||
Regexp.new(source_pattern)
|
|
||||||
rescue RegexpError
|
|
||||||
errors.add :source_pattern, '変な正規表現だね〜(笑)'
|
|
||||||
end
|
|
||||||
end
|
|
||||||
@@ -5,7 +5,6 @@ class PostVersion < ApplicationRecord
|
|||||||
belongs_to :parent, class_name: 'Post', optional: true
|
belongs_to :parent, class_name: 'Post', optional: true
|
||||||
|
|
||||||
validates :url, presence: true
|
validates :url, presence: true
|
||||||
validates :video_ms, numericality: { only_integer: true, greater_than: 0 }, allow_nil: true
|
|
||||||
|
|
||||||
validate :validate_original_created_range
|
validate :validate_original_created_range
|
||||||
|
|
||||||
|
|||||||
@@ -1,52 +1,7 @@
|
|||||||
class Setting < ApplicationRecord
|
class Setting < ApplicationRecord
|
||||||
THEMES = ['system', 'light', 'dark'].freeze
|
|
||||||
|
|
||||||
# These remain in the typed settings schema to preserve existing backend
|
|
||||||
# work. The common `/users/settings` page currently surfaces only `theme`.
|
|
||||||
AUTO_FETCH_MODES = ['auto', 'manual', 'off'].freeze
|
|
||||||
WIKI_EDITOR_MODES = ['split', 'write', 'preview'].freeze
|
|
||||||
|
|
||||||
STRING_ATTRIBUTES = [
|
|
||||||
'theme',
|
|
||||||
'auto_fetch_title',
|
|
||||||
'auto_fetch_thumbnail',
|
|
||||||
'wiki_editor_mode',
|
|
||||||
].freeze
|
|
||||||
INTEGER_ATTRIBUTES = [].freeze
|
|
||||||
BOOLEAN_ATTRIBUTES = [].freeze
|
|
||||||
EDITABLE_ATTRIBUTES =
|
|
||||||
(STRING_ATTRIBUTES + INTEGER_ATTRIBUTES + BOOLEAN_ATTRIBUTES).freeze
|
|
||||||
TYPE_BY_ATTRIBUTE = {
|
|
||||||
'theme' => :string,
|
|
||||||
'auto_fetch_title' => :string,
|
|
||||||
'auto_fetch_thumbnail' => :string,
|
|
||||||
'wiki_editor_mode' => :string,
|
|
||||||
}.freeze
|
|
||||||
|
|
||||||
belongs_to :user
|
belongs_to :user
|
||||||
|
|
||||||
validates :user_id, presence: true
|
validates :user_id, presence: true
|
||||||
validates :user_id, uniqueness: true
|
validates :key, presence: true, length: { maximum: 255 }
|
||||||
|
validates :value, presence: true
|
||||||
validates :theme, inclusion: { in: THEMES }
|
|
||||||
validates :auto_fetch_title, inclusion: { in: AUTO_FETCH_MODES }
|
|
||||||
validates :auto_fetch_thumbnail, inclusion: { in: AUTO_FETCH_MODES }
|
|
||||||
validates :wiki_editor_mode, inclusion: { in: WIKI_EDITOR_MODES }
|
|
||||||
|
|
||||||
def self.defaults
|
|
||||||
{
|
|
||||||
theme: 'system',
|
|
||||||
auto_fetch_title: 'manual',
|
|
||||||
auto_fetch_thumbnail: 'manual',
|
|
||||||
wiki_editor_mode: 'split',
|
|
||||||
}
|
|
||||||
end
|
|
||||||
|
|
||||||
def self.serializable_attributes
|
|
||||||
EDITABLE_ATTRIBUTES.map(&:to_sym)
|
|
||||||
end
|
|
||||||
|
|
||||||
def serializable_hash(options = nil)
|
|
||||||
super({ only: self.class.serializable_attributes }.merge(options || { }))
|
|
||||||
end
|
|
||||||
end
|
end
|
||||||
|
|||||||
+3
-112
@@ -17,16 +17,6 @@ class Tag < ApplicationRecord
|
|||||||
end
|
end
|
||||||
end
|
end
|
||||||
|
|
||||||
class SectionLiteralParseError < ArgumentError
|
|
||||||
attr_reader :tag_name, :literal
|
|
||||||
|
|
||||||
def initialize tag_name, literal
|
|
||||||
@tag_name = tag_name
|
|
||||||
@literal = literal
|
|
||||||
super("invalid section literal for tag #{ tag_name }: #{ literal }")
|
|
||||||
end
|
|
||||||
end
|
|
||||||
|
|
||||||
has_many :post_tags, inverse_of: :tag
|
has_many :post_tags, inverse_of: :tag
|
||||||
has_many :active_post_tags, -> { kept }, class_name: 'PostTag', inverse_of: :tag
|
has_many :active_post_tags, -> { kept }, class_name: 'PostTag', inverse_of: :tag
|
||||||
has_many :post_tags_with_discarded, -> { with_discarded }, class_name: 'PostTag'
|
has_many :post_tags_with_discarded, -> { with_discarded }, class_name: 'PostTag'
|
||||||
@@ -115,60 +105,26 @@ class Tag < ApplicationRecord
|
|||||||
def self.normalise_tags! tag_names, with_tagme: true,
|
def self.normalise_tags! tag_names, with_tagme: true,
|
||||||
with_no_deerjikist: true,
|
with_no_deerjikist: true,
|
||||||
deny_nico: true,
|
deny_nico: true,
|
||||||
deny_deprecated: false,
|
deny_deprecated: false
|
||||||
with_sections: false
|
|
||||||
if deny_nico && tag_names.any? { |n| n.downcase.start_with?('nico:') }
|
if deny_nico && tag_names.any? { |n| n.downcase.start_with?('nico:') }
|
||||||
raise NicoTagNormalisationError
|
raise NicoTagNormalisationError
|
||||||
end
|
end
|
||||||
|
|
||||||
sections = { }
|
|
||||||
tags = tag_names.map do |name|
|
tags = tag_names.map do |name|
|
||||||
raw_name = name
|
|
||||||
pf, cat = CATEGORY_PREFIXES.find { |p, _| name.downcase.start_with?(p) } || ['', nil]
|
pf, cat = CATEGORY_PREFIXES.find { |p, _| name.downcase.start_with?(p) } || ['', nil]
|
||||||
|
name = TagName.canonicalise(name.sub(/\A#{ pf }/i, '')).first
|
||||||
name = name.sub(/\A#{ pf }/i, '')
|
|
||||||
|
|
||||||
sections_by_tag = []
|
|
||||||
while (match = name.match(/\A(\S*?)\[([^\[\]\s]*)-([^\[\]\s]*)\](\S*)\z/))
|
|
||||||
name = "#{ match[1] }#{ match[4] }"
|
|
||||||
next if match[2].empty? && match[3].empty?
|
|
||||||
|
|
||||||
sections_by_tag << normalise_section_range!(
|
|
||||||
begin_raw: match[2],
|
|
||||||
end_raw: match[3],
|
|
||||||
tag_name: name)
|
|
||||||
end
|
|
||||||
|
|
||||||
if name.include?('[') || name.include?(']')
|
|
||||||
raise SectionLiteralParseError.new(raw_name, raw_name)
|
|
||||||
end
|
|
||||||
|
|
||||||
name = TagName.canonicalise(name).first
|
|
||||||
|
|
||||||
find_or_create_by_tag_name!(name, category: (cat || :general)).tap do |tag|
|
find_or_create_by_tag_name!(name, category: (cat || :general)).tap do |tag|
|
||||||
if deny_deprecated && tag.deprecated?
|
if deny_deprecated && tag.deprecated?
|
||||||
raise DeprecatedTagNormalisationError, [tag.name]
|
raise DeprecatedTagNormalisationError, [tag.name]
|
||||||
end
|
end
|
||||||
|
|
||||||
tag.update!(category: cat) if cat && tag.category != cat
|
tag.update!(category: cat) if cat && tag.category != cat
|
||||||
next if sections_by_tag.empty?
|
|
||||||
|
|
||||||
sections[tag.id] ||= []
|
|
||||||
sections[tag.id].concat(sections_by_tag)
|
|
||||||
sections[tag.id] = merge_section_ranges(sections[tag.id])
|
|
||||||
sections.delete(tag.id) if sections[tag.id] == [[0, nil]]
|
|
||||||
end
|
end
|
||||||
end
|
end
|
||||||
|
|
||||||
tags << Tag.tagme if with_tagme && tags.size < 10 && tags.none?(Tag.tagme)
|
tags << Tag.tagme if with_tagme && tags.size < 10 && tags.none?(Tag.tagme)
|
||||||
tags << Tag.no_deerjikist if with_no_deerjikist && tags.all? { |t| !(t.deerjikist?) }
|
tags << Tag.no_deerjikist if with_no_deerjikist && tags.all? { |t| !(t.deerjikist?) }
|
||||||
tags.uniq!(&:id)
|
tags.uniq(&:id)
|
||||||
|
|
||||||
if with_sections
|
|
||||||
{ tags:, sections: }
|
|
||||||
else
|
|
||||||
tags
|
|
||||||
end
|
|
||||||
end
|
end
|
||||||
|
|
||||||
def self.expand_parent_tags tags
|
def self.expand_parent_tags tags
|
||||||
@@ -194,45 +150,6 @@ class Tag < ApplicationRecord
|
|||||||
(result + tags).uniq { |t| t.id }
|
(result + tags).uniq { |t| t.id }
|
||||||
end
|
end
|
||||||
|
|
||||||
def self.normalise_section_range! begin_raw:, end_raw:, tag_name:
|
|
||||||
begin_ms = begin_raw.empty? ? 0 : time_to_ms!(begin_raw, tag_name:)
|
|
||||||
end_ms = end_raw.empty? ? nil : time_to_ms!(end_raw, tag_name:)
|
|
||||||
|
|
||||||
if end_ms
|
|
||||||
begin_ms, end_ms = end_ms, begin_ms if begin_ms > end_ms
|
|
||||||
end_ms = begin_ms + 1 if begin_ms == end_ms
|
|
||||||
end
|
|
||||||
|
|
||||||
[begin_ms, end_ms]
|
|
||||||
end
|
|
||||||
|
|
||||||
def self.merge_section_ranges ranges
|
|
||||||
sorted_ranges = ranges.sort_by { |begin_ms, end_ms| [begin_ms, end_ms || Float::INFINITY] }
|
|
||||||
merged = []
|
|
||||||
|
|
||||||
sorted_ranges.each do |begin_ms, end_ms|
|
|
||||||
if merged.empty?
|
|
||||||
merged << [begin_ms, end_ms]
|
|
||||||
next
|
|
||||||
end
|
|
||||||
|
|
||||||
last_begin_ms, last_end_ms = merged[-1]
|
|
||||||
if last_end_ms.nil? || begin_ms <= last_end_ms
|
|
||||||
merged[-1] = [last_begin_ms, merge_section_end(last_end_ms, end_ms)]
|
|
||||||
else
|
|
||||||
merged << [begin_ms, end_ms]
|
|
||||||
end
|
|
||||||
end
|
|
||||||
|
|
||||||
merged
|
|
||||||
end
|
|
||||||
|
|
||||||
def self.merge_section_end left_end_ms, right_end_ms
|
|
||||||
return nil if left_end_ms.nil? || right_end_ms.nil?
|
|
||||||
|
|
||||||
[left_end_ms, right_end_ms].max
|
|
||||||
end
|
|
||||||
|
|
||||||
def self.find_or_create_by_tag_name! name, category:
|
def self.find_or_create_by_tag_name! name, category:
|
||||||
tn = TagName.find_undiscard_or_create_by!(name: name.to_s.strip)
|
tn = TagName.find_undiscard_or_create_by!(name: name.to_s.strip)
|
||||||
tn = tn.canonical if tn.canonical_id?
|
tn = tn.canonical if tn.canonical_id?
|
||||||
@@ -329,32 +246,6 @@ class Tag < ApplicationRecord
|
|||||||
end
|
end
|
||||||
end
|
end
|
||||||
|
|
||||||
def self.time_to_ms! str, tag_name:
|
|
||||||
match =
|
|
||||||
case str
|
|
||||||
when /\A(?<seconds>\d+)(?:\.(?<ms>\d{1,3}))?\z/
|
|
||||||
{ hours: nil, minutes: nil, seconds: Regexp.last_match[:seconds],
|
|
||||||
ms: Regexp.last_match[:ms] }
|
|
||||||
when /\A(?<minutes>\d+):(?<seconds>[0-5]?\d)(?:\.(?<ms>\d{1,3}))?\z/
|
|
||||||
{ hours: nil, minutes: Regexp.last_match[:minutes],
|
|
||||||
seconds: Regexp.last_match[:seconds],
|
|
||||||
ms: Regexp.last_match[:ms] }
|
|
||||||
when /\A(?<hours>\d+):(?<minutes>[0-5]?\d):(?<seconds>[0-5]?\d)(?:\.(?<ms>\d{1,3}))?\z/
|
|
||||||
{ hours: Regexp.last_match[:hours],
|
|
||||||
minutes: Regexp.last_match[:minutes],
|
|
||||||
seconds: Regexp.last_match[:seconds],
|
|
||||||
ms: Regexp.last_match[:ms] }
|
|
||||||
end
|
|
||||||
|
|
||||||
raise SectionLiteralParseError.new(tag_name, str) unless match
|
|
||||||
|
|
||||||
total_s = match[:seconds].to_i
|
|
||||||
total_s += match[:minutes].to_i * 60 if match[:minutes]
|
|
||||||
total_s += match[:hours].to_i * 3_600 if match[:hours]
|
|
||||||
|
|
||||||
total_s * 1_000 + match[:ms].to_s.ljust(3, '0')[0, 3].to_i
|
|
||||||
end
|
|
||||||
|
|
||||||
def nico_tags_cannot_be_deprecated
|
def nico_tags_cannot_be_deprecated
|
||||||
if nico? && deprecated_at.present?
|
if nico? && deprecated_at.present?
|
||||||
errors.add :deprecated_at, 'ニコタグは廃止できません.'
|
errors.add :deprecated_at, 'ニコタグは廃止できません.'
|
||||||
|
|||||||
@@ -6,7 +6,6 @@ class TagImplication < ApplicationRecord
|
|||||||
validates :parent_tag_id, presence: true
|
validates :parent_tag_id, presence: true
|
||||||
|
|
||||||
validate :parent_tag_mustnt_be_itself
|
validate :parent_tag_mustnt_be_itself
|
||||||
validate :parent_tag_mustnt_create_cycle
|
|
||||||
|
|
||||||
private
|
private
|
||||||
|
|
||||||
@@ -15,27 +14,4 @@ class TagImplication < ApplicationRecord
|
|||||||
errors.add :parent_tag_id, '親タグは子タグと同一であってはなりません.'
|
errors.add :parent_tag_id, '親タグは子タグと同一であってはなりません.'
|
||||||
end
|
end
|
||||||
end
|
end
|
||||||
|
|
||||||
def parent_tag_mustnt_create_cycle
|
|
||||||
return if tag_id.blank? || parent_tag_id.blank?
|
|
||||||
return if errors[:parent_tag_id].present?
|
|
||||||
|
|
||||||
seen = { }
|
|
||||||
stack = [parent_tag_id]
|
|
||||||
|
|
||||||
until stack.empty?
|
|
||||||
current_id = stack.pop
|
|
||||||
next if seen[current_id]
|
|
||||||
|
|
||||||
seen[current_id] = true
|
|
||||||
|
|
||||||
if current_id == tag_id
|
|
||||||
errors.add :parent_tag_id, '親タグに子孫タグを指定すると循環します.'
|
|
||||||
errors.add :base, 'タグの親子関係が循環します.'
|
|
||||||
return
|
|
||||||
end
|
|
||||||
|
|
||||||
stack.concat(TagImplication.where(tag_id: current_id).pluck(:parent_tag_id))
|
|
||||||
end
|
|
||||||
end
|
|
||||||
end
|
end
|
||||||
|
|||||||
@@ -7,8 +7,7 @@ class User < ApplicationRecord
|
|||||||
|
|
||||||
has_many :created_posts,
|
has_many :created_posts,
|
||||||
class_name: 'Post', foreign_key: :uploaded_user_id, dependent: :nullify
|
class_name: 'Post', foreign_key: :uploaded_user_id, dependent: :nullify
|
||||||
has_one :setting, dependent: :destroy
|
has_many :settings
|
||||||
has_many :theme_slots, class_name: 'UserThemeSlot', dependent: :destroy
|
|
||||||
has_many :user_ips, dependent: :destroy
|
has_many :user_ips, dependent: :destroy
|
||||||
has_many :ip_addresses, through: :user_ips
|
has_many :ip_addresses, through: :user_ips
|
||||||
has_many :user_post_views, dependent: :destroy
|
has_many :user_post_views, dependent: :destroy
|
||||||
|
|||||||
@@ -1,27 +0,0 @@
|
|||||||
class UserThemeSlot < ApplicationRecord
|
|
||||||
BASE_THEMES = ['light', 'dark'].freeze
|
|
||||||
SLOT_NOS = [1, 2, 3].freeze
|
|
||||||
|
|
||||||
belongs_to :user
|
|
||||||
|
|
||||||
validates :user_id, presence: true
|
|
||||||
validates :base_theme, presence: true, inclusion: { in: BASE_THEMES }
|
|
||||||
validates :slot_no, presence: true, inclusion: { in: SLOT_NOS }
|
|
||||||
validates :tokens, presence: true
|
|
||||||
validates :user_id, uniqueness: { scope: [:base_theme, :slot_no] }
|
|
||||||
|
|
||||||
validate :tokens_must_be_object
|
|
||||||
|
|
||||||
def serializable_hash(options = nil)
|
|
||||||
hash = { only: [:base_theme, :slot_no, :tokens, :created_at, :updated_at] }
|
|
||||||
super(hash.merge(options || {}))
|
|
||||||
end
|
|
||||||
|
|
||||||
private
|
|
||||||
|
|
||||||
def tokens_must_be_object
|
|
||||||
return if tokens.is_a?(Hash)
|
|
||||||
|
|
||||||
errors.add(:tokens, 'は JSON object で指定してください.')
|
|
||||||
end
|
|
||||||
end
|
|
||||||
@@ -1,23 +1,15 @@
|
|||||||
module VersionRecord
|
module VersionRecord
|
||||||
extend ActiveSupport::Concern
|
extend ActiveSupport::Concern
|
||||||
|
|
||||||
DEFAULT_EVENT_TYPE_MAP = { create: 'create',
|
|
||||||
update: 'update',
|
|
||||||
discard: 'discard',
|
|
||||||
restore: 'restore' }.freeze
|
|
||||||
|
|
||||||
def readonly? = persisted?
|
def readonly? = persisted?
|
||||||
|
|
||||||
included do
|
included do
|
||||||
event_type_map = if const_defined?(:EVENT_TYPE_MAP, false)
|
|
||||||
const_get(:EVENT_TYPE_MAP)
|
|
||||||
else
|
|
||||||
DEFAULT_EVENT_TYPE_MAP
|
|
||||||
end
|
|
||||||
|
|
||||||
belongs_to :created_by_user, class_name: 'User', optional: true
|
belongs_to :created_by_user, class_name: 'User', optional: true
|
||||||
|
|
||||||
enum :event_type, event_type_map, prefix: true, validate: true
|
enum :event_type, { create: 'create',
|
||||||
|
update: 'update',
|
||||||
|
discard: 'discard',
|
||||||
|
restore: 'restore' }, prefix: true, validate: true
|
||||||
|
|
||||||
validates :version_no, presence: true, numericality: { only_integer: true, greater_than: 0 }
|
validates :version_no, presence: true, numericality: { only_integer: true, greater_than: 0 }
|
||||||
validates :event_type, presence: true
|
validates :event_type, presence: true
|
||||||
|
|||||||
@@ -2,9 +2,7 @@
|
|||||||
|
|
||||||
|
|
||||||
module MaterialRepr
|
module MaterialRepr
|
||||||
BASE = { only: [:id, :url, :version_no, :source_kind, :source_uri,
|
BASE = { only: [:id, :url, :created_at, :updated_at],
|
||||||
:source_path, :source_file_id, :normalized_source_key,
|
|
||||||
:created_at, :updated_at],
|
|
||||||
methods: [:content_type],
|
methods: [:content_type],
|
||||||
include: { tag: TagRepr::BASE,
|
include: { tag: TagRepr::BASE,
|
||||||
created_by_user: UserRepr::BASE,
|
created_by_user: UserRepr::BASE,
|
||||||
@@ -17,93 +15,10 @@ module MaterialRepr
|
|||||||
file: if material.file.attached?
|
file: if material.file.attached?
|
||||||
Rails.application.routes.url_helpers.rails_storage_proxy_url(
|
Rails.application.routes.url_helpers.rails_storage_proxy_url(
|
||||||
material.file, host:)
|
material.file, host:)
|
||||||
end,
|
end)
|
||||||
thumbnail: thumbnail_url(material, host:),
|
|
||||||
thumbnail_fallback_text: thumbnail_fallback_text(material),
|
|
||||||
thumbnail_fallback_kind: thumbnail_fallback_kind(material),
|
|
||||||
media_kind: media_kind(material),
|
|
||||||
file_byte_size: material.file_byte_size,
|
|
||||||
export_paths: export_paths(material),
|
|
||||||
export_items: export_items(material))
|
|
||||||
end
|
end
|
||||||
|
|
||||||
def many materials, host:
|
def many materials, host:
|
||||||
materials.map { |m| base(m, host:) }
|
materials.map { |m| base(m, host:) }
|
||||||
end
|
end
|
||||||
|
|
||||||
def list material, host:
|
|
||||||
{ id: material.id,
|
|
||||||
version_no: material.version_no,
|
|
||||||
url: material.url,
|
|
||||||
source_kind: material.source_kind,
|
|
||||||
source_uri: material.source_uri,
|
|
||||||
source_path: material.source_path,
|
|
||||||
source_file_id: material.source_file_id,
|
|
||||||
normalized_source_key: material.normalized_source_key,
|
|
||||||
tag: compact_tag(material.tag),
|
|
||||||
thumbnail: thumbnail_url(material, host:),
|
|
||||||
thumbnail_fallback_text: thumbnail_fallback_text(material),
|
|
||||||
thumbnail_fallback_kind: thumbnail_fallback_kind(material),
|
|
||||||
media_kind: media_kind(material),
|
|
||||||
content_type: material.content_type,
|
|
||||||
file_byte_size: material.file_byte_size,
|
|
||||||
created_at: material.created_at,
|
|
||||||
updated_at: material.updated_at,
|
|
||||||
export_paths: export_paths(material),
|
|
||||||
export_items: export_items(material) }
|
|
||||||
end
|
|
||||||
|
|
||||||
def list_many materials, host:
|
|
||||||
materials.map { |m| list(m, host:) }
|
|
||||||
end
|
|
||||||
|
|
||||||
def export_paths material
|
|
||||||
material.material_export_items.each_with_object({ }) do |item, hash|
|
|
||||||
hash[item.profile] = item.enabled ? item.export_path : ''
|
|
||||||
end
|
|
||||||
end
|
|
||||||
|
|
||||||
def export_items material
|
|
||||||
material.material_export_items.map do |item|
|
|
||||||
{ id: item.id,
|
|
||||||
profile: item.profile,
|
|
||||||
export_path: item.export_path,
|
|
||||||
enabled: item.enabled }
|
|
||||||
end
|
|
||||||
end
|
|
||||||
|
|
||||||
def thumbnail_url material, host:
|
|
||||||
return nil unless material.thumbnail.attached?
|
|
||||||
|
|
||||||
Rails.application.routes.url_helpers.rails_storage_proxy_url(
|
|
||||||
material.thumbnail, host:)
|
|
||||||
end
|
|
||||||
|
|
||||||
def thumbnail_fallback_text material
|
|
||||||
material.tag&.name || material.created_at&.strftime('%Y-%m-%d')
|
|
||||||
end
|
|
||||||
|
|
||||||
def thumbnail_fallback_kind material
|
|
||||||
material.tag.present? ? 'tag_name' : 'created_at'
|
|
||||||
end
|
|
||||||
|
|
||||||
def media_kind material
|
|
||||||
return 'url_only' unless material.file.attached?
|
|
||||||
|
|
||||||
content_type = material.file.blob.content_type.to_s
|
|
||||||
return 'image' if content_type.start_with?('image/')
|
|
||||||
return 'video' if content_type.start_with?('video/')
|
|
||||||
return 'audio' if content_type.start_with?('audio/')
|
|
||||||
|
|
||||||
'file_other'
|
|
||||||
end
|
|
||||||
|
|
||||||
def compact_tag tag
|
|
||||||
return nil unless tag
|
|
||||||
|
|
||||||
{ id: tag.id,
|
|
||||||
name: tag.name,
|
|
||||||
category: tag.category,
|
|
||||||
deprecated_at: tag.deprecated_at }
|
|
||||||
end
|
|
||||||
end
|
end
|
||||||
|
|||||||
@@ -8,7 +8,6 @@ module PostRepr
|
|||||||
:url,
|
:url,
|
||||||
:title,
|
:title,
|
||||||
:thumbnail_base,
|
:thumbnail_base,
|
||||||
:video_ms,
|
|
||||||
:original_created_from,
|
:original_created_from,
|
||||||
:original_created_before,
|
:original_created_before,
|
||||||
:created_at,
|
:created_at,
|
||||||
@@ -19,7 +18,7 @@ module PostRepr
|
|||||||
|
|
||||||
def base post, current_user = nil
|
def base post, current_user = nil
|
||||||
json = common(post)
|
json = common(post)
|
||||||
json['tags'] = tag_json(post)
|
json['tags'] = tag_json(post.tags)
|
||||||
json['uploaded_user'] = post.uploaded_user && UserRepr.base(post.uploaded_user)
|
json['uploaded_user'] = post.uploaded_user && UserRepr.base(post.uploaded_user)
|
||||||
json['viewed'] = current_user ? current_user.viewed?(post) : false
|
json['viewed'] = current_user ? current_user.viewed?(post) : false
|
||||||
json
|
json
|
||||||
@@ -53,16 +52,8 @@ module PostRepr
|
|||||||
.merge('thumbnail' => thumbnail_url(post))
|
.merge('thumbnail' => thumbnail_url(post))
|
||||||
end
|
end
|
||||||
|
|
||||||
def tag_json post
|
def tag_json tags
|
||||||
post
|
tags.reject(&:deprecated?).map { |tag| TagRepr.inline(tag) }
|
||||||
.active_post_tags
|
|
||||||
.reject { _1.tag.deprecated? }
|
|
||||||
.sort_by { _1.tag.name }
|
|
||||||
.map { |post_tag|
|
|
||||||
TagRepr.inline(post_tag.tag).merge(
|
|
||||||
'children' => [],
|
|
||||||
'sections' => post_tag.sections.as_json(only: [:begin_ms, :end_ms]))
|
|
||||||
}
|
|
||||||
end
|
end
|
||||||
|
|
||||||
def thumbnail_url post
|
def thumbnail_url post
|
||||||
|
|||||||
@@ -1,233 +0,0 @@
|
|||||||
require 'json'
|
|
||||||
require 'jwt'
|
|
||||||
require 'net/http'
|
|
||||||
require 'openssl'
|
|
||||||
require 'tempfile'
|
|
||||||
require 'uri'
|
|
||||||
|
|
||||||
|
|
||||||
module GoogleDrive
|
|
||||||
class ApiClient
|
|
||||||
DRIVE_ENDPOINT = 'https://www.googleapis.com/drive/v3'
|
|
||||||
TOKEN_ENDPOINT = 'https://oauth2.googleapis.com/token'
|
|
||||||
FOLDER_MIME_TYPE = 'application/vnd.google-apps.folder'
|
|
||||||
NATIVE_FILE_MIME_TYPE_PREFIX = 'application/vnd.google-apps.'
|
|
||||||
DRIVE_SCOPE = 'https://www.googleapis.com/auth/drive.readonly'
|
|
||||||
|
|
||||||
def initialize service_account_email: ENV['GOOGLE_DRIVE_SERVICE_ACCOUNT_EMAIL'],
|
|
||||||
private_key: ENV['GOOGLE_DRIVE_PRIVATE_KEY'],
|
|
||||||
private_key_path: ENV['GOOGLE_DRIVE_PRIVATE_KEY_PATH'],
|
|
||||||
subject: ENV['GOOGLE_DRIVE_SUBJECT']
|
|
||||||
@service_account_email = service_account_email.to_s
|
|
||||||
@private_key = load_private_key(private_key, private_key_path)
|
|
||||||
@subject = subject.to_s.presence
|
|
||||||
@access_token = nil
|
|
||||||
end
|
|
||||||
|
|
||||||
def list_material_files_under_folder folder_id
|
|
||||||
files = []
|
|
||||||
each_material_file_under_folder(folder_id) { |entry| files << entry }
|
|
||||||
files
|
|
||||||
end
|
|
||||||
|
|
||||||
def each_material_file_under_folder folder_id
|
|
||||||
return enum_for(__method__, folder_id) unless block_given?
|
|
||||||
|
|
||||||
walk_folder(folder_id, nil) do |entry, relative_path|
|
|
||||||
next if entry['mimeType'] == FOLDER_MIME_TYPE
|
|
||||||
next if native_file?(entry['mimeType'])
|
|
||||||
|
|
||||||
yield build_file_entry(entry, relative_path)
|
|
||||||
end
|
|
||||||
end
|
|
||||||
|
|
||||||
def fetch_material_file file_id
|
|
||||||
metadata = get_file(file_id)
|
|
||||||
return nil if native_file?(metadata['mimeType'])
|
|
||||||
|
|
||||||
build_file_entry(metadata, metadata['name'])
|
|
||||||
end
|
|
||||||
|
|
||||||
def download_to_tempfile file_id, filename:
|
|
||||||
tempfile = Tempfile.new(['material-sync', File.extname(filename.to_s)])
|
|
||||||
tempfile.binmode
|
|
||||||
request_binary("/files/#{ file_id }",
|
|
||||||
{ alt: 'media', supportsAllDrives: true }) do |chunk|
|
|
||||||
tempfile.write(chunk.b)
|
|
||||||
end
|
|
||||||
tempfile.rewind
|
|
||||||
tempfile
|
|
||||||
rescue StandardError
|
|
||||||
tempfile&.close!
|
|
||||||
raise
|
|
||||||
end
|
|
||||||
|
|
||||||
def extract_file_id value
|
|
||||||
raw = value.to_s.strip
|
|
||||||
return nil if raw.blank?
|
|
||||||
return raw unless raw.include?('/')
|
|
||||||
|
|
||||||
uri = URI.parse(raw)
|
|
||||||
return uri.query.to_s[%r{(?:^|&)id=([^&]+)}, 1] if uri.query.present?
|
|
||||||
return uri.path[%r{/folders/([^/]+)}, 1] if uri.path.include?('/folders/')
|
|
||||||
|
|
||||||
uri.path[%r{/d/([^/]+)}, 1]
|
|
||||||
rescue URI::InvalidURIError
|
|
||||||
nil
|
|
||||||
end
|
|
||||||
|
|
||||||
private
|
|
||||||
|
|
||||||
def walk_folder folder_id, prefix, &block
|
|
||||||
list_children(folder_id).each do |entry|
|
|
||||||
relative_path = MaterialSyncExportPath.build(prefix:,
|
|
||||||
relative_path: entry['name'])
|
|
||||||
if entry['mimeType'] == FOLDER_MIME_TYPE
|
|
||||||
walk_folder(entry['id'], relative_path, &block)
|
|
||||||
next
|
|
||||||
end
|
|
||||||
|
|
||||||
block.call(entry, relative_path)
|
|
||||||
end
|
|
||||||
end
|
|
||||||
|
|
||||||
def build_file_entry entry, relative_path
|
|
||||||
{ id: entry['id'],
|
|
||||||
name: entry['name'],
|
|
||||||
mime_type: entry['mimeType'],
|
|
||||||
relative_path: MaterialSyncExportPath.normalize_path(relative_path),
|
|
||||||
sha256_checksum: entry['sha256Checksum'],
|
|
||||||
web_view_link: entry['webViewLink'],
|
|
||||||
web_content_link: entry['webContentLink'] }
|
|
||||||
end
|
|
||||||
|
|
||||||
def list_children folder_id
|
|
||||||
files = []
|
|
||||||
page_token = nil
|
|
||||||
|
|
||||||
loop do
|
|
||||||
response =
|
|
||||||
request_json('/files', {
|
|
||||||
q: "'#{ folder_id }' in parents and trashed = false",
|
|
||||||
fields: 'nextPageToken,files(id,name,mimeType,sha256Checksum,' \
|
|
||||||
'webViewLink,webContentLink)',
|
|
||||||
orderBy: 'folder,name',
|
|
||||||
pageSize: 1000,
|
|
||||||
supportsAllDrives: true,
|
|
||||||
includeItemsFromAllDrives: true,
|
|
||||||
pageToken: page_token }.compact)
|
|
||||||
files.concat(response.fetch('files'))
|
|
||||||
page_token = response['nextPageToken']
|
|
||||||
break if page_token.blank?
|
|
||||||
end
|
|
||||||
|
|
||||||
files
|
|
||||||
end
|
|
||||||
|
|
||||||
def get_file file_id
|
|
||||||
request_json("/files/#{ file_id }",
|
|
||||||
fields: 'id,name,mimeType,sha256Checksum,webViewLink,webContentLink',
|
|
||||||
supportsAllDrives: true)
|
|
||||||
end
|
|
||||||
|
|
||||||
def request_json path, params = {}
|
|
||||||
response = request(:get, path, params:)
|
|
||||||
unless response.is_a?(Net::HTTPSuccess)
|
|
||||||
raise "Google Drive API error: #{ response.code } #{ response.body }"
|
|
||||||
end
|
|
||||||
|
|
||||||
JSON.parse(response.body)
|
|
||||||
end
|
|
||||||
|
|
||||||
def request_binary path, params = {}
|
|
||||||
request(:get, path, params:) do |response|
|
|
||||||
unless response.is_a?(Net::HTTPSuccess)
|
|
||||||
raise "Google Drive download error: #{ response.code } #{ response.body }"
|
|
||||||
end
|
|
||||||
|
|
||||||
response.read_body do |chunk|
|
|
||||||
yield chunk.b
|
|
||||||
end
|
|
||||||
end
|
|
||||||
end
|
|
||||||
|
|
||||||
def request method, path, params: nil
|
|
||||||
uri = URI(DRIVE_ENDPOINT + path)
|
|
||||||
uri.query = URI.encode_www_form(params) if params.present?
|
|
||||||
|
|
||||||
klass =
|
|
||||||
case method
|
|
||||||
when :get
|
|
||||||
Net::HTTP::Get
|
|
||||||
else
|
|
||||||
raise ArgumentError, "Unsupported Google Drive request method: #{ method }"
|
|
||||||
end
|
|
||||||
request = klass.new(uri)
|
|
||||||
request['Authorization'] = "Bearer #{ access_token }"
|
|
||||||
|
|
||||||
Net::HTTP.start(uri.host,
|
|
||||||
uri.port,
|
|
||||||
use_ssl: true,
|
|
||||||
open_timeout: 10,
|
|
||||||
read_timeout: 60) do |http|
|
|
||||||
if block_given?
|
|
||||||
http.request(request) do |response|
|
|
||||||
return yield response
|
|
||||||
end
|
|
||||||
else
|
|
||||||
http.request(request)
|
|
||||||
end
|
|
||||||
end
|
|
||||||
end
|
|
||||||
|
|
||||||
def access_token
|
|
||||||
@access_token ||= fetch_access_token
|
|
||||||
end
|
|
||||||
|
|
||||||
def fetch_access_token
|
|
||||||
raise 'GOOGLE_DRIVE_SERVICE_ACCOUNT_EMAIL is required' if @service_account_email.blank?
|
|
||||||
raise 'Google Drive private key is required' if @private_key.blank?
|
|
||||||
|
|
||||||
payload = { iss: @service_account_email,
|
|
||||||
scope: DRIVE_SCOPE,
|
|
||||||
aud: TOKEN_ENDPOINT,
|
|
||||||
exp: 1.hour.from_now.to_i,
|
|
||||||
iat: Time.current.to_i }
|
|
||||||
payload[:sub] = @subject if @subject.present?
|
|
||||||
|
|
||||||
assertion = JWT.encode(payload, @private_key, 'RS256')
|
|
||||||
uri = URI(TOKEN_ENDPOINT)
|
|
||||||
request = Net::HTTP::Post.new(uri)
|
|
||||||
request.set_form_data(
|
|
||||||
grant_type: 'urn:ietf:params:oauth:grant-type:jwt-bearer',
|
|
||||||
assertion:)
|
|
||||||
|
|
||||||
response =
|
|
||||||
Net::HTTP.start(uri.host,
|
|
||||||
uri.port,
|
|
||||||
use_ssl: true,
|
|
||||||
open_timeout: 10,
|
|
||||||
read_timeout: 30) do |http|
|
|
||||||
http.request(request)
|
|
||||||
end
|
|
||||||
unless response.is_a?(Net::HTTPSuccess)
|
|
||||||
raise "Google OAuth error: #{ response.code } #{ response.body }"
|
|
||||||
end
|
|
||||||
|
|
||||||
JSON.parse(response.body).fetch('access_token')
|
|
||||||
end
|
|
||||||
|
|
||||||
def load_private_key private_key, private_key_path
|
|
||||||
raw = private_key.to_s
|
|
||||||
raw = raw.gsub('\n', "\n") if raw.present?
|
|
||||||
raw = File.read(private_key_path) if raw.blank? && private_key_path.present?
|
|
||||||
return nil if raw.blank?
|
|
||||||
|
|
||||||
OpenSSL::PKey::RSA.new(raw)
|
|
||||||
end
|
|
||||||
|
|
||||||
def native_file? mime_type
|
|
||||||
mime_type.to_s.start_with?(NATIVE_FILE_MIME_TYPE_PREFIX)
|
|
||||||
end
|
|
||||||
end
|
|
||||||
end
|
|
||||||
@@ -1,57 +0,0 @@
|
|||||||
require 'digest'
|
|
||||||
require 'json'
|
|
||||||
|
|
||||||
class MaterialFileSha256
|
|
||||||
def self.blob_metadata blob
|
|
||||||
metadata = blob.metadata
|
|
||||||
return metadata if metadata.is_a?(Hash)
|
|
||||||
return { } if metadata.blank?
|
|
||||||
|
|
||||||
JSON.parse(metadata)
|
|
||||||
rescue JSON::ParserError
|
|
||||||
{ }
|
|
||||||
end
|
|
||||||
|
|
||||||
def self.metadata_sha256 blob
|
|
||||||
blob_metadata(blob)['sha256'].presence
|
|
||||||
end
|
|
||||||
|
|
||||||
def self.assign_metadata_sha256! blob, sha256
|
|
||||||
return if sha256.blank?
|
|
||||||
|
|
||||||
metadata = blob_metadata(blob)
|
|
||||||
metadata['sha256'] = sha256
|
|
||||||
blob.metadata = metadata
|
|
||||||
blob.save! if blob.changed?
|
|
||||||
end
|
|
||||||
|
|
||||||
def self.from_blob blob, allow_download: false
|
|
||||||
sha256 = metadata_sha256(blob)
|
|
||||||
return sha256 if sha256.present?
|
|
||||||
return nil unless allow_download
|
|
||||||
|
|
||||||
begin
|
|
||||||
blob.open do |file|
|
|
||||||
sha256 = Digest::SHA256.file(file.path).hexdigest
|
|
||||||
assign_metadata_sha256!(blob, sha256)
|
|
||||||
sha256
|
|
||||||
end
|
|
||||||
rescue ActiveStorage::FileNotFoundError, ArgumentError => error
|
|
||||||
Rails.logger.warn(
|
|
||||||
"MaterialFileSha256.from_blob failed for blob_id=#{blob.id}: " \
|
|
||||||
"#{error.class}: #{error.message}",
|
|
||||||
)
|
|
||||||
nil
|
|
||||||
end
|
|
||||||
end
|
|
||||||
|
|
||||||
def self.from_upload upload
|
|
||||||
tempfile = upload&.tempfile
|
|
||||||
return nil unless tempfile
|
|
||||||
|
|
||||||
tempfile.rewind
|
|
||||||
Digest::SHA256.file(tempfile.path).hexdigest.tap do
|
|
||||||
tempfile.rewind
|
|
||||||
end
|
|
||||||
end
|
|
||||||
end
|
|
||||||
@@ -1,7 +0,0 @@
|
|||||||
class MaterialImportBlockMatcher
|
|
||||||
def self.match_for_sha256 sha256
|
|
||||||
return nil if sha256.blank?
|
|
||||||
|
|
||||||
MaterialImportBlock.find_by(match_kind: 'sha256', sha256:)
|
|
||||||
end
|
|
||||||
end
|
|
||||||
@@ -1,56 +0,0 @@
|
|||||||
class MaterialSyncExportPath
|
|
||||||
class << self
|
|
||||||
def build prefix: nil, relative_path: nil, filename: nil, source_file_id: nil
|
|
||||||
path = normalize_path(relative_path)
|
|
||||||
path = fallback_filename(filename, source_file_id) if path.blank?
|
|
||||||
|
|
||||||
normalized_prefix = normalize_path(prefix)
|
|
||||||
return path if normalized_prefix.blank?
|
|
||||||
return normalized_prefix if path.blank?
|
|
||||||
|
|
||||||
[normalized_prefix, path].join('/')
|
|
||||||
end
|
|
||||||
|
|
||||||
def normalize_path path
|
|
||||||
raw = path.to_s.delete("\0").tr('\\', '/').strip
|
|
||||||
raw = raw.sub(%r{\A[A-Za-z]:/+}, '')
|
|
||||||
raw = raw.sub(%r{\A/+}, '')
|
|
||||||
return nil if raw.blank?
|
|
||||||
|
|
||||||
segments = raw.split('/').filter_map do |segment|
|
|
||||||
normalized = normalize_segment(segment)
|
|
||||||
normalized.presence
|
|
||||||
end
|
|
||||||
return nil if segments.empty?
|
|
||||||
|
|
||||||
segments.join('/')
|
|
||||||
end
|
|
||||||
|
|
||||||
def uniquify path, source_file_id
|
|
||||||
normalized = normalize_path(path)
|
|
||||||
return normalized if normalized.blank? || source_file_id.blank?
|
|
||||||
|
|
||||||
ext = File.extname(normalized)
|
|
||||||
base = ext.present? ? normalized.delete_suffix(ext) : normalized
|
|
||||||
"#{ base }--#{ source_file_id }#{ ext }"
|
|
||||||
end
|
|
||||||
|
|
||||||
private
|
|
||||||
|
|
||||||
def fallback_filename filename, source_file_id
|
|
||||||
name = normalize_path(filename)
|
|
||||||
return name if name.present?
|
|
||||||
return nil if source_file_id.blank?
|
|
||||||
|
|
||||||
source_file_id.to_s
|
|
||||||
end
|
|
||||||
|
|
||||||
def normalize_segment segment
|
|
||||||
cleaned = segment.to_s.delete("\0").tr('\\/', '_').strip
|
|
||||||
return nil if cleaned.blank? || cleaned == '.'
|
|
||||||
return '__' if cleaned == '..'
|
|
||||||
|
|
||||||
cleaned
|
|
||||||
end
|
|
||||||
end
|
|
||||||
end
|
|
||||||
@@ -1,259 +0,0 @@
|
|||||||
require 'digest'
|
|
||||||
|
|
||||||
class MaterialSyncImporter
|
|
||||||
Result = Struct.new(:material, :action, :suppressed, :suppression, keyword_init: true)
|
|
||||||
|
|
||||||
def self.import! attributes
|
|
||||||
new(attributes).import!
|
|
||||||
end
|
|
||||||
|
|
||||||
def initialize attributes
|
|
||||||
@attributes = attributes
|
|
||||||
end
|
|
||||||
|
|
||||||
def import!
|
|
||||||
source = source_attributes
|
|
||||||
key = normalized_source_key(source)
|
|
||||||
raise ArgumentError, 'normalized_source_key is required for material sync' if key.blank?
|
|
||||||
|
|
||||||
suppression = suppression_for(source)
|
|
||||||
if suppression
|
|
||||||
return Result.new(material: nil,
|
|
||||||
action: :suppressed,
|
|
||||||
suppressed: true,
|
|
||||||
suppression:)
|
|
||||||
end
|
|
||||||
|
|
||||||
material =
|
|
||||||
Material
|
|
||||||
.unscoped
|
|
||||||
.find_or_initialize_by(normalized_source_key: key)
|
|
||||||
if unchanged?(material, source)
|
|
||||||
return Result.new(material:,
|
|
||||||
action: :unchanged,
|
|
||||||
suppressed: false,
|
|
||||||
suppression: nil)
|
|
||||||
end
|
|
||||||
|
|
||||||
event_type =
|
|
||||||
if material.new_record?
|
|
||||||
:create
|
|
||||||
elsif material.discarded?
|
|
||||||
:restore
|
|
||||||
else
|
|
||||||
:update
|
|
||||||
end
|
|
||||||
uploaded_blob = uploaded_blob!
|
|
||||||
if @import_block
|
|
||||||
return Result.new(material: nil,
|
|
||||||
action: :suppressed,
|
|
||||||
suppressed: true,
|
|
||||||
suppression: @import_block)
|
|
||||||
end
|
|
||||||
|
|
||||||
Material.transaction do
|
|
||||||
if material.persisted?
|
|
||||||
MaterialVersionRecorder.ensure_snapshot!(material,
|
|
||||||
created_by_user: @attributes[:updated_by_user])
|
|
||||||
end
|
|
||||||
material.assign_attributes(material_attributes_for(material).merge(source))
|
|
||||||
material.discarded_at = nil if material.respond_to?(:discarded_at=)
|
|
||||||
material.file.attach(uploaded_blob) if uploaded_blob
|
|
||||||
material.save!
|
|
||||||
upsert_export_item!(material)
|
|
||||||
MaterialVersionRecorder.record!(material:,
|
|
||||||
event_type:,
|
|
||||||
created_by_user: @attributes[:updated_by_user])
|
|
||||||
end
|
|
||||||
MaterialThumbnailGenerator.generate!(material)
|
|
||||||
|
|
||||||
Result.new(material:,
|
|
||||||
action: event_type == :create ? :imported : :updated,
|
|
||||||
suppressed: false,
|
|
||||||
suppression: nil)
|
|
||||||
rescue StandardError
|
|
||||||
uploaded_blob&.purge_later
|
|
||||||
raise
|
|
||||||
ensure
|
|
||||||
close_tempfile!
|
|
||||||
end
|
|
||||||
|
|
||||||
private
|
|
||||||
|
|
||||||
def source_attributes
|
|
||||||
{ source_kind: @attributes[:source_kind],
|
|
||||||
source_uri: @attributes[:source_uri],
|
|
||||||
source_path: @attributes[:source_path],
|
|
||||||
source_file_id: @attributes[:source_file_id] }
|
|
||||||
end
|
|
||||||
|
|
||||||
def normalized_source_key source
|
|
||||||
MaterialSyncSuppression.normalize_source_key(source_kind: source[:source_kind],
|
|
||||||
source_uri: source[:source_uri],
|
|
||||||
source_path: source[:source_path],
|
|
||||||
source_file_id: source[:source_file_id])
|
|
||||||
end
|
|
||||||
|
|
||||||
def material_attributes_for material
|
|
||||||
attrs =
|
|
||||||
@attributes
|
|
||||||
.except(:source_kind, :source_uri, :source_path, :source_file_id,
|
|
||||||
:file_blob, :file_tempfile, :file_downloader,
|
|
||||||
:filename, :content_type,
|
|
||||||
:file_sha256, :export_path, :profile, :tag, :url)
|
|
||||||
|
|
||||||
if material.new_record?
|
|
||||||
attrs[:tag] = @attributes[:tag] if @attributes.key?(:tag)
|
|
||||||
attrs[:url] = @attributes[:url] if @attributes.key?(:url)
|
|
||||||
return attrs
|
|
||||||
end
|
|
||||||
|
|
||||||
if @attributes.key?(:tag) && @attributes[:tag].present?
|
|
||||||
attrs[:tag] = @attributes[:tag]
|
|
||||||
end
|
|
||||||
if @attributes.key?(:url) && @attributes[:url].present?
|
|
||||||
attrs[:url] = @attributes[:url]
|
|
||||||
end
|
|
||||||
|
|
||||||
attrs
|
|
||||||
end
|
|
||||||
|
|
||||||
def suppression_for source
|
|
||||||
if source[:source_kind] == 'google_drive_file' && source[:source_path].present?
|
|
||||||
return MaterialSyncSuppressionMatcher.match_google_drive_candidate(
|
|
||||||
drive_file_id: source[:source_file_id],
|
|
||||||
relative_path: source[:source_path])
|
|
||||||
end
|
|
||||||
|
|
||||||
MaterialSyncSuppressionMatcher.match(**source)
|
|
||||||
end
|
|
||||||
|
|
||||||
def uploaded_blob!
|
|
||||||
return @attributes[:file_blob] if @attributes[:file_blob]
|
|
||||||
|
|
||||||
tempfile = upload_tempfile
|
|
||||||
return nil unless tempfile
|
|
||||||
|
|
||||||
tempfile.rewind
|
|
||||||
file_sha256 = @attributes[:file_sha256] || Digest::SHA256.file(tempfile.path).hexdigest
|
|
||||||
@attributes[:file_sha256] = file_sha256
|
|
||||||
|
|
||||||
@import_block = MaterialImportBlockMatcher.match_for_sha256(file_sha256)
|
|
||||||
if @import_block
|
|
||||||
tempfile.rewind
|
|
||||||
return nil
|
|
||||||
end
|
|
||||||
|
|
||||||
blob = ActiveStorage::Blob.create_and_upload!(
|
|
||||||
io: tempfile,
|
|
||||||
filename: @attributes[:filename],
|
|
||||||
content_type: @attributes[:content_type])
|
|
||||||
MaterialFileSha256.assign_metadata_sha256!(blob, file_sha256)
|
|
||||||
tempfile.rewind
|
|
||||||
blob
|
|
||||||
end
|
|
||||||
|
|
||||||
def upload_tempfile
|
|
||||||
return @upload_tempfile if defined?(@upload_tempfile)
|
|
||||||
|
|
||||||
@upload_tempfile = @attributes[:file_tempfile]
|
|
||||||
if @upload_tempfile.blank? && @attributes[:file_downloader]
|
|
||||||
@upload_tempfile = @attributes[:file_downloader].call
|
|
||||||
end
|
|
||||||
@upload_tempfile
|
|
||||||
end
|
|
||||||
|
|
||||||
def upsert_export_item! material
|
|
||||||
export_path = resolved_export_path(material)
|
|
||||||
return if export_path.blank?
|
|
||||||
|
|
||||||
item = material.material_export_items.find_or_initialize_by(profile: effective_profile)
|
|
||||||
item.export_path = export_path
|
|
||||||
item.enabled = true
|
|
||||||
item.created_by_user ||= @attributes[:created_by_user]
|
|
||||||
item.save!
|
|
||||||
end
|
|
||||||
|
|
||||||
def export_path_taken_by_other? material, profile, export_path
|
|
||||||
MaterialExportItem
|
|
||||||
.where(profile:, export_path:)
|
|
||||||
.where.not(material_id: material.id)
|
|
||||||
.exists?
|
|
||||||
end
|
|
||||||
|
|
||||||
def close_tempfile!
|
|
||||||
tempfile = upload_tempfile
|
|
||||||
return unless tempfile
|
|
||||||
|
|
||||||
tempfile.close! unless tempfile.closed?
|
|
||||||
rescue StandardError
|
|
||||||
nil
|
|
||||||
end
|
|
||||||
|
|
||||||
def unchanged? material, source
|
|
||||||
return false if material.new_record? || material.discarded?
|
|
||||||
return false unless same_source_attributes?(material, source)
|
|
||||||
return false unless same_export_path?(material)
|
|
||||||
return false unless same_material_attributes?(material)
|
|
||||||
|
|
||||||
same_file_snapshot?(material)
|
|
||||||
end
|
|
||||||
|
|
||||||
def same_source_attributes? material, source
|
|
||||||
material.source_kind == source[:source_kind] \
|
|
||||||
&& material.source_uri == source[:source_uri] \
|
|
||||||
&& material.source_path == source[:source_path] \
|
|
||||||
&& material.source_file_id == source[:source_file_id]
|
|
||||||
end
|
|
||||||
|
|
||||||
def same_export_path? material
|
|
||||||
expected = resolved_export_path(material)
|
|
||||||
current = material.material_export_items.find { |item| item.profile == effective_profile }
|
|
||||||
|
|
||||||
return current.blank? if expected.blank?
|
|
||||||
|
|
||||||
current&.enabled && current.export_path == expected
|
|
||||||
end
|
|
||||||
|
|
||||||
def same_material_attributes? material
|
|
||||||
same_tag_attribute?(material) && same_url_attribute?(material)
|
|
||||||
end
|
|
||||||
|
|
||||||
def same_file_snapshot? material
|
|
||||||
expected_sha256 = @attributes[:file_sha256].to_s.presence
|
|
||||||
|
|
||||||
if expected_sha256.present?
|
|
||||||
return false unless material.file.attached?
|
|
||||||
|
|
||||||
return MaterialFileSha256.metadata_sha256(material.file.blob) == expected_sha256
|
|
||||||
end
|
|
||||||
|
|
||||||
!material.file.attached?
|
|
||||||
end
|
|
||||||
|
|
||||||
def effective_profile
|
|
||||||
@attributes[:profile].presence || 'legacy_drive'
|
|
||||||
end
|
|
||||||
|
|
||||||
def resolved_export_path material
|
|
||||||
export_path = @attributes[:export_path].to_s.strip
|
|
||||||
return export_path if export_path.blank?
|
|
||||||
return export_path unless export_path_taken_by_other?(material, effective_profile, export_path)
|
|
||||||
|
|
||||||
MaterialSyncExportPath.uniquify(export_path, @attributes[:source_file_id])
|
|
||||||
end
|
|
||||||
|
|
||||||
def same_tag_attribute? material
|
|
||||||
return true unless @attributes.key?(:tag)
|
|
||||||
return true if @attributes[:tag].blank?
|
|
||||||
|
|
||||||
material.tag_id == @attributes[:tag].id
|
|
||||||
end
|
|
||||||
|
|
||||||
def same_url_attribute? material
|
|
||||||
return true unless @attributes.key?(:url)
|
|
||||||
return true if @attributes[:url].blank?
|
|
||||||
|
|
||||||
material.url == @attributes[:url]
|
|
||||||
end
|
|
||||||
end
|
|
||||||
@@ -1,251 +0,0 @@
|
|||||||
class MaterialSyncRunner
|
|
||||||
Result = Struct.new(:imported, :updated, :unchanged, :suppressed, :failed,
|
|
||||||
:errors, keyword_init: true)
|
|
||||||
|
|
||||||
def self.sync_enabled!
|
|
||||||
results = MaterialSyncSource.enabled.order(:id).map { |source| new(source).sync! }
|
|
||||||
|
|
||||||
Result.new(imported: results.sum(&:imported),
|
|
||||||
updated: results.sum(&:updated),
|
|
||||||
unchanged: results.sum(&:unchanged),
|
|
||||||
suppressed: results.sum(&:suppressed),
|
|
||||||
failed: results.sum(&:failed),
|
|
||||||
errors: results.flat_map(&:errors))
|
|
||||||
end
|
|
||||||
|
|
||||||
def initialize source
|
|
||||||
@source = source
|
|
||||||
end
|
|
||||||
|
|
||||||
def sync!
|
|
||||||
result = Result.new(imported: 0, updated: 0, unchanged: 0,
|
|
||||||
suppressed: 0, failed: 0, errors: [])
|
|
||||||
|
|
||||||
if @source.source_kind == 'google_drive_path'
|
|
||||||
sync_google_drive_path!(result)
|
|
||||||
else
|
|
||||||
candidates.each do |candidate|
|
|
||||||
next if candidate.blank?
|
|
||||||
|
|
||||||
sync_candidate!(candidate, result)
|
|
||||||
end
|
|
||||||
end
|
|
||||||
|
|
||||||
@source.update!(last_synced_at: Time.current)
|
|
||||||
log_result(result)
|
|
||||||
result
|
|
||||||
rescue NotImplementedError, StandardError => e
|
|
||||||
result.failed += 1
|
|
||||||
result.errors << { source_id: @source.id, error: e.message }
|
|
||||||
log_result(result)
|
|
||||||
result
|
|
||||||
end
|
|
||||||
|
|
||||||
private
|
|
||||||
|
|
||||||
def candidates
|
|
||||||
case @source.source_kind
|
|
||||||
when 'uri'
|
|
||||||
[uri_candidate]
|
|
||||||
when 'google_drive_file'
|
|
||||||
[google_drive_file_candidate]
|
|
||||||
when 'legacy_drive_path'
|
|
||||||
raise NotImplementedError, 'legacy_drive_path material sync is not implemented'
|
|
||||||
else
|
|
||||||
raise NotImplementedError, "Unsupported material sync source_kind: #{ @source.source_kind }"
|
|
||||||
end
|
|
||||||
end
|
|
||||||
|
|
||||||
def uri_candidate
|
|
||||||
{ source_kind: @source.source_kind,
|
|
||||||
source_uri: @source.source_uri,
|
|
||||||
source_path: @source.source_path,
|
|
||||||
source_file_id: @source.source_file_id,
|
|
||||||
url: @source.source_uri,
|
|
||||||
tag: nil,
|
|
||||||
created_by_user: @source.created_by_user,
|
|
||||||
updated_by_user: @source.updated_by_user || @source.created_by_user }
|
|
||||||
end
|
|
||||||
|
|
||||||
def sync_candidate! candidate, result
|
|
||||||
block = MaterialImportBlockMatcher.match_for_sha256(candidate[:file_sha256])
|
|
||||||
if block
|
|
||||||
result.suppressed += 1
|
|
||||||
Rails.logger.info(
|
|
||||||
material_sync_log(action: 'suppressed',
|
|
||||||
reason: block.reason,
|
|
||||||
normalized_source_key: candidate_normalized_source_key(candidate)))
|
|
||||||
return
|
|
||||||
end
|
|
||||||
|
|
||||||
suppression = suppression_for(candidate)
|
|
||||||
|
|
||||||
if suppression
|
|
||||||
result.suppressed += 1
|
|
||||||
Rails.logger.info(
|
|
||||||
material_sync_log(action: 'suppressed',
|
|
||||||
normalized_source_key: suppression.normalized_source_key))
|
|
||||||
return
|
|
||||||
end
|
|
||||||
|
|
||||||
import = MaterialSyncImporter.import!(candidate)
|
|
||||||
result.public_send("#{ import.action }=", result.public_send(import.action) + 1)
|
|
||||||
rescue StandardError => e
|
|
||||||
result.failed += 1
|
|
||||||
result.errors << { source_id: @source.id,
|
|
||||||
normalized_source_key: candidate_normalized_source_key(candidate),
|
|
||||||
error: e.message }
|
|
||||||
Rails.logger.warn(
|
|
||||||
material_sync_log(action: 'failed',
|
|
||||||
error: e.message,
|
|
||||||
normalized_source_key: candidate_normalized_source_key(candidate)))
|
|
||||||
ensure
|
|
||||||
close_candidate_tempfile(candidate)
|
|
||||||
end
|
|
||||||
|
|
||||||
def google_drive_path_candidates
|
|
||||||
folder_id = google_drive_folder_id
|
|
||||||
Enumerator.new do |entries|
|
|
||||||
drive_client.each_material_file_under_folder(folder_id).each do |entry|
|
|
||||||
entries << build_google_drive_candidate(entry)
|
|
||||||
end
|
|
||||||
end
|
|
||||||
end
|
|
||||||
|
|
||||||
def sync_google_drive_path! result
|
|
||||||
folder_id = google_drive_folder_id
|
|
||||||
scanned_count = 0
|
|
||||||
|
|
||||||
drive_client.each_material_file_under_folder(folder_id) do |entry|
|
|
||||||
scanned_count += 1
|
|
||||||
sync_candidate!(build_google_drive_candidate(entry), result)
|
|
||||||
log_google_drive_progress(folder_id, scanned_count, result) if progress_log_scan_count?(scanned_count)
|
|
||||||
end
|
|
||||||
|
|
||||||
log_google_drive_progress(folder_id, scanned_count, result, summary: true)
|
|
||||||
end
|
|
||||||
|
|
||||||
def google_drive_file_candidate
|
|
||||||
entry = drive_client.fetch_material_file(google_drive_file_id)
|
|
||||||
return nil unless entry
|
|
||||||
|
|
||||||
build_google_drive_candidate(entry, single_file: true)
|
|
||||||
end
|
|
||||||
|
|
||||||
def build_google_drive_candidate entry, single_file: false
|
|
||||||
relative_path =
|
|
||||||
if single_file
|
|
||||||
nil
|
|
||||||
else
|
|
||||||
entry[:relative_path]
|
|
||||||
end
|
|
||||||
export_path =
|
|
||||||
MaterialSyncExportPath.build(prefix: @source.export_path_prefix,
|
|
||||||
relative_path:,
|
|
||||||
filename: entry[:name],
|
|
||||||
source_file_id: entry[:id])
|
|
||||||
|
|
||||||
{ source_kind: 'google_drive_file',
|
|
||||||
source_uri: entry[:web_view_link] || google_drive_file_url(entry[:id]),
|
|
||||||
source_path: relative_path.presence || MaterialSyncExportPath.normalize_path(entry[:name]),
|
|
||||||
source_file_id: entry[:id],
|
|
||||||
filename: entry[:name],
|
|
||||||
content_type: entry[:mime_type],
|
|
||||||
file_downloader: lambda {
|
|
||||||
drive_client.download_to_tempfile(entry[:id], filename: entry[:name])
|
|
||||||
},
|
|
||||||
file_sha256: entry[:sha256_checksum],
|
|
||||||
export_path:,
|
|
||||||
profile: @source.profile,
|
|
||||||
tag: nil,
|
|
||||||
url: nil,
|
|
||||||
created_by_user: @source.created_by_user,
|
|
||||||
updated_by_user: @source.updated_by_user || @source.created_by_user }
|
|
||||||
end
|
|
||||||
|
|
||||||
def candidate_normalized_source_key candidate
|
|
||||||
MaterialSyncSuppression.normalize_source_key(source_kind: candidate[:source_kind],
|
|
||||||
source_uri: candidate[:source_uri],
|
|
||||||
source_path: candidate[:source_path],
|
|
||||||
source_file_id: candidate[:source_file_id])
|
|
||||||
end
|
|
||||||
|
|
||||||
def suppression_for candidate
|
|
||||||
if candidate[:source_kind] == 'google_drive_file' && candidate[:source_path].present?
|
|
||||||
return MaterialSyncSuppressionMatcher.match_google_drive_candidate(
|
|
||||||
drive_file_id: candidate[:source_file_id],
|
|
||||||
relative_path: candidate[:source_path])
|
|
||||||
end
|
|
||||||
|
|
||||||
MaterialSyncSuppressionMatcher.match(source_kind: candidate[:source_kind],
|
|
||||||
source_uri: candidate[:source_uri],
|
|
||||||
source_path: candidate[:source_path],
|
|
||||||
source_file_id: candidate[:source_file_id])
|
|
||||||
end
|
|
||||||
|
|
||||||
def drive_client
|
|
||||||
@drive_client ||= GoogleDrive::ApiClient.new
|
|
||||||
end
|
|
||||||
|
|
||||||
def google_drive_folder_id
|
|
||||||
google_drive_target_id.tap do |id|
|
|
||||||
if id.blank?
|
|
||||||
raise ArgumentError, 'google_drive_path source_file_id or source_uri is required'
|
|
||||||
end
|
|
||||||
end
|
|
||||||
end
|
|
||||||
|
|
||||||
def google_drive_file_id
|
|
||||||
google_drive_target_id.tap do |id|
|
|
||||||
if id.blank?
|
|
||||||
raise ArgumentError, 'google_drive_file source_file_id or source_uri is required'
|
|
||||||
end
|
|
||||||
end
|
|
||||||
end
|
|
||||||
|
|
||||||
def google_drive_target_id
|
|
||||||
@source.source_file_id.presence || drive_client.extract_file_id(@source.source_uri)
|
|
||||||
end
|
|
||||||
|
|
||||||
def google_drive_file_url file_id
|
|
||||||
"https://drive.google.com/file/d/#{ file_id }/view"
|
|
||||||
end
|
|
||||||
|
|
||||||
def close_candidate_tempfile candidate
|
|
||||||
tempfile = candidate[:file_tempfile]
|
|
||||||
return unless tempfile
|
|
||||||
|
|
||||||
tempfile.close! unless tempfile.closed?
|
|
||||||
rescue StandardError
|
|
||||||
nil
|
|
||||||
end
|
|
||||||
|
|
||||||
def log_result result
|
|
||||||
Rails.logger.info(material_sync_log(imported: result.imported,
|
|
||||||
updated: result.updated,
|
|
||||||
unchanged: result.unchanged,
|
|
||||||
suppressed: result.suppressed,
|
|
||||||
failed: result.failed))
|
|
||||||
end
|
|
||||||
|
|
||||||
def progress_log_scan_count? scanned_count
|
|
||||||
scanned_count == 1 || (scanned_count % 50).zero?
|
|
||||||
end
|
|
||||||
|
|
||||||
def log_google_drive_progress folder_id, scanned_count, result, summary: false
|
|
||||||
Rails.logger.info(
|
|
||||||
material_sync_log(folder_id:,
|
|
||||||
scanned_count:,
|
|
||||||
imported: result.imported,
|
|
||||||
updated: result.updated,
|
|
||||||
unchanged: result.unchanged,
|
|
||||||
suppressed: result.suppressed,
|
|
||||||
failed: result.failed,
|
|
||||||
progress: summary ? 'summary' : 'scan'))
|
|
||||||
end
|
|
||||||
|
|
||||||
def material_sync_log fields
|
|
||||||
{ material_sync_source_id: @source.id,
|
|
||||||
material_sync_source_name: @source.name }.merge(fields).to_json
|
|
||||||
end
|
|
||||||
end
|
|
||||||
@@ -1,68 +0,0 @@
|
|||||||
class MaterialSyncSuppressionMatcher
|
|
||||||
def self.match source_kind:,
|
|
||||||
source_uri: nil,
|
|
||||||
drive_path: nil,
|
|
||||||
drive_file_id: nil,
|
|
||||||
source_path: nil,
|
|
||||||
source_file_id: nil
|
|
||||||
key = MaterialSyncSuppression.normalize_source_key(source_kind:,
|
|
||||||
source_uri:,
|
|
||||||
drive_path:,
|
|
||||||
drive_file_id:,
|
|
||||||
source_path:,
|
|
||||||
source_file_id:)
|
|
||||||
return nil if key.blank?
|
|
||||||
|
|
||||||
MaterialSyncSuppression.find_by(normalized_source_key: key)
|
|
||||||
end
|
|
||||||
|
|
||||||
def self.match_google_drive_candidate drive_file_id:, relative_path:
|
|
||||||
normalized_path = validate_relative_path!(relative_path)
|
|
||||||
|
|
||||||
exact = [
|
|
||||||
normalize_key('google_drive_file', drive_file_id),
|
|
||||||
normalize_key('google_drive_path', normalized_path),
|
|
||||||
normalize_key('legacy_drive_path', normalized_path),
|
|
||||||
].compact
|
|
||||||
|
|
||||||
suppression =
|
|
||||||
MaterialSyncSuppression.find_by(normalized_source_key: exact)
|
|
||||||
return suppression if suppression
|
|
||||||
|
|
||||||
MaterialSyncSuppression
|
|
||||||
.where(source_kind: ['google_drive_path_prefix', 'legacy_drive_path_prefix'])
|
|
||||||
.where('drive_path = :path OR :path LIKE CONCAT(drive_path, "/%")', path: normalized_path)
|
|
||||||
.order(:id)
|
|
||||||
.first
|
|
||||||
end
|
|
||||||
|
|
||||||
def self.suppressed?(...)
|
|
||||||
match(...).present?
|
|
||||||
end
|
|
||||||
|
|
||||||
def self.validate_relative_path! relative_path
|
|
||||||
raw = relative_path.to_s.delete("\0").tr('\\', '/').strip
|
|
||||||
raise ArgumentError, 'relative_path is required' if raw.blank?
|
|
||||||
raise ArgumentError, 'relative_path must be relative' if raw.start_with?('/')
|
|
||||||
raise ArgumentError, 'relative_path must be relative' if raw.match?(/\A[A-Za-z]:\//)
|
|
||||||
if raw.start_with?('My Drive/', 'マイドライブ/')
|
|
||||||
raise ArgumentError, 'relative_path must be relative to source folder'
|
|
||||||
end
|
|
||||||
raise ArgumentError, 'relative_path must not contain //' if raw.include?('//')
|
|
||||||
raise ArgumentError, 'relative_path must not end with /' if raw.end_with?('/')
|
|
||||||
|
|
||||||
parts = raw.split('/')
|
|
||||||
if parts.any? { |part| part.in?(['.', '..']) }
|
|
||||||
raise ArgumentError, 'relative_path must not contain dot segments'
|
|
||||||
end
|
|
||||||
|
|
||||||
raw
|
|
||||||
end
|
|
||||||
|
|
||||||
def self.normalize_key source_kind, value
|
|
||||||
return nil if value.blank?
|
|
||||||
|
|
||||||
MaterialSyncSuppression.normalize_source_key(source_kind:, source_path: value,
|
|
||||||
source_file_id: value)
|
|
||||||
end
|
|
||||||
end
|
|
||||||
@@ -1,51 +0,0 @@
|
|||||||
class MaterialSyncSuppressionRegistrar
|
|
||||||
def self.create! attributes, created_by_user:
|
|
||||||
new(attributes, created_by_user:).create!
|
|
||||||
end
|
|
||||||
|
|
||||||
def initialize attributes, created_by_user:
|
|
||||||
@attributes = attributes
|
|
||||||
@created_by_user = created_by_user
|
|
||||||
end
|
|
||||||
|
|
||||||
def create!
|
|
||||||
suppression = nil
|
|
||||||
|
|
||||||
MaterialSyncSuppression.transaction do
|
|
||||||
suppression = MaterialSyncSuppression.create!(
|
|
||||||
@attributes.merge(created_by_user: @created_by_user))
|
|
||||||
discard_existing_materials!(suppression)
|
|
||||||
end
|
|
||||||
|
|
||||||
suppression
|
|
||||||
end
|
|
||||||
|
|
||||||
private
|
|
||||||
|
|
||||||
def discard_existing_materials! suppression
|
|
||||||
matching_materials(suppression).find_each do |material|
|
|
||||||
MaterialVersionRecorder.ensure_snapshot!(material, created_by_user: @created_by_user)
|
|
||||||
material.discard!
|
|
||||||
MaterialVersionRecorder.record!(material:,
|
|
||||||
event_type: :discard,
|
|
||||||
created_by_user: @created_by_user)
|
|
||||||
end
|
|
||||||
end
|
|
||||||
|
|
||||||
def matching_materials suppression
|
|
||||||
materials = Material.unscoped.kept
|
|
||||||
|
|
||||||
case suppression.source_kind
|
|
||||||
when 'google_drive_path', 'legacy_drive_path'
|
|
||||||
materials.where(source_path: suppression.drive_path)
|
|
||||||
when 'google_drive_path_prefix', 'legacy_drive_path_prefix'
|
|
||||||
path = suppression.drive_path.to_s
|
|
||||||
materials.where(
|
|
||||||
'source_path = :path OR source_path LIKE :prefix',
|
|
||||||
path:,
|
|
||||||
prefix: "#{ path }/%")
|
|
||||||
else
|
|
||||||
materials.where(normalized_source_key: suppression.normalized_source_key)
|
|
||||||
end
|
|
||||||
end
|
|
||||||
end
|
|
||||||
@@ -1,126 +0,0 @@
|
|||||||
# frozen_string_literal: true
|
|
||||||
|
|
||||||
require 'mini_magick'
|
|
||||||
require 'open3'
|
|
||||||
require 'tempfile'
|
|
||||||
|
|
||||||
|
|
||||||
class MaterialThumbnailGenerator
|
|
||||||
SIZE = '180x180'
|
|
||||||
|
|
||||||
class << self
|
|
||||||
def generate! material
|
|
||||||
new(material).generate!
|
|
||||||
end
|
|
||||||
end
|
|
||||||
|
|
||||||
def initialize material
|
|
||||||
@material = material
|
|
||||||
@ffmpeg_stderr = []
|
|
||||||
end
|
|
||||||
|
|
||||||
def generate!
|
|
||||||
return log_result(:no_file) unless @material.file.attached?
|
|
||||||
return log_result(:unsupported_content_type) unless image? || video?
|
|
||||||
|
|
||||||
@material.file.blob.open do |file|
|
|
||||||
thumbnail = image? ? image_thumbnail(file.path) : video_thumbnail(file.path)
|
|
||||||
return log_result(:generation_failed) unless thumbnail
|
|
||||||
|
|
||||||
return attach_thumbnail(thumbnail)
|
|
||||||
end
|
|
||||||
rescue ActiveStorage::FileNotFoundError => e
|
|
||||||
log_result(:file_not_found, error: e)
|
|
||||||
rescue MiniMagick::Error => e
|
|
||||||
log_result(:mini_magick_error, error: e)
|
|
||||||
rescue ArgumentError, StandardError => e
|
|
||||||
log_result(:generation_failed, error: e)
|
|
||||||
end
|
|
||||||
|
|
||||||
private
|
|
||||||
|
|
||||||
def image? = content_type.start_with?('image/')
|
|
||||||
|
|
||||||
def video? = content_type.start_with?('video/')
|
|
||||||
|
|
||||||
def content_type
|
|
||||||
return nil unless @material.file.attached?
|
|
||||||
|
|
||||||
@material.file.blob.content_type.to_s
|
|
||||||
end
|
|
||||||
|
|
||||||
def image_thumbnail path
|
|
||||||
image = MiniMagick::Image.open(path)
|
|
||||||
image.resize(SIZE)
|
|
||||||
image.format('jpg')
|
|
||||||
image
|
|
||||||
end
|
|
||||||
|
|
||||||
def video_thumbnail path
|
|
||||||
[1, 0].each do |seconds|
|
|
||||||
tempfile = Tempfile.new(['material-thumbnail', '.jpg'])
|
|
||||||
tempfile.close
|
|
||||||
ok = extract_video_frame(path, tempfile.path, seconds)
|
|
||||||
next unless ok && File.size?(tempfile.path)
|
|
||||||
|
|
||||||
return image_thumbnail(tempfile.path)
|
|
||||||
ensure
|
|
||||||
tempfile&.unlink
|
|
||||||
end
|
|
||||||
|
|
||||||
nil
|
|
||||||
end
|
|
||||||
|
|
||||||
def extract_video_frame input_path, output_path, seconds
|
|
||||||
_stdout, stderr, status =
|
|
||||||
Open3.capture3('ffmpeg',
|
|
||||||
'-y',
|
|
||||||
'-ss', seconds.to_s,
|
|
||||||
'-i', input_path,
|
|
||||||
'-frames:v', '1',
|
|
||||||
'-f', 'image2',
|
|
||||||
output_path)
|
|
||||||
@ffmpeg_stderr << stderr if stderr.present?
|
|
||||||
status.success?
|
|
||||||
rescue Errno::ENOENT => e
|
|
||||||
@ffmpeg_stderr << "ffmpeg unavailable: #{ e.message }"
|
|
||||||
false
|
|
||||||
end
|
|
||||||
|
|
||||||
def attach_thumbnail image
|
|
||||||
blob = nil
|
|
||||||
File.open(image.path) do |io|
|
|
||||||
blob = ActiveStorage::Blob.create_and_upload!(
|
|
||||||
io:,
|
|
||||||
filename: 'material-thumbnail.jpg',
|
|
||||||
content_type: 'image/jpeg')
|
|
||||||
end
|
|
||||||
@material.thumbnail.attach(blob)
|
|
||||||
log_result(:attached)
|
|
||||||
rescue StandardError => e
|
|
||||||
blob&.purge_later
|
|
||||||
log_result(:attach_failed, error: e)
|
|
||||||
end
|
|
||||||
|
|
||||||
def log_result result, error: nil
|
|
||||||
log_payload = { material_id: @material.id,
|
|
||||||
file_blob_id: file_blob_id,
|
|
||||||
content_type:,
|
|
||||||
result:,
|
|
||||||
error_class: error&.class&.name,
|
|
||||||
error_message: error&.message,
|
|
||||||
ffmpeg_stderr: @ffmpeg_stderr.join("\n").presence }
|
|
||||||
if [:attached, :no_file, :unsupported_content_type].include?(result)
|
|
||||||
Rails.logger.info("Material thumbnail generation: #{ log_payload.to_json }")
|
|
||||||
else
|
|
||||||
Rails.logger.warn("Material thumbnail generation: #{ log_payload.to_json }")
|
|
||||||
end
|
|
||||||
result
|
|
||||||
end
|
|
||||||
|
|
||||||
def file_blob_id
|
|
||||||
return nil unless @material.file.attached?
|
|
||||||
|
|
||||||
@material.file.blob.id
|
|
||||||
end
|
|
||||||
end
|
|
||||||
@@ -1,73 +0,0 @@
|
|||||||
class MaterialVersionRecorder < VersionRecorder
|
|
||||||
EVENT_TYPES = ['create', 'update', 'discard', 'restore'].freeze
|
|
||||||
|
|
||||||
def self.record! material:, event_type:, created_by_user:, file_snapshot: nil
|
|
||||||
new(material:, event_type:, created_by_user:, file_snapshot:).record!
|
|
||||||
end
|
|
||||||
|
|
||||||
def initialize material:, event_type:, created_by_user:, file_snapshot: nil
|
|
||||||
@file_snapshot = file_snapshot
|
|
||||||
|
|
||||||
super(record: material, event_type:, created_by_user:)
|
|
||||||
end
|
|
||||||
|
|
||||||
def self.ensure_snapshot! material, created_by_user:
|
|
||||||
return if material.material_versions.exists?
|
|
||||||
|
|
||||||
record!(material:, event_type: :create,
|
|
||||||
created_by_user: material.created_by_user || created_by_user)
|
|
||||||
end
|
|
||||||
|
|
||||||
private
|
|
||||||
|
|
||||||
def version_class = MaterialVersion
|
|
||||||
def version_association = :material_versions
|
|
||||||
def record_key = :material
|
|
||||||
|
|
||||||
def snapshot_attributes
|
|
||||||
blob = @record.file.attached? ? @record.file.blob : nil
|
|
||||||
file_snapshot = build_file_snapshot(blob)
|
|
||||||
|
|
||||||
{ url: @record.url,
|
|
||||||
parent: @record.parent,
|
|
||||||
tag: @record.tag,
|
|
||||||
tag_name: @record.tag&.name,
|
|
||||||
tag_category: @record.tag&.category,
|
|
||||||
source_kind: @record.source_kind,
|
|
||||||
source_uri: @record.source_uri,
|
|
||||||
source_path: @record.source_path,
|
|
||||||
source_file_id: @record.source_file_id,
|
|
||||||
normalized_source_key: @record.normalized_source_key,
|
|
||||||
export_paths_json: @record.snapshot_export_paths,
|
|
||||||
discarded_at: @record.discarded_at,
|
|
||||||
file_blob_id: file_snapshot[:file_blob_id],
|
|
||||||
file_filename: file_snapshot[:file_filename],
|
|
||||||
file_content_type: file_snapshot[:file_content_type],
|
|
||||||
file_byte_size: file_snapshot[:file_byte_size],
|
|
||||||
file_checksum: file_snapshot[:file_checksum],
|
|
||||||
file_sha256: file_snapshot[:file_sha256] }
|
|
||||||
end
|
|
||||||
|
|
||||||
def build_file_snapshot blob
|
|
||||||
return @file_snapshot if @file_snapshot
|
|
||||||
return empty_file_snapshot unless blob
|
|
||||||
|
|
||||||
{ file_blob_id: blob.id,
|
|
||||||
file_filename: blob.filename.to_s,
|
|
||||||
file_content_type: blob.content_type,
|
|
||||||
file_byte_size: blob.byte_size,
|
|
||||||
file_checksum: blob.checksum,
|
|
||||||
file_sha256: MaterialFileSha256.metadata_sha256(blob) }
|
|
||||||
end
|
|
||||||
|
|
||||||
def empty_file_snapshot
|
|
||||||
{ file_blob_id: nil,
|
|
||||||
file_filename: nil,
|
|
||||||
file_content_type: nil,
|
|
||||||
file_byte_size: nil,
|
|
||||||
file_checksum: nil,
|
|
||||||
file_sha256: nil }
|
|
||||||
end
|
|
||||||
|
|
||||||
def event_types = self.class::EVENT_TYPES
|
|
||||||
end
|
|
||||||
@@ -1,148 +0,0 @@
|
|||||||
require 'stringio'
|
|
||||||
require 'zlib'
|
|
||||||
|
|
||||||
# Initial implementation keeps every file payload and the final ZIP in memory.
|
|
||||||
# Keep this service boundary stable so job/cached export paths can replace it later.
|
|
||||||
class MaterialZipExporter
|
|
||||||
Entry = Struct.new(:path, :data, :mtime, keyword_init: true)
|
|
||||||
MissingFile = Struct.new(:material_id, :export_path, :blob_id, :filename, keyword_init: true)
|
|
||||||
|
|
||||||
class EmptyExportError < StandardError; end
|
|
||||||
class DuplicatePathError < StandardError; end
|
|
||||||
class MissingFileError < StandardError
|
|
||||||
attr_reader :missing_files
|
|
||||||
|
|
||||||
def initialize missing_files
|
|
||||||
@missing_files = missing_files
|
|
||||||
|
|
||||||
super("Missing files: #{missing_files.map(&:export_path).join(', ')}")
|
|
||||||
end
|
|
||||||
end
|
|
||||||
|
|
||||||
def initialize profile: 'legacy_drive', tag_id: nil
|
|
||||||
@profile = profile.presence || 'legacy_drive'
|
|
||||||
@tag_id = tag_id.presence
|
|
||||||
end
|
|
||||||
|
|
||||||
def export
|
|
||||||
entries = build_entries
|
|
||||||
raise EmptyExportError if entries.empty?
|
|
||||||
|
|
||||||
ZipWriter.write(entries)
|
|
||||||
end
|
|
||||||
|
|
||||||
private
|
|
||||||
|
|
||||||
def build_entries
|
|
||||||
rows = MaterialExportItem
|
|
||||||
.enabled
|
|
||||||
.includes(material: { file_attachment: :blob })
|
|
||||||
.joins(:material)
|
|
||||||
.merge(Material.kept)
|
|
||||||
.where(profile: @profile)
|
|
||||||
.order(:export_path)
|
|
||||||
|
|
||||||
rows = rows.where(materials: { tag_id: @tag_id }) if @tag_id
|
|
||||||
|
|
||||||
missing_files = []
|
|
||||||
|
|
||||||
entries = rows.filter_map do |item|
|
|
||||||
material = item.material
|
|
||||||
next unless material.file.attached?
|
|
||||||
|
|
||||||
data = download_blob(item, missing_files)
|
|
||||||
next unless data
|
|
||||||
|
|
||||||
Entry.new(path: item.export_path,
|
|
||||||
data:,
|
|
||||||
mtime: material.updated_at || Time.current)
|
|
||||||
end
|
|
||||||
|
|
||||||
raise MissingFileError.new(missing_files) if missing_files.any?
|
|
||||||
|
|
||||||
paths = entries.map(&:path)
|
|
||||||
duplicated = paths.find { |path| paths.count(path) > 1 }
|
|
||||||
raise DuplicatePathError, duplicated if duplicated
|
|
||||||
|
|
||||||
entries
|
|
||||||
end
|
|
||||||
|
|
||||||
def download_blob item, missing_files
|
|
||||||
blob = item.material.file.blob
|
|
||||||
|
|
||||||
blob.download
|
|
||||||
rescue ActiveStorage::FileNotFoundError
|
|
||||||
missing_files << MissingFile.new(
|
|
||||||
material_id: item.material_id,
|
|
||||||
export_path: item.export_path,
|
|
||||||
blob_id: blob.id,
|
|
||||||
filename: blob.filename.to_s,
|
|
||||||
)
|
|
||||||
nil
|
|
||||||
end
|
|
||||||
|
|
||||||
class ZipWriter
|
|
||||||
VERSION_NEEDED = 20
|
|
||||||
GP_FLAG = 0x0800
|
|
||||||
COMPRESSION_STORE = 0
|
|
||||||
|
|
||||||
def self.write entries
|
|
||||||
new(entries).write
|
|
||||||
end
|
|
||||||
|
|
||||||
def initialize entries
|
|
||||||
@entries = entries
|
|
||||||
@central_directory = []
|
|
||||||
end
|
|
||||||
|
|
||||||
def write
|
|
||||||
io = StringIO.new(''.b)
|
|
||||||
|
|
||||||
@entries.each do |entry|
|
|
||||||
write_entry(io, entry)
|
|
||||||
end
|
|
||||||
|
|
||||||
central_start = io.pos
|
|
||||||
@central_directory.each { |header| io.write(header) }
|
|
||||||
central_size = io.pos - central_start
|
|
||||||
|
|
||||||
io.write([0x06054b50, 0, 0, @entries.size, @entries.size,
|
|
||||||
central_size, central_start, 0].pack('VvvvvVVv'))
|
|
||||||
io.string
|
|
||||||
end
|
|
||||||
|
|
||||||
private
|
|
||||||
|
|
||||||
def write_entry io, entry
|
|
||||||
path = entry.path.b
|
|
||||||
data = entry.data.b
|
|
||||||
crc32 = Zlib.crc32(data)
|
|
||||||
dos_time, dos_date = dos_timestamp(entry.mtime)
|
|
||||||
offset = io.pos
|
|
||||||
|
|
||||||
local_header = [0x04034b50, VERSION_NEEDED, GP_FLAG, COMPRESSION_STORE,
|
|
||||||
dos_time, dos_date, crc32, data.bytesize, data.bytesize,
|
|
||||||
path.bytesize, 0].pack('VvvvvvVVVvv')
|
|
||||||
|
|
||||||
io.write(local_header)
|
|
||||||
io.write(path)
|
|
||||||
io.write(data)
|
|
||||||
|
|
||||||
@central_directory << central_header(path:, crc32:, size: data.bytesize,
|
|
||||||
dos_time:, dos_date:, offset:)
|
|
||||||
end
|
|
||||||
|
|
||||||
def central_header path:, crc32:, size:, dos_time:, dos_date:, offset:
|
|
||||||
[0x02014b50, VERSION_NEEDED, VERSION_NEEDED, GP_FLAG, COMPRESSION_STORE,
|
|
||||||
dos_time, dos_date, crc32, size, size, path.bytesize, 0, 0, 0, 0, 0,
|
|
||||||
offset].pack('VvvvvvvVVVvvvvvVV') + path
|
|
||||||
end
|
|
||||||
|
|
||||||
def dos_timestamp time
|
|
||||||
local = time.to_time
|
|
||||||
dos_time = (local.hour << 11) | (local.min << 5) | (local.sec / 2)
|
|
||||||
dos_date = ((local.year - 1980) << 9) | (local.month << 5) | local.day
|
|
||||||
[dos_time, dos_date]
|
|
||||||
end
|
|
||||||
end
|
|
||||||
end
|
|
||||||
@@ -1,122 +0,0 @@
|
|||||||
class PostCreator
|
|
||||||
class VideoMsParseError < ArgumentError; end
|
|
||||||
|
|
||||||
def initialize actor:, attributes:
|
|
||||||
@actor = actor
|
|
||||||
@attributes = attributes.symbolize_keys
|
|
||||||
end
|
|
||||||
|
|
||||||
def create!
|
|
||||||
post = Post.new(title: @attributes[:title].presence,
|
|
||||||
url: @attributes[:url],
|
|
||||||
thumbnail_base: @attributes[:thumbnail_base].presence,
|
|
||||||
uploaded_user: @actor,
|
|
||||||
original_created_from: @attributes[:original_created_from].presence,
|
|
||||||
original_created_before: @attributes[:original_created_before].presence)
|
|
||||||
thumbnail = @attributes[:thumbnail]
|
|
||||||
post.thumbnail.attach(Post.resized_thumbnail_attachment(thumbnail)) if thumbnail.present?
|
|
||||||
|
|
||||||
ApplicationRecord.transaction do
|
|
||||||
post.save!
|
|
||||||
Tag.normalise_tags!(tag_names, deny_deprecated: true, with_sections: true) =>
|
|
||||||
{ tags:, sections: }
|
|
||||||
TagVersioning.record_tag_snapshots!(tags, created_by_user: @actor)
|
|
||||||
tags = Tag.expand_parent_tags(tags).reject(&:deprecated?)
|
|
||||||
post.video_ms = normalise_video_ms(tags)
|
|
||||||
validate_video_sections!(post.video_ms, sections)
|
|
||||||
post.save!
|
|
||||||
sync_post_tags!(post, tags, sections)
|
|
||||||
sync_parent_posts!(post, parent_post_ids)
|
|
||||||
PostVersionRecorder.record!(post:, event_type: :create, created_by_user: @actor)
|
|
||||||
end
|
|
||||||
post
|
|
||||||
end
|
|
||||||
|
|
||||||
private
|
|
||||||
|
|
||||||
def tag_names = @attributes[:tags].to_s.split
|
|
||||||
|
|
||||||
def parent_post_ids
|
|
||||||
Array(@attributes[:parent_post_ids]).flat_map { _1.to_s.split }.map { |token|
|
|
||||||
id = Integer(token, exception: false)
|
|
||||||
raise ArgumentError, "親投稿 Id. が不正です: #{ token }" if id.nil? || id <= 0
|
|
||||||
|
|
||||||
id
|
|
||||||
}.uniq
|
|
||||||
end
|
|
||||||
|
|
||||||
def normalise_video_ms tags
|
|
||||||
return nil unless tags.any? { _1.id == Tag.video.id }
|
|
||||||
|
|
||||||
video_ms = @attributes[:video_ms]
|
|
||||||
if video_ms.present?
|
|
||||||
value = Integer(video_ms, exception: false)
|
|
||||||
raise VideoMsParseError unless value&.positive?
|
|
||||||
|
|
||||||
return value
|
|
||||||
end
|
|
||||||
duration = @attributes[:duration]
|
|
||||||
return nil if duration.blank?
|
|
||||||
|
|
||||||
return duration.to_i if duration.is_a?(Numeric) && duration.to_i.positive?
|
|
||||||
|
|
||||||
value = Tag.time_to_ms!(duration.to_s, tag_name: '動画時間')
|
|
||||||
raise VideoMsParseError unless value.positive?
|
|
||||||
|
|
||||||
value
|
|
||||||
rescue Tag::SectionLiteralParseError
|
|
||||||
raise VideoMsParseError
|
|
||||||
end
|
|
||||||
|
|
||||||
def validate_video_sections! video_ms, sections
|
|
||||||
return unless video_ms
|
|
||||||
|
|
||||||
sections.each_value do |ranges|
|
|
||||||
ranges.each do |begin_ms, end_ms|
|
|
||||||
post = Post.new
|
|
||||||
if begin_ms >= video_ms
|
|
||||||
post.errors.add :video_ms, 'タグ区間の開始が動画時間以上です.'
|
|
||||||
raise ActiveRecord::RecordInvalid, post
|
|
||||||
end
|
|
||||||
if end_ms && end_ms > video_ms
|
|
||||||
post.errors.add :video_ms, 'タグ区間の終端が動画時間を超えてゐます.'
|
|
||||||
raise ActiveRecord::RecordInvalid, post
|
|
||||||
end
|
|
||||||
end
|
|
||||||
end
|
|
||||||
end
|
|
||||||
|
|
||||||
def sync_post_tags! post, desired_tags, sections
|
|
||||||
desired_ids = desired_tags.map(&:id).to_set
|
|
||||||
current_ids = post.tags.pluck(:id).to_set
|
|
||||||
Tag.where(id: desired_ids - current_ids).find_each do |tag|
|
|
||||||
PostTag.create_or_find_by!(post:, tag:, created_user: @actor)
|
|
||||||
end
|
|
||||||
PostTagSection.where(post_id: post.id).destroy_all
|
|
||||||
sections.each do |tag_id, ranges|
|
|
||||||
ranges.each do |begin_ms, end_ms|
|
|
||||||
PostTagSection.create!(post_id: post.id,
|
|
||||||
tag_id:,
|
|
||||||
begin_ms:,
|
|
||||||
end_ms:)
|
|
||||||
end
|
|
||||||
end
|
|
||||||
PostTag.where(post_id: post.id,
|
|
||||||
tag_id: (current_ids - desired_ids).to_a).kept.find_each do |post_tag|
|
|
||||||
post_tag.discard_by!(@actor)
|
|
||||||
end
|
|
||||||
end
|
|
||||||
|
|
||||||
def sync_parent_posts! post, ids
|
|
||||||
if ids.include?(post.id)
|
|
||||||
post.errors.add :parent_post_ids, '自分自身を親投稿にはできません.'
|
|
||||||
raise ActiveRecord::RecordInvalid, post
|
|
||||||
end
|
|
||||||
missing = ids - Post.where(id: ids).pluck(:id)
|
|
||||||
if missing.present?
|
|
||||||
post.errors.add :parent_post_ids, "存在しない親投稿 Id. があります: #{ missing.join(' ') }"
|
|
||||||
raise ActiveRecord::RecordInvalid, post
|
|
||||||
end
|
|
||||||
ids.each { |parent_post_id| PostImplication.create_or_find_by!(post:, parent_post_id:) }
|
|
||||||
end
|
|
||||||
end
|
|
||||||
@@ -1,485 +0,0 @@
|
|||||||
require 'time'
|
|
||||||
require 'timeout'
|
|
||||||
|
|
||||||
class PostImportPreviewer
|
|
||||||
FIELDS = [
|
|
||||||
'title',
|
|
||||||
'thumbnail_base',
|
|
||||||
'original_created_from',
|
|
||||||
'original_created_before',
|
|
||||||
'duration',
|
|
||||||
'tags',
|
|
||||||
'parent_post_ids'].freeze
|
|
||||||
FETCH_WARNING_FIELDS = ['url', 'title', 'thumbnail_base'].freeze
|
|
||||||
TITLE_FETCH_WARNING = 'タイトルを取得できませんでした.'.freeze
|
|
||||||
THUMBNAIL_FETCH_WARNING = 'サムネールを取得できませんでした.'.freeze
|
|
||||||
METADATA_FETCH_WARNING = 'メタデータを取得できませんでした.'.freeze
|
|
||||||
|
|
||||||
def preview_rows rows:, fetch_metadata: true, metadata_cache: { }
|
|
||||||
prepared_rows = rows.map { prepare_row(_1) }
|
|
||||||
url_counts = prepared_rows.filter_map { _1[:normal_url] }.tally
|
|
||||||
existing_posts =
|
|
||||||
Post.where(url: prepared_rows.map { _1[:normal_url] }.compact.uniq).index_by(&:url)
|
|
||||||
existing_parent_ids = preload_parent_ids(prepared_rows)
|
|
||||||
preload_metadata!(
|
|
||||||
prepared_rows,
|
|
||||||
fetch_metadata,
|
|
||||||
metadata_cache,
|
|
||||||
existing_posts,
|
|
||||||
url_counts)
|
|
||||||
known_tags =
|
|
||||||
preload_known_tags(
|
|
||||||
prepared_rows,
|
|
||||||
fetch_metadata,
|
|
||||||
metadata_cache,
|
|
||||||
existing_posts,
|
|
||||||
url_counts)
|
|
||||||
prepared_rows.map { |row|
|
|
||||||
preview_row(row,
|
|
||||||
fetch_metadata:,
|
|
||||||
metadata_cache:,
|
|
||||||
existing_posts:,
|
|
||||||
url_counts:,
|
|
||||||
known_tags:,
|
|
||||||
existing_parent_ids:)
|
|
||||||
}
|
|
||||||
end
|
|
||||||
|
|
||||||
def normalised_url value
|
|
||||||
PostUrlNormaliser.normalise(value)
|
|
||||||
end
|
|
||||||
|
|
||||||
private
|
|
||||||
|
|
||||||
def prepare_row row
|
|
||||||
source = row.symbolize_keys
|
|
||||||
url = source[:url].to_s.strip
|
|
||||||
normal_url = normalised_url(url)
|
|
||||||
source.merge(url_text: url, normal_url:, url_error: validate_url_safety(normal_url))
|
|
||||||
end
|
|
||||||
|
|
||||||
def preview_row row,
|
|
||||||
fetch_metadata:,
|
|
||||||
metadata_cache:,
|
|
||||||
existing_posts:,
|
|
||||||
url_counts:,
|
|
||||||
known_tags:,
|
|
||||||
existing_parent_ids:
|
|
||||||
attributes = initial_attributes(row)
|
|
||||||
provenance = initial_provenance(row)
|
|
||||||
tag_sources = initial_tag_sources(row, attributes, provenance)
|
|
||||||
field_warnings = initial_field_warnings(row)
|
|
||||||
base_warnings = initial_base_warnings(row)
|
|
||||||
url = row[:url_text]
|
|
||||||
provenance['url'] = 'manual'
|
|
||||||
normal_url = row[:normal_url]
|
|
||||||
url_for_metadata = normal_url || url
|
|
||||||
existing_post = normal_url.present? ? existing_posts[normal_url] : nil
|
|
||||||
|
|
||||||
validation_errors = {}
|
|
||||||
validation_errors[:url] = ['URL が不正です.'] if normal_url.blank?
|
|
||||||
if row[:url_error].present?
|
|
||||||
validation_errors[:url] = [row[:url_error]]
|
|
||||||
end
|
|
||||||
if normal_url.present? && url_counts[normal_url].to_i > 1
|
|
||||||
validation_errors[:url] = ['URL が重複しています.']
|
|
||||||
end
|
|
||||||
|
|
||||||
if row[:metadata_url].present? && row[:metadata_url] != url_for_metadata
|
|
||||||
clear_automatic_values!(attributes, provenance, tag_sources)
|
|
||||||
field_warnings = { }
|
|
||||||
base_warnings = [ ]
|
|
||||||
end
|
|
||||||
|
|
||||||
if validation_errors.blank? && normal_url.present? && existing_post
|
|
||||||
attributes['tags'] = merged_tags(tag_sources, provenance['tags'])
|
|
||||||
warnings_present = field_warnings.values.any?(&:present?) || base_warnings.present?
|
|
||||||
return { source_row: row[:source_row],
|
|
||||||
url: normal_url,
|
|
||||||
attributes:,
|
|
||||||
provenance:,
|
|
||||||
tag_sources:,
|
|
||||||
metadata_url: url_for_metadata,
|
|
||||||
skip_reason: 'existing',
|
|
||||||
existing_post_id: existing_post.id,
|
|
||||||
field_warnings:,
|
|
||||||
base_warnings:,
|
|
||||||
validation_errors:,
|
|
||||||
status: warnings_present ? 'warning' : 'ready' }
|
|
||||||
end
|
|
||||||
|
|
||||||
should_fetch = validation_errors.blank?
|
|
||||||
should_fetch &&= should_fetch_metadata?(fetch_metadata, row[:source_row].to_i)
|
|
||||||
if should_fetch
|
|
||||||
clear_fetch_warnings!(field_warnings)
|
|
||||||
metadata = metadata_for(url_for_metadata, metadata_cache)
|
|
||||||
if metadata[:validation_errors].present?
|
|
||||||
validation_errors.merge!(metadata[:validation_errors])
|
|
||||||
else
|
|
||||||
apply_metadata!(attributes, provenance, tag_sources, metadata[:data])
|
|
||||||
apply_fetch_warnings!(field_warnings, metadata[:warnings])
|
|
||||||
end
|
|
||||||
end
|
|
||||||
|
|
||||||
attributes['url'] = url
|
|
||||||
validate_basic_data(attributes, validation_errors)
|
|
||||||
validate_preview_tags(merged_tags(tag_sources, provenance['tags']),
|
|
||||||
validation_errors,
|
|
||||||
field_warnings,
|
|
||||||
known_tags)
|
|
||||||
validate_parents(attributes['parent_post_ids'], validation_errors, existing_parent_ids)
|
|
||||||
attributes.delete('url')
|
|
||||||
attributes['tags'] = merged_tags(tag_sources, provenance['tags'])
|
|
||||||
|
|
||||||
warnings_present = field_warnings.values.any?(&:present?) || base_warnings.present?
|
|
||||||
{ source_row: row[:source_row],
|
|
||||||
url: validation_errors[:url].present? ? url : (normal_url || url),
|
|
||||||
attributes:,
|
|
||||||
provenance:,
|
|
||||||
tag_sources:,
|
|
||||||
metadata_url: url_for_metadata,
|
|
||||||
skip_reason: nil,
|
|
||||||
existing_post_id: nil,
|
|
||||||
field_warnings:,
|
|
||||||
base_warnings:,
|
|
||||||
validation_errors:,
|
|
||||||
status: validation_errors.present? ? 'error' : (warnings_present ? 'warning' : 'ready') }
|
|
||||||
end
|
|
||||||
|
|
||||||
def should_fetch_metadata? fetch_metadata, source_row
|
|
||||||
case fetch_metadata
|
|
||||||
when true then true
|
|
||||||
when Integer then fetch_metadata == source_row
|
|
||||||
when false, nil then false
|
|
||||||
else raise ArgumentError, 'メタデータ取得対象が不正です.'
|
|
||||||
end
|
|
||||||
end
|
|
||||||
|
|
||||||
def initial_attributes row
|
|
||||||
attributes = row[:attributes]&.stringify_keys || { }
|
|
||||||
FIELDS.to_h { |field| [field, attributes[field].to_s] }
|
|
||||||
end
|
|
||||||
|
|
||||||
def initial_provenance row
|
|
||||||
provenance = row[:provenance]&.stringify_keys || { }
|
|
||||||
FIELDS.to_h { |field| [field, provenance[field].presence || 'automatic'] }
|
|
||||||
.merge('url' => 'manual')
|
|
||||||
end
|
|
||||||
|
|
||||||
def initial_tag_sources row, attributes, provenance
|
|
||||||
sources = row[:tag_sources]&.stringify_keys || { 'automatic' => '', 'manual' => '' }
|
|
||||||
sources['automatic'] = sources['automatic'].to_s
|
|
||||||
sources['manual'] =
|
|
||||||
provenance['tags'] == 'manual' ? attributes['tags'].to_s : sources['manual'].to_s
|
|
||||||
sources
|
|
||||||
end
|
|
||||||
|
|
||||||
def initial_field_warnings row
|
|
||||||
(row[:field_warnings] || { })
|
|
||||||
.stringify_keys
|
|
||||||
.transform_values { |value| Array(value).map(&:to_s) }
|
|
||||||
end
|
|
||||||
|
|
||||||
def initial_base_warnings row
|
|
||||||
Array(row[:base_warnings]).map(&:to_s)
|
|
||||||
end
|
|
||||||
|
|
||||||
def clear_automatic_values! attributes, provenance, tag_sources
|
|
||||||
['title', 'thumbnail_base', 'original_created_from',
|
|
||||||
'original_created_before', 'duration'].each do |field|
|
|
||||||
attributes[field] = '' if provenance[field] == 'automatic'
|
|
||||||
end
|
|
||||||
tag_sources['automatic'] = ''
|
|
||||||
attributes['tags'] = merged_tags(tag_sources, provenance['tags'])
|
|
||||||
end
|
|
||||||
|
|
||||||
def clear_fetch_warnings! field_warnings
|
|
||||||
FETCH_WARNING_FIELDS.each do |field|
|
|
||||||
field_warnings.delete(field)
|
|
||||||
end
|
|
||||||
end
|
|
||||||
|
|
||||||
def metadata_for url, cache
|
|
||||||
cache[url] ||= fetch_metadata(url)
|
|
||||||
end
|
|
||||||
|
|
||||||
def fetch_metadata url
|
|
||||||
return { data: { }, warnings: { 'url' => ['URL が空です.'] },
|
|
||||||
validation_errors: { } } if url.blank?
|
|
||||||
|
|
||||||
data = sanitise_metadata(PostMetadataFetcher.fetch(url).stringify_keys.compact)
|
|
||||||
warnings = { }
|
|
||||||
add_field_warning!(warnings, 'title', TITLE_FETCH_WARNING) if data['title'].blank?
|
|
||||||
if data['thumbnail_base'].blank?
|
|
||||||
add_field_warning!(warnings, 'thumbnail_base', THUMBNAIL_FETCH_WARNING)
|
|
||||||
end
|
|
||||||
{ data:, warnings:, validation_errors: { } }
|
|
||||||
rescue Preview::UrlSafety::UnsafeUrl => e
|
|
||||||
Rails.logger.info(
|
|
||||||
"post_import_metadata_fetch_unsafe_url "\
|
|
||||||
"#{ { error: e.class.name, message: e.message }.to_json }")
|
|
||||||
{ data: { }, warnings: { }, validation_errors: { url: [e.message] } }
|
|
||||||
rescue Preview::HttpFetcher::FetchFailed,
|
|
||||||
Preview::HttpFetcher::ResponseTooLarge => e
|
|
||||||
Rails.logger.info(
|
|
||||||
"post_import_metadata_fetch_failure "\
|
|
||||||
"#{ { error: e.class.name, message: e.message }.to_json }")
|
|
||||||
{ data: { }, warnings: { 'url' => [METADATA_FETCH_WARNING] }, validation_errors: { } }
|
|
||||||
end
|
|
||||||
|
|
||||||
def sanitise_metadata metadata
|
|
||||||
{
|
|
||||||
'title' => sanitise_metadata_title(metadata['title']),
|
|
||||||
'thumbnail_base' => sanitise_metadata_url(metadata['thumbnail_base']),
|
|
||||||
'original_created_from' => sanitise_metadata_time(metadata['original_created_from']),
|
|
||||||
'original_created_before' => sanitise_metadata_time(metadata['original_created_before']),
|
|
||||||
'duration' => sanitise_metadata_duration(metadata['duration']),
|
|
||||||
'tags' => metadata['tags'].to_s.presence }.compact
|
|
||||||
end
|
|
||||||
|
|
||||||
def sanitise_metadata_title value
|
|
||||||
value.is_a?(String) ? value.presence : nil
|
|
||||||
end
|
|
||||||
|
|
||||||
def sanitise_metadata_url value
|
|
||||||
return nil unless value.is_a?(String)
|
|
||||||
|
|
||||||
PostUrlNormaliser.normalise(value)
|
|
||||||
end
|
|
||||||
|
|
||||||
def sanitise_metadata_time value
|
|
||||||
return nil unless value.is_a?(String)
|
|
||||||
|
|
||||||
time = Time.iso8601(value)
|
|
||||||
time.nsec.zero? ? time.iso8601 : time.iso8601(9)
|
|
||||||
rescue ArgumentError, TypeError
|
|
||||||
nil
|
|
||||||
end
|
|
||||||
|
|
||||||
def sanitise_metadata_duration value
|
|
||||||
return nil unless value.is_a?(Numeric)
|
|
||||||
|
|
||||||
value.positive? ? value.to_i : nil
|
|
||||||
end
|
|
||||||
|
|
||||||
def preload_metadata! prepared_rows, fetch_metadata, metadata_cache, existing_posts, url_counts
|
|
||||||
urls = prepared_rows.filter_map { |row|
|
|
||||||
next unless row[:normal_url].present?
|
|
||||||
next if row[:url_error].present?
|
|
||||||
next unless should_fetch_metadata?(fetch_metadata, row[:source_row].to_i)
|
|
||||||
next if url_counts[row[:normal_url]].to_i > 1
|
|
||||||
next if existing_posts.key?(row[:normal_url])
|
|
||||||
|
|
||||||
row[:normal_url]
|
|
||||||
}.uniq
|
|
||||||
urls = urls.reject { metadata_cache.key?(_1) }
|
|
||||||
return if urls.empty?
|
|
||||||
|
|
||||||
url_queue = Queue.new
|
|
||||||
result_queue = Queue.new
|
|
||||||
urls.each { url_queue << _1 }
|
|
||||||
workers = [urls.length, 4].min.times.map {
|
|
||||||
Thread.new {
|
|
||||||
Rails.application.executor.wrap do
|
|
||||||
loop do
|
|
||||||
url = url_queue.pop(true)
|
|
||||||
result_queue << [url, safe_fetch_metadata(url)]
|
|
||||||
rescue ThreadError
|
|
||||||
break
|
|
||||||
end
|
|
||||||
end
|
|
||||||
}
|
|
||||||
}
|
|
||||||
Timeout.timeout(15) { workers.each(&:join) }
|
|
||||||
rescue Timeout::Error
|
|
||||||
workers&.each(&:kill)
|
|
||||||
ensure
|
|
||||||
workers&.each(&:join)
|
|
||||||
while result_queue&.size.to_i.positive?
|
|
||||||
url, result = result_queue.pop
|
|
||||||
metadata_cache[url] = result
|
|
||||||
end
|
|
||||||
urls&.each do |url|
|
|
||||||
metadata_cache[url] ||= { data: { }, warnings: { 'url' => [METADATA_FETCH_WARNING] } }
|
|
||||||
end
|
|
||||||
end
|
|
||||||
|
|
||||||
def safe_fetch_metadata url
|
|
||||||
fetch_metadata(url)
|
|
||||||
rescue Preview::UrlSafety::UnsafeUrl => e
|
|
||||||
Rails.logger.info(
|
|
||||||
"post_import_metadata_fetch_unsafe_url "\
|
|
||||||
"#{ { error: e.class.name, message: e.message }.to_json }")
|
|
||||||
{ data: { }, warnings: { }, validation_errors: { url: [e.message] } }
|
|
||||||
rescue StandardError => e
|
|
||||||
Rails.logger.error(
|
|
||||||
"post_import_metadata_fetch_unexpected_failure "\
|
|
||||||
"#{ { error: e.class.name, message: e.message }.to_json }")
|
|
||||||
{ data: { }, warnings: { 'url' => [METADATA_FETCH_WARNING] }, validation_errors: { } }
|
|
||||||
end
|
|
||||||
|
|
||||||
def preload_known_tags prepared_rows, fetch_metadata, metadata_cache, existing_posts, url_counts
|
|
||||||
names = prepared_rows.flat_map { |row|
|
|
||||||
attributes = initial_attributes(row)
|
|
||||||
provenance = initial_provenance(row)
|
|
||||||
tag_sources = initial_tag_sources(row, attributes, provenance)
|
|
||||||
if metadata_url_changed?(row)
|
|
||||||
clear_automatic_values!(attributes, provenance, tag_sources)
|
|
||||||
end
|
|
||||||
if should_apply_metadata_to_row?(row, fetch_metadata, existing_posts, url_counts)
|
|
||||||
metadata = metadata_for(row[:normal_url], metadata_cache)
|
|
||||||
apply_metadata!(attributes, provenance, tag_sources, metadata[:data])
|
|
||||||
end
|
|
||||||
preview_tag_names(merged_tags(tag_sources, provenance['tags']))
|
|
||||||
}.compact.uniq
|
|
||||||
return { } if names.empty?
|
|
||||||
|
|
||||||
Tag.joins(:tag_name)
|
|
||||||
.where(tag_names: { name: names })
|
|
||||||
.includes(:tag_name)
|
|
||||||
.to_a
|
|
||||||
.index_by(&:name)
|
|
||||||
end
|
|
||||||
|
|
||||||
def preload_parent_ids prepared_rows
|
|
||||||
ids = prepared_rows.flat_map { |row|
|
|
||||||
attributes = initial_attributes(row)
|
|
||||||
preview_parent_ids(attributes['parent_post_ids'])
|
|
||||||
}.uniq
|
|
||||||
return { } if ids.empty?
|
|
||||||
|
|
||||||
Post.where(id: ids).pluck(:id).to_h { [_1, true] }
|
|
||||||
end
|
|
||||||
|
|
||||||
def metadata_url_changed? row
|
|
||||||
row[:metadata_url].present? && row[:metadata_url] != (row[:normal_url] || row[:url_text])
|
|
||||||
end
|
|
||||||
|
|
||||||
def should_apply_metadata_to_row? row, fetch_metadata, existing_posts, url_counts
|
|
||||||
normal_url = row[:normal_url]
|
|
||||||
return false if normal_url.blank?
|
|
||||||
return false if row[:url_error].present?
|
|
||||||
return false if url_counts[normal_url].to_i > 1
|
|
||||||
return false if existing_posts.key?(normal_url)
|
|
||||||
|
|
||||||
should_fetch_metadata?(fetch_metadata, row[:source_row].to_i)
|
|
||||||
end
|
|
||||||
|
|
||||||
def validate_url_safety normal_url
|
|
||||||
return nil if normal_url.blank?
|
|
||||||
|
|
||||||
Preview::UrlSafety.validate(normal_url)
|
|
||||||
nil
|
|
||||||
rescue Preview::UrlSafety::UnsafeUrl => e
|
|
||||||
e.message
|
|
||||||
end
|
|
||||||
|
|
||||||
def apply_metadata! attributes, provenance, tag_sources, metadata
|
|
||||||
metadata.each do |field, value|
|
|
||||||
if field == 'tags'
|
|
||||||
next if provenance['tags'] == 'manual'
|
|
||||||
|
|
||||||
tag_sources['automatic'] = value.to_s
|
|
||||||
attributes['tags'] = merged_tags(tag_sources)
|
|
||||||
next
|
|
||||||
end
|
|
||||||
next unless provenance[field] == 'automatic'
|
|
||||||
|
|
||||||
attributes[field] = value
|
|
||||||
provenance[field] = 'automatic'
|
|
||||||
end
|
|
||||||
end
|
|
||||||
|
|
||||||
def apply_fetch_warnings! field_warnings, warnings
|
|
||||||
warnings.each do |field, values|
|
|
||||||
values.each do |value|
|
|
||||||
add_field_warning!(field_warnings, field, value)
|
|
||||||
end
|
|
||||||
end
|
|
||||||
end
|
|
||||||
|
|
||||||
def add_field_warning! field_warnings, field, message
|
|
||||||
field_warnings[field] ||= []
|
|
||||||
field_warnings[field] << message unless field_warnings[field].include?(message)
|
|
||||||
end
|
|
||||||
|
|
||||||
def merged_tags sources, origin = nil
|
|
||||||
return sources['manual'].to_s if origin == 'manual'
|
|
||||||
|
|
||||||
sources['automatic'].to_s
|
|
||||||
end
|
|
||||||
|
|
||||||
def preview_tag_names raw
|
|
||||||
names = raw.to_s.split
|
|
||||||
return [] if names.empty?
|
|
||||||
if names.any? { _1.downcase.start_with?('nico:') }
|
|
||||||
return []
|
|
||||||
end
|
|
||||||
|
|
||||||
names.map { |name| TagName.canonicalise(name.sub(/\[.*\]\z/, '')).first }
|
|
||||||
rescue Tag::SectionLiteralParseError
|
|
||||||
[]
|
|
||||||
end
|
|
||||||
|
|
||||||
def validate_preview_tags raw, errors, field_warnings, known_tags
|
|
||||||
names = raw.to_s.split
|
|
||||||
return if names.empty?
|
|
||||||
if names.any? { _1.downcase.start_with?('nico:') }
|
|
||||||
errors[:tags] = ['ニコニコ・タグは直接指定できません.']
|
|
||||||
return
|
|
||||||
end
|
|
||||||
|
|
||||||
parsed = names.map { |name| TagName.canonicalise(name.sub(/\[.*\]\z/, '')).first }
|
|
||||||
existing = parsed.filter_map { known_tags[_1] }
|
|
||||||
deprecated = existing.select(&:deprecated?).map(&:name)
|
|
||||||
errors[:tags] = ["廃止済みタグがあります: #{ deprecated.join(' ') }"] if deprecated.present?
|
|
||||||
known = existing.reject(&:deprecated?).map(&:name)
|
|
||||||
new_tags = parsed.uniq - known
|
|
||||||
if new_tags.present?
|
|
||||||
add_field_warning!(field_warnings, 'tags', "新規タグを作成します: #{ new_tags.join(' ') }")
|
|
||||||
end
|
|
||||||
rescue Tag::SectionLiteralParseError
|
|
||||||
errors[:tags] = ['タグ区間の記法が不正です.']
|
|
||||||
end
|
|
||||||
|
|
||||||
def validate_basic_data attributes, errors
|
|
||||||
post = Post.new(title: attributes['title'].presence,
|
|
||||||
url: attributes['url'],
|
|
||||||
thumbnail_base: attributes['thumbnail_base'].presence,
|
|
||||||
original_created_from: attributes['original_created_from'].presence,
|
|
||||||
original_created_before: attributes['original_created_before'].presence,
|
|
||||||
video_ms: parse_duration(attributes['duration'], errors))
|
|
||||||
post.valid?
|
|
||||||
post.errors.each do |error|
|
|
||||||
next if error.attribute == :url && error.type == :taken
|
|
||||||
|
|
||||||
(errors[error.attribute] ||= []) << error.message
|
|
||||||
end
|
|
||||||
end
|
|
||||||
|
|
||||||
def parse_duration value, errors
|
|
||||||
return nil if value.blank?
|
|
||||||
|
|
||||||
value.is_a?(Numeric) ? value.to_i : Tag.time_to_ms!(value.to_s, tag_name: '動画時間')
|
|
||||||
rescue Tag::SectionLiteralParseError
|
|
||||||
errors[:video_ms] = ['動画時間の記法が不正です.']
|
|
||||||
nil
|
|
||||||
end
|
|
||||||
|
|
||||||
def preview_parent_ids raw
|
|
||||||
raw.to_s.split.map { Integer(_1, exception: false) }.compact
|
|
||||||
end
|
|
||||||
|
|
||||||
def validate_parents raw, errors, existing_parent_ids
|
|
||||||
ids = raw.to_s.split.map { Integer(_1, exception: false) }
|
|
||||||
return if ids.compact.empty? && raw.to_s.blank?
|
|
||||||
|
|
||||||
if ids.any? { _1.nil? || _1 <= 0 }
|
|
||||||
errors[:parent_post_ids] = ['親投稿 Id. が不正です.']
|
|
||||||
return
|
|
||||||
end
|
|
||||||
if ids.uniq.any? { !existing_parent_ids[_1] }
|
|
||||||
errors[:parent_post_ids] = ['存在しない親投稿 Id. があります.']
|
|
||||||
end
|
|
||||||
end
|
|
||||||
end
|
|
||||||
@@ -1,189 +0,0 @@
|
|||||||
class PostImportRowNormaliser
|
|
||||||
ORIGINS = ['automatic', 'manual'].freeze
|
|
||||||
STRING_FIELDS = [
|
|
||||||
'title',
|
|
||||||
'thumbnail_base',
|
|
||||||
'original_created_from',
|
|
||||||
'original_created_before',
|
|
||||||
'tags',
|
|
||||||
'parent_post_ids'].freeze
|
|
||||||
FLEXIBLE_FIELDS = ['duration', 'video_ms'].freeze
|
|
||||||
ATTRIBUTE_FIELDS = (STRING_FIELDS + FLEXIBLE_FIELDS).freeze
|
|
||||||
|
|
||||||
def self.normalise! rows, allow_warning_fields: false
|
|
||||||
raise ArgumentError, '取込行の形式が不正です.' unless rows.is_a?(Array)
|
|
||||||
raise ArgumentError, '取込件数が多すぎます.' if rows.length > PostImportUrlListParser::MAX_ROWS
|
|
||||||
|
|
||||||
normalised_rows = rows.map { normalise_row!(_1, allow_warning_fields:) }
|
|
||||||
source_rows = normalised_rows.map { _1['source_row'] }
|
|
||||||
raise ArgumentError, '元行番号が重複しています.' if source_rows.uniq.length != source_rows.length
|
|
||||||
if normalised_rows.sum { row_bytesize(_1) } > PostImportUrlListParser::MAX_BYTES
|
|
||||||
raise ArgumentError, '取込データが大きすぎます.'
|
|
||||||
end
|
|
||||||
|
|
||||||
normalised_rows
|
|
||||||
end
|
|
||||||
|
|
||||||
def self.normalise_row! row, allow_warning_fields:
|
|
||||||
unless row.is_a?(Hash) || row.is_a?(ActionController::Parameters)
|
|
||||||
raise ArgumentError, '取込行の形式が不正です.'
|
|
||||||
end
|
|
||||||
|
|
||||||
parameters =
|
|
||||||
row.is_a?(ActionController::Parameters) ? row : ActionController::Parameters.new(row)
|
|
||||||
permitted = parameters.permit(*permitted_keys(allow_warning_fields))
|
|
||||||
normalised = permitted.to_h.deep_transform_keys { _1.to_s.underscore }
|
|
||||||
|
|
||||||
normalised['source_row'] = normalise_source_row!(normalised['source_row'])
|
|
||||||
normalise_url!(normalised['url'])
|
|
||||||
normalise_metadata_url!(normalised['metadata_url'])
|
|
||||||
normalise_attributes!(normalised.fetch('attributes', { }))
|
|
||||||
normalise_provenance!(normalised.fetch('provenance', { }))
|
|
||||||
normalise_tag_sources!(normalised['tag_sources'])
|
|
||||||
normalise_warning_values!(normalised, allow_warning_fields:)
|
|
||||||
if row_bytesize(normalised) > PostImportUrlListParser::MAX_BYTES
|
|
||||||
raise ArgumentError, '取込行が大きすぎます.'
|
|
||||||
end
|
|
||||||
|
|
||||||
normalised
|
|
||||||
end
|
|
||||||
|
|
||||||
def self.permitted_keys allow_warning_fields
|
|
||||||
keys = [
|
|
||||||
:source_row,
|
|
||||||
:sourceRow,
|
|
||||||
:url,
|
|
||||||
:metadata_url,
|
|
||||||
:metadataUrl,
|
|
||||||
{ attributes: {} },
|
|
||||||
{ provenance: {} },
|
|
||||||
{ tag_sources: {} },
|
|
||||||
{ tagSources: {} }]
|
|
||||||
return keys unless allow_warning_fields
|
|
||||||
|
|
||||||
keys + [
|
|
||||||
{ field_warnings: {} },
|
|
||||||
{ fieldWarnings: {} },
|
|
||||||
{ base_warnings: [] },
|
|
||||||
{ baseWarnings: [] }]
|
|
||||||
end
|
|
||||||
private_class_method :permitted_keys
|
|
||||||
|
|
||||||
def self.normalise_source_row! value
|
|
||||||
source_row = Integer(value, exception: false)
|
|
||||||
raise ArgumentError, '元行番号が不正です.' if source_row.nil? || source_row <= 0
|
|
||||||
|
|
||||||
source_row
|
|
||||||
end
|
|
||||||
private_class_method :normalise_source_row!
|
|
||||||
|
|
||||||
def self.normalise_url! value
|
|
||||||
unless value.is_a?(String)
|
|
||||||
raise ArgumentError, 'URL の形式が不正です.'
|
|
||||||
end
|
|
||||||
if value.bytesize > PostImportUrlListParser::MAX_URL_BYTES
|
|
||||||
raise ArgumentError, 'URL が長すぎます.'
|
|
||||||
end
|
|
||||||
end
|
|
||||||
private_class_method :normalise_url!
|
|
||||||
|
|
||||||
def self.normalise_metadata_url! value
|
|
||||||
raise ArgumentError, 'metadata_url の形式が不正です.' unless value.nil? || value.is_a?(String)
|
|
||||||
if value.to_s.bytesize > PostImportUrlListParser::MAX_URL_BYTES
|
|
||||||
raise ArgumentError, 'metadata_url が長すぎます.'
|
|
||||||
end
|
|
||||||
end
|
|
||||||
private_class_method :normalise_metadata_url!
|
|
||||||
|
|
||||||
def self.normalise_attributes! attributes
|
|
||||||
raise ArgumentError, 'attributes の形式が不正です.' unless attributes.is_a?(Hash)
|
|
||||||
raise ArgumentError, '取込項目が不正です.' unless (attributes.keys - ATTRIBUTE_FIELDS).empty?
|
|
||||||
|
|
||||||
attributes.each do |key, value|
|
|
||||||
case key
|
|
||||||
when *STRING_FIELDS
|
|
||||||
raise ArgumentError, '取込項目の型が不正です.' unless value.nil? || value.is_a?(String)
|
|
||||||
when *FLEXIBLE_FIELDS
|
|
||||||
unless value.nil? || value.is_a?(String) || value.is_a?(Numeric)
|
|
||||||
raise ArgumentError, '取込項目の型が不正です.'
|
|
||||||
end
|
|
||||||
end
|
|
||||||
if value.to_s.bytesize > PostImportUrlListParser::MAX_URL_BYTES
|
|
||||||
raise ArgumentError, '取込項目が大きすぎます.'
|
|
||||||
end
|
|
||||||
end
|
|
||||||
end
|
|
||||||
private_class_method :normalise_attributes!
|
|
||||||
|
|
||||||
def self.normalise_provenance! provenance
|
|
||||||
unless provenance.is_a?(Hash)
|
|
||||||
raise ArgumentError, 'provenance の形式が不正です.'
|
|
||||||
end
|
|
||||||
|
|
||||||
allowed = ATTRIBUTE_FIELDS + ['url']
|
|
||||||
unless (provenance.keys - allowed).empty?
|
|
||||||
raise ArgumentError, '値の由来が不正です.'
|
|
||||||
end
|
|
||||||
unless provenance.values.all? { ORIGINS.include?(_1) }
|
|
||||||
raise ArgumentError, '値の由来が不正です.'
|
|
||||||
end
|
|
||||||
end
|
|
||||||
private_class_method :normalise_provenance!
|
|
||||||
|
|
||||||
def self.normalise_tag_sources! tag_sources
|
|
||||||
return if tag_sources.nil?
|
|
||||||
|
|
||||||
unless tag_sources.is_a?(Hash)
|
|
||||||
raise ArgumentError, 'タグ由来の形式が不正です.'
|
|
||||||
end
|
|
||||||
unless (tag_sources.keys - ORIGINS).empty?
|
|
||||||
raise ArgumentError, 'タグ由来の形式が不正です.'
|
|
||||||
end
|
|
||||||
unless tag_sources.values.all? { _1.is_a?(String) }
|
|
||||||
raise ArgumentError, 'タグ由来の形式が不正です.'
|
|
||||||
end
|
|
||||||
if tag_sources.values.any? { _1.bytesize > PostImportUrlListParser::MAX_URL_BYTES }
|
|
||||||
raise ArgumentError, 'タグ由来が大きすぎます.'
|
|
||||||
end
|
|
||||||
if tag_sources.values.sum(&:bytesize) > PostImportUrlListParser::MAX_URL_BYTES
|
|
||||||
raise ArgumentError, 'タグ由来が大きすぎます.'
|
|
||||||
end
|
|
||||||
end
|
|
||||||
private_class_method :normalise_tag_sources!
|
|
||||||
|
|
||||||
def self.normalise_warning_values! normalised, allow_warning_fields:
|
|
||||||
return unless allow_warning_fields
|
|
||||||
|
|
||||||
field_warnings = normalised['field_warnings']
|
|
||||||
unless field_warnings.nil? || field_warnings.is_a?(Hash)
|
|
||||||
raise ArgumentError, '警告の形式が不正です.'
|
|
||||||
end
|
|
||||||
field_warnings&.each do |key, values|
|
|
||||||
unless ATTRIBUTE_FIELDS.include?(key) || key == 'url'
|
|
||||||
raise ArgumentError, '警告の形式が不正です.'
|
|
||||||
end
|
|
||||||
unless values.is_a?(Array) && values.all? { _1.is_a?(String) }
|
|
||||||
raise ArgumentError, '警告の形式が不正です.'
|
|
||||||
end
|
|
||||||
if values.any? { _1.bytesize > PostImportUrlListParser::MAX_URL_BYTES }
|
|
||||||
raise ArgumentError, '警告が大きすぎます.'
|
|
||||||
end
|
|
||||||
end
|
|
||||||
|
|
||||||
base_warnings = normalised['base_warnings']
|
|
||||||
return if base_warnings.nil?
|
|
||||||
|
|
||||||
unless base_warnings.is_a?(Array) && base_warnings.all? { _1.is_a?(String) }
|
|
||||||
raise ArgumentError, '警告の形式が不正です.'
|
|
||||||
end
|
|
||||||
if base_warnings.any? { _1.bytesize > PostImportUrlListParser::MAX_URL_BYTES }
|
|
||||||
raise ArgumentError, '警告が大きすぎます.'
|
|
||||||
end
|
|
||||||
end
|
|
||||||
private_class_method :normalise_warning_values!
|
|
||||||
|
|
||||||
def self.row_bytesize row
|
|
||||||
row.to_json.bytesize
|
|
||||||
end
|
|
||||||
private_class_method :row_bytesize
|
|
||||||
end
|
|
||||||
@@ -1,104 +0,0 @@
|
|||||||
class PostImportRunner
|
|
||||||
def initialize actor:, rows:
|
|
||||||
@actor = actor
|
|
||||||
@rows = rows
|
|
||||||
end
|
|
||||||
|
|
||||||
def run
|
|
||||||
normalised_rows = PostImportRowNormaliser.normalise!(@rows)
|
|
||||||
previews = PostImportPreviewer.new.preview_rows(rows: normalised_rows,
|
|
||||||
fetch_metadata: false)
|
|
||||||
preview_map = previews.index_by { _1[:source_row] }
|
|
||||||
|
|
||||||
results = normalised_rows.map do |row|
|
|
||||||
run_row(row, preview_map.fetch(row['source_row']))
|
|
||||||
end
|
|
||||||
|
|
||||||
{ created: results.count { _1[:status] == 'created' },
|
|
||||||
skipped: results.count { _1[:status] == 'skipped' },
|
|
||||||
failed: results.count { _1[:status] == 'failed' },
|
|
||||||
rows: results }
|
|
||||||
end
|
|
||||||
|
|
||||||
private
|
|
||||||
|
|
||||||
def run_row row, preview
|
|
||||||
attributes = row.fetch('attributes', { }).transform_keys { _1.to_s.underscore }
|
|
||||||
return { source_row: row['source_row'],
|
|
||||||
status: 'failed',
|
|
||||||
errors: preview[:validation_errors],
|
|
||||||
recoverable: true } if preview[:validation_errors].present?
|
|
||||||
if preview[:skip_reason] == 'existing'
|
|
||||||
return { source_row: row['source_row'],
|
|
||||||
status: 'skipped',
|
|
||||||
existing_post_id: preview[:existing_post_id] }
|
|
||||||
end
|
|
||||||
|
|
||||||
attributes['tags'] = preview[:attributes]['tags']
|
|
||||||
attributes['url'] = row['url']
|
|
||||||
post = PostCreator.new(actor: @actor, attributes:).create!
|
|
||||||
{ source_row: row['source_row'], status: 'created', post: PostRepr.base(post) }
|
|
||||||
rescue ActiveRecord::RecordInvalid => e
|
|
||||||
existing_post = existing_post_for_race(row, e.record)
|
|
||||||
if existing_post
|
|
||||||
return { source_row: row['source_row'],
|
|
||||||
status: 'skipped',
|
|
||||||
existing_post_id: existing_post.id }
|
|
||||||
end
|
|
||||||
|
|
||||||
{ source_row: row['source_row'],
|
|
||||||
status: 'failed',
|
|
||||||
errors: e.record.errors.to_hash,
|
|
||||||
recoverable: true }
|
|
||||||
rescue ActiveRecord::RecordNotUnique => e
|
|
||||||
raise unless url_record_not_unique?(e)
|
|
||||||
|
|
||||||
existing_post = existing_post_for_race(row)
|
|
||||||
raise unless existing_post
|
|
||||||
|
|
||||||
{ source_row: row['source_row'],
|
|
||||||
status: 'skipped',
|
|
||||||
existing_post_id: existing_post.id }
|
|
||||||
rescue Tag::NicoTagNormalisationError
|
|
||||||
{ source_row: row['source_row'],
|
|
||||||
status: 'failed',
|
|
||||||
errors: { tags: ['ニコニコ・タグは直接指定できません.'] },
|
|
||||||
recoverable: true }
|
|
||||||
rescue Tag::DeprecatedTagNormalisationError
|
|
||||||
{ source_row: row['source_row'],
|
|
||||||
status: 'failed',
|
|
||||||
errors: { tags: ['廃止済みタグは付与できません.'] },
|
|
||||||
recoverable: true }
|
|
||||||
rescue PostCreator::VideoMsParseError
|
|
||||||
{ source_row: row['source_row'],
|
|
||||||
status: 'failed',
|
|
||||||
errors: { duration: ['動画時間の記法が不正です.'] },
|
|
||||||
recoverable: true }
|
|
||||||
rescue ArgumentError
|
|
||||||
{ source_row: row['source_row'],
|
|
||||||
status: 'failed',
|
|
||||||
errors: { base: ['入力値が不正です.'] },
|
|
||||||
recoverable: true }
|
|
||||||
rescue StandardError => e
|
|
||||||
Rails.logger.error("post_import_runner_failure #{ { error: e.class.name,
|
|
||||||
message: e.message }.to_json }")
|
|
||||||
{ source_row: row['source_row'],
|
|
||||||
status: 'failed',
|
|
||||||
errors: { base: ['登録中にエラーが発生しました.'] } }
|
|
||||||
end
|
|
||||||
|
|
||||||
def existing_post_for_race row, record = nil
|
|
||||||
if record && !(record.errors.of_kind?(:url, :taken))
|
|
||||||
return nil
|
|
||||||
end
|
|
||||||
|
|
||||||
normal_url = PostUrlNormaliser.normalise(row['url'])
|
|
||||||
return nil if normal_url.blank?
|
|
||||||
|
|
||||||
Post.find_by(url: normal_url)
|
|
||||||
end
|
|
||||||
|
|
||||||
def url_record_not_unique? error
|
|
||||||
error.message.include?('index_posts_on_url')
|
|
||||||
end
|
|
||||||
end
|
|
||||||
@@ -1,25 +0,0 @@
|
|||||||
class PostImportUrlListParser
|
|
||||||
MAX_ROWS = 100
|
|
||||||
MAX_BYTES = 1.megabyte
|
|
||||||
MAX_URL_BYTES = 20.kilobytes
|
|
||||||
|
|
||||||
def self.parse source
|
|
||||||
raw = source.to_s
|
|
||||||
raise ArgumentError, '入力が大きすぎます.' if raw.bytesize > MAX_BYTES
|
|
||||||
|
|
||||||
rows = raw.split(/\r\n|\n|\r/).each_with_index.filter_map { |line, index|
|
|
||||||
url = line.strip
|
|
||||||
next if url.blank?
|
|
||||||
|
|
||||||
if url.bytesize > MAX_URL_BYTES
|
|
||||||
raise ArgumentError, "#{ index + 1 } 行目: URL が長すぎます."
|
|
||||||
end
|
|
||||||
|
|
||||||
{ source_row: index + 1, url: }
|
|
||||||
}
|
|
||||||
raise ArgumentError, 'URL を入力してください.' if rows.empty?
|
|
||||||
raise ArgumentError, "取込件数は #{ MAX_ROWS } 件までです." if rows.length > MAX_ROWS
|
|
||||||
|
|
||||||
rows
|
|
||||||
end
|
|
||||||
end
|
|
||||||
@@ -1,131 +0,0 @@
|
|||||||
require 'time'
|
|
||||||
|
|
||||||
class PostMetadataFetcher
|
|
||||||
TIMESTAMP_PATTERN =
|
|
||||||
Regexp.new(
|
|
||||||
'\A(\d{4})-(\d{2})-(\d{2})T(\d{2})' \
|
|
||||||
'(?::(\d{2})(?::(\d{2})(?:\.(\d+))?)?)?' \
|
|
||||||
'(Z|[+-]\d{2}:?\d{2})?\z')
|
|
||||||
|
|
||||||
def self.fetch raw_url
|
|
||||||
uri, = Preview::UrlSafety.validate(raw_url)
|
|
||||||
response = Preview::HttpFetcher.fetch(
|
|
||||||
uri.to_s,
|
|
||||||
max_bytes: Preview::ThumbnailFetcher::HTML_MAX_BYTES)
|
|
||||||
metadata = Preview::HtmlMetadataExtractor.extract(response)
|
|
||||||
document = Nokogiri::HTML.parse(response.body)
|
|
||||||
content = lambda { |name|
|
|
||||||
document
|
|
||||||
.at_css("meta[property='#{ name }'], meta[name='#{ name }']")
|
|
||||||
&.[]('content')
|
|
||||||
&.strip
|
|
||||||
&.presence
|
|
||||||
}
|
|
||||||
duration = content.call('og:video:duration') || content.call('video:duration')
|
|
||||||
published = content.call('article:published_time') || content.call('date')
|
|
||||||
created_range = original_created_range(published)
|
|
||||||
platform_tags = platform_tags(uri)
|
|
||||||
{ title: metadata[:title],
|
|
||||||
thumbnail_base:
|
|
||||||
Preview::KnownSiteExtractor.thumbnail_url(uri) || metadata[:image_url],
|
|
||||||
original_created_from: serialise_time(created_range&.first),
|
|
||||||
original_created_before: serialise_time(created_range&.last),
|
|
||||||
duration: duration&.to_f&.then { _1.positive? ? (_1 * 1_000).round : nil },
|
|
||||||
tags: platform_tags.join(' ') }
|
|
||||||
end
|
|
||||||
|
|
||||||
def self.platform_tags uri
|
|
||||||
return ['動画', 'YouTube'] if Preview::KnownSiteExtractor.youtube_video_id(uri)
|
|
||||||
return ['動画', 'ニコニコ'] if Preview::KnownSiteExtractor.niconico_video_id(uri)
|
|
||||||
|
|
||||||
[]
|
|
||||||
end
|
|
||||||
|
|
||||||
def self.original_created_range value
|
|
||||||
return nil if value.blank?
|
|
||||||
|
|
||||||
raw = value.to_s.strip
|
|
||||||
from, before =
|
|
||||||
case raw
|
|
||||||
when /\A(\d{4})\z/
|
|
||||||
year = Regexp.last_match(1).to_i
|
|
||||||
from = Time.zone.local(year, 1, 1)
|
|
||||||
[from, from + 1.year]
|
|
||||||
when /\A(\d{4})-(\d{2})\z/
|
|
||||||
year = Regexp.last_match(1).to_i
|
|
||||||
month = Regexp.last_match(2).to_i
|
|
||||||
from = Time.zone.local(year, month, 1)
|
|
||||||
[from, from + 1.month]
|
|
||||||
when /\A(\d{4})-(\d{2})-(\d{2})\z/
|
|
||||||
year = Regexp.last_match(1).to_i
|
|
||||||
month = Regexp.last_match(2).to_i
|
|
||||||
day = Regexp.last_match(3).to_i
|
|
||||||
from = Time.zone.local(year, month, day)
|
|
||||||
[from, from + 1.day]
|
|
||||||
else
|
|
||||||
parse_timestamp_range(raw)
|
|
||||||
end
|
|
||||||
return nil if from.nil? || before.nil?
|
|
||||||
|
|
||||||
[from, before]
|
|
||||||
rescue ArgumentError, TypeError
|
|
||||||
nil
|
|
||||||
end
|
|
||||||
|
|
||||||
def self.parse_timestamp_range raw
|
|
||||||
match = raw.match(TIMESTAMP_PATTERN)
|
|
||||||
return nil unless match
|
|
||||||
|
|
||||||
year = match[1].to_i
|
|
||||||
month = match[2].to_i
|
|
||||||
day = match[3].to_i
|
|
||||||
hour = match[4].to_i
|
|
||||||
minute = match[5]&.to_i || 0
|
|
||||||
second = match[6]&.to_i || 0
|
|
||||||
fraction = match[7]
|
|
||||||
offset = match[8]
|
|
||||||
whole_second = second + fractional_seconds(fraction)
|
|
||||||
from =
|
|
||||||
if offset.present?
|
|
||||||
Time.new(year, month, day, hour, minute, whole_second, parse_offset(offset)).in_time_zone
|
|
||||||
else
|
|
||||||
Time.zone.local(year, month, day, hour, minute, whole_second)
|
|
||||||
end
|
|
||||||
before =
|
|
||||||
if fraction.present?
|
|
||||||
from + (10**(-fraction.length))
|
|
||||||
elsif match[6].present?
|
|
||||||
from + 1.second
|
|
||||||
elsif match[5].present?
|
|
||||||
from + 1.minute
|
|
||||||
else
|
|
||||||
from + 1.hour
|
|
||||||
end
|
|
||||||
[from, before]
|
|
||||||
end
|
|
||||||
|
|
||||||
def self.parse_offset value
|
|
||||||
return '+00:00' if value == 'Z'
|
|
||||||
|
|
||||||
value.match?(/\A[+-]\d{2}:\d{2}\z/) ? value : "#{ value[0, 3] }:#{ value[3, 2] }"
|
|
||||||
end
|
|
||||||
|
|
||||||
def self.fractional_seconds value
|
|
||||||
return 0 if value.blank?
|
|
||||||
|
|
||||||
Rational(value.to_i, 10**value.length)
|
|
||||||
end
|
|
||||||
|
|
||||||
def self.serialise_time value
|
|
||||||
return nil if value.nil?
|
|
||||||
|
|
||||||
value.nsec.zero? ? value.iso8601 : value.iso8601(9)
|
|
||||||
end
|
|
||||||
|
|
||||||
private_class_method :platform_tags,
|
|
||||||
:original_created_range,
|
|
||||||
:parse_timestamp_range,
|
|
||||||
:parse_offset,
|
|
||||||
:fractional_seconds,
|
|
||||||
:serialise_time
|
|
||||||
end
|
|
||||||
@@ -1,13 +0,0 @@
|
|||||||
class PostUrlNormaliser
|
|
||||||
def self.normalise raw_url
|
|
||||||
value = raw_url.to_s.strip
|
|
||||||
uri = URI.parse(value)
|
|
||||||
return nil unless uri.is_a?(URI::HTTP) && uri.host.present?
|
|
||||||
|
|
||||||
uri.host = uri.host.downcase
|
|
||||||
uri.path = uri.path.sub(/\/\z/, '') if uri.path.present?
|
|
||||||
PostUrlSanitisationRule.sanitise(uri.to_s)
|
|
||||||
rescue URI::InvalidURIError
|
|
||||||
nil
|
|
||||||
end
|
|
||||||
end
|
|
||||||
@@ -23,7 +23,6 @@ class PostVersionRecorder < VersionRecorder
|
|||||||
{ title: @record.title,
|
{ title: @record.title,
|
||||||
url: @record.url,
|
url: @record.url,
|
||||||
thumbnail_base: @record.thumbnail_base,
|
thumbnail_base: @record.thumbnail_base,
|
||||||
video_ms: @record.video_ms,
|
|
||||||
tags: @record.snapshot_tag_names.join(' '),
|
tags: @record.snapshot_tag_names.join(' '),
|
||||||
parent_post_ids: @record.snapshot_parent_post_ids.join(' '),
|
parent_post_ids: @record.snapshot_parent_post_ids.join(' '),
|
||||||
original_created_from: @record.original_created_from,
|
original_created_from: @record.original_created_from,
|
||||||
|
|||||||
@@ -1,21 +0,0 @@
|
|||||||
module Preview
|
|
||||||
class HtmlMetadataExtractor
|
|
||||||
IMAGE_SELECTORS = [
|
|
||||||
'meta[property="og:image"]',
|
|
||||||
'meta[name="twitter:image"]',
|
|
||||||
'meta[name="thumbnail"]'
|
|
||||||
].freeze
|
|
||||||
|
|
||||||
def self.extract(response)
|
|
||||||
document = Nokogiri::HTML.parse(response.body)
|
|
||||||
image_url = IMAGE_SELECTORS.filter_map {
|
|
||||||
document.at_css(_1)&.[]('content')&.strip.presence
|
|
||||||
}.first
|
|
||||||
|
|
||||||
{ title: document.at_css('title')&.text&.strip,
|
|
||||||
image_url: image_url ? URI.join(response.url, image_url).to_s : nil }
|
|
||||||
rescue URI::InvalidURIError
|
|
||||||
{ title: document.at_css('title')&.text&.strip, image_url: nil }
|
|
||||||
end
|
|
||||||
end
|
|
||||||
end
|
|
||||||
@@ -1,127 +0,0 @@
|
|||||||
require 'net/http'
|
|
||||||
require 'json'
|
|
||||||
|
|
||||||
module Preview
|
|
||||||
class HttpFetcher
|
|
||||||
class FetchFailed < StandardError; end
|
|
||||||
class FetchTimeout < FetchFailed; end
|
|
||||||
class ResponseTooLarge < FetchFailed; end
|
|
||||||
|
|
||||||
MAX_REDIRECTS = 5
|
|
||||||
DEFAULT_MAX_BYTES = 5.megabytes
|
|
||||||
|
|
||||||
Response = Data.define(:body, :content_type, :url)
|
|
||||||
|
|
||||||
def self.fetch(raw_url,
|
|
||||||
max_bytes: DEFAULT_MAX_BYTES,
|
|
||||||
redirects: MAX_REDIRECTS)
|
|
||||||
uri, addresses = UrlSafety.validate(raw_url)
|
|
||||||
response = request(uri, addresses.first, max_bytes)
|
|
||||||
|
|
||||||
if response.is_a?(Net::HTTPRedirection)
|
|
||||||
location = response['location']
|
|
||||||
if redirects.zero?
|
|
||||||
log_failure(:redirect_limit,
|
|
||||||
url: uri.to_s,
|
|
||||||
redirects:,
|
|
||||||
location:,
|
|
||||||
content_type: response['content-type'],
|
|
||||||
content_length: response['content-length'])
|
|
||||||
raise FetchFailed, 'redirect が多すぎます.'
|
|
||||||
end
|
|
||||||
|
|
||||||
if location.blank?
|
|
||||||
log_failure(:blank_redirect_location,
|
|
||||||
url: uri.to_s,
|
|
||||||
redirects:,
|
|
||||||
content_type: response['content-type'],
|
|
||||||
content_length: response['content-length'])
|
|
||||||
raise FetchFailed, 'redirect 先が不正です.'
|
|
||||||
end
|
|
||||||
|
|
||||||
redirect_url =
|
|
||||||
begin
|
|
||||||
URI.join(uri, location).to_s
|
|
||||||
rescue URI::InvalidURIError => e
|
|
||||||
log_failure(:invalid_redirect_location,
|
|
||||||
url: uri.to_s,
|
|
||||||
redirects:,
|
|
||||||
location:,
|
|
||||||
error: e.class.name,
|
|
||||||
message: e.message)
|
|
||||||
raise FetchFailed, 'redirect 先が不正です.'
|
|
||||||
end
|
|
||||||
|
|
||||||
return fetch(redirect_url, max_bytes:, redirects: redirects - 1)
|
|
||||||
end
|
|
||||||
|
|
||||||
unless response.is_a?(Net::HTTPSuccess)
|
|
||||||
log_failure(:http_status,
|
|
||||||
url: uri.to_s,
|
|
||||||
code: response.code,
|
|
||||||
content_type: response['content-type'],
|
|
||||||
content_length: response['content-length'])
|
|
||||||
raise FetchFailed, "外部サーバーが HTTP #{ response.code } を返しました."
|
|
||||||
end
|
|
||||||
|
|
||||||
Response.new(response.body, response['content-type'].to_s, uri.to_s)
|
|
||||||
rescue Net::OpenTimeout, Net::ReadTimeout, Timeout::Error => e
|
|
||||||
log_failure(:timeout, url: uri&.to_s || raw_url, error: e.class.name, message: e.message)
|
|
||||||
raise FetchTimeout, e.message
|
|
||||||
rescue SocketError, SystemCallError, OpenSSL::SSL::SSLError, EOFError => e
|
|
||||||
log_failure(:network_error,
|
|
||||||
url: uri&.to_s || raw_url,
|
|
||||||
error: e.class.name,
|
|
||||||
message: e.message)
|
|
||||||
raise FetchFailed, e.message
|
|
||||||
end
|
|
||||||
|
|
||||||
def self.request(uri, ip_address, max_bytes)
|
|
||||||
http = Net::HTTP.new(uri.host, uri.port)
|
|
||||||
http.ipaddr = ip_address
|
|
||||||
http.use_ssl = uri.scheme == 'https'
|
|
||||||
http.open_timeout = 5
|
|
||||||
http.read_timeout = 8
|
|
||||||
http.write_timeout = 5
|
|
||||||
|
|
||||||
request = Net::HTTP::Get.new(uri)
|
|
||||||
request['User-Agent'] = 'BTRC-Hub thumbnail preview'
|
|
||||||
request['Accept'] = 'text/html,image/*;q=0.9,*/*;q=0.1'
|
|
||||||
|
|
||||||
http.request(request) do |response|
|
|
||||||
length = response['content-length'].to_i
|
|
||||||
if length > max_bytes
|
|
||||||
log_failure(:response_too_large,
|
|
||||||
url: uri.to_s,
|
|
||||||
content_type: response['content-type'],
|
|
||||||
content_length: response['content-length'],
|
|
||||||
max_bytes:)
|
|
||||||
raise ResponseTooLarge, '外部データが大きすぎます.'
|
|
||||||
end
|
|
||||||
|
|
||||||
body = +''
|
|
||||||
response.read_body do |chunk|
|
|
||||||
body << chunk
|
|
||||||
next unless body.bytesize > max_bytes
|
|
||||||
|
|
||||||
log_failure(:response_too_large,
|
|
||||||
url: uri.to_s,
|
|
||||||
content_type: response['content-type'],
|
|
||||||
content_length: response['content-length'],
|
|
||||||
bytes_read: body.bytesize,
|
|
||||||
max_bytes:)
|
|
||||||
raise ResponseTooLarge, '外部データが大きすぎます.'
|
|
||||||
end
|
|
||||||
response.instance_variable_set(:@body, body)
|
|
||||||
response.instance_variable_set(:@read, true)
|
|
||||||
return response
|
|
||||||
end
|
|
||||||
end
|
|
||||||
|
|
||||||
def self.log_failure(reason, **payload)
|
|
||||||
Rails.logger.warn("preview_http_fetcher_failure #{ { reason:, **payload }.to_json }")
|
|
||||||
end
|
|
||||||
|
|
||||||
private_class_method :request, :log_failure
|
|
||||||
end
|
|
||||||
end
|
|
||||||
@@ -1,34 +0,0 @@
|
|||||||
module Preview
|
|
||||||
class KnownSiteExtractor
|
|
||||||
def self.thumbnail_url(uri)
|
|
||||||
youtube_thumbnail(uri)
|
|
||||||
end
|
|
||||||
|
|
||||||
def self.youtube_video_id(uri)
|
|
||||||
case uri.host&.downcase
|
|
||||||
when 'youtu.be'
|
|
||||||
uri.path.split('/').reject(&:blank?).first
|
|
||||||
when 'www.youtube.com', 'youtube.com', 'm.youtube.com'
|
|
||||||
uri.path == '/watch' ? URI.decode_www_form(uri.query.to_s).to_h['v'] : nil
|
|
||||||
end&.then { _1 if _1.match?(/\A[A-Za-z0-9_-]{6,20}\z/) }
|
|
||||||
end
|
|
||||||
|
|
||||||
def self.niconico_video_id(uri)
|
|
||||||
case uri.host&.downcase
|
|
||||||
when 'www.nicovideo.jp', 'nicovideo.jp'
|
|
||||||
uri.path[%r{\A/watch/(sm\d+)\z}, 1]
|
|
||||||
when 'nico.ms'
|
|
||||||
uri.path[%r{\A/(sm\d+)\z}, 1]
|
|
||||||
end
|
|
||||||
end
|
|
||||||
|
|
||||||
def self.youtube_thumbnail(uri)
|
|
||||||
id = youtube_video_id(uri)
|
|
||||||
return unless id
|
|
||||||
|
|
||||||
"https://i.ytimg.com/vi/#{ id }/hqdefault.jpg"
|
|
||||||
end
|
|
||||||
|
|
||||||
private_class_method :youtube_thumbnail
|
|
||||||
end
|
|
||||||
end
|
|
||||||
@@ -1,95 +0,0 @@
|
|||||||
module Preview
|
|
||||||
class ThumbnailFetcher
|
|
||||||
class GenerationFailed < StandardError; end
|
|
||||||
ALLOWED_IMAGE_CONTENT_TYPES = [
|
|
||||||
'image/jpeg', 'image/png', 'image/gif', 'image/webp'
|
|
||||||
].freeze
|
|
||||||
HTML_MAX_BYTES = 1.megabyte
|
|
||||||
NICONICO_XML_MAX_BYTES = 256.kilobytes
|
|
||||||
|
|
||||||
def self.fetch(raw_url)
|
|
||||||
uri, = UrlSafety.validate(raw_url)
|
|
||||||
|
|
||||||
known_url = KnownSiteExtractor.thumbnail_url(uri)
|
|
||||||
image = fetch_image_or_nil(known_url) if known_url
|
|
||||||
return image if image
|
|
||||||
|
|
||||||
niconico_url = niconico_thumbnail_url(uri)
|
|
||||||
image = fetch_image_or_nil(niconico_url) if niconico_url
|
|
||||||
return image if image
|
|
||||||
|
|
||||||
page = HttpFetcher.fetch(uri.to_s, max_bytes: HTML_MAX_BYTES)
|
|
||||||
metadata = HtmlMetadataExtractor.extract(page)
|
|
||||||
raise GenerationFailed, 'サムネール画像が見つかりませんでした.' if metadata[:image_url].blank?
|
|
||||||
|
|
||||||
fetch_image!(metadata[:image_url])
|
|
||||||
end
|
|
||||||
|
|
||||||
def self.title(raw_url)
|
|
||||||
uri, = UrlSafety.validate(raw_url)
|
|
||||||
HtmlMetadataExtractor.extract(
|
|
||||||
HttpFetcher.fetch(uri.to_s, max_bytes: HTML_MAX_BYTES))[:title]
|
|
||||||
end
|
|
||||||
|
|
||||||
def self.fetch_image_or_nil(url)
|
|
||||||
return nil if url.blank?
|
|
||||||
|
|
||||||
response = HttpFetcher.fetch(url)
|
|
||||||
return nil unless allowed_image_content_type?(response.content_type)
|
|
||||||
|
|
||||||
response.body
|
|
||||||
rescue HttpFetcher::FetchTimeout
|
|
||||||
raise
|
|
||||||
rescue HttpFetcher::FetchFailed
|
|
||||||
nil
|
|
||||||
end
|
|
||||||
|
|
||||||
def self.fetch_image!(url)
|
|
||||||
response = HttpFetcher.fetch(url)
|
|
||||||
unless allowed_image_content_type?(response.content_type)
|
|
||||||
raise GenerationFailed, 'サムネール画像が見つかりませんでした.'
|
|
||||||
end
|
|
||||||
|
|
||||||
response.body
|
|
||||||
rescue HttpFetcher::FetchTimeout
|
|
||||||
raise
|
|
||||||
rescue HttpFetcher::ResponseTooLarge
|
|
||||||
raise
|
|
||||||
rescue HttpFetcher::FetchFailed
|
|
||||||
raise GenerationFailed, 'サムネール画像を取得できませんでした.'
|
|
||||||
end
|
|
||||||
|
|
||||||
def self.niconico_thumbnail_url(uri)
|
|
||||||
video_id = KnownSiteExtractor.niconico_video_id(uri)
|
|
||||||
return nil if video_id.blank?
|
|
||||||
|
|
||||||
response = HttpFetcher.fetch("https://ext.nicovideo.jp/api/getthumbinfo/#{ video_id }",
|
|
||||||
max_bytes: NICONICO_XML_MAX_BYTES)
|
|
||||||
xml = Nokogiri::XML(response.body)
|
|
||||||
return nil unless xml.at_xpath('/nicovideo_thumb_response/@status')&.value == 'ok'
|
|
||||||
|
|
||||||
xml.at_xpath('//thumbnail_url')&.text&.strip.presence
|
|
||||||
rescue HttpFetcher::FetchFailed, HttpFetcher::FetchTimeout => e
|
|
||||||
Rails.logger.info("preview_niconico_getthumbinfo_fallback #{ { url: uri.to_s,
|
|
||||||
video_id:,
|
|
||||||
error: e.class.name,
|
|
||||||
message: e.message }.to_json }")
|
|
||||||
nil
|
|
||||||
rescue Nokogiri::XML::SyntaxError => e
|
|
||||||
Rails.logger.info("preview_niconico_getthumbinfo_fallback #{ { url: uri.to_s,
|
|
||||||
video_id:,
|
|
||||||
error: e.class.name,
|
|
||||||
message: e.message }.to_json }")
|
|
||||||
nil
|
|
||||||
end
|
|
||||||
|
|
||||||
def self.allowed_image_content_type?(content_type)
|
|
||||||
mime_type = content_type.to_s.split(';', 2).first.downcase.strip
|
|
||||||
ALLOWED_IMAGE_CONTENT_TYPES.include?(mime_type)
|
|
||||||
end
|
|
||||||
|
|
||||||
private_class_method :fetch_image_or_nil, :fetch_image!,
|
|
||||||
:niconico_thumbnail_url,
|
|
||||||
:allowed_image_content_type?
|
|
||||||
end
|
|
||||||
end
|
|
||||||
@@ -1,55 +0,0 @@
|
|||||||
require 'resolv'
|
|
||||||
require 'ipaddr'
|
|
||||||
require 'uri'
|
|
||||||
|
|
||||||
module Preview
|
|
||||||
class UrlSafety
|
|
||||||
class UnsafeUrl < StandardError; end
|
|
||||||
|
|
||||||
FORBIDDEN_NETWORKS = [
|
|
||||||
'0.0.0.0/8', '10.0.0.0/8', '100.64.0.0/10', '127.0.0.0/8',
|
|
||||||
'169.254.0.0/16', '172.16.0.0/12', '192.0.0.0/24',
|
|
||||||
'192.0.2.0/24', '192.168.0.0/16', '198.18.0.0/15',
|
|
||||||
'198.51.100.0/24', '203.0.113.0/24', '224.0.0.0/4',
|
|
||||||
'240.0.0.0/4', '::/128', '::1/128', 'fc00::/7', 'fe80::/10',
|
|
||||||
'ff00::/8', '2001:db8::/32', '::ffff:0:0/96'
|
|
||||||
].map { IPAddr.new(_1) }.freeze
|
|
||||||
|
|
||||||
def self.validate(raw_url)
|
|
||||||
value = raw_url.to_s.strip
|
|
||||||
if value.match?(/\A[a-z][a-z0-9+\-.]*:/i)
|
|
||||||
unless value.match?(/\Ahttps?:\/\//i)
|
|
||||||
raise UnsafeUrl, 'http または https の URL を指定してください.'
|
|
||||||
end
|
|
||||||
else
|
|
||||||
value = "http://#{ value }"
|
|
||||||
end
|
|
||||||
uri = URI.parse(value)
|
|
||||||
|
|
||||||
unless ['http', 'https'].include?(uri.scheme&.downcase) && uri.host.present?
|
|
||||||
raise UnsafeUrl, 'http または https の URL を指定してください.'
|
|
||||||
end
|
|
||||||
raise UnsafeUrl, 'userinfo つき URL は使用できません.' if uri.userinfo.present?
|
|
||||||
|
|
||||||
addresses = Resolv.getaddresses(uri.host)
|
|
||||||
raise UnsafeUrl, 'URL のホストを解決できません.' if addresses.empty?
|
|
||||||
|
|
||||||
parsed_addresses = addresses.map { IPAddr.new(_1) }
|
|
||||||
if parsed_addresses.any? { |address| forbidden?(address) }
|
|
||||||
raise UnsafeUrl, '安全でない接続先は使用できません.'
|
|
||||||
end
|
|
||||||
|
|
||||||
[uri, parsed_addresses.map(&:to_s)]
|
|
||||||
rescue Resolv::ResolvError
|
|
||||||
raise UnsafeUrl, 'URL のホストを解決できません.'
|
|
||||||
rescue URI::InvalidURIError, IPAddr::InvalidAddressError
|
|
||||||
raise UnsafeUrl, 'URL が不正です.'
|
|
||||||
end
|
|
||||||
|
|
||||||
def self.forbidden?(address)
|
|
||||||
FORBIDDEN_NETWORKS.any? { _1.include?(address) }
|
|
||||||
end
|
|
||||||
|
|
||||||
private_class_method :forbidden?
|
|
||||||
end
|
|
||||||
end
|
|
||||||
@@ -73,7 +73,7 @@ class VersionRecorder
|
|||||||
end
|
end
|
||||||
|
|
||||||
def validate_event_type!
|
def validate_event_type!
|
||||||
return if event_types.include?(@event_type)
|
return if EVENT_TYPES.include?(@event_type)
|
||||||
|
|
||||||
raise ArgumentError, "Invalid event_type: #{ @event_type }"
|
raise ArgumentError, "Invalid event_type: #{ @event_type }"
|
||||||
end
|
end
|
||||||
@@ -84,5 +84,4 @@ class VersionRecorder
|
|||||||
def snapshot_attributes = raise NotImplementedError
|
def snapshot_attributes = raise NotImplementedError
|
||||||
|
|
||||||
def record_class = @record.class
|
def record_class = @record.class
|
||||||
def event_types = self.class::EVENT_TYPES
|
|
||||||
end
|
end
|
||||||
|
|||||||
@@ -33,12 +33,6 @@ Rails.application.routes.draw do
|
|||||||
get :thumbnail
|
get :thumbnail
|
||||||
end
|
end
|
||||||
|
|
||||||
scope 'posts/import', controller: :post_imports do
|
|
||||||
post :preview
|
|
||||||
post :validate
|
|
||||||
post '', action: :create
|
|
||||||
end
|
|
||||||
|
|
||||||
resources :wiki_pages, path: 'wiki', only: [:index, :show, :create, :update] do
|
resources :wiki_pages, path: 'wiki', only: [:index, :show, :create, :update] do
|
||||||
collection do
|
collection do
|
||||||
get :search
|
get :search
|
||||||
@@ -87,11 +81,6 @@ Rails.application.routes.draw do
|
|||||||
end
|
end
|
||||||
end
|
end
|
||||||
|
|
||||||
get 'users/settings', to: 'user_settings#show'
|
|
||||||
patch 'users/settings', to: 'user_settings#update'
|
|
||||||
get 'users/theme_slots', to: 'user_theme_slots#index'
|
|
||||||
put 'users/theme_slots/:base_theme/:slot_no', to: 'user_theme_slots#update'
|
|
||||||
|
|
||||||
resources :users, only: [:create, :update] do
|
resources :users, only: [:create, :update] do
|
||||||
collection do
|
collection do
|
||||||
post :verify
|
post :verify
|
||||||
@@ -124,8 +113,5 @@ Rails.application.routes.draw do
|
|||||||
resources :skip_events, controller: :theatre_skip_events, only: [:index]
|
resources :skip_events, controller: :theatre_skip_events, only: [:index]
|
||||||
end
|
end
|
||||||
|
|
||||||
get 'materials/download.zip', to: 'materials#download'
|
|
||||||
get 'materials/versions', to: 'material_versions#index'
|
|
||||||
resources :material_sync_suppressions, path: 'materials/suppressions', only: [:index, :create]
|
|
||||||
resources :materials, only: [:index, :show, :create, :update, :destroy]
|
resources :materials, only: [:index, :show, :create, :update, :destroy]
|
||||||
end
|
end
|
||||||
|
|||||||
@@ -1,9 +0,0 @@
|
|||||||
class AddVideoMsToPosts < ActiveRecord::Migration[8.0]
|
|
||||||
def change
|
|
||||||
add_column :posts, :video_ms, :integer
|
|
||||||
add_index :posts, [:video_ms, :id], name: 'idx_posts_video_ms_id'
|
|
||||||
|
|
||||||
add_check_constraint :posts, 'video_ms IS NULL OR video_ms > 0',
|
|
||||||
name: 'chk_posts_video_ms_positive'
|
|
||||||
end
|
|
||||||
end
|
|
||||||
@@ -1,19 +0,0 @@
|
|||||||
class CreatePostTagSections < ActiveRecord::Migration[8.0]
|
|
||||||
def change
|
|
||||||
create_table :post_tag_sections, primary_key: [:post_id, :tag_id, :begin_ms] do |t|
|
|
||||||
t.references :post, null: false, foreign_key: true, index: false
|
|
||||||
t.references :tag, null: false, foreign_key: true, index: false
|
|
||||||
t.integer :begin_ms, null: false
|
|
||||||
t.integer :end_ms, null: true
|
|
||||||
t.timestamps
|
|
||||||
|
|
||||||
t.index [:post_id, :begin_ms], name: 'idx_post_tag_sections_post_id_begin_ms'
|
|
||||||
|
|
||||||
t.check_constraint 'begin_ms >= 0',
|
|
||||||
name: 'chk_post_tag_sections_begin_ms_natural'
|
|
||||||
|
|
||||||
t.check_constraint 'end_ms IS NULL OR begin_ms < end_ms',
|
|
||||||
name: 'chk_post_tag_sections_end_ms_after_begin_ms'
|
|
||||||
end
|
|
||||||
end
|
|
||||||
end
|
|
||||||
@@ -1,9 +0,0 @@
|
|||||||
class AddVideoMsToPostVersions < ActiveRecord::Migration[8.0]
|
|
||||||
def change
|
|
||||||
add_column :post_versions, :video_ms, :integer
|
|
||||||
add_index :post_versions, [:video_ms, :post_id], name: 'idx_post_versions_video_ms_post_id'
|
|
||||||
|
|
||||||
add_check_constraint :post_versions, 'video_ms IS NULL OR video_ms > 0',
|
|
||||||
name: 'chk_post_versions_video_ms_positive'
|
|
||||||
end
|
|
||||||
end
|
|
||||||
@@ -1,88 +0,0 @@
|
|||||||
class EnhanceMaterialManagement < ActiveRecord::Migration[8.0]
|
|
||||||
def up
|
|
||||||
change_table :materials, bulk: true do |t|
|
|
||||||
t.integer :version_no, null: false, default: 1
|
|
||||||
end
|
|
||||||
|
|
||||||
change_table :material_versions, bulk: true do |t|
|
|
||||||
t.string :event_type
|
|
||||||
t.string :tag_name
|
|
||||||
t.string :tag_category
|
|
||||||
t.json :export_paths_json
|
|
||||||
t.bigint :file_blob_id
|
|
||||||
t.string :file_filename
|
|
||||||
t.string :file_content_type
|
|
||||||
t.bigint :file_byte_size
|
|
||||||
t.string :file_checksum
|
|
||||||
t.string :file_sha256
|
|
||||||
end
|
|
||||||
|
|
||||||
execute <<~SQL.squish
|
|
||||||
UPDATE material_versions
|
|
||||||
SET event_type = CASE
|
|
||||||
WHEN version_no = 1 THEN 'create'
|
|
||||||
ELSE 'update'
|
|
||||||
END
|
|
||||||
WHERE event_type IS NULL
|
|
||||||
SQL
|
|
||||||
|
|
||||||
change_column_null :material_versions, :event_type, false
|
|
||||||
add_index :material_versions, :file_blob_id
|
|
||||||
|
|
||||||
add_check_constraint :material_versions,
|
|
||||||
"event_type IN ('create', 'update', 'discard', 'restore')",
|
|
||||||
name: 'material_versions_event_type_valid'
|
|
||||||
|
|
||||||
create_table :material_export_items do |t|
|
|
||||||
t.references :material, null: false, foreign_key: true
|
|
||||||
t.string :profile, null: false, default: 'legacy_drive'
|
|
||||||
t.string :export_path, null: false
|
|
||||||
t.boolean :enabled, null: false, default: true
|
|
||||||
t.references :created_by_user, foreign_key: { to_table: :users }
|
|
||||||
t.timestamps
|
|
||||||
|
|
||||||
t.index [:profile, :export_path], unique: true
|
|
||||||
t.index [:material_id, :profile], unique: true
|
|
||||||
end
|
|
||||||
|
|
||||||
create_table :material_import_blocks do |t|
|
|
||||||
t.string :match_kind, null: false
|
|
||||||
t.string :sha256
|
|
||||||
t.string :external_path_pattern
|
|
||||||
t.string :reason, null: false
|
|
||||||
t.text :note
|
|
||||||
t.references :created_by_user, foreign_key: { to_table: :users }
|
|
||||||
t.timestamps
|
|
||||||
end
|
|
||||||
|
|
||||||
execute <<~SQL.squish
|
|
||||||
UPDATE materials
|
|
||||||
SET version_no = COALESCE(
|
|
||||||
(SELECT MAX(material_versions.version_no)
|
|
||||||
FROM material_versions
|
|
||||||
WHERE material_versions.material_id = materials.id),
|
|
||||||
1)
|
|
||||||
SQL
|
|
||||||
end
|
|
||||||
|
|
||||||
def down
|
|
||||||
drop_table :material_import_blocks
|
|
||||||
drop_table :material_export_items
|
|
||||||
|
|
||||||
remove_check_constraint :material_versions, name: 'material_versions_event_type_valid'
|
|
||||||
|
|
||||||
remove_index :material_versions, :file_blob_id
|
|
||||||
remove_column :material_versions, :event_type
|
|
||||||
remove_column :material_versions, :tag_name
|
|
||||||
remove_column :material_versions, :tag_category
|
|
||||||
remove_column :material_versions, :export_paths_json
|
|
||||||
remove_column :material_versions, :file_blob_id
|
|
||||||
remove_column :material_versions, :file_filename
|
|
||||||
remove_column :material_versions, :file_content_type
|
|
||||||
remove_column :material_versions, :file_byte_size
|
|
||||||
remove_column :material_versions, :file_checksum
|
|
||||||
remove_column :material_versions, :file_sha256
|
|
||||||
|
|
||||||
remove_column :materials, :version_no
|
|
||||||
end
|
|
||||||
end
|
|
||||||
@@ -1,118 +0,0 @@
|
|||||||
class ReconcileMaterialSyncSuppressions < ActiveRecord::Migration[8.0]
|
|
||||||
MATERIAL_VERSION_EVENT_CONSTRAINT = 'material_versions_event_type_valid'
|
|
||||||
|
|
||||||
def up
|
|
||||||
ensure_material_sync_suppressions!
|
|
||||||
ensure_material_source_columns!
|
|
||||||
ensure_material_active_source_key!
|
|
||||||
remove_material_file_suppression!
|
|
||||||
remove_material_version_file_suppression!
|
|
||||||
reconcile_material_version_event_constraint!
|
|
||||||
end
|
|
||||||
|
|
||||||
def down
|
|
||||||
raise ActiveRecord::IrreversibleMigration
|
|
||||||
end
|
|
||||||
|
|
||||||
private
|
|
||||||
|
|
||||||
def ensure_material_sync_suppressions!
|
|
||||||
return if table_exists?(:material_sync_suppressions)
|
|
||||||
|
|
||||||
create_table :material_sync_suppressions do |t|
|
|
||||||
t.string :source_kind, null: false
|
|
||||||
t.string :source_uri
|
|
||||||
t.string :drive_path
|
|
||||||
t.string :drive_file_id
|
|
||||||
t.string :normalized_source_key, null: false
|
|
||||||
t.string :reason, null: false
|
|
||||||
t.references :created_by_user, foreign_key: { to_table: :users }
|
|
||||||
t.timestamps
|
|
||||||
|
|
||||||
t.index :normalized_source_key, unique: true
|
|
||||||
t.index :source_kind
|
|
||||||
t.index :drive_file_id
|
|
||||||
end
|
|
||||||
end
|
|
||||||
|
|
||||||
def ensure_material_source_columns!
|
|
||||||
add_column :materials, :source_kind, :string unless column_exists?(:materials, :source_kind)
|
|
||||||
add_column :materials, :source_uri, :string unless column_exists?(:materials, :source_uri)
|
|
||||||
add_column :materials, :source_path, :string unless column_exists?(:materials, :source_path)
|
|
||||||
unless column_exists?(:materials, :source_file_id)
|
|
||||||
add_column :materials, :source_file_id, :string
|
|
||||||
end
|
|
||||||
unless column_exists?(:materials, :normalized_source_key)
|
|
||||||
add_column :materials, :normalized_source_key, :string
|
|
||||||
end
|
|
||||||
|
|
||||||
if index_exists?(:materials, :normalized_source_key)
|
|
||||||
remove_index :materials, column: :normalized_source_key
|
|
||||||
end
|
|
||||||
end
|
|
||||||
|
|
||||||
def ensure_material_active_source_key!
|
|
||||||
unless column_exists?(:materials, :active_normalized_source_key)
|
|
||||||
change_table :materials do |t|
|
|
||||||
t.virtual :active_normalized_source_key,
|
|
||||||
type: :string,
|
|
||||||
as: 'IF(discarded_at IS NULL, normalized_source_key, NULL)',
|
|
||||||
stored: false
|
|
||||||
end
|
|
||||||
end
|
|
||||||
|
|
||||||
return if index_exists?(:materials,
|
|
||||||
:active_normalized_source_key,
|
|
||||||
name: 'index_materials_on_active_normalized_source_key')
|
|
||||||
|
|
||||||
add_index :materials,
|
|
||||||
:active_normalized_source_key,
|
|
||||||
unique: true,
|
|
||||||
name: 'index_materials_on_active_normalized_source_key'
|
|
||||||
end
|
|
||||||
|
|
||||||
def remove_material_file_suppression!
|
|
||||||
if foreign_key_exists?(:materials, :users, column: :file_suppressed_by_user_id)
|
|
||||||
remove_foreign_key :materials, column: :file_suppressed_by_user_id
|
|
||||||
end
|
|
||||||
|
|
||||||
if index_exists?(:materials,
|
|
||||||
:file_suppressed_by_user_id,
|
|
||||||
name: 'index_materials_on_file_suppressed_by_user_id')
|
|
||||||
remove_index :materials, name: 'index_materials_on_file_suppressed_by_user_id'
|
|
||||||
end
|
|
||||||
|
|
||||||
remove_column :materials, :file_suppressed_at if column_exists?(
|
|
||||||
:materials,
|
|
||||||
:file_suppressed_at)
|
|
||||||
remove_column :materials, :file_suppressed_by_user_id if column_exists?(
|
|
||||||
:materials,
|
|
||||||
:file_suppressed_by_user_id)
|
|
||||||
remove_column :materials, :file_suppression_reason if column_exists?(
|
|
||||||
:materials,
|
|
||||||
:file_suppression_reason)
|
|
||||||
end
|
|
||||||
|
|
||||||
def remove_material_version_file_suppression!
|
|
||||||
remove_column :material_versions, :file_suppressed_at if column_exists?(
|
|
||||||
:material_versions,
|
|
||||||
:file_suppressed_at)
|
|
||||||
remove_column :material_versions, :file_suppression_reason if column_exists?(
|
|
||||||
:material_versions,
|
|
||||||
:file_suppression_reason)
|
|
||||||
end
|
|
||||||
|
|
||||||
def reconcile_material_version_event_constraint!
|
|
||||||
constraint = check_constraints(:material_versions).find do |candidate|
|
|
||||||
candidate.name == MATERIAL_VERSION_EVENT_CONSTRAINT
|
|
||||||
end
|
|
||||||
return if constraint && !constraint.expression.include?('suppress')
|
|
||||||
|
|
||||||
if constraint
|
|
||||||
remove_check_constraint :material_versions, name: MATERIAL_VERSION_EVENT_CONSTRAINT
|
|
||||||
end
|
|
||||||
add_check_constraint :material_versions,
|
|
||||||
"event_type IN ('create', 'update', 'discard', 'restore')",
|
|
||||||
name: MATERIAL_VERSION_EVENT_CONSTRAINT
|
|
||||||
end
|
|
||||||
end
|
|
||||||
@@ -1,22 +0,0 @@
|
|||||||
class CreateMaterialSyncSources < ActiveRecord::Migration[8.0]
|
|
||||||
def change
|
|
||||||
create_table :material_sync_sources do |t|
|
|
||||||
t.string :name, null: false
|
|
||||||
t.string :source_kind, null: false
|
|
||||||
t.string :source_uri
|
|
||||||
t.string :source_path
|
|
||||||
t.string :source_file_id
|
|
||||||
t.string :profile, null: false, default: 'legacy_drive'
|
|
||||||
t.boolean :enabled, null: false, default: true
|
|
||||||
t.string :default_tag_name
|
|
||||||
t.string :export_path_prefix
|
|
||||||
t.datetime :last_synced_at
|
|
||||||
t.references :created_by_user, foreign_key: { to_table: :users }
|
|
||||||
t.references :updated_by_user, foreign_key: { to_table: :users }
|
|
||||||
t.timestamps
|
|
||||||
|
|
||||||
t.index :enabled
|
|
||||||
t.index :source_kind
|
|
||||||
end
|
|
||||||
end
|
|
||||||
end
|
|
||||||
@@ -1,19 +0,0 @@
|
|||||||
class AddSourceSnapshotToMaterialVersions < ActiveRecord::Migration[8.0]
|
|
||||||
def change
|
|
||||||
unless column_exists?(:material_versions, :source_kind)
|
|
||||||
add_column :material_versions, :source_kind, :string
|
|
||||||
end
|
|
||||||
unless column_exists?(:material_versions, :source_uri)
|
|
||||||
add_column :material_versions, :source_uri, :string
|
|
||||||
end
|
|
||||||
unless column_exists?(:material_versions, :source_path)
|
|
||||||
add_column :material_versions, :source_path, :string
|
|
||||||
end
|
|
||||||
unless column_exists?(:material_versions, :source_file_id)
|
|
||||||
add_column :material_versions, :source_file_id, :string
|
|
||||||
end
|
|
||||||
unless column_exists?(:material_versions, :normalized_source_key)
|
|
||||||
add_column :material_versions, :normalized_source_key, :string
|
|
||||||
end
|
|
||||||
end
|
|
||||||
end
|
|
||||||
@@ -1,171 +0,0 @@
|
|||||||
require 'json'
|
|
||||||
|
|
||||||
class BackfillInitialMaterialVersions < ActiveRecord::Migration[8.0]
|
|
||||||
class MigrationMaterial < ActiveRecord::Base
|
|
||||||
self.table_name = 'materials'
|
|
||||||
end
|
|
||||||
|
|
||||||
class MigrationMaterialExportItem < ActiveRecord::Base
|
|
||||||
self.table_name = 'material_export_items'
|
|
||||||
end
|
|
||||||
|
|
||||||
class MigrationMaterialVersion < ActiveRecord::Base
|
|
||||||
self.table_name = 'material_versions'
|
|
||||||
end
|
|
||||||
|
|
||||||
class MigrationStorageAttachment < ActiveRecord::Base
|
|
||||||
self.table_name = 'active_storage_attachments'
|
|
||||||
end
|
|
||||||
|
|
||||||
class MigrationStorageBlob < ActiveRecord::Base
|
|
||||||
self.table_name = 'active_storage_blobs'
|
|
||||||
end
|
|
||||||
|
|
||||||
class MigrationTag < ActiveRecord::Base
|
|
||||||
self.table_name = 'tags'
|
|
||||||
end
|
|
||||||
|
|
||||||
class MigrationTagName < ActiveRecord::Base
|
|
||||||
self.table_name = 'tag_names'
|
|
||||||
end
|
|
||||||
|
|
||||||
def up
|
|
||||||
say_with_time 'Backfilling initial material versions' do
|
|
||||||
materials_without_versions.find_in_batches(batch_size: 500) do |materials|
|
|
||||||
material_ids = materials.map(&:id)
|
|
||||||
tags_by_id = load_tags(materials.map(&:tag_id).compact.uniq)
|
|
||||||
export_paths_by_material_id = load_export_paths(material_ids)
|
|
||||||
file_snapshots_by_material_id = load_file_snapshots(material_ids)
|
|
||||||
|
|
||||||
rows = materials.map do |material|
|
|
||||||
tag = tags_by_id[material.tag_id]
|
|
||||||
file_snapshot = file_snapshots_by_material_id[material.id] || empty_file_snapshot
|
|
||||||
|
|
||||||
{ material_id: material.id,
|
|
||||||
version_no: 1,
|
|
||||||
event_type: 'create',
|
|
||||||
url: material.url,
|
|
||||||
parent_id: material.parent_id,
|
|
||||||
tag_id: material.tag_id,
|
|
||||||
tag_name: tag&.fetch(:name, nil),
|
|
||||||
tag_category: tag&.fetch(:category, nil),
|
|
||||||
source_kind: material.source_kind,
|
|
||||||
source_uri: material.source_uri,
|
|
||||||
source_path: material.source_path,
|
|
||||||
source_file_id: material.source_file_id,
|
|
||||||
normalized_source_key: material.normalized_source_key,
|
|
||||||
export_paths_json: export_paths_by_material_id[material.id] || { },
|
|
||||||
discarded_at: material.discarded_at,
|
|
||||||
file_blob_id: file_snapshot[:file_blob_id],
|
|
||||||
file_filename: file_snapshot[:file_filename],
|
|
||||||
file_content_type: file_snapshot[:file_content_type],
|
|
||||||
file_byte_size: file_snapshot[:file_byte_size],
|
|
||||||
file_checksum: file_snapshot[:file_checksum],
|
|
||||||
file_sha256: file_snapshot[:file_sha256],
|
|
||||||
created_by_user_id: material.created_by_user_id,
|
|
||||||
updated_by_user_id: material.updated_by_user_id,
|
|
||||||
created_at: material.created_at,
|
|
||||||
updated_at: material.created_at }
|
|
||||||
end
|
|
||||||
|
|
||||||
MigrationMaterialVersion.insert_all!(rows) if rows.present?
|
|
||||||
end
|
|
||||||
end
|
|
||||||
|
|
||||||
execute <<~SQL.squish
|
|
||||||
UPDATE materials
|
|
||||||
SET version_no = COALESCE(
|
|
||||||
(
|
|
||||||
SELECT MAX(material_versions.version_no)
|
|
||||||
FROM material_versions
|
|
||||||
WHERE material_versions.material_id = materials.id
|
|
||||||
),
|
|
||||||
1
|
|
||||||
)
|
|
||||||
SQL
|
|
||||||
end
|
|
||||||
|
|
||||||
def down
|
|
||||||
raise ActiveRecord::IrreversibleMigration
|
|
||||||
end
|
|
||||||
|
|
||||||
private
|
|
||||||
|
|
||||||
def materials_without_versions
|
|
||||||
MigrationMaterial.where.not(id: MigrationMaterialVersion.select(:material_id))
|
|
||||||
end
|
|
||||||
|
|
||||||
def load_tags tag_ids
|
|
||||||
return { } if tag_ids.empty?
|
|
||||||
|
|
||||||
tag_name_ids_by_tag_id = MigrationTag.where(id: tag_ids).pluck(:id, :tag_name_id).to_h
|
|
||||||
tag_names_by_id =
|
|
||||||
MigrationTagName
|
|
||||||
.where(id: tag_name_ids_by_tag_id.values.uniq)
|
|
||||||
.pluck(:id, :name)
|
|
||||||
.to_h
|
|
||||||
categories_by_tag_id = MigrationTag.where(id: tag_ids).pluck(:id, :category).to_h
|
|
||||||
|
|
||||||
tag_ids.each_with_object({ }) do |tag_id, hash|
|
|
||||||
hash[tag_id] = { name: tag_names_by_id[tag_name_ids_by_tag_id[tag_id]],
|
|
||||||
category: categories_by_tag_id[tag_id] }
|
|
||||||
end
|
|
||||||
end
|
|
||||||
|
|
||||||
def load_export_paths material_ids
|
|
||||||
MigrationMaterialExportItem
|
|
||||||
.where(material_id: material_ids)
|
|
||||||
.order(:material_id, :profile)
|
|
||||||
.pluck(:material_id, :profile, :export_path)
|
|
||||||
.each_with_object({ }) do |(material_id, profile, export_path), hash|
|
|
||||||
hash[material_id] ||= { }
|
|
||||||
hash[material_id][profile] = export_path
|
|
||||||
end
|
|
||||||
end
|
|
||||||
|
|
||||||
def load_file_snapshots material_ids
|
|
||||||
attachments_by_material_id =
|
|
||||||
MigrationStorageAttachment
|
|
||||||
.where(record_type: 'Material', name: 'file', record_id: material_ids)
|
|
||||||
.pluck(:record_id, :blob_id)
|
|
||||||
.to_h
|
|
||||||
|
|
||||||
blobs_by_id =
|
|
||||||
MigrationStorageBlob
|
|
||||||
.where(id: attachments_by_material_id.values.uniq)
|
|
||||||
.index_by(&:id)
|
|
||||||
|
|
||||||
attachments_by_material_id.each_with_object({ }) do |(material_id, blob_id), hash|
|
|
||||||
blob = blobs_by_id[blob_id]
|
|
||||||
next unless blob
|
|
||||||
|
|
||||||
hash[material_id] = {
|
|
||||||
file_blob_id: blob.id,
|
|
||||||
file_filename: blob.filename.to_s,
|
|
||||||
file_content_type: blob.content_type,
|
|
||||||
file_byte_size: blob.byte_size,
|
|
||||||
file_checksum: blob.checksum,
|
|
||||||
file_sha256: blob_metadata(blob)['sha256']
|
|
||||||
}
|
|
||||||
end
|
|
||||||
end
|
|
||||||
|
|
||||||
def empty_file_snapshot
|
|
||||||
{ file_blob_id: nil,
|
|
||||||
file_filename: nil,
|
|
||||||
file_content_type: nil,
|
|
||||||
file_byte_size: nil,
|
|
||||||
file_checksum: nil,
|
|
||||||
file_sha256: nil }
|
|
||||||
end
|
|
||||||
|
|
||||||
def blob_metadata blob
|
|
||||||
metadata = blob.metadata
|
|
||||||
return metadata if metadata.is_a?(Hash)
|
|
||||||
return { } if metadata.blank?
|
|
||||||
|
|
||||||
JSON.parse(metadata)
|
|
||||||
rescue JSON::ParserError
|
|
||||||
{ }
|
|
||||||
end
|
|
||||||
end
|
|
||||||
@@ -1,31 +0,0 @@
|
|||||||
class RebuildSettingsAsTypedUserSettings < ActiveRecord::Migration[8.0]
|
|
||||||
def change
|
|
||||||
remove_foreign_key :settings, :users if foreign_key_exists?(:settings, :users)
|
|
||||||
remove_index :settings, :user_id if index_exists?(:settings, :user_id)
|
|
||||||
|
|
||||||
remove_column :settings, :key, :string if column_exists?(:settings, :key)
|
|
||||||
remove_column :settings, :value, :json if column_exists?(:settings, :value)
|
|
||||||
|
|
||||||
change_column_null :settings, :user_id, false
|
|
||||||
|
|
||||||
add_column :settings, :theme, :string, null: false, default: 'system'
|
|
||||||
add_column :settings,
|
|
||||||
:auto_fetch_title,
|
|
||||||
:string,
|
|
||||||
null: false,
|
|
||||||
default: 'manual'
|
|
||||||
add_column :settings,
|
|
||||||
:auto_fetch_thumbnail,
|
|
||||||
:string,
|
|
||||||
null: false,
|
|
||||||
default: 'manual'
|
|
||||||
add_column :settings,
|
|
||||||
:wiki_editor_mode,
|
|
||||||
:string,
|
|
||||||
null: false,
|
|
||||||
default: 'split'
|
|
||||||
|
|
||||||
add_index :settings, :user_id, unique: true
|
|
||||||
add_foreign_key :settings, :users
|
|
||||||
end
|
|
||||||
end
|
|
||||||
@@ -1,22 +0,0 @@
|
|||||||
class CreateUserThemeSlots < ActiveRecord::Migration[8.0]
|
|
||||||
def change
|
|
||||||
create_table :user_theme_slots do |t|
|
|
||||||
t.references :user, null: false, foreign_key: true
|
|
||||||
t.string :base_theme, null: false
|
|
||||||
t.integer :slot_no, null: false
|
|
||||||
t.json :tokens, null: false
|
|
||||||
t.timestamps
|
|
||||||
end
|
|
||||||
|
|
||||||
add_index :user_theme_slots,
|
|
||||||
[:user_id, :base_theme, :slot_no],
|
|
||||||
unique: true,
|
|
||||||
name: 'index_user_theme_slots_on_user_theme_and_slot'
|
|
||||||
add_check_constraint :user_theme_slots,
|
|
||||||
"base_theme IN ('light', 'dark')",
|
|
||||||
name: 'user_theme_slots_base_theme_valid'
|
|
||||||
add_check_constraint :user_theme_slots,
|
|
||||||
'slot_no BETWEEN 1 AND 3',
|
|
||||||
name: 'user_theme_slots_slot_no_valid'
|
|
||||||
end
|
|
||||||
end
|
|
||||||
@@ -1,61 +0,0 @@
|
|||||||
class CreatePostUrlSanitisationRules < ActiveRecord::Migration[8.0]
|
|
||||||
class PostUrlSanitisationRule < ActiveRecord::Base
|
|
||||||
self.table_name = 'post_url_sanitisation_rules'
|
|
||||||
end
|
|
||||||
|
|
||||||
def up
|
|
||||||
create_table :post_url_sanitisation_rules, id: :integer, primary_key: :priority do |t|
|
|
||||||
t.string :source_pattern, null: false
|
|
||||||
t.string :replacement, null: false
|
|
||||||
t.timestamps
|
|
||||||
t.datetime :discarded_at
|
|
||||||
|
|
||||||
t.index :source_pattern, unique: true
|
|
||||||
t.index :discarded_at
|
|
||||||
end
|
|
||||||
|
|
||||||
now = Time.current
|
|
||||||
|
|
||||||
PostUrlSanitisationRule.insert_all!([
|
|
||||||
{ priority: 10,
|
|
||||||
source_pattern: '\Ahttps?://youtu\.be/([^/?#]+)(?:[?#].*)?\z',
|
|
||||||
replacement: 'https://www.youtube.com/watch?v=\1',
|
|
||||||
created_at: now,
|
|
||||||
updated_at: now },
|
|
||||||
{ priority: 20,
|
|
||||||
source_pattern: '\Ahttps?://(?:www\.|m\.)?youtube\.com/live/([^/?#]+)(?:[?#].*)?\z',
|
|
||||||
replacement: 'https://www.youtube.com/watch?v=\1',
|
|
||||||
created_at: now,
|
|
||||||
updated_at: now },
|
|
||||||
{ priority: 30,
|
|
||||||
source_pattern: '\Ahttps?://(?:www\.|m\.)?youtube\.com/shorts/([^/?#]+)(?:[?#].*)?\z',
|
|
||||||
replacement: 'https://www.youtube.com/watch?v=\1',
|
|
||||||
created_at: now,
|
|
||||||
updated_at: now },
|
|
||||||
{ priority: 40,
|
|
||||||
source_pattern: '\Ahttps?://(?:www\.|m\.)?youtube\.com/embed/([^/?#]+)(?:[?#].*)?\z',
|
|
||||||
replacement: 'https://www.youtube.com/watch?v=\1',
|
|
||||||
created_at: now,
|
|
||||||
updated_at: now },
|
|
||||||
{ priority: 50,
|
|
||||||
source_pattern:
|
|
||||||
'\Ahttps?://(?:www\.|m\.)?youtube\.com/watch\?(?:[^#&]+&)*v=([^&#]+)(?:[&#].*)?\z',
|
|
||||||
replacement: 'https://www.youtube.com/watch?v=\1',
|
|
||||||
created_at: now,
|
|
||||||
updated_at: now },
|
|
||||||
{ priority: 60,
|
|
||||||
source_pattern: '\Ahttps?://nico\.ms/([^/?#]+)(?:[?#].*)?\z',
|
|
||||||
replacement: 'https://www.nicovideo.jp/watch/\1',
|
|
||||||
created_at: now,
|
|
||||||
updated_at: now },
|
|
||||||
{ priority: 70,
|
|
||||||
source_pattern: '\Ahttps?://(?:www\.)?nicovideo\.jp/watch/([^?#/]+)(?:[?#].*)?\z',
|
|
||||||
replacement: 'https://www.nicovideo.jp/watch/\1',
|
|
||||||
created_at: now,
|
|
||||||
updated_at: now }])
|
|
||||||
end
|
|
||||||
|
|
||||||
def down
|
|
||||||
drop_table :post_url_sanitisation_rules
|
|
||||||
end
|
|
||||||
end
|
|
||||||
生成ファイル
+19
-160
@@ -10,7 +10,7 @@
|
|||||||
#
|
#
|
||||||
# It's strongly recommended that you check this file into your version control system.
|
# It's strongly recommended that you check this file into your version control system.
|
||||||
|
|
||||||
ActiveRecord::Schema[8.0].define(version: 2026_07_13_000000) do
|
ActiveRecord::Schema[8.0].define(version: 2026_06_21_000000) do
|
||||||
create_table "active_storage_attachments", charset: "utf8mb4", collation: "utf8mb4_0900_ai_ci", force: :cascade do |t|
|
create_table "active_storage_attachments", charset: "utf8mb4", collation: "utf8mb4_0900_ai_ci", force: :cascade do |t|
|
||||||
t.string "name", null: false
|
t.string "name", null: false
|
||||||
t.string "record_type", null: false
|
t.string "record_type", null: false
|
||||||
@@ -130,69 +130,6 @@ ActiveRecord::Schema[8.0].define(version: 2026_07_13_000000) do
|
|||||||
t.index ["ip_address"], name: "index_ip_addresses_on_ip_address", unique: true
|
t.index ["ip_address"], name: "index_ip_addresses_on_ip_address", unique: true
|
||||||
end
|
end
|
||||||
|
|
||||||
create_table "material_export_items", charset: "utf8mb4", collation: "utf8mb4_0900_ai_ci", force: :cascade do |t|
|
|
||||||
t.bigint "material_id", null: false
|
|
||||||
t.string "profile", default: "legacy_drive", null: false
|
|
||||||
t.string "export_path", null: false
|
|
||||||
t.boolean "enabled", default: true, null: false
|
|
||||||
t.bigint "created_by_user_id"
|
|
||||||
t.datetime "created_at", null: false
|
|
||||||
t.datetime "updated_at", null: false
|
|
||||||
t.index ["created_by_user_id"], name: "index_material_export_items_on_created_by_user_id"
|
|
||||||
t.index ["material_id", "profile"], name: "index_material_export_items_on_material_id_and_profile", unique: true
|
|
||||||
t.index ["material_id"], name: "index_material_export_items_on_material_id"
|
|
||||||
t.index ["profile", "export_path"], name: "index_material_export_items_on_profile_and_export_path", unique: true
|
|
||||||
end
|
|
||||||
|
|
||||||
create_table "material_import_blocks", charset: "utf8mb4", collation: "utf8mb4_0900_ai_ci", force: :cascade do |t|
|
|
||||||
t.string "match_kind", null: false
|
|
||||||
t.string "sha256"
|
|
||||||
t.string "external_path_pattern"
|
|
||||||
t.string "reason", null: false
|
|
||||||
t.text "note"
|
|
||||||
t.bigint "created_by_user_id"
|
|
||||||
t.datetime "created_at", null: false
|
|
||||||
t.datetime "updated_at", null: false
|
|
||||||
t.index ["created_by_user_id"], name: "index_material_import_blocks_on_created_by_user_id"
|
|
||||||
end
|
|
||||||
|
|
||||||
create_table "material_sync_sources", charset: "utf8mb4", collation: "utf8mb4_0900_ai_ci", force: :cascade do |t|
|
|
||||||
t.string "name", null: false
|
|
||||||
t.string "source_kind", null: false
|
|
||||||
t.string "source_uri"
|
|
||||||
t.string "source_path"
|
|
||||||
t.string "source_file_id"
|
|
||||||
t.string "profile", default: "legacy_drive", null: false
|
|
||||||
t.boolean "enabled", default: true, null: false
|
|
||||||
t.string "default_tag_name"
|
|
||||||
t.string "export_path_prefix"
|
|
||||||
t.datetime "last_synced_at"
|
|
||||||
t.bigint "created_by_user_id"
|
|
||||||
t.bigint "updated_by_user_id"
|
|
||||||
t.datetime "created_at", null: false
|
|
||||||
t.datetime "updated_at", null: false
|
|
||||||
t.index ["created_by_user_id"], name: "index_material_sync_sources_on_created_by_user_id"
|
|
||||||
t.index ["enabled"], name: "index_material_sync_sources_on_enabled"
|
|
||||||
t.index ["source_kind"], name: "index_material_sync_sources_on_source_kind"
|
|
||||||
t.index ["updated_by_user_id"], name: "index_material_sync_sources_on_updated_by_user_id"
|
|
||||||
end
|
|
||||||
|
|
||||||
create_table "material_sync_suppressions", charset: "utf8mb4", collation: "utf8mb4_0900_ai_ci", force: :cascade do |t|
|
|
||||||
t.string "source_kind", null: false
|
|
||||||
t.string "source_uri"
|
|
||||||
t.string "drive_path"
|
|
||||||
t.string "drive_file_id"
|
|
||||||
t.string "normalized_source_key", null: false
|
|
||||||
t.string "reason", null: false
|
|
||||||
t.bigint "created_by_user_id"
|
|
||||||
t.datetime "created_at", null: false
|
|
||||||
t.datetime "updated_at", null: false
|
|
||||||
t.index ["created_by_user_id"], name: "index_material_sync_suppressions_on_created_by_user_id"
|
|
||||||
t.index ["drive_file_id"], name: "index_material_sync_suppressions_on_drive_file_id"
|
|
||||||
t.index ["normalized_source_key"], name: "index_material_sync_suppressions_on_normalized_source_key", unique: true
|
|
||||||
t.index ["source_kind"], name: "index_material_sync_suppressions_on_source_kind"
|
|
||||||
end
|
|
||||||
|
|
||||||
create_table "material_versions", charset: "utf8mb4", collation: "utf8mb4_0900_ai_ci", force: :cascade do |t|
|
create_table "material_versions", charset: "utf8mb4", collation: "utf8mb4_0900_ai_ci", force: :cascade do |t|
|
||||||
t.bigint "material_id", null: false
|
t.bigint "material_id", null: false
|
||||||
t.integer "version_no", null: false
|
t.integer "version_no", null: false
|
||||||
@@ -204,31 +141,14 @@ ActiveRecord::Schema[8.0].define(version: 2026_07_13_000000) do
|
|||||||
t.datetime "created_at", null: false
|
t.datetime "created_at", null: false
|
||||||
t.datetime "updated_at", null: false
|
t.datetime "updated_at", null: false
|
||||||
t.datetime "discarded_at"
|
t.datetime "discarded_at"
|
||||||
t.string "event_type", null: false
|
|
||||||
t.string "tag_name"
|
|
||||||
t.string "tag_category"
|
|
||||||
t.json "export_paths_json"
|
|
||||||
t.bigint "file_blob_id"
|
|
||||||
t.string "file_filename"
|
|
||||||
t.string "file_content_type"
|
|
||||||
t.bigint "file_byte_size"
|
|
||||||
t.string "file_checksum"
|
|
||||||
t.string "file_sha256"
|
|
||||||
t.string "source_kind"
|
|
||||||
t.string "source_uri"
|
|
||||||
t.string "source_path"
|
|
||||||
t.string "source_file_id"
|
|
||||||
t.string "normalized_source_key"
|
|
||||||
t.index ["created_by_user_id"], name: "index_material_versions_on_created_by_user_id"
|
t.index ["created_by_user_id"], name: "index_material_versions_on_created_by_user_id"
|
||||||
t.index ["discarded_at"], name: "index_material_versions_on_discarded_at"
|
t.index ["discarded_at"], name: "index_material_versions_on_discarded_at"
|
||||||
t.index ["file_blob_id"], name: "index_material_versions_on_file_blob_id"
|
|
||||||
t.index ["material_id", "version_no"], name: "index_material_versions_on_material_id_and_version_no", unique: true
|
t.index ["material_id", "version_no"], name: "index_material_versions_on_material_id_and_version_no", unique: true
|
||||||
t.index ["material_id"], name: "index_material_versions_on_material_id"
|
t.index ["material_id"], name: "index_material_versions_on_material_id"
|
||||||
t.index ["parent_id"], name: "index_material_versions_on_parent_id"
|
t.index ["parent_id"], name: "index_material_versions_on_parent_id"
|
||||||
t.index ["tag_id"], name: "index_material_versions_on_tag_id"
|
t.index ["tag_id"], name: "index_material_versions_on_tag_id"
|
||||||
t.index ["updated_by_user_id"], name: "index_material_versions_on_updated_by_user_id"
|
t.index ["updated_by_user_id"], name: "index_material_versions_on_updated_by_user_id"
|
||||||
t.index ["url"], name: "index_material_versions_on_url"
|
t.index ["url"], name: "index_material_versions_on_url"
|
||||||
t.check_constraint "`event_type` in (_utf8mb4'create',_utf8mb4'update',_utf8mb4'discard',_utf8mb4'restore')", name: "material_versions_event_type_valid"
|
|
||||||
end
|
end
|
||||||
|
|
||||||
create_table "materials", charset: "utf8mb4", collation: "utf8mb4_0900_ai_ci", force: :cascade do |t|
|
create_table "materials", charset: "utf8mb4", collation: "utf8mb4_0900_ai_ci", force: :cascade do |t|
|
||||||
@@ -241,14 +161,6 @@ ActiveRecord::Schema[8.0].define(version: 2026_07_13_000000) do
|
|||||||
t.datetime "updated_at", null: false
|
t.datetime "updated_at", null: false
|
||||||
t.datetime "discarded_at"
|
t.datetime "discarded_at"
|
||||||
t.virtual "active_url", type: :string, as: "if((`discarded_at` is null),`url`,NULL)"
|
t.virtual "active_url", type: :string, as: "if((`discarded_at` is null),`url`,NULL)"
|
||||||
t.integer "version_no", default: 1, null: false
|
|
||||||
t.string "source_kind"
|
|
||||||
t.string "source_uri"
|
|
||||||
t.string "source_path"
|
|
||||||
t.string "source_file_id"
|
|
||||||
t.string "normalized_source_key"
|
|
||||||
t.virtual "active_normalized_source_key", type: :string, as: "if((`discarded_at` is null),`normalized_source_key`,NULL)"
|
|
||||||
t.index ["active_normalized_source_key"], name: "index_materials_on_active_normalized_source_key", unique: true
|
|
||||||
t.index ["active_url"], name: "index_materials_on_active_url", unique: true
|
t.index ["active_url"], name: "index_materials_on_active_url", unique: true
|
||||||
t.index ["created_by_user_id"], name: "index_materials_on_created_by_user_id"
|
t.index ["created_by_user_id"], name: "index_materials_on_created_by_user_id"
|
||||||
t.index ["discarded_at"], name: "index_materials_on_discarded_at"
|
t.index ["discarded_at"], name: "index_materials_on_discarded_at"
|
||||||
@@ -298,19 +210,6 @@ ActiveRecord::Schema[8.0].define(version: 2026_07_13_000000) do
|
|||||||
t.index ["target_post_id"], name: "index_post_similarities_on_target_post_id"
|
t.index ["target_post_id"], name: "index_post_similarities_on_target_post_id"
|
||||||
end
|
end
|
||||||
|
|
||||||
create_table "post_tag_sections", primary_key: ["post_id", "tag_id", "begin_ms"], charset: "utf8mb4", collation: "utf8mb4_0900_ai_ci", force: :cascade do |t|
|
|
||||||
t.bigint "post_id", null: false
|
|
||||||
t.bigint "tag_id", null: false
|
|
||||||
t.integer "begin_ms", null: false
|
|
||||||
t.integer "end_ms"
|
|
||||||
t.datetime "created_at", null: false
|
|
||||||
t.datetime "updated_at", null: false
|
|
||||||
t.index ["post_id", "begin_ms"], name: "idx_post_tag_sections_post_id_begin_ms"
|
|
||||||
t.index ["tag_id"], name: "fk_rails_8be3847903"
|
|
||||||
t.check_constraint "(`end_ms` is null) or (`begin_ms` < `end_ms`)", name: "chk_post_tag_sections_end_ms_after_begin_ms"
|
|
||||||
t.check_constraint "`begin_ms` >= 0", name: "chk_post_tag_sections_begin_ms_natural"
|
|
||||||
end
|
|
||||||
|
|
||||||
create_table "post_tags", charset: "utf8mb4", collation: "utf8mb4_0900_ai_ci", force: :cascade do |t|
|
create_table "post_tags", charset: "utf8mb4", collation: "utf8mb4_0900_ai_ci", force: :cascade do |t|
|
||||||
t.bigint "post_id", null: false
|
t.bigint "post_id", null: false
|
||||||
t.bigint "tag_id", null: false
|
t.bigint "tag_id", null: false
|
||||||
@@ -331,16 +230,6 @@ ActiveRecord::Schema[8.0].define(version: 2026_07_13_000000) do
|
|||||||
t.index ["tag_id"], name: "index_post_tags_on_tag_id"
|
t.index ["tag_id"], name: "index_post_tags_on_tag_id"
|
||||||
end
|
end
|
||||||
|
|
||||||
create_table "post_url_sanitisation_rules", primary_key: "priority", id: :integer, charset: "utf8mb4", collation: "utf8mb4_0900_ai_ci", force: :cascade do |t|
|
|
||||||
t.string "source_pattern", null: false
|
|
||||||
t.string "replacement", null: false
|
|
||||||
t.datetime "created_at", null: false
|
|
||||||
t.datetime "updated_at", null: false
|
|
||||||
t.datetime "discarded_at"
|
|
||||||
t.index ["discarded_at"], name: "index_post_url_sanitisation_rules_on_discarded_at"
|
|
||||||
t.index ["source_pattern"], name: "index_post_url_sanitisation_rules_on_source_pattern", unique: true
|
|
||||||
end
|
|
||||||
|
|
||||||
create_table "post_versions", charset: "utf8mb4", collation: "utf8mb4_0900_ai_ci", force: :cascade do |t|
|
create_table "post_versions", charset: "utf8mb4", collation: "utf8mb4_0900_ai_ci", force: :cascade do |t|
|
||||||
t.bigint "post_id", null: false
|
t.bigint "post_id", null: false
|
||||||
t.integer "version_no", null: false
|
t.integer "version_no", null: false
|
||||||
@@ -354,12 +243,9 @@ ActiveRecord::Schema[8.0].define(version: 2026_07_13_000000) do
|
|||||||
t.datetime "original_created_before"
|
t.datetime "original_created_before"
|
||||||
t.datetime "created_at", null: false
|
t.datetime "created_at", null: false
|
||||||
t.bigint "created_by_user_id"
|
t.bigint "created_by_user_id"
|
||||||
t.integer "video_ms"
|
|
||||||
t.index ["created_by_user_id"], name: "index_post_versions_on_created_by_user_id"
|
t.index ["created_by_user_id"], name: "index_post_versions_on_created_by_user_id"
|
||||||
t.index ["post_id", "version_no"], name: "index_post_versions_on_post_id_and_version_no", unique: true
|
t.index ["post_id", "version_no"], name: "index_post_versions_on_post_id_and_version_no", unique: true
|
||||||
t.index ["post_id"], name: "index_post_versions_on_post_id"
|
t.index ["post_id"], name: "index_post_versions_on_post_id"
|
||||||
t.index ["video_ms", "post_id"], name: "idx_post_versions_video_ms_post_id"
|
|
||||||
t.check_constraint "(`video_ms` is null) or (`video_ms` > 0)", name: "chk_post_versions_video_ms_positive"
|
|
||||||
t.check_constraint "`event_type` in (_utf8mb4'create',_utf8mb4'update',_utf8mb4'discard',_utf8mb4'restore')", name: "post_versions_event_type_valid"
|
t.check_constraint "`event_type` in (_utf8mb4'create',_utf8mb4'update',_utf8mb4'discard',_utf8mb4'restore')", name: "post_versions_event_type_valid"
|
||||||
t.check_constraint "`version_no` > 0", name: "post_versions_version_no_positive"
|
t.check_constraint "`version_no` > 0", name: "post_versions_version_no_positive"
|
||||||
end
|
end
|
||||||
@@ -374,36 +260,18 @@ ActiveRecord::Schema[8.0].define(version: 2026_07_13_000000) do
|
|||||||
t.datetime "original_created_before"
|
t.datetime "original_created_before"
|
||||||
t.datetime "updated_at", null: false
|
t.datetime "updated_at", null: false
|
||||||
t.integer "version_no", null: false
|
t.integer "version_no", null: false
|
||||||
t.integer "video_ms"
|
|
||||||
t.index ["uploaded_user_id"], name: "index_posts_on_uploaded_user_id"
|
t.index ["uploaded_user_id"], name: "index_posts_on_uploaded_user_id"
|
||||||
t.index ["url"], name: "index_posts_on_url", unique: true
|
t.index ["url"], name: "index_posts_on_url", unique: true
|
||||||
t.index ["video_ms", "id"], name: "idx_posts_video_ms_id"
|
|
||||||
t.check_constraint "(`video_ms` is null) or (`video_ms` > 0)", name: "chk_posts_video_ms_positive"
|
|
||||||
t.check_constraint "`version_no` > 0", name: "chk_posts_version_no_positive"
|
t.check_constraint "`version_no` > 0", name: "chk_posts_version_no_positive"
|
||||||
end
|
end
|
||||||
|
|
||||||
create_table "settings", charset: "utf8mb4", collation: "utf8mb4_0900_ai_ci", force: :cascade do |t|
|
create_table "settings", charset: "utf8mb4", collation: "utf8mb4_0900_ai_ci", force: :cascade do |t|
|
||||||
t.bigint "user_id", null: false
|
t.bigint "user_id", null: false
|
||||||
|
t.string "key", null: false
|
||||||
|
t.json "value", null: false
|
||||||
t.datetime "created_at", null: false
|
t.datetime "created_at", null: false
|
||||||
t.datetime "updated_at", null: false
|
t.datetime "updated_at", null: false
|
||||||
t.string "theme", default: "system", null: false
|
t.index ["user_id"], name: "index_settings_on_user_id"
|
||||||
t.string "auto_fetch_title", default: "manual", null: false
|
|
||||||
t.string "auto_fetch_thumbnail", default: "manual", null: false
|
|
||||||
t.string "wiki_editor_mode", default: "split", null: false
|
|
||||||
t.index ["user_id"], name: "index_settings_on_user_id", unique: true
|
|
||||||
end
|
|
||||||
|
|
||||||
create_table "wiki_assets", charset: "utf8mb4", collation: "utf8mb4_0900_ai_ci", force: :cascade do |t|
|
|
||||||
t.bigint "wiki_page_id", null: false
|
|
||||||
t.integer "no", null: false
|
|
||||||
t.string "alt_text"
|
|
||||||
t.binary "sha256", limit: 32, null: false
|
|
||||||
t.bigint "created_by_user_id", null: false
|
|
||||||
t.datetime "created_at", null: false
|
|
||||||
t.datetime "updated_at", null: false
|
|
||||||
t.index ["created_by_user_id"], name: "index_wiki_assets_on_created_by_user_id"
|
|
||||||
t.index ["wiki_page_id", "no"], name: "index_wiki_assets_on_wiki_page_id_and_no", unique: true
|
|
||||||
t.index ["wiki_page_id", "sha256"], name: "index_wiki_assets_on_wiki_page_id_and_sha256", unique: true
|
|
||||||
end
|
end
|
||||||
|
|
||||||
create_table "tag_implications", charset: "utf8mb4", collation: "utf8mb4_0900_ai_ci", force: :cascade do |t|
|
create_table "tag_implications", charset: "utf8mb4", collation: "utf8mb4_0900_ai_ci", force: :cascade do |t|
|
||||||
@@ -451,9 +319,9 @@ ActiveRecord::Schema[8.0].define(version: 2026_07_13_000000) do
|
|||||||
t.string "event_type", null: false
|
t.string "event_type", null: false
|
||||||
t.string "name", null: false
|
t.string "name", null: false
|
||||||
t.string "category", null: false
|
t.string "category", null: false
|
||||||
|
t.datetime "deprecated_at"
|
||||||
t.text "aliases", null: false
|
t.text "aliases", null: false
|
||||||
t.text "parent_tag_ids", null: false
|
t.text "parent_tag_ids", null: false
|
||||||
t.datetime "deprecated_at"
|
|
||||||
t.datetime "created_at", null: false
|
t.datetime "created_at", null: false
|
||||||
t.bigint "created_by_user_id"
|
t.bigint "created_by_user_id"
|
||||||
t.index ["created_at"], name: "index_tag_versions_on_created_at"
|
t.index ["created_at"], name: "index_tag_versions_on_created_at"
|
||||||
@@ -466,10 +334,10 @@ ActiveRecord::Schema[8.0].define(version: 2026_07_13_000000) do
|
|||||||
create_table "tags", charset: "utf8mb4", collation: "utf8mb4_0900_ai_ci", force: :cascade do |t|
|
create_table "tags", charset: "utf8mb4", collation: "utf8mb4_0900_ai_ci", force: :cascade do |t|
|
||||||
t.bigint "tag_name_id", null: false
|
t.bigint "tag_name_id", null: false
|
||||||
t.string "category", default: "general", null: false
|
t.string "category", default: "general", null: false
|
||||||
t.datetime "deprecated_at"
|
|
||||||
t.datetime "created_at", null: false
|
t.datetime "created_at", null: false
|
||||||
t.datetime "updated_at", null: false
|
t.datetime "updated_at", null: false
|
||||||
t.integer "post_count", default: 0, null: false
|
t.integer "post_count", default: 0, null: false
|
||||||
|
t.datetime "deprecated_at"
|
||||||
t.datetime "discarded_at"
|
t.datetime "discarded_at"
|
||||||
t.integer "version_no", null: false
|
t.integer "version_no", null: false
|
||||||
t.index ["deprecated_at"], name: "index_tags_on_deprecated_at"
|
t.index ["deprecated_at"], name: "index_tags_on_deprecated_at"
|
||||||
@@ -591,19 +459,6 @@ ActiveRecord::Schema[8.0].define(version: 2026_07_13_000000) do
|
|||||||
t.index ["post_id"], name: "index_user_post_views_on_post_id"
|
t.index ["post_id"], name: "index_user_post_views_on_post_id"
|
||||||
end
|
end
|
||||||
|
|
||||||
create_table "user_theme_slots", charset: "utf8mb4", collation: "utf8mb4_0900_ai_ci", force: :cascade do |t|
|
|
||||||
t.bigint "user_id", null: false
|
|
||||||
t.string "base_theme", null: false
|
|
||||||
t.integer "slot_no", null: false
|
|
||||||
t.json "tokens", null: false
|
|
||||||
t.datetime "created_at", null: false
|
|
||||||
t.datetime "updated_at", null: false
|
|
||||||
t.index ["user_id", "base_theme", "slot_no"], name: "index_user_theme_slots_on_user_theme_and_slot", unique: true
|
|
||||||
t.index ["user_id"], name: "index_user_theme_slots_on_user_id"
|
|
||||||
t.check_constraint "`base_theme` in (_utf8mb4'light',_utf8mb4'dark')", name: "user_theme_slots_base_theme_valid"
|
|
||||||
t.check_constraint "`slot_no` between 1 and 3", name: "user_theme_slots_slot_no_valid"
|
|
||||||
end
|
|
||||||
|
|
||||||
create_table "users", charset: "utf8mb4", collation: "utf8mb4_0900_ai_ci", force: :cascade do |t|
|
create_table "users", charset: "utf8mb4", collation: "utf8mb4_0900_ai_ci", force: :cascade do |t|
|
||||||
t.string "name"
|
t.string "name"
|
||||||
t.string "inheritance_code", limit: 64, null: false
|
t.string "inheritance_code", limit: 64, null: false
|
||||||
@@ -614,6 +469,19 @@ ActiveRecord::Schema[8.0].define(version: 2026_07_13_000000) do
|
|||||||
t.index ["banned_at"], name: "index_users_on_banned_at"
|
t.index ["banned_at"], name: "index_users_on_banned_at"
|
||||||
end
|
end
|
||||||
|
|
||||||
|
create_table "wiki_assets", charset: "utf8mb4", collation: "utf8mb4_0900_ai_ci", force: :cascade do |t|
|
||||||
|
t.bigint "wiki_page_id", null: false
|
||||||
|
t.integer "no", null: false
|
||||||
|
t.string "alt_text"
|
||||||
|
t.binary "sha256", limit: 32, null: false
|
||||||
|
t.bigint "created_by_user_id", null: false
|
||||||
|
t.datetime "created_at", null: false
|
||||||
|
t.datetime "updated_at", null: false
|
||||||
|
t.index ["created_by_user_id"], name: "index_wiki_assets_on_created_by_user_id"
|
||||||
|
t.index ["wiki_page_id", "no"], name: "index_wiki_assets_on_wiki_page_id_and_no", unique: true
|
||||||
|
t.index ["wiki_page_id", "sha256"], name: "index_wiki_assets_on_wiki_page_id_and_sha256", unique: true
|
||||||
|
end
|
||||||
|
|
||||||
create_table "wiki_lines", charset: "utf8mb4", collation: "utf8mb4_0900_ai_ci", force: :cascade do |t|
|
create_table "wiki_lines", charset: "utf8mb4", collation: "utf8mb4_0900_ai_ci", force: :cascade do |t|
|
||||||
t.string "sha256", limit: 64, null: false
|
t.string "sha256", limit: 64, null: false
|
||||||
t.text "body", null: false
|
t.text "body", null: false
|
||||||
@@ -699,12 +567,6 @@ ActiveRecord::Schema[8.0].define(version: 2026_07_13_000000) do
|
|||||||
add_foreign_key "gekanator_question_suggestions", "users"
|
add_foreign_key "gekanator_question_suggestions", "users"
|
||||||
add_foreign_key "gekanator_questions", "gekanator_question_suggestions"
|
add_foreign_key "gekanator_questions", "gekanator_question_suggestions"
|
||||||
add_foreign_key "gekanator_questions", "users", column: "created_by_id"
|
add_foreign_key "gekanator_questions", "users", column: "created_by_id"
|
||||||
add_foreign_key "material_export_items", "materials"
|
|
||||||
add_foreign_key "material_export_items", "users", column: "created_by_user_id"
|
|
||||||
add_foreign_key "material_import_blocks", "users", column: "created_by_user_id"
|
|
||||||
add_foreign_key "material_sync_sources", "users", column: "created_by_user_id"
|
|
||||||
add_foreign_key "material_sync_sources", "users", column: "updated_by_user_id"
|
|
||||||
add_foreign_key "material_sync_suppressions", "users", column: "created_by_user_id"
|
|
||||||
add_foreign_key "material_versions", "materials"
|
add_foreign_key "material_versions", "materials"
|
||||||
add_foreign_key "material_versions", "materials", column: "parent_id"
|
add_foreign_key "material_versions", "materials", column: "parent_id"
|
||||||
add_foreign_key "material_versions", "tags"
|
add_foreign_key "material_versions", "tags"
|
||||||
@@ -722,8 +584,6 @@ ActiveRecord::Schema[8.0].define(version: 2026_07_13_000000) do
|
|||||||
add_foreign_key "post_implications", "posts", column: "parent_post_id"
|
add_foreign_key "post_implications", "posts", column: "parent_post_id"
|
||||||
add_foreign_key "post_similarities", "posts"
|
add_foreign_key "post_similarities", "posts"
|
||||||
add_foreign_key "post_similarities", "posts", column: "target_post_id"
|
add_foreign_key "post_similarities", "posts", column: "target_post_id"
|
||||||
add_foreign_key "post_tag_sections", "posts"
|
|
||||||
add_foreign_key "post_tag_sections", "tags"
|
|
||||||
add_foreign_key "post_tags", "posts"
|
add_foreign_key "post_tags", "posts"
|
||||||
add_foreign_key "post_tags", "tags"
|
add_foreign_key "post_tags", "tags"
|
||||||
add_foreign_key "post_tags", "users", column: "created_user_id"
|
add_foreign_key "post_tags", "users", column: "created_user_id"
|
||||||
@@ -763,7 +623,6 @@ ActiveRecord::Schema[8.0].define(version: 2026_07_13_000000) do
|
|||||||
add_foreign_key "user_ips", "users"
|
add_foreign_key "user_ips", "users"
|
||||||
add_foreign_key "user_post_views", "posts"
|
add_foreign_key "user_post_views", "posts"
|
||||||
add_foreign_key "user_post_views", "users"
|
add_foreign_key "user_post_views", "users"
|
||||||
add_foreign_key "user_theme_slots", "users"
|
|
||||||
add_foreign_key "wiki_assets", "users", column: "created_by_user_id"
|
add_foreign_key "wiki_assets", "users", column: "created_by_user_id"
|
||||||
add_foreign_key "wiki_assets", "wiki_pages"
|
add_foreign_key "wiki_assets", "wiki_pages"
|
||||||
add_foreign_key "wiki_pages", "tag_names"
|
add_foreign_key "wiki_pages", "tag_names"
|
||||||
|
|||||||
@@ -7,58 +7,3 @@
|
|||||||
# ["Action", "Comedy", "Drama", "Horror"].each do |genre_name|
|
# ["Action", "Comedy", "Drama", "Horror"].each do |genre_name|
|
||||||
# MovieGenre.find_or_create_by!(name: genre_name)
|
# MovieGenre.find_or_create_by!(name: genre_name)
|
||||||
# end
|
# end
|
||||||
|
|
||||||
post_url_sanitisation_rules = [
|
|
||||||
{ priority: 10,
|
|
||||||
source_pattern: '\Ahttps?://youtu\.be/([^/?#]+)(?:[?#].*)?\z',
|
|
||||||
replacement: 'https://www.youtube.com/watch?v=\1' },
|
|
||||||
{ priority: 20,
|
|
||||||
source_pattern: '\Ahttps?://(?:www\.|m\.)?youtube\.com/live/([^/?#]+)(?:[?#].*)?\z',
|
|
||||||
replacement: 'https://www.youtube.com/watch?v=\1' },
|
|
||||||
{ priority: 30,
|
|
||||||
source_pattern: '\Ahttps?://(?:www\.|m\.)?youtube\.com/shorts/([^/?#]+)(?:[?#].*)?\z',
|
|
||||||
replacement: 'https://www.youtube.com/watch?v=\1' },
|
|
||||||
{ priority: 40,
|
|
||||||
source_pattern: '\Ahttps?://(?:www\.|m\.)?youtube\.com/embed/([^/?#]+)(?:[?#].*)?\z',
|
|
||||||
replacement: 'https://www.youtube.com/watch?v=\1' },
|
|
||||||
{ priority: 50,
|
|
||||||
source_pattern:
|
|
||||||
'\Ahttps?://(?:www\.|m\.)?youtube\.com/watch\?(?:[^#&]+&)*v=([^&#]+)(?:[&#].*)?\z',
|
|
||||||
replacement: 'https://www.youtube.com/watch?v=\1' },
|
|
||||||
{ priority: 60,
|
|
||||||
source_pattern: '\Ahttps?://nico\.ms/([^/?#]+)(?:[?#].*)?\z',
|
|
||||||
replacement: 'https://www.nicovideo.jp/watch/\1' },
|
|
||||||
{ priority: 70,
|
|
||||||
source_pattern: '\Ahttps?://(?:www\.)?nicovideo\.jp/watch/([^?#/]+)(?:[?#].*)?\z',
|
|
||||||
replacement: 'https://www.nicovideo.jp/watch/\1' }
|
|
||||||
]
|
|
||||||
|
|
||||||
post_url_sanitisation_rule_scope = PostUrlSanitisationRule.unscoped
|
|
||||||
|
|
||||||
post_url_sanitisation_rules.each do |attributes|
|
|
||||||
priority = attributes.fetch(:priority)
|
|
||||||
source_pattern = attributes.fetch(:source_pattern)
|
|
||||||
|
|
||||||
next if post_url_sanitisation_rule_scope.exists?(priority:)
|
|
||||||
next if post_url_sanitisation_rule_scope.exists?(source_pattern:)
|
|
||||||
|
|
||||||
post_url_sanitisation_rule_scope.create!(attributes)
|
|
||||||
end
|
|
||||||
|
|
||||||
material_sync_source_uri = ENV['MATERIAL_SYNC_SOURCE_URI']
|
|
||||||
material_sync_source_file_id = ENV['MATERIAL_SYNC_SOURCE_FILE_ID']
|
|
||||||
|
|
||||||
if material_sync_source_uri.present? || material_sync_source_file_id.present?
|
|
||||||
source = MaterialSyncSource.find_or_initialize_by(
|
|
||||||
name: ENV.fetch('MATERIAL_SYNC_SOURCE_NAME', 'Legacy URI material sync'))
|
|
||||||
source.assign_attributes(
|
|
||||||
source_kind: ENV.fetch('MATERIAL_SYNC_SOURCE_KIND', 'uri'),
|
|
||||||
source_uri: material_sync_source_uri,
|
|
||||||
source_path: ENV['MATERIAL_SYNC_SOURCE_PATH'],
|
|
||||||
source_file_id: material_sync_source_file_id,
|
|
||||||
profile: ENV.fetch('MATERIAL_SYNC_SOURCE_PROFILE', 'legacy_drive'),
|
|
||||||
enabled: ENV.fetch('MATERIAL_SYNC_SOURCE_ENABLED', 'true') != 'false',
|
|
||||||
default_tag_name: ENV['MATERIAL_SYNC_SOURCE_TAG_NAME'],
|
|
||||||
export_path_prefix: ENV['MATERIAL_SYNC_EXPORT_PATH_PREFIX'])
|
|
||||||
source.save!
|
|
||||||
end
|
|
||||||
|
|||||||
+1102
ファイル差分が大きすぎるため省略します
差分を読込み
@@ -0,0 +1,15 @@
|
|||||||
|
{
|
||||||
|
"name": "lib",
|
||||||
|
"version": "1.0.0",
|
||||||
|
"main": "screenshot.js",
|
||||||
|
"scripts": {
|
||||||
|
"test": "echo \"Error: no test specified\" && exit 1"
|
||||||
|
},
|
||||||
|
"keywords": [],
|
||||||
|
"author": "",
|
||||||
|
"license": "ISC",
|
||||||
|
"description": "",
|
||||||
|
"dependencies": {
|
||||||
|
"puppeteer": "^24.10.0"
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,18 @@
|
|||||||
|
const puppeteer = require ('puppeteer')
|
||||||
|
const fs = require ('fs')
|
||||||
|
|
||||||
|
|
||||||
|
void (async () => {
|
||||||
|
const url = process.argv[2]
|
||||||
|
const output = process.argv[3]
|
||||||
|
|
||||||
|
const browser = await puppeteer.launch ({
|
||||||
|
args: ['--no-sandbox', '--disable-setuid-sandbox'] })
|
||||||
|
|
||||||
|
const page = await browser.newPage ()
|
||||||
|
await page.setViewport ({ width: 960, height: 960 })
|
||||||
|
await page.goto (url, { waitUntil: 'networkidle2', timeout: 15000 })
|
||||||
|
|
||||||
|
await page.screenshot ({ path: output })
|
||||||
|
await browser.close ()
|
||||||
|
}) ()
|
||||||
@@ -1,51 +0,0 @@
|
|||||||
namespace :materials do
|
|
||||||
desc '素材同期'
|
|
||||||
task sync: :environment do
|
|
||||||
result = MaterialSyncRunner.sync_enabled!
|
|
||||||
message = [
|
|
||||||
"materials:sync imported=#{ result.imported }",
|
|
||||||
"updated=#{ result.updated }",
|
|
||||||
"unchanged=#{ result.unchanged }",
|
|
||||||
"suppressed=#{ result.suppressed }",
|
|
||||||
"failed=#{ result.failed }"
|
|
||||||
].join(' ')
|
|
||||||
|
|
||||||
Rails.logger.info(message)
|
|
||||||
puts message
|
|
||||||
|
|
||||||
result.errors.each do |error|
|
|
||||||
Rails.logger.warn(error.to_json)
|
|
||||||
warn error.to_json
|
|
||||||
end
|
|
||||||
end
|
|
||||||
|
|
||||||
namespace :thumbnails do
|
|
||||||
desc '素材サムネールを補完'
|
|
||||||
task backfill: :environment do
|
|
||||||
counts = Hash.new(0)
|
|
||||||
|
|
||||||
Material
|
|
||||||
.with_attached_file
|
|
||||||
.with_attached_thumbnail
|
|
||||||
.find_each do |material|
|
|
||||||
next unless material.file.attached?
|
|
||||||
next if material.thumbnail.attached?
|
|
||||||
|
|
||||||
result = MaterialThumbnailGenerator.generate!(material)
|
|
||||||
counts[result] += 1
|
|
||||||
end
|
|
||||||
|
|
||||||
message =
|
|
||||||
"materials:thumbnails:backfill " \
|
|
||||||
"attached=#{ counts[:attached] } " \
|
|
||||||
"no_file=#{ counts[:no_file] } " \
|
|
||||||
"unsupported_content_type=#{ counts[:unsupported_content_type] } " \
|
|
||||||
"generation_failed=#{ counts[:generation_failed] } " \
|
|
||||||
"attach_failed=#{ counts[:attach_failed] } " \
|
|
||||||
"file_not_found=#{ counts[:file_not_found] } " \
|
|
||||||
"mini_magick_error=#{ counts[:mini_magick_error] }"
|
|
||||||
Rails.logger.info(message)
|
|
||||||
puts message
|
|
||||||
end
|
|
||||||
end
|
|
||||||
end
|
|
||||||
@@ -1,65 +0,0 @@
|
|||||||
require 'rails_helper'
|
|
||||||
|
|
||||||
RSpec.describe 'database seeds' do
|
|
||||||
before do
|
|
||||||
PostUrlSanitisationRule.unscoped.delete_all
|
|
||||||
end
|
|
||||||
|
|
||||||
it 'registers the initial post URL sanitisation rules' do
|
|
||||||
load_seeds
|
|
||||||
|
|
||||||
urls = {
|
|
||||||
'https://youtu.be/abc123?si=share' => youtube_url('abc123'),
|
|
||||||
'https://www.youtube.com/live/abc123?t=10' => youtube_url('abc123'),
|
|
||||||
'https://youtube.com/shorts/abc123?feature=share' => youtube_url('abc123'),
|
|
||||||
'https://m.youtube.com/embed/abc123' => youtube_url('abc123'),
|
|
||||||
'https://youtube.com/watch?feature=share&v=abc123&t=10' => youtube_url('abc123'),
|
|
||||||
'https://nico.ms/sm123?from=share#fragment' => nico_url('sm123'),
|
|
||||||
'https://www.nicovideo.jp/watch/sm123?ref=share#fragment' => nico_url('sm123')
|
|
||||||
}
|
|
||||||
|
|
||||||
expect(PostUrlSanitisationRule.count).to eq(7)
|
|
||||||
urls.each do |url, canonical_url|
|
|
||||||
expect(PostUrlSanitisationRule.sanitise(url)).to eq(canonical_url)
|
|
||||||
end
|
|
||||||
end
|
|
||||||
|
|
||||||
it 'does not overwrite or restore an existing rule' do
|
|
||||||
rule = PostUrlSanitisationRule.create!(
|
|
||||||
priority: 10,
|
|
||||||
source_pattern: '\Ahttps://example\.com/custom\z',
|
|
||||||
replacement: 'https://example.com/replacement'
|
|
||||||
)
|
|
||||||
rule.discard!
|
|
||||||
original_attributes = rule.reload.attributes
|
|
||||||
|
|
||||||
2.times { load_seeds }
|
|
||||||
|
|
||||||
persisted_rule = PostUrlSanitisationRule.unscoped.find(10)
|
|
||||||
expect(persisted_rule.attributes).to eq(original_attributes)
|
|
||||||
expect(PostUrlSanitisationRule.unscoped.count).to eq(7)
|
|
||||||
end
|
|
||||||
|
|
||||||
it 'does not duplicate a rule moved to another priority' do
|
|
||||||
source_pattern = '\Ahttps?://youtu\.be/([^/?#]+)(?:[?#].*)?\z'
|
|
||||||
PostUrlSanitisationRule.create!(
|
|
||||||
priority: 80,
|
|
||||||
source_pattern:,
|
|
||||||
replacement: 'https://example.com/custom/\1'
|
|
||||||
)
|
|
||||||
|
|
||||||
load_seeds
|
|
||||||
|
|
||||||
rules = PostUrlSanitisationRule.unscoped
|
|
||||||
expect(rules.where(source_pattern:).count).to eq(1)
|
|
||||||
expect(rules.find(80).replacement).to eq('https://example.com/custom/\1')
|
|
||||||
end
|
|
||||||
|
|
||||||
def load_seeds
|
|
||||||
load Rails.root.join('db/seeds.rb')
|
|
||||||
end
|
|
||||||
|
|
||||||
def youtube_url(video_id) = "https://www.youtube.com/watch?v=#{ video_id }"
|
|
||||||
|
|
||||||
def nico_url(video_id) = "https://www.nicovideo.jp/watch/#{ video_id }"
|
|
||||||
end
|
|
||||||
@@ -1,8 +0,0 @@
|
|||||||
FactoryBot.define do
|
|
||||||
factory :post_tag_section do
|
|
||||||
association :post
|
|
||||||
association :tag
|
|
||||||
begin_ms { 1_000 }
|
|
||||||
end_ms { 2_000 }
|
|
||||||
end
|
|
||||||
end
|
|
||||||
@@ -1,6 +0,0 @@
|
|||||||
FactoryBot.define do
|
|
||||||
factory :post_tag do
|
|
||||||
association :post
|
|
||||||
association :tag
|
|
||||||
end
|
|
||||||
end
|
|
||||||
@@ -1,8 +0,0 @@
|
|||||||
FactoryBot.define do
|
|
||||||
factory :post do
|
|
||||||
sequence(:url) { |n| "https://example.com/factory-post-#{ n }" }
|
|
||||||
title { 'factory post' }
|
|
||||||
thumbnail_base { nil }
|
|
||||||
uploaded_user { nil }
|
|
||||||
end
|
|
||||||
end
|
|
||||||
@@ -1,57 +0,0 @@
|
|||||||
require 'rails_helper'
|
|
||||||
|
|
||||||
RSpec.describe MaterialExportItem, type: :model do
|
|
||||||
let(:user) { create(:user, :member) }
|
|
||||||
let(:tag) { Tag.create!(tag_name: TagName.create!(name: 'export_item'), category: :material) }
|
|
||||||
let(:material) do
|
|
||||||
Material.create!(tag:, url: 'https://example.com/material',
|
|
||||||
created_by_user: user, updated_by_user: user)
|
|
||||||
end
|
|
||||||
|
|
||||||
it 'rejects blank export_path' do
|
|
||||||
item = described_class.new(material:, profile: 'legacy_drive', export_path: '')
|
|
||||||
|
|
||||||
expect(item).not_to be_valid
|
|
||||||
expect(item.errors[:export_path]).to be_present
|
|
||||||
end
|
|
||||||
|
|
||||||
it 'rejects absolute export_path' do
|
|
||||||
item = described_class.new(material:, profile: 'legacy_drive',
|
|
||||||
export_path: '/素材/a.png')
|
|
||||||
|
|
||||||
expect(item).not_to be_valid
|
|
||||||
expect(item.errors[:export_path]).to be_present
|
|
||||||
end
|
|
||||||
|
|
||||||
it 'rejects parent traversal export_path' do
|
|
||||||
item = described_class.new(material:, profile: 'legacy_drive',
|
|
||||||
export_path: '素材/../a.png')
|
|
||||||
|
|
||||||
expect(item).not_to be_valid
|
|
||||||
expect(item.errors[:export_path]).to be_present
|
|
||||||
end
|
|
||||||
|
|
||||||
it 'rejects double slash export_path' do
|
|
||||||
item = described_class.new(material:, profile: 'legacy_drive',
|
|
||||||
export_path: '素材//a.png')
|
|
||||||
|
|
||||||
expect(item).not_to be_valid
|
|
||||||
expect(item.errors[:export_path]).to be_present
|
|
||||||
end
|
|
||||||
|
|
||||||
it 'rejects dot segment export_path' do
|
|
||||||
item = described_class.new(material:, profile: 'legacy_drive',
|
|
||||||
export_path: './素材/a.png')
|
|
||||||
|
|
||||||
expect(item).not_to be_valid
|
|
||||||
expect(item.errors[:export_path]).to be_present
|
|
||||||
end
|
|
||||||
|
|
||||||
it 'rejects trailing slash export_path' do
|
|
||||||
item = described_class.new(material:, profile: 'legacy_drive',
|
|
||||||
export_path: '素材/a/')
|
|
||||||
|
|
||||||
expect(item).not_to be_valid
|
|
||||||
expect(item.errors[:export_path]).to be_present
|
|
||||||
end
|
|
||||||
end
|
|
||||||
@@ -1,19 +0,0 @@
|
|||||||
require 'rails_helper'
|
|
||||||
|
|
||||||
RSpec.describe Material, type: :model do
|
|
||||||
let(:user) { create(:user, :member) }
|
|
||||||
|
|
||||||
it 'allows only material source kinds, not suppression prefix kinds' do
|
|
||||||
material = described_class.new(url: 'https://example.com/material',
|
|
||||||
source_kind: 'google_drive_path_prefix',
|
|
||||||
source_path: '素材/危険',
|
|
||||||
created_by_user: user,
|
|
||||||
updated_by_user: user)
|
|
||||||
|
|
||||||
expect(material).not_to be_valid
|
|
||||||
expect(material.errors[:source_kind]).to be_present
|
|
||||||
|
|
||||||
material.source_kind = 'google_drive_path'
|
|
||||||
expect(material).to be_valid
|
|
||||||
end
|
|
||||||
end
|
|
||||||
@@ -1,68 +0,0 @@
|
|||||||
require 'rails_helper'
|
|
||||||
|
|
||||||
RSpec.describe Post, type: :model do
|
|
||||||
before do
|
|
||||||
PostUrlSanitisationRule.unscoped.delete_all
|
|
||||||
end
|
|
||||||
|
|
||||||
describe 'URL normalisation' do
|
|
||||||
it 'normalises the HTTP URL before applying sanitisation rules' do
|
|
||||||
PostUrlSanitisationRule.create!(
|
|
||||||
priority: 10,
|
|
||||||
source_pattern: '\\Ahttps://example\\.com/videos/([^/]+)\\z',
|
|
||||||
replacement: 'https://example.com/watch/\\1'
|
|
||||||
)
|
|
||||||
|
|
||||||
post = described_class.create!(
|
|
||||||
title: 'normalised URL',
|
|
||||||
url: ' https://EXAMPLE.com/videos/123/ '
|
|
||||||
)
|
|
||||||
|
|
||||||
expect(post.url).to eq('https://example.com/watch/123')
|
|
||||||
end
|
|
||||||
|
|
||||||
it 'does not normalise an unchanged URL when another attribute changes' do
|
|
||||||
post = create(:post)
|
|
||||||
post.update_column(:url, 'https://EXAMPLE.com/unchanged/')
|
|
||||||
|
|
||||||
post.update!(title: 'updated title')
|
|
||||||
|
|
||||||
expect(post.reload.url).to eq('https://EXAMPLE.com/unchanged/')
|
|
||||||
end
|
|
||||||
|
|
||||||
it 'validates the sanitised URL length' do
|
|
||||||
path = 'a' * 375
|
|
||||||
|
|
||||||
PostUrlSanitisationRule.create!(
|
|
||||||
priority: 10,
|
|
||||||
source_pattern: '\\Ahttps://example\\.com/(a+)\\z',
|
|
||||||
replacement: 'https://example.com/\\1\\1'
|
|
||||||
)
|
|
||||||
|
|
||||||
post = described_class.new(
|
|
||||||
title: 'long URL',
|
|
||||||
url: "https://example.com/#{ path }"
|
|
||||||
)
|
|
||||||
|
|
||||||
expect(post).to be_invalid
|
|
||||||
expect(post.errors.details.fetch(:url)).to include(
|
|
||||||
error: :too_long,
|
|
||||||
count: 768
|
|
||||||
)
|
|
||||||
end
|
|
||||||
|
|
||||||
it 'validates uniqueness after sanitisation' do
|
|
||||||
PostUrlSanitisationRule.create!(
|
|
||||||
priority: 10,
|
|
||||||
source_pattern: '\\Ahttps://example\\.com/alias\\z',
|
|
||||||
replacement: 'https://example.com/canonical'
|
|
||||||
)
|
|
||||||
create(:post, url: 'https://example.com/canonical')
|
|
||||||
|
|
||||||
post = described_class.new(title: 'duplicate URL', url: 'https://example.com/alias')
|
|
||||||
|
|
||||||
expect(post).to be_invalid
|
|
||||||
expect(post.errors.details.fetch(:url)).to include(error: :taken, value: post.url)
|
|
||||||
end
|
|
||||||
end
|
|
||||||
end
|
|
||||||
@@ -1,41 +0,0 @@
|
|||||||
RSpec.describe PostTag, type: :model do
|
|
||||||
describe '#sections' do
|
|
||||||
it 'loads sections by post_id and tag_id' do
|
|
||||||
post_tag = create(:post_tag)
|
|
||||||
section = create(:post_tag_section,
|
|
||||||
post: post_tag.post,
|
|
||||||
tag: post_tag.tag,
|
|
||||||
begin_ms: 1000,
|
|
||||||
end_ms: 2000)
|
|
||||||
|
|
||||||
expect(post_tag.sections).to contain_exactly(section)
|
|
||||||
end
|
|
||||||
|
|
||||||
it 'does not load sections for another tag on the same post' do
|
|
||||||
post = create(:post)
|
|
||||||
tag = create(:tag)
|
|
||||||
other_tag = create(:tag)
|
|
||||||
|
|
||||||
post_tag = create(:post_tag, post:, tag:)
|
|
||||||
create(:post_tag_section,
|
|
||||||
post:,
|
|
||||||
tag: other_tag,
|
|
||||||
begin_ms: 1000,
|
|
||||||
end_ms: 2000)
|
|
||||||
|
|
||||||
expect(post_tag.sections).to be_empty
|
|
||||||
end
|
|
||||||
|
|
||||||
it 'allows open-ended sections' do
|
|
||||||
post_tag = create(:post_tag)
|
|
||||||
section = create(:post_tag_section,
|
|
||||||
post: post_tag.post,
|
|
||||||
tag: post_tag.tag,
|
|
||||||
begin_ms: 1000,
|
|
||||||
end_ms: nil)
|
|
||||||
|
|
||||||
expect(section).to be_valid
|
|
||||||
expect(post_tag.sections).to contain_exactly(section)
|
|
||||||
end
|
|
||||||
end
|
|
||||||
end
|
|
||||||
@@ -1,265 +0,0 @@
|
|||||||
require 'rails_helper'
|
|
||||||
|
|
||||||
RSpec.describe PostUrlSanitisationRule, type: :model do
|
|
||||||
before do
|
|
||||||
described_class.unscoped.delete_all
|
|
||||||
end
|
|
||||||
|
|
||||||
describe 'validations' do
|
|
||||||
it 'requires a source pattern' do
|
|
||||||
rule = described_class.new(priority: 10, source_pattern: nil, replacement: '')
|
|
||||||
|
|
||||||
expect(rule).to be_invalid
|
|
||||||
expect(rule.errors.details.fetch(:source_pattern)).to eq([{ error: :blank }])
|
|
||||||
end
|
|
||||||
|
|
||||||
it 'requires a unique source pattern' do
|
|
||||||
described_class.create!(priority: 10, source_pattern: 'source', replacement: 'first')
|
|
||||||
rule = described_class.new(
|
|
||||||
priority: 20,
|
|
||||||
source_pattern: 'source',
|
|
||||||
replacement: 'second'
|
|
||||||
)
|
|
||||||
|
|
||||||
expect(rule).to be_invalid
|
|
||||||
expect(rule.errors.details.fetch(:source_pattern)).to include(
|
|
||||||
error: :taken,
|
|
||||||
value: 'source'
|
|
||||||
)
|
|
||||||
end
|
|
||||||
|
|
||||||
it 'rejects an invalid regexp' do
|
|
||||||
rule = described_class.new(priority: 10, source_pattern: '[', replacement: '')
|
|
||||||
|
|
||||||
expect(rule).to be_invalid
|
|
||||||
expect(rule.errors[:source_pattern]).to include('変な正規表現だね〜(笑)')
|
|
||||||
end
|
|
||||||
end
|
|
||||||
|
|
||||||
describe '.sanitise' do
|
|
||||||
it 'applies each active rule once in priority order' do
|
|
||||||
described_class.create!(priority: 30, source_pattern: 'c', replacement: 'd')
|
|
||||||
described_class.create!(priority: 10, source_pattern: 'a', replacement: 'aa')
|
|
||||||
described_class.create!(priority: 20, source_pattern: 'aa', replacement: 'c')
|
|
||||||
discarded = described_class.create!(
|
|
||||||
priority: 5,
|
|
||||||
source_pattern: '.',
|
|
||||||
replacement: 'discarded'
|
|
||||||
)
|
|
||||||
discarded.discard!
|
|
||||||
|
|
||||||
expect(described_class.sanitise('a')).to eq('d')
|
|
||||||
end
|
|
||||||
|
|
||||||
it 'canonicalises the initial YouTube and Nico URL forms' do
|
|
||||||
create_initial_rules
|
|
||||||
|
|
||||||
urls = {
|
|
||||||
'https://youtu.be/abc123?si=share' => youtube_url('abc123'),
|
|
||||||
'https://www.youtube.com/live/abc123?t=10' => youtube_url('abc123'),
|
|
||||||
'https://youtube.com/shorts/abc123?feature=share' => youtube_url('abc123'),
|
|
||||||
'https://m.youtube.com/embed/abc123' => youtube_url('abc123'),
|
|
||||||
'https://youtube.com/watch?feature=share&v=abc123&t=10' => youtube_url('abc123'),
|
|
||||||
'https://nico.ms/sm123?from=share#fragment' => nico_url('sm123'),
|
|
||||||
'https://www.nicovideo.jp/watch/sm123?ref=share#fragment' => nico_url('sm123')
|
|
||||||
}
|
|
||||||
|
|
||||||
urls.each do |url, canonical_url|
|
|
||||||
expect(described_class.sanitise(url)).to eq(canonical_url)
|
|
||||||
end
|
|
||||||
end
|
|
||||||
end
|
|
||||||
|
|
||||||
describe '.apply!' do
|
|
||||||
it 'locks posts in ID order and updates through temporary URLs' do
|
|
||||||
first = create(:post, url: 'https://example.com/source/1')
|
|
||||||
second = create(:post, url: 'https://example.com/source/2')
|
|
||||||
create_source_rule
|
|
||||||
sql = capture_sql { described_class.apply! }
|
|
||||||
|
|
||||||
lock_sql = sql.find { _1.match?(/SELECT .*posts.*FOR UPDATE/i) }
|
|
||||||
expect(lock_sql).to match(/ORDER BY .*posts.*id.* ASC/i)
|
|
||||||
expect(sql.grep(/post-url-sanitising\.invalid/).size).to eq(2)
|
|
||||||
expect(first.reload.url).to eq('https://example.com/canonical/1')
|
|
||||||
expect(second.reload.url).to eq('https://example.com/canonical/2')
|
|
||||||
end
|
|
||||||
|
|
||||||
it 'loads and compiles rules only once' do
|
|
||||||
create(:post, url: 'https://example.com/source/1')
|
|
||||||
create(:post, url: 'https://example.com/source/2')
|
|
||||||
create_source_rule
|
|
||||||
|
|
||||||
expect(described_class).to receive(:rules).once.and_call_original
|
|
||||||
|
|
||||||
described_class.apply!
|
|
||||||
end
|
|
||||||
|
|
||||||
it 'does not change post versions, version_no, or updated_at' do
|
|
||||||
post = create(:post, url: 'https://example.com/source/1')
|
|
||||||
post.update_columns(version_no: 7, updated_at: 1.day.ago)
|
|
||||||
original_updated_at = post.reload.updated_at
|
|
||||||
create_source_rule
|
|
||||||
|
|
||||||
expect { described_class.apply! }.not_to change(PostVersion, :count)
|
|
||||||
|
|
||||||
post.reload
|
|
||||||
expect(post.url).to eq('https://example.com/canonical/1')
|
|
||||||
expect(post.version_no).to eq(7)
|
|
||||||
expect(post.updated_at).to eq(original_updated_at)
|
|
||||||
end
|
|
||||||
|
|
||||||
invalid_urls = {
|
|
||||||
'a blank URL' => '',
|
|
||||||
'an unparseable URL' => 'https://[',
|
|
||||||
'a non-HTTP URL' => 'ftp://example.com/file',
|
|
||||||
'an HTTP URL without a host' => 'https:/path'
|
|
||||||
}
|
|
||||||
|
|
||||||
invalid_urls.each do |description, sanitised_url|
|
|
||||||
it "rolls back every update when sanitisation produces #{ description }" do
|
|
||||||
valid_post = create(:post, url: 'https://example.com/source/1')
|
|
||||||
invalid_post = create(:post, url: 'https://example.com/invalid')
|
|
||||||
create_source_rule
|
|
||||||
described_class.create!(
|
|
||||||
priority: 20,
|
|
||||||
source_pattern: '\\Ahttps://example\\.com/invalid\\z',
|
|
||||||
replacement: sanitised_url
|
|
||||||
)
|
|
||||||
|
|
||||||
expect { described_class.apply! }
|
|
||||||
.to raise_error(described_class::InvalidUrlError) { |error|
|
|
||||||
expect(error.invalid_rows).to eq([
|
|
||||||
{ post_id: invalid_post.id,
|
|
||||||
original_url: 'https://example.com/invalid',
|
|
||||||
sanitised_url: }
|
|
||||||
])
|
|
||||||
}
|
|
||||||
expect(valid_post.reload.url).to eq('https://example.com/source/1')
|
|
||||||
expect(invalid_post.reload.url).to eq('https://example.com/invalid')
|
|
||||||
end
|
|
||||||
end
|
|
||||||
|
|
||||||
it 'rolls back every update when sanitisation produces a URL longer than 768 characters' do
|
|
||||||
path = 'a' * 375
|
|
||||||
original_url = "https://example.com/#{ path }"
|
|
||||||
sanitised_url = "https://example.com/#{ path }#{ path }"
|
|
||||||
valid_post = create(:post, url: 'https://example.com/source/1')
|
|
||||||
invalid_post = create(:post, url: original_url)
|
|
||||||
create_source_rule
|
|
||||||
described_class.create!(
|
|
||||||
priority: 20,
|
|
||||||
source_pattern: '\\Ahttps://example\\.com/(a+)\\z',
|
|
||||||
replacement: 'https://example.com/\\1\\1'
|
|
||||||
)
|
|
||||||
|
|
||||||
expect { described_class.apply! }
|
|
||||||
.to raise_error(described_class::InvalidUrlError) { |error|
|
|
||||||
expect(error.invalid_rows).to eq([
|
|
||||||
{ post_id: invalid_post.id,
|
|
||||||
original_url:,
|
|
||||||
sanitised_url: }
|
|
||||||
])
|
|
||||||
}
|
|
||||||
expect(valid_post.reload.url).to eq('https://example.com/source/1')
|
|
||||||
expect(invalid_post.reload.url).to eq(original_url)
|
|
||||||
end
|
|
||||||
|
|
||||||
it 'rejects sanitised URL collisions case-insensitively without changing posts' do
|
|
||||||
source = create(:post, url: 'https://example.com/source/Foo')
|
|
||||||
canonical = create(:post, url: 'https://example.com/canonical/foo')
|
|
||||||
create_source_rule
|
|
||||||
|
|
||||||
expect { described_class.apply! }
|
|
||||||
.to raise_error(described_class::UrlConflictError) { |error|
|
|
||||||
expect(error.conflicts).to contain_exactly(
|
|
||||||
{ url: 'https://example.com/canonical/Foo',
|
|
||||||
post_id: source.id,
|
|
||||||
original_url: 'https://example.com/source/Foo' },
|
|
||||||
{ url: 'https://example.com/canonical/foo',
|
|
||||||
post_id: canonical.id,
|
|
||||||
original_url: 'https://example.com/canonical/foo' }
|
|
||||||
)
|
|
||||||
}
|
|
||||||
expect(source.reload.url).to eq('https://example.com/source/Foo')
|
|
||||||
expect(canonical.reload.url).to eq('https://example.com/canonical/foo')
|
|
||||||
expect(Post.count).to eq(2)
|
|
||||||
end
|
|
||||||
|
|
||||||
it 'converts a persisted URL constraint race into UrlConflictError' do
|
|
||||||
post = create(:post, url: 'https://example.com/source/1')
|
|
||||||
create_source_rule
|
|
||||||
conflict = { url: 'https://example.com/canonical/1',
|
|
||||||
post_id: post.id,
|
|
||||||
original_url: post.url }
|
|
||||||
database_error = ActiveRecord::RecordNotUnique.new('duplicate URL')
|
|
||||||
allow_any_instance_of(ActiveRecord::Relation)
|
|
||||||
.to receive(:update_all).and_raise(database_error)
|
|
||||||
allow(described_class).to receive(:build_persisted_conflicts).and_return([conflict])
|
|
||||||
|
|
||||||
expect { described_class.apply! }
|
|
||||||
.to raise_error(described_class::UrlConflictError) { |error|
|
|
||||||
expect(error.conflicts).to eq([conflict])
|
|
||||||
expect(error.cause).to equal(database_error)
|
|
||||||
}
|
|
||||||
expect(post.reload.url).to eq('https://example.com/source/1')
|
|
||||||
end
|
|
||||||
|
|
||||||
it 're-raises an unidentified RecordNotUnique error' do
|
|
||||||
post = create(:post, url: 'https://example.com/source/1')
|
|
||||||
create_source_rule
|
|
||||||
database_error = ActiveRecord::RecordNotUnique.new('another unique constraint')
|
|
||||||
allow_any_instance_of(ActiveRecord::Relation)
|
|
||||||
.to receive(:update_all).and_raise(database_error)
|
|
||||||
allow(described_class).to receive(:build_persisted_conflicts).and_return([])
|
|
||||||
|
|
||||||
expect { described_class.apply! }.to raise_error(database_error)
|
|
||||||
expect(post.reload.url).to eq('https://example.com/source/1')
|
|
||||||
end
|
|
||||||
end
|
|
||||||
|
|
||||||
def capture_sql
|
|
||||||
statements = []
|
|
||||||
subscriber = lambda do |_name, _start, _finish, _id, payload|
|
|
||||||
binds = payload.fetch(:binds, []).map { _1.value_for_database.to_s }
|
|
||||||
statements << ([payload.fetch(:sql)] + binds).join(' ')
|
|
||||||
end
|
|
||||||
ActiveSupport::Notifications.subscribed(subscriber, 'sql.active_record') { yield }
|
|
||||||
statements
|
|
||||||
end
|
|
||||||
|
|
||||||
def create_source_rule
|
|
||||||
described_class.create!(
|
|
||||||
priority: 10,
|
|
||||||
source_pattern: '\\Ahttps://example\\.com/source/(.+)\\z',
|
|
||||||
replacement: 'https://example.com/canonical/\\1'
|
|
||||||
)
|
|
||||||
end
|
|
||||||
|
|
||||||
def create_initial_rules
|
|
||||||
rules = [
|
|
||||||
['\\Ahttps?://youtu\\.be/([^/?#]+)(?:[?#].*)?\\z', youtube_url('\\1')],
|
|
||||||
['\\Ahttps?://(?:www\\.|m\\.)?youtube\\.com/live/([^/?#]+)(?:[?#].*)?\\z',
|
|
||||||
youtube_url('\\1')],
|
|
||||||
['\\Ahttps?://(?:www\\.|m\\.)?youtube\\.com/shorts/([^/?#]+)(?:[?#].*)?\\z',
|
|
||||||
youtube_url('\\1')],
|
|
||||||
['\\Ahttps?://(?:www\\.|m\\.)?youtube\\.com/embed/([^/?#]+)(?:[?#].*)?\\z',
|
|
||||||
youtube_url('\\1')],
|
|
||||||
['\\Ahttps?://(?:www\\.|m\\.)?youtube\\.com/watch\\?(?:[^#&]+&)*' \
|
|
||||||
'v=([^&#]+)(?:[&#].*)?\\z', youtube_url('\\1')],
|
|
||||||
['\\Ahttps?://nico\\.ms/([^/?#]+)(?:[?#].*)?\\z', nico_url('\\1')],
|
|
||||||
['\\Ahttps?://(?:www\\.)?nicovideo\\.jp/watch/([^?#/]+)(?:[?#].*)?\\z',
|
|
||||||
nico_url('\\1')]
|
|
||||||
]
|
|
||||||
rules.each_with_index do |(source_pattern, replacement), index|
|
|
||||||
described_class.create!(
|
|
||||||
priority: (index + 1) * 10,
|
|
||||||
source_pattern:,
|
|
||||||
replacement:
|
|
||||||
)
|
|
||||||
end
|
|
||||||
end
|
|
||||||
|
|
||||||
def youtube_url(video_id) = "https://www.youtube.com/watch?v=#{ video_id }"
|
|
||||||
|
|
||||||
def nico_url(video_id) = "https://www.nicovideo.jp/watch/#{ video_id }"
|
|
||||||
end
|
|
||||||
@@ -1,34 +0,0 @@
|
|||||||
require 'rails_helper'
|
|
||||||
|
|
||||||
RSpec.describe Setting, type: :model do
|
|
||||||
it 'accepts the default typed user settings' do
|
|
||||||
setting = described_class.new({ user: create(:user) }.merge(described_class.defaults))
|
|
||||||
|
|
||||||
expect(setting).to be_valid
|
|
||||||
end
|
|
||||||
|
|
||||||
it 'requires one settings row per user' do
|
|
||||||
user = create(:user)
|
|
||||||
described_class.create!({ user: }.merge(described_class.defaults))
|
|
||||||
|
|
||||||
duplicate = described_class.new({ user: }.merge(described_class.defaults))
|
|
||||||
|
|
||||||
expect(duplicate).not_to be_valid
|
|
||||||
expect(duplicate.errors[:user_id]).to be_present
|
|
||||||
end
|
|
||||||
|
|
||||||
it 'validates enum-like settings columns' do
|
|
||||||
setting = described_class.new(
|
|
||||||
{ user: create(:user),
|
|
||||||
theme: 'neon',
|
|
||||||
auto_fetch_title: 'sometimes',
|
|
||||||
auto_fetch_thumbnail: 'sometimes',
|
|
||||||
wiki_editor_mode: 'sideways' })
|
|
||||||
|
|
||||||
expect(setting).not_to be_valid
|
|
||||||
expect(setting.errors[:theme]).to be_present
|
|
||||||
expect(setting.errors[:auto_fetch_title]).to be_present
|
|
||||||
expect(setting.errors[:auto_fetch_thumbnail]).to be_present
|
|
||||||
expect(setting.errors[:wiki_editor_mode]).to be_present
|
|
||||||
end
|
|
||||||
end
|
|
||||||
@@ -1,36 +0,0 @@
|
|||||||
require 'rails_helper'
|
|
||||||
|
|
||||||
RSpec.describe TagImplication, type: :model do
|
|
||||||
it 'rejects a parent tag that would create a cycle' do
|
|
||||||
child = create(:tag, name: 'tag_implication_cycle_child')
|
|
||||||
parent = create(:tag, name: 'tag_implication_cycle_parent')
|
|
||||||
|
|
||||||
described_class.create!(tag: child, parent_tag: parent)
|
|
||||||
|
|
||||||
implication = described_class.new(tag: parent, parent_tag: child)
|
|
||||||
|
|
||||||
expect(implication).not_to be_valid
|
|
||||||
expect(implication.errors[:parent_tag_id]).to include(
|
|
||||||
'親タグに子孫タグを指定すると循環します.'
|
|
||||||
)
|
|
||||||
expect(implication.errors[:base]).to be_present
|
|
||||||
end
|
|
||||||
|
|
||||||
it 'terminates even when existing data already contains a cycle' do
|
|
||||||
child = create(:tag, name: 'tag_implication_existing_cycle_child')
|
|
||||||
parent = create(:tag, name: 'tag_implication_existing_cycle_parent')
|
|
||||||
ancestor = create(:tag, name: 'tag_implication_existing_cycle_ancestor')
|
|
||||||
|
|
||||||
described_class.create!(tag: parent, parent_tag: ancestor)
|
|
||||||
described_class.insert_all!(
|
|
||||||
[
|
|
||||||
{ tag_id: ancestor.id, parent_tag_id: parent.id,
|
|
||||||
created_at: Time.current, updated_at: Time.current }
|
|
||||||
]
|
|
||||||
)
|
|
||||||
|
|
||||||
implication = described_class.new(tag: child, parent_tag: parent)
|
|
||||||
|
|
||||||
expect(implication).to be_valid
|
|
||||||
end
|
|
||||||
end
|
|
||||||
@@ -19,95 +19,6 @@ RSpec.describe Tag, type: :model do
|
|||||||
expect(error.tag_names).to eq([deprecated_tag.name])
|
expect(error.tag_names).to eq([deprecated_tag.name])
|
||||||
}
|
}
|
||||||
end
|
end
|
||||||
|
|
||||||
it 'rejects invalid section literals instead of treating them as zero' do
|
|
||||||
expect {
|
|
||||||
described_class.normalise_tags!(
|
|
||||||
['normalise_invalid_section[1:aa-2:00]'],
|
|
||||||
with_sections: true
|
|
||||||
)
|
|
||||||
}.to raise_error(Tag::SectionLiteralParseError)
|
|
||||||
end
|
|
||||||
|
|
||||||
it 'parses open-ended section literals' do
|
|
||||||
result = described_class.normalise_tags!(
|
|
||||||
['伊地知ニジカ[1:00-]'],
|
|
||||||
with_sections: true
|
|
||||||
)
|
|
||||||
|
|
||||||
tag = result.fetch(:tags).find { _1.name == '伊地知ニジカ' }
|
|
||||||
expect(result.fetch(:sections).fetch(tag.id)).to eq([[60_000, nil]])
|
|
||||||
end
|
|
||||||
|
|
||||||
it 'parses omitted begin as zero' do
|
|
||||||
result = described_class.normalise_tags!(
|
|
||||||
['伊地知ニジカ[-1:00]'],
|
|
||||||
with_sections: true
|
|
||||||
)
|
|
||||||
|
|
||||||
tag = result.fetch(:tags).find { _1.name == '伊地知ニジカ' }
|
|
||||||
expect(result.fetch(:sections).fetch(tag.id)).to eq([[0, 60_000]])
|
|
||||||
end
|
|
||||||
|
|
||||||
it 'treats fully open section literals as plain tags' do
|
|
||||||
result = described_class.normalise_tags!(
|
|
||||||
['伊地知ニジカ[-]'],
|
|
||||||
with_sections: true
|
|
||||||
)
|
|
||||||
|
|
||||||
tag = result.fetch(:tags).find { _1.name == '伊地知ニジカ' }
|
|
||||||
expect(result.fetch(:sections)[tag.id]).to be_nil
|
|
||||||
end
|
|
||||||
|
|
||||||
it 'treats [0:00-] as a plain tag' do
|
|
||||||
result = described_class.normalise_tags!(
|
|
||||||
['伊地知ニジカ[0:00-]'],
|
|
||||||
with_sections: true
|
|
||||||
)
|
|
||||||
|
|
||||||
tag = result.fetch(:tags).find { _1.name == '伊地知ニジカ' }
|
|
||||||
expect(result.fetch(:sections)[tag.id]).to be_nil
|
|
||||||
end
|
|
||||||
|
|
||||||
it 'expands zero-width sections to one millisecond' do
|
|
||||||
result = described_class.normalise_tags!(
|
|
||||||
['伊地知ニジカ[1:00-1:00]'],
|
|
||||||
with_sections: true
|
|
||||||
)
|
|
||||||
|
|
||||||
tag = result.fetch(:tags).find { _1.name == '伊地知ニジカ' }
|
|
||||||
expect(result.fetch(:sections).fetch(tag.id)).to eq([[60_000, 60_001]])
|
|
||||||
end
|
|
||||||
|
|
||||||
it 'swaps reversed section boundaries' do
|
|
||||||
result = described_class.normalise_tags!(
|
|
||||||
['伊地知ニジカ[2:00-1:00]'],
|
|
||||||
with_sections: true
|
|
||||||
)
|
|
||||||
|
|
||||||
tag = result.fetch(:tags).find { _1.name == '伊地知ニジカ' }
|
|
||||||
expect(result.fetch(:sections).fetch(tag.id)).to eq([[60_000, 120_000]])
|
|
||||||
end
|
|
||||||
|
|
||||||
it 'merges open-ended sections over later bounded sections' do
|
|
||||||
result = described_class.normalise_tags!(
|
|
||||||
['伊地知ニジカ[1:00-][2:00-3:00]'],
|
|
||||||
with_sections: true
|
|
||||||
)
|
|
||||||
|
|
||||||
tag = result.fetch(:tags).find { _1.name == '伊地知ニジカ' }
|
|
||||||
expect(result.fetch(:sections).fetch(tag.id)).to eq([[60_000, nil]])
|
|
||||||
end
|
|
||||||
|
|
||||||
it 'merges adjacent bounded and open-ended sections' do
|
|
||||||
result = described_class.normalise_tags!(
|
|
||||||
['伊地知ニジカ[1:00-3:00][3:00-]'],
|
|
||||||
with_sections: true
|
|
||||||
)
|
|
||||||
|
|
||||||
tag = result.fetch(:tags).find { _1.name == '伊地知ニジカ' }
|
|
||||||
expect(result.fetch(:sections).fetch(tag.id)).to eq([[60_000, nil]])
|
|
||||||
end
|
|
||||||
end
|
end
|
||||||
|
|
||||||
describe '.expand_parent_tags' do
|
describe '.expand_parent_tags' do
|
||||||
@@ -143,17 +54,7 @@ RSpec.describe Tag, type: :model do
|
|||||||
first = create(:tag, name: 'expand_cycle_first')
|
first = create(:tag, name: 'expand_cycle_first')
|
||||||
second = create(:tag, name: 'expand_cycle_second')
|
second = create(:tag, name: 'expand_cycle_second')
|
||||||
TagImplication.create!(tag: first, parent_tag: second)
|
TagImplication.create!(tag: first, parent_tag: second)
|
||||||
now = Time.current
|
TagImplication.create!(tag: second, parent_tag: first)
|
||||||
TagImplication.insert_all!(
|
|
||||||
[
|
|
||||||
{
|
|
||||||
tag_id: second.id,
|
|
||||||
parent_tag_id: first.id,
|
|
||||||
created_at: now,
|
|
||||||
updated_at: now
|
|
||||||
}
|
|
||||||
]
|
|
||||||
)
|
|
||||||
|
|
||||||
expect(described_class.expand_parent_tags([first])).to contain_exactly(first, second)
|
expect(described_class.expand_parent_tags([first])).to contain_exactly(first, second)
|
||||||
end
|
end
|
||||||
|
|||||||
@@ -1,40 +0,0 @@
|
|||||||
require 'rails_helper'
|
|
||||||
|
|
||||||
RSpec.describe UserThemeSlot, type: :model do
|
|
||||||
it 'accepts a theme slot for one user, base theme, and slot number' do
|
|
||||||
slot = described_class.new(user: create(:user),
|
|
||||||
base_theme: 'light',
|
|
||||||
slot_no: 1,
|
|
||||||
tokens: { 'background' => '0 0% 100%' })
|
|
||||||
|
|
||||||
expect(slot).to be_valid
|
|
||||||
end
|
|
||||||
|
|
||||||
it 'requires unique slot numbers per user and base theme' do
|
|
||||||
user = create(:user)
|
|
||||||
described_class.create!(user:,
|
|
||||||
base_theme: 'dark',
|
|
||||||
slot_no: 2,
|
|
||||||
tokens: { 'background' => '222.2 84% 4.9%' })
|
|
||||||
|
|
||||||
duplicate = described_class.new(user:,
|
|
||||||
base_theme: 'dark',
|
|
||||||
slot_no: 2,
|
|
||||||
tokens: { 'background' => '0 0% 100%' })
|
|
||||||
|
|
||||||
expect(duplicate).not_to be_valid
|
|
||||||
expect(duplicate.errors[:user_id]).to be_present
|
|
||||||
end
|
|
||||||
|
|
||||||
it 'validates base theme, slot number, and token object shape' do
|
|
||||||
slot = described_class.new(user: create(:user),
|
|
||||||
base_theme: 'system',
|
|
||||||
slot_no: 4,
|
|
||||||
tokens: 'not-object')
|
|
||||||
|
|
||||||
expect(slot).not_to be_valid
|
|
||||||
expect(slot.errors[:base_theme]).to be_present
|
|
||||||
expect(slot.errors[:slot_no]).to be_present
|
|
||||||
expect(slot.errors[:tokens]).to be_present
|
|
||||||
end
|
|
||||||
end
|
|
||||||
@@ -1,10 +1,7 @@
|
|||||||
require 'rails_helper'
|
require 'rails_helper'
|
||||||
|
|
||||||
RSpec.describe 'Materials API', type: :request do
|
RSpec.describe 'Materials API', type: :request do
|
||||||
include ActiveJob::TestHelper
|
|
||||||
|
|
||||||
let!(:member_user) { create(:user, :member) }
|
let!(:member_user) { create(:user, :member) }
|
||||||
let!(:admin_user) { create(:user, :admin) }
|
|
||||||
let!(:guest_user) { create(:user) }
|
let!(:guest_user) { create(:user) }
|
||||||
|
|
||||||
def dummy_upload(filename: 'dummy.png', type: 'image/png', body: 'dummy')
|
def dummy_upload(filename: 'dummy.png', type: 'image/png', body: 'dummy')
|
||||||
@@ -16,29 +13,22 @@ RSpec.describe 'Materials API', type: :request do
|
|||||||
end
|
end
|
||||||
|
|
||||||
def build_material(tag:, user:, parent: nil, file: dummy_upload, url: nil)
|
def build_material(tag:, user:, parent: nil, file: dummy_upload, url: nil)
|
||||||
Material.new(tag:, parent:, url:,
|
Material.new(tag:, parent:, url:, created_by_user: user, updated_by_user: user).tap do |material|
|
||||||
created_by_user: user,
|
|
||||||
updated_by_user: user).tap do |material|
|
|
||||||
material.file.attach(file) if file
|
material.file.attach(file) if file
|
||||||
material.save!
|
material.save!
|
||||||
end
|
end
|
||||||
end
|
end
|
||||||
|
|
||||||
describe 'GET /materials' do
|
describe 'GET /materials' do
|
||||||
let!(:tag_a) do
|
let!(:tag_a) { Tag.create!(tag_name: TagName.create!(name: 'material_index_a'), category: :material) }
|
||||||
Tag.create!(tag_name: TagName.create!(name: 'material_index_a'), category: :material)
|
let!(:tag_b) { Tag.create!(tag_name: TagName.create!(name: 'material_index_b'), category: :material) }
|
||||||
end
|
|
||||||
let!(:tag_b) do
|
|
||||||
Tag.create!(tag_name: TagName.create!(name: 'material_index_b'), category: :material)
|
|
||||||
end
|
|
||||||
|
|
||||||
let!(:material_a) do
|
let!(:material_a) do
|
||||||
build_material(tag: tag_a, user: member_user, file: dummy_upload(filename: 'a.png'))
|
build_material(tag: tag_a, user: member_user, file: dummy_upload(filename: 'a.png'))
|
||||||
end
|
end
|
||||||
|
|
||||||
let!(:material_b) do
|
let!(:material_b) do
|
||||||
build_material(tag: tag_b, user: member_user, parent: material_a,
|
build_material(tag: tag_b, user: member_user, parent: material_a, file: dummy_upload(filename: 'b.png'))
|
||||||
file: dummy_upload(filename: 'b.png'))
|
|
||||||
end
|
end
|
||||||
|
|
||||||
before do
|
before do
|
||||||
@@ -64,36 +54,27 @@ RSpec.describe 'Materials API', type: :request do
|
|||||||
'name' => 'material_index_b',
|
'name' => 'material_index_b',
|
||||||
'category' => 'material'
|
'category' => 'material'
|
||||||
)
|
)
|
||||||
|
expect(row['created_by_user']).to include(
|
||||||
|
'id' => member_user.id,
|
||||||
|
'name' => member_user.name
|
||||||
|
)
|
||||||
expect(row['content_type']).to eq('image/png')
|
expect(row['content_type']).to eq('image/png')
|
||||||
expect(row['media_kind']).to eq('image')
|
|
||||||
end
|
end
|
||||||
|
|
||||||
it 'filters materials by q' do
|
it 'filters materials by tag_id' do
|
||||||
get '/materials', params: { q: 'material_index_a' }
|
get '/materials', params: { tag_id: material_a.tag_id }
|
||||||
|
|
||||||
expect(response).to have_http_status(:ok)
|
expect(response).to have_http_status(:ok)
|
||||||
expect(json['count']).to eq(1)
|
expect(json['count']).to eq(1)
|
||||||
expect(response_materials.map { |m| m['id'] }).to eq([material_a.id])
|
expect(response_materials.map { |m| m['id'] }).to eq([material_a.id])
|
||||||
end
|
end
|
||||||
|
|
||||||
it 'filters materials by tag_state' do
|
it 'filters materials by parent_id' do
|
||||||
untagged =
|
get '/materials', params: { parent_id: material_a.id }
|
||||||
build_material(tag: nil, user: member_user,
|
|
||||||
file: dummy_upload(filename: 'untagged.png'))
|
|
||||||
|
|
||||||
get '/materials', params: { tag_state: 'untagged' }
|
|
||||||
|
|
||||||
expect(response).to have_http_status(:ok)
|
expect(response).to have_http_status(:ok)
|
||||||
expect(json['count']).to eq(1)
|
expect(json['count']).to eq(1)
|
||||||
expect(response_materials.map { |m| m['id'] }).to eq([untagged.id])
|
expect(response_materials.map { |m| m['id'] }).to eq([material_b.id])
|
||||||
end
|
|
||||||
|
|
||||||
it 'filters materials by media_kind' do
|
|
||||||
get '/materials', params: { media_kind: 'image' }
|
|
||||||
|
|
||||||
expect(response).to have_http_status(:ok)
|
|
||||||
expect(json['count']).to eq(2)
|
|
||||||
expect(response_materials.map { |m| m['media_kind'] }).to all(eq('image'))
|
|
||||||
end
|
end
|
||||||
|
|
||||||
it 'paginates and keeps total count' do
|
it 'paginates and keeps total count' do
|
||||||
@@ -113,74 +94,10 @@ RSpec.describe 'Materials API', type: :request do
|
|||||||
expect(response_materials.size).to eq(1)
|
expect(response_materials.size).to eq(1)
|
||||||
expect(response_materials.first['id']).to eq(material_b.id)
|
expect(response_materials.first['id']).to eq(material_b.id)
|
||||||
end
|
end
|
||||||
|
|
||||||
it 'filters by descendant tags and returns stable parent tag groups' do
|
|
||||||
root =
|
|
||||||
Tag.create!(tag_name: TagName.create!(name: 'material_scope_root'),
|
|
||||||
category: :material)
|
|
||||||
child_b =
|
|
||||||
Tag.create!(tag_name: TagName.create!(name: 'material_scope_b'),
|
|
||||||
category: :material)
|
|
||||||
child_a =
|
|
||||||
Tag.create!(tag_name: TagName.create!(name: 'material_scope_a'),
|
|
||||||
category: :material)
|
|
||||||
deprecated =
|
|
||||||
Tag.create!(tag_name: TagName.create!(name: 'material_scope_old'),
|
|
||||||
category: :material,
|
|
||||||
deprecated_at: Time.current)
|
|
||||||
grandchild =
|
|
||||||
Tag.create!(tag_name: TagName.create!(name: 'material_scope_grandchild'),
|
|
||||||
category: :material)
|
|
||||||
root_material =
|
|
||||||
build_material(tag: root, user: member_user,
|
|
||||||
file: dummy_upload(filename: 'root.png'))
|
|
||||||
child_a_material =
|
|
||||||
build_material(tag: child_a, user: member_user,
|
|
||||||
file: dummy_upload(filename: 'child_a.png'))
|
|
||||||
grandchild_material =
|
|
||||||
build_material(tag: grandchild, user: member_user,
|
|
||||||
file: dummy_upload(filename: 'grandchild.png'))
|
|
||||||
|
|
||||||
TagImplication.create!(parent_tag: root, tag: child_b)
|
|
||||||
TagImplication.create!(parent_tag: root, tag: child_a)
|
|
||||||
TagImplication.create!(parent_tag: child_b, tag: deprecated)
|
|
||||||
TagImplication.create!(parent_tag: deprecated, tag: grandchild)
|
|
||||||
|
|
||||||
get '/materials',
|
|
||||||
params: { tag_id: root.id, include_descendants: '1',
|
|
||||||
group_by: 'parent_tag', sort: 'id', direction: 'asc' }
|
|
||||||
|
|
||||||
expect(response).to have_http_status(:ok)
|
|
||||||
expect(response_materials.map { |m| m['id'] })
|
|
||||||
.to include(root_material.id, child_a_material.id, grandchild_material.id)
|
|
||||||
expect(json['tag_scope']).to eq(
|
|
||||||
'tag' => {
|
|
||||||
'id' => root.id,
|
|
||||||
'name' => 'material_scope_root',
|
|
||||||
'category' => 'material'
|
|
||||||
},
|
|
||||||
'include_descendants' => true
|
|
||||||
)
|
|
||||||
expect(json['groups'].map { |group| group.dig('tag', 'name') })
|
|
||||||
.to eq(['material_scope_a', 'material_scope_b', 'material_scope_root'])
|
|
||||||
expect(json['groups'].find { |group| group.dig('tag', 'id') == child_b.id }
|
|
||||||
.fetch('material_ids')).to eq([grandchild_material.id])
|
|
||||||
end
|
|
||||||
|
|
||||||
it 'returns nil tag_scope when tag_id is unknown' do
|
|
||||||
get '/materials', params: { tag_id: 999_999, group_by: 'parent_tag' }
|
|
||||||
|
|
||||||
expect(response).to have_http_status(:ok)
|
|
||||||
expect(json['tag_scope']).to be_nil
|
|
||||||
expect(json['groups']).to eq([])
|
|
||||||
expect(response_materials).to eq([])
|
|
||||||
end
|
|
||||||
end
|
end
|
||||||
|
|
||||||
describe 'GET /materials/:id' do
|
describe 'GET /materials/:id' do
|
||||||
let!(:tag) do
|
let!(:tag) { Tag.create!(tag_name: TagName.create!(name: 'material_show'), category: :material) }
|
||||||
Tag.create!(tag_name: TagName.create!(name: 'material_show'), category: :material)
|
|
||||||
end
|
|
||||||
let!(:material) do
|
let!(:material) do
|
||||||
build_material(tag:, user: member_user, file: dummy_upload(filename: 'show.png'))
|
build_material(tag:, user: member_user, file: dummy_upload(filename: 'show.png'))
|
||||||
end
|
end
|
||||||
@@ -221,22 +138,9 @@ RSpec.describe 'Materials API', type: :request do
|
|||||||
end
|
end
|
||||||
end
|
end
|
||||||
|
|
||||||
context 'when logged in but not member' do
|
context 'when logged in' do
|
||||||
before { sign_in_as(guest_user) }
|
before { sign_in_as(guest_user) }
|
||||||
|
|
||||||
it 'returns 403' do
|
|
||||||
post '/materials', params: {
|
|
||||||
tag: 'material_create_guest_forbidden',
|
|
||||||
file: dummy_upload
|
|
||||||
}
|
|
||||||
|
|
||||||
expect(response).to have_http_status(:forbidden)
|
|
||||||
end
|
|
||||||
end
|
|
||||||
|
|
||||||
context 'when member' do
|
|
||||||
before { sign_in_as(member_user) }
|
|
||||||
|
|
||||||
it 'returns 422 when tag is blank' do
|
it 'returns 422 when tag is blank' do
|
||||||
post '/materials', params: { tag: ' ', file: dummy_upload }
|
post '/materials', params: { tag: ' ', file: dummy_upload }
|
||||||
|
|
||||||
@@ -258,68 +162,24 @@ RSpec.describe 'Materials API', type: :request do
|
|||||||
expect do
|
expect do
|
||||||
post '/materials', params: {
|
post '/materials', params: {
|
||||||
tag: 'material_create_new',
|
tag: 'material_create_new',
|
||||||
file: dummy_upload(filename: 'created.png'),
|
file: dummy_upload(filename: 'created.png')
|
||||||
export_paths: { legacy_drive: '伊地知ニジカ/created.png' }
|
|
||||||
}
|
}
|
||||||
end.to change(Material, :count).by(1)
|
end.to change(Material, :count).by(1)
|
||||||
.and change(Tag, :count).by(1)
|
.and change(Tag, :count).by(1)
|
||||||
.and change(TagName, :count).by(1)
|
.and change(TagName, :count).by(1)
|
||||||
.and change(MaterialVersion, :count).by(1)
|
|
||||||
|
|
||||||
expect(response).to have_http_status(:created)
|
expect(response).to have_http_status(:created)
|
||||||
|
|
||||||
material = Material.order(:id).last
|
material = Material.order(:id).last
|
||||||
expect(material.tag.name).to eq('material_create_new')
|
expect(material.tag.name).to eq('material_create_new')
|
||||||
expect(material.tag.category).to eq('material')
|
expect(material.tag.category).to eq('material')
|
||||||
expect(material.created_by_user).to eq(member_user)
|
expect(material.created_by_user).to eq(guest_user)
|
||||||
expect(material.updated_by_user).to eq(member_user)
|
expect(material.updated_by_user).to eq(guest_user)
|
||||||
expect(material.file.attached?).to be(true)
|
expect(material.file.attached?).to be(true)
|
||||||
expect(material.version_no).to eq(1)
|
|
||||||
expect(material.material_versions.first.event_type).to eq('create')
|
|
||||||
expect(material.material_export_items.first.export_path).to eq('伊地知ニジカ/created.png')
|
|
||||||
expect(material.material_versions.first.export_paths_json).to eq(
|
|
||||||
'legacy_drive' => '伊地知ニジカ/created.png'
|
|
||||||
)
|
|
||||||
|
|
||||||
expect(json['id']).to eq(material.id)
|
expect(json['id']).to eq(material.id)
|
||||||
expect(json.dig('tag', 'name')).to eq('material_create_new')
|
expect(json.dig('tag', 'name')).to eq('material_create_new')
|
||||||
expect(json['content_type']).to eq('image/png')
|
expect(json['content_type']).to eq('image/png')
|
||||||
expect(json.dig('export_paths', 'legacy_drive')).to eq('伊地知ニジカ/created.png')
|
|
||||||
end
|
|
||||||
|
|
||||||
it 'creates a create tag_version for a newly created material tag' do
|
|
||||||
expect do
|
|
||||||
post '/materials', params: {
|
|
||||||
tag: 'material_create_versioned_tag',
|
|
||||||
file: dummy_upload(filename: 'created.png')
|
|
||||||
}
|
|
||||||
end.to change(TagVersion, :count).by(1)
|
|
||||||
|
|
||||||
expect(response).to have_http_status(:created)
|
|
||||||
|
|
||||||
tag = Tag.joins(:tag_name).find_by!(tag_names: { name: 'material_create_versioned_tag' })
|
|
||||||
version = tag.tag_versions.order(:version_no).last
|
|
||||||
|
|
||||||
expect(version.event_type).to eq('create')
|
|
||||||
expect(version.name).to eq('material_create_versioned_tag')
|
|
||||||
expect(version.category).to eq('material')
|
|
||||||
expect(version.created_by_user).to eq(member_user)
|
|
||||||
end
|
|
||||||
|
|
||||||
it 'snapshots attached file metadata and sha256' do
|
|
||||||
post '/materials', params: {
|
|
||||||
tag: 'material_create_file_version',
|
|
||||||
file: dummy_upload(filename: 'created.png', body: 'sha-body')
|
|
||||||
}
|
|
||||||
|
|
||||||
expect(response).to have_http_status(:created)
|
|
||||||
|
|
||||||
version = Material.order(:id).last.material_versions.first
|
|
||||||
expect(version.file_blob_id).to be_present
|
|
||||||
expect(version.file_filename).to eq('created.png')
|
|
||||||
expect(version.file_content_type).to eq('image/png')
|
|
||||||
expect(version.file_byte_size).to eq('sha-body'.bytesize)
|
|
||||||
expect(version.file_sha256).to eq(Digest::SHA256.hexdigest('sha-body'))
|
|
||||||
end
|
end
|
||||||
|
|
||||||
it 'returns 422 when the existing tag is not material/character' do
|
it 'returns 422 when the existing tag is not material/character' do
|
||||||
@@ -359,33 +219,11 @@ RSpec.describe 'Materials API', type: :request do
|
|||||||
expect(response).to have_http_status(:created)
|
expect(response).to have_http_status(:created)
|
||||||
expect(json['url']).to eq('https://example.com/material-source')
|
expect(json['url']).to eq('https://example.com/material-source')
|
||||||
end
|
end
|
||||||
|
|
||||||
it 'rejects sha256-blocked file upload' do
|
|
||||||
sha256 = Digest::SHA256.hexdigest('blocked-body')
|
|
||||||
MaterialImportBlock.create!(match_kind: 'sha256',
|
|
||||||
sha256:,
|
|
||||||
reason: 'copyright_high_risk',
|
|
||||||
created_by_user: admin_user)
|
|
||||||
|
|
||||||
expect do
|
|
||||||
post '/materials', params: {
|
|
||||||
tag: 'material_blocked_create',
|
|
||||||
file: dummy_upload(filename: 'blocked.png', body: 'blocked-body')
|
|
||||||
}
|
|
||||||
end.not_to change(Material, :count)
|
|
||||||
|
|
||||||
expect(response).to have_http_status(:unprocessable_entity)
|
|
||||||
expect(json.fetch('errors')).to include(
|
|
||||||
'file' => ['抑止された素材です: copyright_high_risk']
|
|
||||||
)
|
|
||||||
end
|
|
||||||
end
|
end
|
||||||
end
|
end
|
||||||
|
|
||||||
describe 'PUT /materials/:id' do
|
describe 'PUT /materials/:id' do
|
||||||
let!(:tag) do
|
let!(:tag) { Tag.create!(tag_name: TagName.create!(name: 'material_update_old'), category: :material) }
|
||||||
Tag.create!(tag_name: TagName.create!(name: 'material_update_old'), category: :material)
|
|
||||||
end
|
|
||||||
let!(:material) do
|
let!(:material) do
|
||||||
build_material(tag:, user: member_user, file: dummy_upload(filename: 'old.png'))
|
build_material(tag:, user: member_user, file: dummy_upload(filename: 'old.png'))
|
||||||
end
|
end
|
||||||
@@ -439,26 +277,25 @@ RSpec.describe 'Materials API', type: :request do
|
|||||||
'tag' => ['タグは必須です.'])
|
'tag' => ['タグは必須です.'])
|
||||||
end
|
end
|
||||||
|
|
||||||
it 'keeps the existing file when file and url are omitted' do
|
it 'returns 422 when both file and url are blank' do
|
||||||
put "/materials/#{ material.id }", params: {
|
put "/materials/#{ material.id }", params: {
|
||||||
tag: 'material_update_no_payload'
|
tag: 'material_update_no_payload'
|
||||||
}
|
}
|
||||||
|
|
||||||
expect(response).to have_http_status(:ok)
|
expect(response).to have_http_status(:unprocessable_entity)
|
||||||
expect(material.reload.file.attached?).to be(true)
|
expect(json.fetch('errors')).to include(
|
||||||
|
'file' => ['ファイルまたは URL は必須です.'],
|
||||||
|
'url' => ['ファイルまたは URL は必須です.'])
|
||||||
end
|
end
|
||||||
|
|
||||||
it 'updates tag, url, file, and updated_by_user' do
|
it 'updates tag, url, file, and updated_by_user' do
|
||||||
old_blob_id = material.file.blob.id
|
old_blob_id = material.file.blob.id
|
||||||
|
|
||||||
expect do
|
put "/materials/#{ material.id }", params: {
|
||||||
put "/materials/#{ material.id }", params: {
|
tag: 'material_update_new',
|
||||||
tag: 'material_update_new',
|
url: 'https://example.com/updated-source',
|
||||||
url: 'https://example.com/updated-source',
|
file: dummy_upload(filename: 'updated.jpg', type: 'image/jpeg')
|
||||||
file: dummy_upload(filename: 'updated.jpg', type: 'image/jpeg'),
|
}
|
||||||
export_paths: { legacy_drive: '伊地知ニジカ/updated.jpg' }
|
|
||||||
}
|
|
||||||
end.to change(MaterialVersion, :count).by(2)
|
|
||||||
|
|
||||||
expect(response).to have_http_status(:ok)
|
expect(response).to have_http_status(:ok)
|
||||||
|
|
||||||
@@ -469,15 +306,8 @@ RSpec.describe 'Materials API', type: :request do
|
|||||||
expect(material.updated_by_user).to eq(member_user)
|
expect(material.updated_by_user).to eq(member_user)
|
||||||
expect(material.file.attached?).to be(true)
|
expect(material.file.attached?).to be(true)
|
||||||
expect(material.file.blob.id).not_to eq(old_blob_id)
|
expect(material.file.blob.id).not_to eq(old_blob_id)
|
||||||
expect(ActiveStorage::Blob.where(id: old_blob_id).exists?).to be(true)
|
|
||||||
expect(material.file.blob.filename.to_s).to eq('updated.jpg')
|
expect(material.file.blob.filename.to_s).to eq('updated.jpg')
|
||||||
expect(material.file.blob.content_type).to eq('image/jpeg')
|
expect(material.file.blob.content_type).to eq('image/jpeg')
|
||||||
expect(material.version_no).to eq(2)
|
|
||||||
expect(material.material_versions.order(:version_no).last.event_type).to eq('update')
|
|
||||||
expect(material.material_export_items.first.export_path).to eq('伊地知ニジカ/updated.jpg')
|
|
||||||
expect(material.material_versions.order(:version_no).last.export_paths_json).to eq(
|
|
||||||
'legacy_drive' => '伊地知ニジカ/updated.jpg'
|
|
||||||
)
|
|
||||||
|
|
||||||
expect(json['id']).to eq(material.id)
|
expect(json['id']).to eq(material.id)
|
||||||
expect(json['file']).to be_present
|
expect(json['file']).to be_present
|
||||||
@@ -485,74 +315,7 @@ RSpec.describe 'Materials API', type: :request do
|
|||||||
expect(json.dig('tag', 'name')).to eq('material_update_new')
|
expect(json.dig('tag', 'name')).to eq('material_update_new')
|
||||||
end
|
end
|
||||||
|
|
||||||
it 'creates a create tag_version when update creates a new tag' do
|
it 'purges the existing file when file is omitted and url is provided' do
|
||||||
expect do
|
|
||||||
put "/materials/#{ material.id }", params: {
|
|
||||||
tag: 'material_update_versioned_tag',
|
|
||||||
file: dummy_upload(filename: 'updated.png')
|
|
||||||
}
|
|
||||||
end.to change(Tag, :count).by(1)
|
|
||||||
.and change(TagName, :count).by(1)
|
|
||||||
.and change(TagVersion, :count).by(1)
|
|
||||||
|
|
||||||
expect(response).to have_http_status(:ok)
|
|
||||||
|
|
||||||
tag = Tag.joins(:tag_name).find_by!(tag_names: { name: 'material_update_versioned_tag' })
|
|
||||||
version = tag.tag_versions.order(:version_no).last
|
|
||||||
|
|
||||||
expect(version.event_type).to eq('create')
|
|
||||||
expect(version.name).to eq('material_update_versioned_tag')
|
|
||||||
expect(version.category).to eq('material')
|
|
||||||
expect(version.created_by_user).to eq(member_user)
|
|
||||||
end
|
|
||||||
|
|
||||||
it 'backfills a create tag_version for an existing material tag without history' do
|
|
||||||
existing_tag =
|
|
||||||
Tag.create!(tag_name: TagName.create!(name: 'material_update_existing_no_history'),
|
|
||||||
category: :material)
|
|
||||||
|
|
||||||
expect(existing_tag.tag_versions).to be_empty
|
|
||||||
|
|
||||||
expect do
|
|
||||||
put "/materials/#{ material.id }", params: {
|
|
||||||
tag: 'material_update_existing_no_history',
|
|
||||||
file: dummy_upload(filename: 'updated.png')
|
|
||||||
}
|
|
||||||
end.to change(TagVersion, :count).by(1)
|
|
||||||
|
|
||||||
expect(response).to have_http_status(:ok)
|
|
||||||
|
|
||||||
version = existing_tag.reload.tag_versions.order(:version_no).last
|
|
||||||
expect(version.event_type).to eq('create')
|
|
||||||
expect(version.name).to eq('material_update_existing_no_history')
|
|
||||||
expect(version.category).to eq('material')
|
|
||||||
expect(version.created_by_user).to eq(member_user)
|
|
||||||
end
|
|
||||||
|
|
||||||
it 'backfills a create tag_version for an existing character tag without history' do
|
|
||||||
existing_tag =
|
|
||||||
Tag.create!(tag_name: TagName.create!(name: 'material_update_character_no_history'),
|
|
||||||
category: :character)
|
|
||||||
|
|
||||||
expect(existing_tag.tag_versions).to be_empty
|
|
||||||
|
|
||||||
expect do
|
|
||||||
put "/materials/#{ material.id }", params: {
|
|
||||||
tag: 'material_update_character_no_history',
|
|
||||||
file: dummy_upload(filename: 'updated.png')
|
|
||||||
}
|
|
||||||
end.to change(TagVersion, :count).by(1)
|
|
||||||
|
|
||||||
expect(response).to have_http_status(:ok)
|
|
||||||
|
|
||||||
version = existing_tag.reload.tag_versions.order(:version_no).last
|
|
||||||
expect(version.event_type).to eq('create')
|
|
||||||
expect(version.name).to eq('material_update_character_no_history')
|
|
||||||
expect(version.category).to eq('character')
|
|
||||||
expect(version.created_by_user).to eq(member_user)
|
|
||||||
end
|
|
||||||
|
|
||||||
it 'detaches the existing file without purging blob when url replaces file' do
|
|
||||||
old_blob_id = material.file.blob.id
|
old_blob_id = material.file.blob.id
|
||||||
|
|
||||||
put "/materials/#{ material.id }", params: {
|
put "/materials/#{ material.id }", params: {
|
||||||
@@ -568,7 +331,9 @@ RSpec.describe 'Materials API', type: :request do
|
|||||||
expect(material.updated_by_user).to eq(member_user)
|
expect(material.updated_by_user).to eq(member_user)
|
||||||
expect(material.file.attached?).to be(false)
|
expect(material.file.attached?).to be(false)
|
||||||
|
|
||||||
expect(ActiveStorage::Blob.where(id: old_blob_id).exists?).to be(true)
|
expect(
|
||||||
|
ActiveStorage::Blob.where(id: old_blob_id).exists?
|
||||||
|
).to be(false)
|
||||||
|
|
||||||
expect(json['id']).to eq(material.id)
|
expect(json['id']).to eq(material.id)
|
||||||
expect(json['file']).to be_nil
|
expect(json['file']).to be_nil
|
||||||
@@ -576,208 +341,11 @@ RSpec.describe 'Materials API', type: :request do
|
|||||||
expect(json.dig('tag', 'name')).to eq('material_update_remove_file')
|
expect(json.dig('tag', 'name')).to eq('material_update_remove_file')
|
||||||
expect(json['url']).to eq('https://example.com/updated-source')
|
expect(json['url']).to eq('https://example.com/updated-source')
|
||||||
end
|
end
|
||||||
|
|
||||||
it 'does not increase version for the same snapshot update' do
|
|
||||||
MaterialVersionRecorder.record!(material:, event_type: :create,
|
|
||||||
created_by_user: member_user)
|
|
||||||
TagVersioning.ensure_snapshot!(tag, created_by_user: member_user)
|
|
||||||
|
|
||||||
expect do
|
|
||||||
put "/materials/#{ material.id }", params: {
|
|
||||||
tag: 'material_update_old'
|
|
||||||
}
|
|
||||||
end.not_to change(MaterialVersion, :count)
|
|
||||||
|
|
||||||
expect(tag.reload.tag_versions.count).to eq(1)
|
|
||||||
|
|
||||||
expect(response).to have_http_status(:ok)
|
|
||||||
expect(material.reload.version_no).to eq(1)
|
|
||||||
end
|
|
||||||
|
|
||||||
it 'records update version when only export_path changes' do
|
|
||||||
MaterialVersionRecorder.record!(material:, event_type: :create,
|
|
||||||
created_by_user: member_user)
|
|
||||||
|
|
||||||
expect do
|
|
||||||
put "/materials/#{ material.id }", params: {
|
|
||||||
tag: 'material_update_old',
|
|
||||||
export_paths: { legacy_drive: '素材/only-path.png' }
|
|
||||||
}
|
|
||||||
end.to change(MaterialVersion, :count).by(1)
|
|
||||||
|
|
||||||
expect(response).to have_http_status(:ok)
|
|
||||||
expect(material.reload.material_export_items.first.export_path).to eq('素材/only-path.png')
|
|
||||||
expect(material.material_versions.order(:version_no).last.export_paths_json).to eq(
|
|
||||||
'legacy_drive' => '素材/only-path.png'
|
|
||||||
)
|
|
||||||
end
|
|
||||||
|
|
||||||
it 'removes export_path item when blank is submitted' do
|
|
||||||
MaterialExportItem.create!(material:, profile: 'legacy_drive',
|
|
||||||
export_path: '素材/remove.png',
|
|
||||||
created_by_user: member_user)
|
|
||||||
MaterialVersionRecorder.record!(material:, event_type: :create,
|
|
||||||
created_by_user: member_user)
|
|
||||||
|
|
||||||
expect do
|
|
||||||
put "/materials/#{ material.id }", params: {
|
|
||||||
tag: 'material_update_old',
|
|
||||||
export_paths: { legacy_drive: '' }
|
|
||||||
}
|
|
||||||
end.to change(MaterialExportItem, :count).by(-1)
|
|
||||||
.and change(MaterialVersion, :count).by(1)
|
|
||||||
|
|
||||||
expect(response).to have_http_status(:ok)
|
|
||||||
expect(material.reload.material_export_items).to be_empty
|
|
||||||
expect(material.material_versions.order(:version_no).last.export_paths_json).to eq({})
|
|
||||||
end
|
|
||||||
|
|
||||||
it 'rejects sha256-blocked replacement file' do
|
|
||||||
sha256 = Digest::SHA256.hexdigest('blocked-update')
|
|
||||||
MaterialImportBlock.create!(match_kind: 'sha256',
|
|
||||||
sha256:,
|
|
||||||
reason: 'source_owner_request',
|
|
||||||
created_by_user: admin_user)
|
|
||||||
|
|
||||||
put "/materials/#{ material.id }", params: {
|
|
||||||
tag: 'material_update_old',
|
|
||||||
file: dummy_upload(filename: 'blocked.png', body: 'blocked-update')
|
|
||||||
}
|
|
||||||
|
|
||||||
expect(response).to have_http_status(:unprocessable_entity)
|
|
||||||
expect(material.reload.file.blob.filename.to_s).to eq('old.png')
|
|
||||||
end
|
|
||||||
end
|
|
||||||
end
|
|
||||||
|
|
||||||
describe 'GET /materials/download.zip' do
|
|
||||||
let!(:tag_a) { Tag.create!(tag_name: TagName.create!(name: 'zip_a'), category: :material) }
|
|
||||||
let!(:tag_b) { Tag.create!(tag_name: TagName.create!(name: 'zip_b'), category: :material) }
|
|
||||||
let!(:material_a) do
|
|
||||||
build_material(tag: tag_a, user: member_user,
|
|
||||||
file: dummy_upload(filename: 'a.png', body: 'zip-a'))
|
|
||||||
end
|
|
||||||
let!(:material_b) do
|
|
||||||
build_material(tag: tag_b, user: member_user,
|
|
||||||
file: dummy_upload(filename: 'b.png', body: 'zip-b'))
|
|
||||||
end
|
|
||||||
|
|
||||||
before do
|
|
||||||
MaterialExportItem.create!(material: material_a, profile: 'legacy_drive',
|
|
||||||
export_path: '素材/a.png',
|
|
||||||
created_by_user: member_user)
|
|
||||||
MaterialExportItem.create!(material: material_b, profile: 'legacy_drive',
|
|
||||||
export_path: '素材/b.png',
|
|
||||||
created_by_user: member_user)
|
|
||||||
end
|
|
||||||
|
|
||||||
it 'uses material_export_items.export_path as ZIP entry paths' do
|
|
||||||
get '/materials/download.zip', params: { profile: 'legacy_drive' }
|
|
||||||
|
|
||||||
expect(response).to have_http_status(:ok)
|
|
||||||
expect(response.media_type).to eq('application/zip')
|
|
||||||
expect(response.body.b).to include('素材/a.png'.b)
|
|
||||||
expect(response.body.b).to include('素材/b.png'.b)
|
|
||||||
end
|
|
||||||
|
|
||||||
it 'filters by tag_id' do
|
|
||||||
get '/materials/download.zip', params: { profile: 'legacy_drive', tag_id: tag_a.id }
|
|
||||||
|
|
||||||
expect(response).to have_http_status(:ok)
|
|
||||||
expect(response.body.b).to include('素材/a.png'.b)
|
|
||||||
expect(response.body.b).not_to include('素材/b.png'.b)
|
|
||||||
end
|
|
||||||
|
|
||||||
it 'does not include disabled export items' do
|
|
||||||
material_b.material_export_items.first.update!(enabled: false)
|
|
||||||
|
|
||||||
get '/materials/download.zip', params: { profile: 'legacy_drive' }
|
|
||||||
|
|
||||||
expect(response).to have_http_status(:ok)
|
|
||||||
expect(response.body.b).to include('素材/a.png'.b)
|
|
||||||
expect(response.body.b).not_to include('素材/b.png'.b)
|
|
||||||
end
|
|
||||||
end
|
|
||||||
|
|
||||||
describe 'GET /materials/versions' do
|
|
||||||
let!(:tag) do
|
|
||||||
Tag.create!(tag_name: TagName.create!(name: 'material_history'), category: :material)
|
|
||||||
end
|
|
||||||
let!(:material) do
|
|
||||||
build_material(tag:, user: member_user, file: dummy_upload(filename: 'history.png'))
|
|
||||||
end
|
|
||||||
|
|
||||||
it 'returns material versions in reverse chronological order' do
|
|
||||||
MaterialVersionRecorder.record!(material:, event_type: :create,
|
|
||||||
created_by_user: member_user)
|
|
||||||
material.update!(url: 'https://example.com/history')
|
|
||||||
MaterialVersionRecorder.record!(material:, event_type: :update,
|
|
||||||
created_by_user: admin_user)
|
|
||||||
|
|
||||||
get '/materials/versions', params: { material_id: material.id }
|
|
||||||
|
|
||||||
expect(response).to have_http_status(:ok)
|
|
||||||
expect(json['count']).to eq(2)
|
|
||||||
versions = json.fetch('versions')
|
|
||||||
expect(versions.map { |v| v['version_no'] }).to eq([2, 1])
|
|
||||||
expect(versions.first).to include(
|
|
||||||
'material_id' => material.id,
|
|
||||||
'event_type' => 'update',
|
|
||||||
'tag_name' => 'material_history',
|
|
||||||
'file_filename' => 'history.png',
|
|
||||||
'url' => 'https://example.com/history'
|
|
||||||
)
|
|
||||||
expect(versions.first['created_by_user']).to include('id' => admin_user.id)
|
|
||||||
end
|
|
||||||
|
|
||||||
it 'filters material versions by tag and event_type' do
|
|
||||||
MaterialVersionRecorder.record!(material:, event_type: :create,
|
|
||||||
created_by_user: member_user)
|
|
||||||
|
|
||||||
get '/materials/versions', params: {
|
|
||||||
tag: 'material_history',
|
|
||||||
event_type: 'create'
|
|
||||||
}
|
|
||||||
|
|
||||||
expect(response).to have_http_status(:ok)
|
|
||||||
expect(json.fetch('versions').map { |v| v['material_id'] }).to include(material.id)
|
|
||||||
expect(json.fetch('versions').map { |v| v['event_type'] }).to all(eq('create'))
|
|
||||||
end
|
|
||||||
end
|
|
||||||
|
|
||||||
describe 'material sync suppressions' do
|
|
||||||
it 'requires member permission for index' do
|
|
||||||
sign_out
|
|
||||||
|
|
||||||
get '/materials/suppressions'
|
|
||||||
|
|
||||||
expect(response).to have_http_status(:unauthorized)
|
|
||||||
end
|
|
||||||
|
|
||||||
it 'creates source suppression as member' do
|
|
||||||
sign_in_as(member_user)
|
|
||||||
|
|
||||||
expect do
|
|
||||||
post '/materials/suppressions', params: {
|
|
||||||
source_kind: 'google_drive_path_prefix',
|
|
||||||
drive_path: '伊地知ニジカ/危険素材',
|
|
||||||
reason: 'copyright_high_risk'
|
|
||||||
}
|
|
||||||
end.to change(MaterialSyncSuppression, :count).by(1)
|
|
||||||
|
|
||||||
expect(response).to have_http_status(:created)
|
|
||||||
expect(json).to include(
|
|
||||||
'source_kind' => 'google_drive_path_prefix',
|
|
||||||
'drive_path' => '伊地知ニジカ/危険素材',
|
|
||||||
'normalized_source_key' => 'google_drive_path_prefix:伊地知ニジカ/危険素材'
|
|
||||||
)
|
|
||||||
end
|
end
|
||||||
end
|
end
|
||||||
|
|
||||||
describe 'DELETE /materials/:id' do
|
describe 'DELETE /materials/:id' do
|
||||||
let!(:tag) do
|
let!(:tag) { Tag.create!(tag_name: TagName.create!(name: 'material_destroy'), category: :material) }
|
||||||
Tag.create!(tag_name: TagName.create!(name: 'material_destroy'), category: :material)
|
|
||||||
end
|
|
||||||
let!(:material) do
|
let!(:material) do
|
||||||
build_material(tag:, user: member_user, file: dummy_upload(filename: 'destroy.png'))
|
build_material(tag:, user: member_user, file: dummy_upload(filename: 'destroy.png'))
|
||||||
end
|
end
|
||||||
|
|||||||
@@ -1,112 +0,0 @@
|
|||||||
require 'rails_helper'
|
|
||||||
|
|
||||||
RSpec.describe 'Post imports API', type: :request do
|
|
||||||
let(:member) { create(:user, :member) }
|
|
||||||
|
|
||||||
before do
|
|
||||||
allow(Preview::UrlSafety).to receive(:validate) do |url|
|
|
||||||
[URI.parse(url), ['8.8.8.8']]
|
|
||||||
end
|
|
||||||
allow(PostMetadataFetcher).to receive(:fetch).and_return(
|
|
||||||
title: 'fetched title',
|
|
||||||
thumbnail_base: nil,
|
|
||||||
tags: ''
|
|
||||||
)
|
|
||||||
end
|
|
||||||
|
|
||||||
describe 'POST /posts/import/preview' do
|
|
||||||
it 'requires a member' do
|
|
||||||
sign_out
|
|
||||||
post '/posts/import/preview', params: { source: 'https://example.com/post' }
|
|
||||||
expect(response).to have_http_status(:unauthorized)
|
|
||||||
|
|
||||||
sign_in_as(create(:user, :guest))
|
|
||||||
post '/posts/import/preview', params: { source: 'https://example.com/post' }
|
|
||||||
expect(response).to have_http_status(:forbidden)
|
|
||||||
end
|
|
||||||
|
|
||||||
it 'parses a URL list and returns preview rows' do
|
|
||||||
sign_in_as(member)
|
|
||||||
|
|
||||||
post '/posts/import/preview', params: {
|
|
||||||
source: " https://example.com/one \r\n\r\nhttps://example.com/two"
|
|
||||||
}
|
|
||||||
|
|
||||||
expect(response).to have_http_status(:ok)
|
|
||||||
expect(json.fetch('rows').map { _1.fetch('source_row') }).to eq([1, 3])
|
|
||||||
expect(json.fetch('rows').map { _1.fetch('url') }).to eq([
|
|
||||||
'https://example.com/one',
|
|
||||||
'https://example.com/two'
|
|
||||||
])
|
|
||||||
end
|
|
||||||
|
|
||||||
it 'returns a safe 400 response for an invalid source' do
|
|
||||||
sign_in_as(member)
|
|
||||||
|
|
||||||
post '/posts/import/preview', params: { source: '' }
|
|
||||||
|
|
||||||
expect(response).to have_http_status(:bad_request)
|
|
||||||
expect(json.fetch('message')).to eq('URL を入力してください.')
|
|
||||||
end
|
|
||||||
end
|
|
||||||
|
|
||||||
describe 'POST /posts/import/validate' do
|
|
||||||
it 'accepts camel-case row properties and returns their warnings' do
|
|
||||||
sign_in_as(member)
|
|
||||||
|
|
||||||
post '/posts/import/validate', params: {
|
|
||||||
rows: [{
|
|
||||||
sourceRow: '1',
|
|
||||||
url: 'https://example.com/post',
|
|
||||||
metadataUrl: 'https://example.com/post',
|
|
||||||
attributes: { title: 'manual title' },
|
|
||||||
provenance: { url: 'manual', title: 'manual' },
|
|
||||||
tagSources: { automatic: '', manual: '' },
|
|
||||||
fieldWarnings: { title: ['old warning'] },
|
|
||||||
baseWarnings: ['base warning']
|
|
||||||
}],
|
|
||||||
changed_row: -1
|
|
||||||
}
|
|
||||||
|
|
||||||
expect(response).to have_http_status(:ok)
|
|
||||||
result = json.fetch('rows').first
|
|
||||||
expect(result.fetch('source_row')).to eq(1)
|
|
||||||
expect(result.fetch('tag_sources')).to eq('automatic' => '', 'manual' => '')
|
|
||||||
expect(result.fetch('field_warnings')).to eq('title' => ['old warning'])
|
|
||||||
expect(result.fetch('base_warnings')).to eq(['base warning'])
|
|
||||||
end
|
|
||||||
|
|
||||||
it 'rejects a non-array rows value with 400' do
|
|
||||||
sign_in_as(member)
|
|
||||||
|
|
||||||
post '/posts/import/validate', params: { rows: { sourceRow: 1 }, changed_row: -1 }
|
|
||||||
|
|
||||||
expect(response).to have_http_status(:bad_request)
|
|
||||||
expect(json.fetch('message')).to eq('取込行の形式が不正です.')
|
|
||||||
end
|
|
||||||
end
|
|
||||||
|
|
||||||
describe 'POST /posts/import' do
|
|
||||||
it 'returns a formal skipped result for an existing post' do
|
|
||||||
existing = create(:post, url: 'https://example.com/existing')
|
|
||||||
sign_in_as(member)
|
|
||||||
|
|
||||||
post '/posts/import', params: {
|
|
||||||
rows: [{
|
|
||||||
sourceRow: 1,
|
|
||||||
url: existing.url,
|
|
||||||
attributes: { title: 'ignored' },
|
|
||||||
provenance: { url: 'manual', title: 'manual' },
|
|
||||||
tagSources: { automatic: '', manual: '' }
|
|
||||||
}]
|
|
||||||
}
|
|
||||||
|
|
||||||
expect(response).to have_http_status(:ok)
|
|
||||||
expect(json).to include('created' => 0, 'skipped' => 1, 'failed' => 0)
|
|
||||||
expect(json.fetch('rows').first).to include(
|
|
||||||
'status' => 'skipped',
|
|
||||||
'existing_post_id' => existing.id
|
|
||||||
)
|
|
||||||
end
|
|
||||||
end
|
|
||||||
end
|
|
||||||
@@ -1,5 +1,4 @@
|
|||||||
require 'rails_helper'
|
require 'rails_helper'
|
||||||
require 'base64'
|
|
||||||
require 'set'
|
require 'set'
|
||||||
|
|
||||||
include ActiveSupport::Testing::TimeHelpers
|
include ActiveSupport::Testing::TimeHelpers
|
||||||
@@ -9,11 +8,6 @@ RSpec.describe 'Posts API', type: :request do
|
|||||||
# resized_thumbnail! が MiniMagick 依存でコケやすいので request spec ではスタブしとくのが無難。
|
# resized_thumbnail! が MiniMagick 依存でコケやすいので request spec ではスタブしとくのが無難。
|
||||||
before do
|
before do
|
||||||
allow_any_instance_of(Post).to receive(:resized_thumbnail!).and_return(true)
|
allow_any_instance_of(Post).to receive(:resized_thumbnail!).and_return(true)
|
||||||
allow(Post).to receive(:resized_thumbnail_attachment).and_return(
|
|
||||||
io: StringIO.new('dummy'),
|
|
||||||
filename: 'resized_thumbnail.jpg',
|
|
||||||
content_type: 'image/jpeg'
|
|
||||||
)
|
|
||||||
end
|
end
|
||||||
|
|
||||||
def create_nico_tag!(name)
|
def create_nico_tag!(name)
|
||||||
@@ -25,18 +19,6 @@ RSpec.describe 'Posts API', type: :request do
|
|||||||
Rack::Test::UploadedFile.new(StringIO.new('dummy'), 'image/jpeg', original_filename: 'dummy.jpg')
|
Rack::Test::UploadedFile.new(StringIO.new('dummy'), 'image/jpeg', original_filename: 'dummy.jpg')
|
||||||
end
|
end
|
||||||
|
|
||||||
def real_thumbnail_upload
|
|
||||||
gif =
|
|
||||||
Base64.decode64(
|
|
||||||
'R0lGODdhAQABAIABAP///wAAACwAAAAAAQABAAACAkQBADs=')
|
|
||||||
|
|
||||||
Rack::Test::UploadedFile.new(
|
|
||||||
StringIO.new(gif),
|
|
||||||
'image/gif',
|
|
||||||
original_filename: 'thumbnail.gif'
|
|
||||||
)
|
|
||||||
end
|
|
||||||
|
|
||||||
def post_write_params params = { }
|
def post_write_params params = { }
|
||||||
{ parent_post_ids: '' }.merge(params)
|
{ parent_post_ids: '' }.merge(params)
|
||||||
end
|
end
|
||||||
@@ -54,7 +36,6 @@ RSpec.describe 'Posts API', type: :request do
|
|||||||
title: post.title,
|
title: post.title,
|
||||||
url: post.url,
|
url: post.url,
|
||||||
thumbnail_base: post.thumbnail_base,
|
thumbnail_base: post.thumbnail_base,
|
||||||
video_ms: post.video_ms,
|
|
||||||
tags: post.snapshot_tag_names.join(' '),
|
tags: post.snapshot_tag_names.join(' '),
|
||||||
parent_post_ids: post.snapshot_parent_post_ids.join(' '),
|
parent_post_ids: post.snapshot_parent_post_ids.join(' '),
|
||||||
original_created_from: post.original_created_from,
|
original_created_from: post.original_created_from,
|
||||||
@@ -146,22 +127,6 @@ RSpec.describe 'Posts API', type: :request do
|
|||||||
expect(all_tag_names).to include("spec_tag")
|
expect(all_tag_names).to include("spec_tag")
|
||||||
end
|
end
|
||||||
|
|
||||||
it 'keeps children and sections keys in non-detail tag responses' do
|
|
||||||
PostTagSection.create!(post: hit_post, tag:, begin_ms: 1_000, end_ms: nil)
|
|
||||||
|
|
||||||
get '/posts'
|
|
||||||
|
|
||||||
expect(response).to have_http_status(:ok)
|
|
||||||
|
|
||||||
hit_json = json.fetch('posts').find { |post| post['id'] == hit_post.id }
|
|
||||||
tag_json = hit_json.fetch('tags').find { |item| item['name'] == 'spec_tag' }
|
|
||||||
|
|
||||||
expect(tag_json.fetch('children')).to eq([])
|
|
||||||
expect(tag_json.fetch('sections')).to eq([
|
|
||||||
{ 'begin_ms' => 1_000, 'end_ms' => nil }
|
|
||||||
])
|
|
||||||
end
|
|
||||||
|
|
||||||
context "when q is provided" do
|
context "when q is provided" do
|
||||||
it "filters posts by q (hit case)" do
|
it "filters posts by q (hit case)" do
|
||||||
get "/posts", params: { tags: "spec_tag" }
|
get "/posts", params: { tags: "spec_tag" }
|
||||||
@@ -731,66 +696,6 @@ RSpec.describe 'Posts API', type: :request do
|
|||||||
expect(json['tags'][0]).to have_key('name')
|
expect(json['tags'][0]).to have_key('name')
|
||||||
end
|
end
|
||||||
|
|
||||||
it '201 when posting manually with a thumbnail' do
|
|
||||||
sign_in_as(member)
|
|
||||||
allow(Post).to receive(:resized_thumbnail_attachment).and_call_original
|
|
||||||
|
|
||||||
post '/posts', params: post_write_params(
|
|
||||||
title: 'thumbnail post',
|
|
||||||
url: 'https://example.com/thumbnail-post',
|
|
||||||
tags: 'spec_tag',
|
|
||||||
thumbnail: real_thumbnail_upload
|
|
||||||
)
|
|
||||||
|
|
||||||
expect(response).to have_http_status(:created)
|
|
||||||
post_record = Post.find(json.fetch('id'))
|
|
||||||
expect(post_record.thumbnail).to be_attached
|
|
||||||
expect(post_record.thumbnail.blob.content_type).to eq('image/jpeg')
|
|
||||||
expect { post_record.thumbnail.download }.not_to raise_error
|
|
||||||
end
|
|
||||||
|
|
||||||
it 'resizes the thumbnail before the create transaction begins' do
|
|
||||||
sign_in_as(member)
|
|
||||||
|
|
||||||
open_transactions = []
|
|
||||||
baseline_open_transactions = Post.connection.open_transactions
|
|
||||||
allow(Post).to receive(:resized_thumbnail_attachment) do |_upload|
|
|
||||||
open_transactions << Post.connection.open_transactions
|
|
||||||
{ io: StringIO.new('dummy'),
|
|
||||||
filename: 'resized_thumbnail.jpg',
|
|
||||||
content_type: 'image/jpeg' }
|
|
||||||
end
|
|
||||||
|
|
||||||
post '/posts', params: post_write_params(
|
|
||||||
title: 'transaction post',
|
|
||||||
url: 'https://example.com/transaction-post',
|
|
||||||
tags: 'spec_tag',
|
|
||||||
thumbnail: dummy_upload
|
|
||||||
)
|
|
||||||
|
|
||||||
expect(response).to have_http_status(:created)
|
|
||||||
expect(open_transactions).to eq([baseline_open_transactions])
|
|
||||||
end
|
|
||||||
|
|
||||||
it 'returns 422 and does not create a post when thumbnail resize fails' do
|
|
||||||
sign_in_as(member)
|
|
||||||
allow(Post).to receive(:resized_thumbnail_attachment).and_raise(MiniMagick::Error)
|
|
||||||
|
|
||||||
expect {
|
|
||||||
post '/posts', params: post_write_params(
|
|
||||||
title: 'broken thumbnail post',
|
|
||||||
url: 'https://example.com/broken-thumbnail-post',
|
|
||||||
tags: 'spec_tag',
|
|
||||||
thumbnail: dummy_upload
|
|
||||||
)
|
|
||||||
}.not_to change(Post, :count)
|
|
||||||
|
|
||||||
expect(response).to have_http_status(:unprocessable_entity)
|
|
||||||
expect(json.fetch('errors')).to include(
|
|
||||||
'thumbnail' => ['サムネイル画像の変換に失敗しました.']
|
|
||||||
)
|
|
||||||
end
|
|
||||||
|
|
||||||
it '201 and creates post + tags when member and tags have aliases' do
|
it '201 and creates post + tags when member and tags have aliases' do
|
||||||
sign_in_as(member)
|
sign_in_as(member)
|
||||||
|
|
||||||
@@ -862,198 +767,6 @@ RSpec.describe 'Posts API', type: :request do
|
|||||||
expect(saved_names).not_to include('deprecated_parent', 'deprecated_grandparent')
|
expect(saved_names).not_to include('deprecated_parent', 'deprecated_grandparent')
|
||||||
end
|
end
|
||||||
|
|
||||||
it 'returns validation error for an invalid section literal' do
|
|
||||||
sign_in_as(member)
|
|
||||||
|
|
||||||
post '/posts', params: post_write_params(
|
|
||||||
title: 'invalid section literal',
|
|
||||||
url: 'https://example.com/invalid-section-literal',
|
|
||||||
tags: 'spec_tag[1:aa-2:00]',
|
|
||||||
thumbnail: dummy_upload
|
|
||||||
)
|
|
||||||
|
|
||||||
expect(response).to have_http_status(:unprocessable_entity)
|
|
||||||
expect(json).to include(
|
|
||||||
'type' => 'validation_error',
|
|
||||||
'message' => '入力内容を確認してください.'
|
|
||||||
)
|
|
||||||
expect(json.fetch('errors')).to include(
|
|
||||||
'tags' => ['タグ区間の記法が不正です.']
|
|
||||||
)
|
|
||||||
end
|
|
||||||
|
|
||||||
it 'creates a video post with duration' do
|
|
||||||
sign_in_as(member)
|
|
||||||
|
|
||||||
post '/posts', params: post_write_params(
|
|
||||||
title: 'video post',
|
|
||||||
url: 'https://example.com/video-post',
|
|
||||||
tags: '動画 spec_tag',
|
|
||||||
duration: '3:00.500',
|
|
||||||
thumbnail: dummy_upload
|
|
||||||
)
|
|
||||||
|
|
||||||
expect(response).to have_http_status(:created)
|
|
||||||
expect(Post.find(json.fetch('id')).video_ms).to eq(180_500)
|
|
||||||
expect(json.fetch('video_ms')).to eq(180_500)
|
|
||||||
end
|
|
||||||
|
|
||||||
it 'creates a video post with number input duration seconds' do
|
|
||||||
sign_in_as(member)
|
|
||||||
|
|
||||||
post '/posts', params: post_write_params(
|
|
||||||
title: 'video post seconds',
|
|
||||||
url: 'https://example.com/video-post-seconds',
|
|
||||||
tags: '動画 spec_tag',
|
|
||||||
duration: '180.5',
|
|
||||||
thumbnail: dummy_upload
|
|
||||||
)
|
|
||||||
|
|
||||||
expect(response).to have_http_status(:created)
|
|
||||||
expect(Post.find(json.fetch('id')).video_ms).to eq(180_500)
|
|
||||||
expect(json.fetch('video_ms')).to eq(180_500)
|
|
||||||
end
|
|
||||||
|
|
||||||
it 'clears video_ms when the saved tags do not include 動画' do
|
|
||||||
sign_in_as(member)
|
|
||||||
|
|
||||||
post '/posts', params: post_write_params(
|
|
||||||
title: 'non video post',
|
|
||||||
url: 'https://example.com/non-video-post',
|
|
||||||
tags: 'spec_tag',
|
|
||||||
duration: '3:00',
|
|
||||||
thumbnail: dummy_upload
|
|
||||||
)
|
|
||||||
|
|
||||||
expect(response).to have_http_status(:created)
|
|
||||||
expect(Post.find(json.fetch('id')).video_ms).to be_nil
|
|
||||||
end
|
|
||||||
|
|
||||||
it 'returns validation error when a bounded section exceeds duration' do
|
|
||||||
sign_in_as(member)
|
|
||||||
|
|
||||||
post '/posts', params: post_write_params(
|
|
||||||
title: 'too long section',
|
|
||||||
url: 'https://example.com/too-long-section',
|
|
||||||
tags: '動画 伊地知ニジカ[2:50-3:10]',
|
|
||||||
duration: '3:00',
|
|
||||||
thumbnail: dummy_upload
|
|
||||||
)
|
|
||||||
|
|
||||||
expect(response).to have_http_status(:unprocessable_entity)
|
|
||||||
expect(json.fetch('errors')).to include(
|
|
||||||
'video_ms' => ['タグ区間の終端が動画時間を超えてゐます.']
|
|
||||||
)
|
|
||||||
end
|
|
||||||
|
|
||||||
it 'saves open-ended sections with end_ms NULL' do
|
|
||||||
sign_in_as(member)
|
|
||||||
|
|
||||||
post '/posts', params: post_write_params(
|
|
||||||
title: 'open ended section literal',
|
|
||||||
url: 'https://example.com/open-ended-section-literal',
|
|
||||||
tags: '動画 伊地知ニジカ[1:00-]',
|
|
||||||
duration: '3:00',
|
|
||||||
thumbnail: dummy_upload
|
|
||||||
)
|
|
||||||
|
|
||||||
expect(response).to have_http_status(:created)
|
|
||||||
|
|
||||||
created_post = Post.find(json.fetch('id'))
|
|
||||||
tag = Tag.joins(:tag_name).find_by!(tag_names: { name: '伊地知ニジカ' })
|
|
||||||
section = PostTagSection.find_by!(post: created_post, tag:)
|
|
||||||
|
|
||||||
expect(section.begin_ms).to eq(60_000)
|
|
||||||
expect(section.end_ms).to be_nil
|
|
||||||
end
|
|
||||||
|
|
||||||
it 'does not save sections for [-]' do
|
|
||||||
sign_in_as(member)
|
|
||||||
|
|
||||||
post '/posts', params: post_write_params(
|
|
||||||
title: 'fully open section literal',
|
|
||||||
url: 'https://example.com/fully-open-section-literal',
|
|
||||||
tags: '伊地知ニジカ[-]',
|
|
||||||
thumbnail: dummy_upload
|
|
||||||
)
|
|
||||||
|
|
||||||
expect(response).to have_http_status(:created)
|
|
||||||
|
|
||||||
created_post = Post.find(json.fetch('id'))
|
|
||||||
tag = Tag.joins(:tag_name).find_by!(tag_names: { name: '伊地知ニジカ' })
|
|
||||||
expect(PostTagSection.find_by(post: created_post, tag:)).to be_nil
|
|
||||||
end
|
|
||||||
|
|
||||||
it 'does not save sections for [0:00-]' do
|
|
||||||
sign_in_as(member)
|
|
||||||
|
|
||||||
post '/posts', params: post_write_params(
|
|
||||||
title: 'zero open ended section literal',
|
|
||||||
url: 'https://example.com/zero-open-ended-section-literal',
|
|
||||||
tags: '伊地知ニジカ[0:00-]',
|
|
||||||
thumbnail: dummy_upload
|
|
||||||
)
|
|
||||||
|
|
||||||
expect(response).to have_http_status(:created)
|
|
||||||
|
|
||||||
created_post = Post.find(json.fetch('id'))
|
|
||||||
tag = Tag.joins(:tag_name).find_by!(tag_names: { name: '伊地知ニジカ' })
|
|
||||||
expect(PostTagSection.find_by(post: created_post, tag:)).to be_nil
|
|
||||||
end
|
|
||||||
|
|
||||||
it 'returns end_ms null for open-ended sections in show response' do
|
|
||||||
sign_in_as(member)
|
|
||||||
|
|
||||||
post '/posts', params: post_write_params(
|
|
||||||
title: 'show open ended section literal',
|
|
||||||
url: 'https://example.com/show-open-ended-section-literal',
|
|
||||||
tags: '動画 伊地知ニジカ[1:00-]',
|
|
||||||
duration: '3:00',
|
|
||||||
thumbnail: dummy_upload
|
|
||||||
)
|
|
||||||
|
|
||||||
expect(response).to have_http_status(:created)
|
|
||||||
|
|
||||||
get "/posts/#{ json.fetch('id') }"
|
|
||||||
|
|
||||||
expect(response).to have_http_status(:ok)
|
|
||||||
tag_json = json.fetch('tags').find { |item| item['name'] == '伊地知ニジカ' }
|
|
||||||
expect(tag_json.fetch('sections')).to eq([
|
|
||||||
{ 'begin_ms' => 60_000, 'end_ms' => nil }
|
|
||||||
])
|
|
||||||
end
|
|
||||||
|
|
||||||
it 'allows open-ended sections when begin is within duration' do
|
|
||||||
sign_in_as(member)
|
|
||||||
|
|
||||||
post '/posts', params: post_write_params(
|
|
||||||
title: 'valid open ended section literal',
|
|
||||||
url: 'https://example.com/valid-open-ended-section-literal',
|
|
||||||
tags: '動画 伊地知ニジカ[1:00-]',
|
|
||||||
duration: '3:00',
|
|
||||||
thumbnail: dummy_upload
|
|
||||||
)
|
|
||||||
|
|
||||||
expect(response).to have_http_status(:created)
|
|
||||||
end
|
|
||||||
|
|
||||||
it 'rejects open-ended sections when begin equals duration' do
|
|
||||||
sign_in_as(member)
|
|
||||||
|
|
||||||
post '/posts', params: post_write_params(
|
|
||||||
title: 'invalid open ended section literal',
|
|
||||||
url: 'https://example.com/invalid-open-ended-section-literal',
|
|
||||||
tags: '動画 伊地知ニジカ[3:00-]',
|
|
||||||
duration: '3:00',
|
|
||||||
thumbnail: dummy_upload
|
|
||||||
)
|
|
||||||
|
|
||||||
expect(response).to have_http_status(:unprocessable_entity)
|
|
||||||
expect(json.fetch('errors')).to include(
|
|
||||||
'video_ms' => ['タグ区間の開始が動画時間以上です.']
|
|
||||||
)
|
|
||||||
end
|
|
||||||
|
|
||||||
context "when nico tag already exists in tags" do
|
context "when nico tag already exists in tags" do
|
||||||
before do
|
before do
|
||||||
Tag.find_undiscard_or_create_by!(
|
Tag.find_undiscard_or_create_by!(
|
||||||
|
|||||||
@@ -1,46 +1,28 @@
|
|||||||
require 'rails_helper'
|
require "rails_helper"
|
||||||
|
|
||||||
|
|
||||||
RSpec.describe 'Preview', type: :request do
|
RSpec.describe "Preview", type: :request do
|
||||||
describe 'GET /preview/title' do
|
describe "GET /preview/title" do
|
||||||
it '401 unless logged in' do
|
it "401 unless logged in" do
|
||||||
sign_out
|
sign_out
|
||||||
get '/preview/title', params: { url: 'example.com' }
|
get "/preview/title", params: { url: "example.com" }
|
||||||
expect(response).to have_http_status(:unauthorized)
|
expect(response).to have_http_status(:unauthorized)
|
||||||
end
|
end
|
||||||
|
|
||||||
it '403 when logged in as guest' do
|
it "400 when url blank" do
|
||||||
sign_in_as(create(:user, :guest))
|
sign_in_as(create(:user))
|
||||||
get '/preview/title', params: { url: 'example.com' }
|
get "/preview/title", params: { url: "" }
|
||||||
expect(response).to have_http_status(:forbidden)
|
|
||||||
end
|
|
||||||
|
|
||||||
it '400 when url blank' do
|
|
||||||
sign_in_as(create(:user, :member))
|
|
||||||
get '/preview/title', params: { url: '' }
|
|
||||||
expect(response).to have_http_status(:bad_request)
|
expect(response).to have_http_status(:bad_request)
|
||||||
end
|
end
|
||||||
|
|
||||||
it 'returns parsed title' do
|
it "returns parsed title (stubbing URI.open)" do
|
||||||
sign_in_as(create(:user, :member))
|
sign_in_as(create(:user))
|
||||||
allow(Preview::ThumbnailFetcher)
|
fake_html = "<html><head><title> Hello </title></head></html>"
|
||||||
.to receive(:title)
|
allow(URI).to receive(:open).and_return(StringIO.new(fake_html))
|
||||||
.with('example.com')
|
|
||||||
.and_return('Hello')
|
|
||||||
|
|
||||||
get '/preview/title', params: { url: 'example.com' }
|
get "/preview/title", params: { url: "example.com" }
|
||||||
expect(response).to have_http_status(:ok)
|
expect(response).to have_http_status(:ok)
|
||||||
expect(json['title']).to eq('Hello')
|
expect(json["title"]).to eq("Hello")
|
||||||
end
|
|
||||||
|
|
||||||
it '413 when fetched response is too large' do
|
|
||||||
sign_in_as(create(:user, :member))
|
|
||||||
allow(Preview::ThumbnailFetcher)
|
|
||||||
.to receive(:title)
|
|
||||||
.and_raise(Preview::HttpFetcher::ResponseTooLarge, '外部データが大きすぎます.')
|
|
||||||
|
|
||||||
get '/preview/title', params: { url: 'example.com' }
|
|
||||||
expect(response).to have_http_status(:payload_too_large)
|
|
||||||
end
|
end
|
||||||
end
|
end
|
||||||
end
|
end
|
||||||
|
|||||||
@@ -56,17 +56,6 @@ RSpec.describe "TagChildren", type: :request do
|
|||||||
|
|
||||||
expect(response).to have_http_status(:no_content)
|
expect(response).to have_http_status(:no_content)
|
||||||
end
|
end
|
||||||
|
|
||||||
it 'returns 422 and does not create relation when the new link makes a cycle' do
|
|
||||||
TagImplication.create!(tag: parent, parent_tag: child)
|
|
||||||
|
|
||||||
expect {
|
|
||||||
do_request
|
|
||||||
}.not_to change(TagImplication, :count)
|
|
||||||
|
|
||||||
expect(response).to have_http_status(:unprocessable_entity)
|
|
||||||
expect(TagImplication.where(tag: child, parent_tag: parent)).not_to exist
|
|
||||||
end
|
|
||||||
end
|
end
|
||||||
|
|
||||||
context "when Tag.find raises (invalid ids)" do
|
context "when Tag.find raises (invalid ids)" do
|
||||||
|
|||||||
@@ -581,58 +581,6 @@ RSpec.describe 'Tags API', type: :request do
|
|||||||
expect(response).to have_http_status(:ok)
|
expect(response).to have_http_status(:ok)
|
||||||
expect(wiki_page.reload.wiki_versions.count).to eq(before_wiki_version_count)
|
expect(wiki_page.reload.wiki_versions.count).to eq(before_wiki_version_count)
|
||||||
end
|
end
|
||||||
|
|
||||||
it 'full update で旧名 alias が残った tag を PATCH で旧名へ戻せる' do
|
|
||||||
put "/tags/#{ tag.id }", params: {
|
|
||||||
name: 'patch_roundtrip_target',
|
|
||||||
category: 'general',
|
|
||||||
aliases: 'unko',
|
|
||||||
parent_tags: '',
|
|
||||||
deprecated: '0',
|
|
||||||
}
|
|
||||||
|
|
||||||
expect(response).to have_http_status(:ok)
|
|
||||||
expect(TagName.find_by!(name: 'spec_tag').canonical).to eq(tag.reload.tag_name)
|
|
||||||
|
|
||||||
patch "/tags/#{ tag.id }", params: { name: 'spec_tag' }
|
|
||||||
|
|
||||||
expect(response).to have_http_status(:ok)
|
|
||||||
|
|
||||||
tag.reload
|
|
||||||
|
|
||||||
expect(tag.name).to eq('spec_tag')
|
|
||||||
expect(tag.tag_name.canonical_id).to be_nil
|
|
||||||
expect(TagName.find_by!(name: 'patch_roundtrip_target').canonical).to eq(tag.tag_name)
|
|
||||||
end
|
|
||||||
|
|
||||||
it '別 tag の正規名には変更できない' do
|
|
||||||
wiki_page =
|
|
||||||
Wiki::Commit.create_content!(
|
|
||||||
tag_name: tag.tag_name,
|
|
||||||
body: 'patch collision wiki',
|
|
||||||
created_by_user: member_user,
|
|
||||||
message: 'init')
|
|
||||||
|
|
||||||
patch "/tags/#{ tag.id }", params: { name: 'unknown' }
|
|
||||||
|
|
||||||
expect(response).to have_http_status(:unprocessable_entity)
|
|
||||||
expect(json.fetch('errors')).to include(
|
|
||||||
'name' => ['その名前は既に使はれてゐます.']
|
|
||||||
)
|
|
||||||
|
|
||||||
expect(tag.reload.name).to eq('spec_tag')
|
|
||||||
expect(tag.tag_name.aliases.map(&:name)).to contain_exactly('unko')
|
|
||||||
expect(wiki_page.reload.tag_name).to eq(tag.tag_name)
|
|
||||||
end
|
|
||||||
|
|
||||||
it 'system tag の name は変更できない' do
|
|
||||||
system_tag = Tag.bot
|
|
||||||
|
|
||||||
patch "/tags/#{ system_tag.id }", params: { name: 'patch_system_tag_renamed' }
|
|
||||||
|
|
||||||
expect(response).to have_http_status(:unprocessable_entity)
|
|
||||||
expect(system_tag.reload.name).to eq('bot操作')
|
|
||||||
end
|
|
||||||
end
|
end
|
||||||
end
|
end
|
||||||
|
|
||||||
@@ -797,17 +745,7 @@ RSpec.describe 'Tags API', type: :request do
|
|||||||
)
|
)
|
||||||
TagImplication.create!(tag: first, parent_tag: root_material)
|
TagImplication.create!(tag: first, parent_tag: root_material)
|
||||||
TagImplication.create!(tag: second, parent_tag: first)
|
TagImplication.create!(tag: second, parent_tag: first)
|
||||||
now = Time.current
|
TagImplication.create!(tag: first, parent_tag: second)
|
||||||
TagImplication.insert_all!(
|
|
||||||
[
|
|
||||||
{
|
|
||||||
tag_id: first.id,
|
|
||||||
parent_tag_id: second.id,
|
|
||||||
created_at: now,
|
|
||||||
updated_at: now
|
|
||||||
}
|
|
||||||
]
|
|
||||||
)
|
|
||||||
|
|
||||||
get '/tags/with-depth', params: { parent: root_material.id }
|
get '/tags/with-depth', params: { parent: root_material.id }
|
||||||
|
|
||||||
@@ -1091,42 +1029,6 @@ RSpec.describe 'Tags API', type: :request do
|
|||||||
expect(versions.second.created_by_user_id).to eq(member_user.id)
|
expect(versions.second.created_by_user_id).to eq(member_user.id)
|
||||||
end
|
end
|
||||||
|
|
||||||
it '同じ tag の旧 alias へ戻しても rename できる' do
|
|
||||||
put "/tags/#{ tag.id }", params: {
|
|
||||||
name: 'put_roundtrip_b',
|
|
||||||
category: 'general',
|
|
||||||
aliases: 'unko',
|
|
||||||
parent_tags: '',
|
|
||||||
deprecated: '0',
|
|
||||||
}
|
|
||||||
|
|
||||||
expect(response).to have_http_status(:ok)
|
|
||||||
|
|
||||||
put "/tags/#{ tag.id }", params: {
|
|
||||||
name: 'spec_tag',
|
|
||||||
category: 'general',
|
|
||||||
aliases: 'unko',
|
|
||||||
parent_tags: '',
|
|
||||||
deprecated: '0',
|
|
||||||
}
|
|
||||||
|
|
||||||
expect(response).to have_http_status(:ok)
|
|
||||||
|
|
||||||
tag.reload
|
|
||||||
|
|
||||||
expect(tag.name).to eq('spec_tag')
|
|
||||||
expect(TagName.find_by!(name: 'put_roundtrip_b').canonical).to eq(tag.tag_name)
|
|
||||||
expect(tag.tag_name.aliases.map(&:name)).to contain_exactly('put_roundtrip_b', 'unko')
|
|
||||||
expect(tag.tag_name.aliases.map(&:name)).not_to include('spec_tag')
|
|
||||||
expect(alias_tn.reload.canonical).to eq(tag.tag_name)
|
|
||||||
|
|
||||||
version = tag.tag_versions.order(:version_no).last
|
|
||||||
|
|
||||||
expect(version.event_type).to eq('update')
|
|
||||||
expect(version.name).to eq('spec_tag')
|
|
||||||
expect(version.aliases.split).to contain_exactly('put_roundtrip_b', 'unko')
|
|
||||||
end
|
|
||||||
|
|
||||||
it 'parent tag の snapshot も作成する' do
|
it 'parent tag の snapshot も作成する' do
|
||||||
old_parent = Tag.create!(
|
old_parent = Tag.create!(
|
||||||
tag_name: TagName.create!(name: 'put_snapshot_old_parent'),
|
tag_name: TagName.create!(name: 'put_snapshot_old_parent'),
|
||||||
@@ -1251,48 +1153,6 @@ RSpec.describe 'Tags API', type: :request do
|
|||||||
)
|
)
|
||||||
end
|
end
|
||||||
|
|
||||||
it 'wiki を持つ tag を旧 alias へ戻しても wiki を新 canonical へ移す' do
|
|
||||||
wiki_page =
|
|
||||||
Wiki::Commit.create_content!(
|
|
||||||
tag_name: tag.tag_name,
|
|
||||||
body: 'wiki body before',
|
|
||||||
created_by_user: member_user,
|
|
||||||
message: 'init')
|
|
||||||
|
|
||||||
expect {
|
|
||||||
put "/tags/#{ tag.id }", params: {
|
|
||||||
name: 'put_wiki_roundtrip_b',
|
|
||||||
category: 'general',
|
|
||||||
aliases: 'unko',
|
|
||||||
parent_tags: '',
|
|
||||||
deprecated: '0',
|
|
||||||
}
|
|
||||||
|
|
||||||
put "/tags/#{ tag.id }", params: {
|
|
||||||
name: 'spec_tag',
|
|
||||||
category: 'general',
|
|
||||||
aliases: 'unko',
|
|
||||||
parent_tags: '',
|
|
||||||
deprecated: '0',
|
|
||||||
}
|
|
||||||
}
|
|
||||||
.to change(TagVersion, :count).by(3)
|
|
||||||
.and change(WikiVersion, :count).by(2)
|
|
||||||
|
|
||||||
expect(response).to have_http_status(:ok)
|
|
||||||
|
|
||||||
tag.reload
|
|
||||||
|
|
||||||
expect(wiki_page.reload.tag_name).to eq(tag.tag_name)
|
|
||||||
expect(TagName.find_by!(name: 'put_wiki_roundtrip_b').wiki_page).to be_nil
|
|
||||||
expect(TagName.find_by!(name: 'put_wiki_roundtrip_b').canonical).to eq(tag.tag_name)
|
|
||||||
|
|
||||||
versions = wiki_page.wiki_versions.order(:version_no).last(2)
|
|
||||||
|
|
||||||
expect(versions.map(&:event_type)).to eq(['update', 'update'])
|
|
||||||
expect(versions.map(&:title)).to eq(['put_wiki_roundtrip_b', 'spec_tag'])
|
|
||||||
end
|
|
||||||
|
|
||||||
it '別名を他 tag から奪った場合、奪はれた側の tag version も作成する' do
|
it '別名を他 tag から奪った場合、奪はれた側の tag version も作成する' do
|
||||||
old_owner = Tag.create!(
|
old_owner = Tag.create!(
|
||||||
tag_name: TagName.create!(name: 'put_alias_old_owner'),
|
tag_name: TagName.create!(name: 'put_alias_old_owner'),
|
||||||
@@ -1331,49 +1191,9 @@ RSpec.describe 'Tags API', type: :request do
|
|||||||
expect(old_owner_versions.second.aliases.split).not_to include('put_stolen_alias')
|
expect(old_owner_versions.second.aliases.split).not_to include('put_stolen_alias')
|
||||||
end
|
end
|
||||||
|
|
||||||
it '別 tag の alias 名を rename で奪へる' do
|
|
||||||
old_owner = Tag.create!(
|
|
||||||
tag_name: TagName.create!(name: 'put_alias_collision_owner'),
|
|
||||||
category: :general
|
|
||||||
)
|
|
||||||
stolen_alias = TagName.create!(
|
|
||||||
name: 'put_alias_collision_name',
|
|
||||||
canonical: old_owner.tag_name
|
|
||||||
)
|
|
||||||
wiki_page =
|
|
||||||
Wiki::Commit.create_content!(
|
|
||||||
tag_name: tag.tag_name,
|
|
||||||
body: 'put collision wiki',
|
|
||||||
created_by_user: member_user,
|
|
||||||
message: 'init')
|
|
||||||
|
|
||||||
put "/tags/#{ tag.id }", params: {
|
|
||||||
name: 'put_alias_collision_name',
|
|
||||||
category: 'general',
|
|
||||||
aliases: 'unko',
|
|
||||||
parent_tags: '',
|
|
||||||
deprecated: '0',
|
|
||||||
}
|
|
||||||
|
|
||||||
expect(response).to have_http_status(:ok)
|
|
||||||
|
|
||||||
tag.reload
|
|
||||||
old_owner.reload
|
|
||||||
stolen_alias.reload
|
|
||||||
|
|
||||||
expect(tag.name).to eq('put_alias_collision_name')
|
|
||||||
expect(stolen_alias.canonical_id).to be_nil
|
|
||||||
expect(TagName.find_by!(name: 'spec_tag').canonical).to eq(tag.tag_name)
|
|
||||||
expect(old_owner.tag_name.aliases.map(&:name)).not_to include('put_alias_collision_name')
|
|
||||||
|
|
||||||
old_owner_versions = old_owner.tag_versions.order(:version_no)
|
|
||||||
|
|
||||||
expect(old_owner_versions.last.event_type).to eq('update')
|
|
||||||
expect(old_owner_versions.last.aliases.split).not_to include('put_alias_collision_name')
|
|
||||||
expect(wiki_page.reload.tag_name).to eq(tag.tag_name)
|
|
||||||
end
|
|
||||||
|
|
||||||
it 'parent_tags に指定すると循環する tag は 422 にする' do
|
it 'parent_tags に指定すると循環する tag は 422 にする' do
|
||||||
|
pending '#332 で対応予定'
|
||||||
|
|
||||||
child = Tag.create!(
|
child = Tag.create!(
|
||||||
tag_name: TagName.create!(name: 'put_cycle_child'),
|
tag_name: TagName.create!(name: 'put_cycle_child'),
|
||||||
category: :general
|
category: :general
|
||||||
|
|||||||
@@ -1,98 +0,0 @@
|
|||||||
require 'rails_helper'
|
|
||||||
|
|
||||||
RSpec.describe 'user settings', type: :request do
|
|
||||||
describe 'GET /users/settings' do
|
|
||||||
it 'requires a current user' do
|
|
||||||
sign_out
|
|
||||||
|
|
||||||
get '/users/settings'
|
|
||||||
|
|
||||||
expect(response).to have_http_status(:unauthorized)
|
|
||||||
end
|
|
||||||
|
|
||||||
it 'returns defaults and creates a missing settings row' do
|
|
||||||
user = create(:user)
|
|
||||||
sign_in_as(user)
|
|
||||||
|
|
||||||
expect {
|
|
||||||
get '/users/settings'
|
|
||||||
}.to change(Setting, :count).by(1)
|
|
||||||
|
|
||||||
expect(response).to have_http_status(:ok)
|
|
||||||
expect(json).to eq(
|
|
||||||
'theme' => 'system',
|
|
||||||
'auto_fetch_title' => 'manual',
|
|
||||||
'auto_fetch_thumbnail' => 'manual',
|
|
||||||
'wiki_editor_mode' => 'split')
|
|
||||||
expect(user.reload.setting).to be_present
|
|
||||||
end
|
|
||||||
end
|
|
||||||
|
|
||||||
describe 'PATCH /users/settings' do
|
|
||||||
it 'requires a current user' do
|
|
||||||
sign_out
|
|
||||||
|
|
||||||
patch '/users/settings', params: { theme: 'dark' }, as: :json
|
|
||||||
|
|
||||||
expect(response).to have_http_status(:unauthorized)
|
|
||||||
end
|
|
||||||
|
|
||||||
it 'updates multiple typed settings at once' do
|
|
||||||
user = create(:user)
|
|
||||||
sign_in_as(user)
|
|
||||||
|
|
||||||
patch '/users/settings',
|
|
||||||
params: {
|
|
||||||
theme: 'dark',
|
|
||||||
auto_fetch_title: 'auto',
|
|
||||||
auto_fetch_thumbnail: 'off',
|
|
||||||
wiki_editor_mode: 'preview' },
|
|
||||||
as: :json
|
|
||||||
|
|
||||||
expect(response).to have_http_status(:ok)
|
|
||||||
expect(json).to include(
|
|
||||||
'theme' => 'dark',
|
|
||||||
'auto_fetch_title' => 'auto',
|
|
||||||
'auto_fetch_thumbnail' => 'off',
|
|
||||||
'wiki_editor_mode' => 'preview')
|
|
||||||
expect(user.reload.setting).to have_attributes(
|
|
||||||
theme: 'dark',
|
|
||||||
auto_fetch_title: 'auto',
|
|
||||||
auto_fetch_thumbnail: 'off',
|
|
||||||
wiki_editor_mode: 'preview')
|
|
||||||
end
|
|
||||||
|
|
||||||
it 'does not treat Rails parameter wrapping as an editable setting' do
|
|
||||||
user = create(:user)
|
|
||||||
sign_in_as(user)
|
|
||||||
|
|
||||||
patch '/users/settings',
|
|
||||||
params: { theme: 'light', user_setting: { theme: 'dark' } },
|
|
||||||
as: :json
|
|
||||||
|
|
||||||
expect(response).to have_http_status(:ok)
|
|
||||||
expect(user.reload.setting.theme).to eq('light')
|
|
||||||
end
|
|
||||||
|
|
||||||
it 'returns validation_error for type mismatches' do
|
|
||||||
sign_in_as(create(:user))
|
|
||||||
|
|
||||||
patch '/users/settings', params: { theme: 1 }, as: :json
|
|
||||||
|
|
||||||
expect(response).to have_http_status(:unprocessable_entity)
|
|
||||||
expect(json).to include('type' => 'validation_error')
|
|
||||||
expect(json.fetch('errors')).to include(
|
|
||||||
'theme' => ['値の型が不正です.'])
|
|
||||||
end
|
|
||||||
|
|
||||||
it 'returns validation_error for invalid enum values' do
|
|
||||||
sign_in_as(create(:user))
|
|
||||||
|
|
||||||
patch '/users/settings', params: { theme: 'neon' }, as: :json
|
|
||||||
|
|
||||||
expect(response).to have_http_status(:unprocessable_entity)
|
|
||||||
expect(json).to include('type' => 'validation_error')
|
|
||||||
expect(json.fetch('errors').fetch('theme')).to be_present
|
|
||||||
end
|
|
||||||
end
|
|
||||||
end
|
|
||||||
@@ -1,104 +0,0 @@
|
|||||||
require 'rails_helper'
|
|
||||||
|
|
||||||
RSpec.describe 'user theme slots', type: :request do
|
|
||||||
describe 'GET /users/theme_slots' do
|
|
||||||
it 'requires a current user' do
|
|
||||||
sign_out
|
|
||||||
|
|
||||||
get '/users/theme_slots'
|
|
||||||
|
|
||||||
expect(response).to have_http_status(:unauthorized)
|
|
||||||
end
|
|
||||||
|
|
||||||
it 'returns only the current user slots in stable order' do
|
|
||||||
user = create(:user)
|
|
||||||
other_user = create(:user)
|
|
||||||
sign_in_as(user)
|
|
||||||
UserThemeSlot.create!(user:, base_theme: 'light', slot_no: 2, tokens: { 'a' => 1 })
|
|
||||||
UserThemeSlot.create!(user:, base_theme: 'dark', slot_no: 1, tokens: { 'b' => 2 })
|
|
||||||
UserThemeSlot.create!(user: other_user,
|
|
||||||
base_theme: 'dark',
|
|
||||||
slot_no: 3,
|
|
||||||
tokens: { 'c' => 3 })
|
|
||||||
|
|
||||||
get '/users/theme_slots'
|
|
||||||
|
|
||||||
expect(response).to have_http_status(:ok)
|
|
||||||
expect(json.map { |slot| [slot.fetch('base_theme'), slot.fetch('slot_no')] }).to eq(
|
|
||||||
[['dark', 1], ['light', 2]])
|
|
||||||
expect(json.map { |slot| slot.fetch('tokens') }).to eq(
|
|
||||||
[{ 'b' => 2 }, { 'a' => 1 }])
|
|
||||||
end
|
|
||||||
end
|
|
||||||
|
|
||||||
describe 'PUT /users/theme_slots/:base_theme/:slot_no' do
|
|
||||||
it 'requires a current user' do
|
|
||||||
sign_out
|
|
||||||
|
|
||||||
put '/users/theme_slots/light/1',
|
|
||||||
params: { tokens: { background: '0 0% 100%' } },
|
|
||||||
as: :json
|
|
||||||
|
|
||||||
expect(response).to have_http_status(:unauthorized)
|
|
||||||
end
|
|
||||||
|
|
||||||
it 'creates a theme slot' do
|
|
||||||
user = create(:user)
|
|
||||||
sign_in_as(user)
|
|
||||||
|
|
||||||
expect {
|
|
||||||
put '/users/theme_slots/light/1',
|
|
||||||
params: { tokens: { background: '0 0% 100%', tagColours: { nico: '#ffffff' } } },
|
|
||||||
as: :json
|
|
||||||
}.to change(UserThemeSlot, :count).by(1)
|
|
||||||
|
|
||||||
expect(response).to have_http_status(:ok)
|
|
||||||
expect(json).to include('base_theme' => 'light', 'slot_no' => 1)
|
|
||||||
expect(json.fetch('tokens')).to include(
|
|
||||||
'background' => '0 0% 100%',
|
|
||||||
'tagColours' => { 'nico' => '#ffffff' })
|
|
||||||
expect(user.theme_slots.find_by!(base_theme: 'light', slot_no: 1).tokens).to include(
|
|
||||||
'background' => '0 0% 100%')
|
|
||||||
end
|
|
||||||
|
|
||||||
it 'updates an existing theme slot instead of creating a duplicate' do
|
|
||||||
user = create(:user)
|
|
||||||
sign_in_as(user)
|
|
||||||
UserThemeSlot.create!(user:, base_theme: 'dark', slot_no: 2, tokens: { 'old' => true })
|
|
||||||
|
|
||||||
expect {
|
|
||||||
put '/users/theme_slots/dark/2',
|
|
||||||
params: { tokens: { background: '222.2 84% 4.9%' } },
|
|
||||||
as: :json
|
|
||||||
}.not_to change(UserThemeSlot, :count)
|
|
||||||
|
|
||||||
expect(response).to have_http_status(:ok)
|
|
||||||
expect(user.theme_slots.find_by!(base_theme: 'dark', slot_no: 2).tokens).to eq(
|
|
||||||
'background' => '222.2 84% 4.9%')
|
|
||||||
end
|
|
||||||
|
|
||||||
it 'rejects invalid path parameters and token shapes' do
|
|
||||||
sign_in_as(create(:user))
|
|
||||||
|
|
||||||
put '/users/theme_slots/system/4',
|
|
||||||
params: { tokens: 'not-object' },
|
|
||||||
as: :json
|
|
||||||
|
|
||||||
expect(response).to have_http_status(:unprocessable_entity)
|
|
||||||
expect(json.fetch('errors')).to include(
|
|
||||||
'base_theme' => ['値が不正です.'])
|
|
||||||
end
|
|
||||||
|
|
||||||
it 'rejects non-object tokens for valid slots' do
|
|
||||||
sign_in_as(create(:user))
|
|
||||||
|
|
||||||
put '/users/theme_slots/light/1',
|
|
||||||
params: { tokens: 'not-object' },
|
|
||||||
as: :json
|
|
||||||
|
|
||||||
expect(response).to have_http_status(:unprocessable_entity)
|
|
||||||
expect(json.fetch('errors')).to include(
|
|
||||||
'tokens' => ['JSON object で指定してください.'])
|
|
||||||
end
|
|
||||||
end
|
|
||||||
end
|
|
||||||
@@ -1,23 +0,0 @@
|
|||||||
require 'rails_helper'
|
|
||||||
|
|
||||||
RSpec.describe MaterialFileSha256 do
|
|
||||||
describe '.metadata_sha256' do
|
|
||||||
it 'reads sha256 from Hash metadata' do
|
|
||||||
blob = instance_double(ActiveStorage::Blob, metadata: { 'sha256' => 'abc123' })
|
|
||||||
|
|
||||||
expect(described_class.metadata_sha256(blob)).to eq('abc123')
|
|
||||||
end
|
|
||||||
|
|
||||||
it 'reads sha256 from JSON string metadata' do
|
|
||||||
blob = instance_double(ActiveStorage::Blob, metadata: '{"sha256":"abc123"}')
|
|
||||||
|
|
||||||
expect(described_class.metadata_sha256(blob)).to eq('abc123')
|
|
||||||
end
|
|
||||||
|
|
||||||
it 'returns nil for invalid metadata' do
|
|
||||||
blob = instance_double(ActiveStorage::Blob, metadata: '{')
|
|
||||||
|
|
||||||
expect(described_class.metadata_sha256(blob)).to be_nil
|
|
||||||
end
|
|
||||||
end
|
|
||||||
end
|
|
||||||
@@ -1,128 +0,0 @@
|
|||||||
require 'rails_helper'
|
|
||||||
|
|
||||||
RSpec.describe MaterialSyncImporter do
|
|
||||||
let(:user) { create(:user, :member) }
|
|
||||||
let(:tag) { Tag.create!(tag_name: TagName.create!(name: 'sync_tag'), category: :material) }
|
|
||||||
|
|
||||||
def tempfile_for body
|
|
||||||
Tempfile.new(['material-sync-importer', '.png']).tap do |file|
|
|
||||||
file.binmode
|
|
||||||
file.write(body)
|
|
||||||
file.rewind
|
|
||||||
end
|
|
||||||
end
|
|
||||||
|
|
||||||
it 'creates an untagged material from a sync candidate with file_downloader' do
|
|
||||||
result =
|
|
||||||
described_class.import!(
|
|
||||||
source_kind: 'google_drive_file',
|
|
||||||
source_path: '素材/a.png',
|
|
||||||
source_file_id: 'drive-file-a',
|
|
||||||
normalized_source_key: 'google_drive_file:drive-file-a',
|
|
||||||
filename: 'a.png',
|
|
||||||
content_type: 'image/png',
|
|
||||||
file_sha256: Digest::SHA256.hexdigest('sync-body'),
|
|
||||||
file_downloader: -> { tempfile_for('sync-body') },
|
|
||||||
export_path: '素材/a.png',
|
|
||||||
tag: nil,
|
|
||||||
url: nil,
|
|
||||||
created_by_user: user,
|
|
||||||
updated_by_user: user)
|
|
||||||
|
|
||||||
material = result.material.reload
|
|
||||||
expect(result.action).to eq(:imported)
|
|
||||||
expect(material.tag).to be_nil
|
|
||||||
expect(material.url).to be_nil
|
|
||||||
expect(material.file).to be_attached
|
|
||||||
expect(material.file.blob.filename.to_s).to eq('a.png')
|
|
||||||
expect(material.material_export_items.first.export_path).to eq('素材/a.png')
|
|
||||||
end
|
|
||||||
|
|
||||||
it 'does not clear a human-assigned tag or URL on existing Drive sync' do
|
|
||||||
material =
|
|
||||||
Material.create!(tag:, url: 'https://example.com/human',
|
|
||||||
source_kind: 'google_drive_file',
|
|
||||||
source_path: '素材/a.png',
|
|
||||||
source_file_id: 'drive-file-a',
|
|
||||||
created_by_user: user,
|
|
||||||
updated_by_user: user)
|
|
||||||
material.file.attach(
|
|
||||||
io: StringIO.new('same-body'),
|
|
||||||
filename: 'a.png',
|
|
||||||
content_type: 'image/png')
|
|
||||||
material.file.blob.metadata['sha256'] = Digest::SHA256.hexdigest('same-body')
|
|
||||||
material.file.blob.save!
|
|
||||||
MaterialExportItem.create!(material:,
|
|
||||||
profile: 'legacy_drive',
|
|
||||||
export_path: '素材/a.png',
|
|
||||||
created_by_user: user)
|
|
||||||
|
|
||||||
result =
|
|
||||||
described_class.import!(
|
|
||||||
source_kind: 'google_drive_file',
|
|
||||||
source_path: '素材/a.png',
|
|
||||||
source_file_id: 'drive-file-a',
|
|
||||||
filename: 'a.png',
|
|
||||||
content_type: 'image/png',
|
|
||||||
file_sha256: Digest::SHA256.hexdigest('same-body'),
|
|
||||||
export_path: '素材/a.png',
|
|
||||||
tag: nil,
|
|
||||||
url: nil,
|
|
||||||
updated_by_user: user)
|
|
||||||
|
|
||||||
expect(result.action).to eq(:unchanged)
|
|
||||||
expect(material.reload.tag).to eq(tag)
|
|
||||||
expect(material.url).to eq('https://example.com/human')
|
|
||||||
end
|
|
||||||
|
|
||||||
it 'suppresses a Google Drive candidate by relative path prefix' do
|
|
||||||
MaterialSyncSuppression.create!(source_kind: 'google_drive_path_prefix',
|
|
||||||
drive_path: '素材/危険',
|
|
||||||
reason: 'copyright_high_risk',
|
|
||||||
created_by_user: user)
|
|
||||||
|
|
||||||
result =
|
|
||||||
described_class.import!(
|
|
||||||
source_kind: 'google_drive_file',
|
|
||||||
source_path: '素材/危険/a.png',
|
|
||||||
source_file_id: 'drive-file-b',
|
|
||||||
filename: 'a.png',
|
|
||||||
content_type: 'image/png',
|
|
||||||
file_sha256: Digest::SHA256.hexdigest('blocked-body'),
|
|
||||||
file_downloader: -> { tempfile_for('blocked-body') },
|
|
||||||
export_path: '素材/危険/a.png',
|
|
||||||
updated_by_user: user)
|
|
||||||
|
|
||||||
expect(result.action).to eq(:suppressed)
|
|
||||||
expect(result.suppression.normalized_source_key)
|
|
||||||
.to eq('google_drive_path_prefix:素材/危険')
|
|
||||||
expect(Material.find_by(source_file_id: 'drive-file-b')).to be_nil
|
|
||||||
end
|
|
||||||
|
|
||||||
it 'rechecks import blocks after downloaded file sha256 is known' do
|
|
||||||
MaterialImportBlock.create!(match_kind: 'sha256',
|
|
||||||
sha256: Digest::SHA256.hexdigest('blocked-body'),
|
|
||||||
reason: 'copyright_high_risk',
|
|
||||||
created_by_user: user)
|
|
||||||
blob_count = ActiveStorage::Blob.count
|
|
||||||
|
|
||||||
expect do
|
|
||||||
result =
|
|
||||||
described_class.import!(
|
|
||||||
source_kind: 'google_drive_file',
|
|
||||||
source_path: '素材/blocked.png',
|
|
||||||
source_file_id: 'drive-file-blocked',
|
|
||||||
filename: 'blocked.png',
|
|
||||||
content_type: 'image/png',
|
|
||||||
file_downloader: -> { tempfile_for('blocked-body') },
|
|
||||||
export_path: '素材/blocked.png',
|
|
||||||
updated_by_user: user)
|
|
||||||
|
|
||||||
expect(result.action).to eq(:suppressed)
|
|
||||||
expect(result.suppression.reason).to eq('copyright_high_risk')
|
|
||||||
end.not_to change(Material, :count)
|
|
||||||
|
|
||||||
expect(ActiveStorage::Blob.count).to eq(blob_count)
|
|
||||||
expect(Material.find_by(source_file_id: 'drive-file-blocked')).to be_nil
|
|
||||||
end
|
|
||||||
end
|
|
||||||
@@ -1,89 +0,0 @@
|
|||||||
require 'rails_helper'
|
|
||||||
|
|
||||||
RSpec.describe MaterialSyncRunner do
|
|
||||||
let(:user) { create(:user, :member) }
|
|
||||||
let(:source) do
|
|
||||||
MaterialSyncSource.create!(
|
|
||||||
name: 'Drive source',
|
|
||||||
source_kind: 'google_drive_path',
|
|
||||||
source_file_id: 'folder-123',
|
|
||||||
profile: 'legacy_drive',
|
|
||||||
created_by_user: user,
|
|
||||||
updated_by_user: user)
|
|
||||||
end
|
|
||||||
let(:drive_client) { instance_double(GoogleDrive::ApiClient) }
|
|
||||||
|
|
||||||
describe '#sync!' do
|
|
||||||
it 'imports google drive path candidates as they are yielded' do
|
|
||||||
first_entry = { id: 'file-1',
|
|
||||||
name: 'a.png',
|
|
||||||
mime_type: 'image/png',
|
|
||||||
relative_path: '素材/a.png',
|
|
||||||
sha256_checksum: 'sha-a',
|
|
||||||
web_view_link: 'https://drive.google.com/file/d/file-1/view',
|
|
||||||
web_content_link: nil }
|
|
||||||
second_entry = { id: 'file-2',
|
|
||||||
name: 'b.png',
|
|
||||||
mime_type: 'image/png',
|
|
||||||
relative_path: '素材/b.png',
|
|
||||||
sha256_checksum: 'sha-b',
|
|
||||||
web_view_link: 'https://drive.google.com/file/d/file-2/view',
|
|
||||||
web_content_link: nil }
|
|
||||||
yielded = []
|
|
||||||
|
|
||||||
allow(GoogleDrive::ApiClient).to receive(:new).and_return(drive_client)
|
|
||||||
allow(drive_client).to receive(:each_material_file_under_folder) do |folder_id, &block|
|
|
||||||
expect(folder_id).to eq('folder-123')
|
|
||||||
block.call(first_entry)
|
|
||||||
expect(yielded).to eq(['素材/a.png'])
|
|
||||||
block.call(second_entry)
|
|
||||||
end
|
|
||||||
allow(MaterialSyncImporter).to receive(:import!) do |candidate|
|
|
||||||
yielded << candidate.fetch(:source_path)
|
|
||||||
instance_double(MaterialSyncImporter::Result,
|
|
||||||
action: yielded.last == '素材/a.png' ? :imported : :updated)
|
|
||||||
end
|
|
||||||
|
|
||||||
result = described_class.new(source).sync!
|
|
||||||
|
|
||||||
expect(yielded).to eq(['素材/a.png', '素材/b.png'])
|
|
||||||
expect(result.imported).to eq(1)
|
|
||||||
expect(result.updated).to eq(1)
|
|
||||||
expect(result.unchanged).to eq(0)
|
|
||||||
expect(source.reload.last_synced_at).to be_present
|
|
||||||
end
|
|
||||||
|
|
||||||
it 'logs google drive path progress at first item and summary' do
|
|
||||||
entry = { id: 'file-1',
|
|
||||||
name: 'a.png',
|
|
||||||
mime_type: 'image/png',
|
|
||||||
relative_path: '素材/a.png',
|
|
||||||
sha256_checksum: 'sha-a',
|
|
||||||
web_view_link: 'https://drive.google.com/file/d/file-1/view',
|
|
||||||
web_content_link: nil }
|
|
||||||
logged = []
|
|
||||||
|
|
||||||
allow(GoogleDrive::ApiClient).to receive(:new).and_return(drive_client)
|
|
||||||
allow(drive_client).to receive(:each_material_file_under_folder) do |_folder_id, &block|
|
|
||||||
block.call(entry)
|
|
||||||
end
|
|
||||||
allow(MaterialSyncImporter).to receive(:import!)
|
|
||||||
.and_return(instance_double(MaterialSyncImporter::Result, action: :imported))
|
|
||||||
allow(Rails.logger).to receive(:info) { |message| logged << JSON.parse(message) }
|
|
||||||
|
|
||||||
described_class.new(source).sync!
|
|
||||||
|
|
||||||
progress_logs = logged.select { |row| row['folder_id'] == 'folder-123' }
|
|
||||||
expect(progress_logs.map { |row| row['progress'] }).to eq(['scan', 'summary'])
|
|
||||||
expect(progress_logs.last).to include(
|
|
||||||
'material_sync_source_id' => source.id,
|
|
||||||
'material_sync_source_name' => 'Drive source',
|
|
||||||
'scanned_count' => 1,
|
|
||||||
'imported' => 1,
|
|
||||||
'updated' => 0,
|
|
||||||
'unchanged' => 0,
|
|
||||||
'suppressed' => 0,
|
|
||||||
'failed' => 0)
|
|
||||||
end
|
|
||||||
end
|
|
||||||
end
|
|
||||||
@@ -1,187 +0,0 @@
|
|||||||
require 'rails_helper'
|
|
||||||
|
|
||||||
RSpec.describe PostImportPreviewer do
|
|
||||||
before do
|
|
||||||
allow(Preview::UrlSafety).to receive(:validate) do |url|
|
|
||||||
[URI.parse(url), ['8.8.8.8']]
|
|
||||||
end
|
|
||||||
end
|
|
||||||
|
|
||||||
def row(source_row:, url:, attributes: { }, provenance: { }, **values)
|
|
||||||
{
|
|
||||||
source_row:,
|
|
||||||
url:,
|
|
||||||
attributes:,
|
|
||||||
provenance:
|
|
||||||
}.merge(values)
|
|
||||||
end
|
|
||||||
|
|
||||||
describe '#preview_rows' do
|
|
||||||
it 'marks an existing post for skipping without fetching metadata' do
|
|
||||||
existing = create(:post, url: 'https://example.com/existing')
|
|
||||||
expect(PostMetadataFetcher).not_to receive(:fetch)
|
|
||||||
|
|
||||||
result = described_class.new.preview_rows(rows: [
|
|
||||||
row(source_row: 1, url: 'https://EXAMPLE.com/existing/')
|
|
||||||
]).first
|
|
||||||
|
|
||||||
expect(result).to include(
|
|
||||||
url: 'https://example.com/existing',
|
|
||||||
skip_reason: 'existing',
|
|
||||||
existing_post_id: existing.id,
|
|
||||||
validation_errors: {}
|
|
||||||
)
|
|
||||||
expect(result.fetch(:field_warnings)).to eq({})
|
|
||||||
end
|
|
||||||
|
|
||||||
it 'reports list duplicates as URL errors instead of existing skips' do
|
|
||||||
create(:post, url: 'https://example.com/duplicate')
|
|
||||||
expect(PostMetadataFetcher).not_to receive(:fetch)
|
|
||||||
|
|
||||||
results = described_class.new.preview_rows(rows: [
|
|
||||||
row(source_row: 1, url: 'https://example.com/duplicate'),
|
|
||||||
row(source_row: 2, url: 'https://EXAMPLE.com/duplicate/')
|
|
||||||
])
|
|
||||||
|
|
||||||
expect(results.map { _1[:validation_errors] }).to all(
|
|
||||||
include(url: ['URL が重複しています.'])
|
|
||||||
)
|
|
||||||
expect(results).to all(include(skip_reason: nil, existing_post_id: nil))
|
|
||||||
end
|
|
||||||
|
|
||||||
it 'does not fetch metadata for an unsafe URL' do
|
|
||||||
allow(Preview::UrlSafety).to receive(:validate)
|
|
||||||
.and_raise(Preview::UrlSafety::UnsafeUrl, '安全でない接続先は使用できません.')
|
|
||||||
expect(PostMetadataFetcher).not_to receive(:fetch)
|
|
||||||
|
|
||||||
result = described_class.new.preview_rows(rows: [
|
|
||||||
row(source_row: 1, url: 'https://unsafe.example/post')
|
|
||||||
]).first
|
|
||||||
|
|
||||||
expect(result.fetch(:validation_errors)).to include(
|
|
||||||
url: ['安全でない接続先は使用できません.']
|
|
||||||
)
|
|
||||||
end
|
|
||||||
|
|
||||||
it 'applies metadata to automatic fields and recognises metadata tags' do
|
|
||||||
Tag.create!(name: 'known-tag', category: :general)
|
|
||||||
allow(PostMetadataFetcher).to receive(:fetch).and_return(
|
|
||||||
title: 'metadata title',
|
|
||||||
thumbnail_base: 'https://example.com/thumb.jpg',
|
|
||||||
duration: 2_000,
|
|
||||||
tags: 'known-tag'
|
|
||||||
)
|
|
||||||
|
|
||||||
result = described_class.new.preview_rows(rows: [
|
|
||||||
row(source_row: 1, url: 'https://example.com/new')
|
|
||||||
]).first
|
|
||||||
|
|
||||||
expect(result.fetch(:attributes)).to include(
|
|
||||||
'title' => 'metadata title',
|
|
||||||
'thumbnail_base' => 'https://example.com/thumb.jpg',
|
|
||||||
'duration' => 2_000,
|
|
||||||
'tags' => 'known-tag'
|
|
||||||
)
|
|
||||||
expect(result.fetch(:field_warnings)).not_to have_key('tags')
|
|
||||||
expect(PostMetadataFetcher).to have_received(:fetch).once
|
|
||||||
end
|
|
||||||
|
|
||||||
it 'preserves manual empty values when metadata is available' do
|
|
||||||
allow(PostMetadataFetcher).to receive(:fetch).and_return(
|
|
||||||
title: 'metadata title',
|
|
||||||
tags: 'metadata-tag'
|
|
||||||
)
|
|
||||||
|
|
||||||
result = described_class.new.preview_rows(rows: [
|
|
||||||
row(
|
|
||||||
source_row: 1,
|
|
||||||
url: 'https://example.com/manual',
|
|
||||||
attributes: { title: '', tags: '' },
|
|
||||||
provenance: { title: 'manual', tags: 'manual' },
|
|
||||||
tag_sources: { automatic: 'old-tag', manual: '' }
|
|
||||||
)
|
|
||||||
]).first
|
|
||||||
|
|
||||||
expect(result.fetch(:attributes)).to include('title' => '', 'tags' => '')
|
|
||||||
expect(result.fetch(:tag_sources)).to include('manual' => '')
|
|
||||||
end
|
|
||||||
|
|
||||||
it 'clears warnings derived from an old metadata URL' do
|
|
||||||
allow(PostMetadataFetcher).to receive(:fetch).and_return(
|
|
||||||
title: 'new title',
|
|
||||||
thumbnail_base: 'https://example.com/new-thumb.jpg',
|
|
||||||
tags: ''
|
|
||||||
)
|
|
||||||
|
|
||||||
result = described_class.new.preview_rows(rows: [
|
|
||||||
row(
|
|
||||||
source_row: 1,
|
|
||||||
url: 'https://example.com/new-url',
|
|
||||||
attributes: { title: 'old title', tags: 'old-tag' },
|
|
||||||
provenance: { title: 'automatic', tags: 'automatic' },
|
|
||||||
tag_sources: { automatic: 'old-tag', manual: '' },
|
|
||||||
metadata_url: 'https://example.com/old-url',
|
|
||||||
field_warnings: { tags: ['old tag warning'], url: ['old fetch warning'] },
|
|
||||||
base_warnings: ['old base warning']
|
|
||||||
)
|
|
||||||
]).first
|
|
||||||
|
|
||||||
expect(result.fetch(:attributes)).to include('title' => 'new title', 'tags' => '')
|
|
||||||
expect(result.fetch(:base_warnings)).to eq([])
|
|
||||||
expect(result.fetch(:field_warnings).values.flatten)
|
|
||||||
.not_to include('old tag warning', 'old fetch warning')
|
|
||||||
end
|
|
||||||
|
|
||||||
it 'isolates an unexpected metadata failure to the affected URL' do
|
|
||||||
allow(PostMetadataFetcher).to receive(:fetch) do |url|
|
|
||||||
raise 'fetch failure' if url.include?('failure')
|
|
||||||
|
|
||||||
{ title: 'successful title', tags: '' }
|
|
||||||
end
|
|
||||||
allow(Rails.logger).to receive(:error)
|
|
||||||
|
|
||||||
results = described_class.new.preview_rows(rows: [
|
|
||||||
row(source_row: 1, url: 'https://example.com/failure'),
|
|
||||||
row(source_row: 2, url: 'https://example.com/success')
|
|
||||||
])
|
|
||||||
|
|
||||||
expect(results[0].fetch(:field_warnings)).to include(
|
|
||||||
'url' => [described_class::METADATA_FETCH_WARNING]
|
|
||||||
)
|
|
||||||
expect(results[1].fetch(:attributes)).to include('title' => 'successful title')
|
|
||||||
end
|
|
||||||
|
|
||||||
it 'turns an unsafe URL detected during metadata fetching into a URL error' do
|
|
||||||
allow(PostMetadataFetcher).to receive(:fetch)
|
|
||||||
.and_raise(Preview::UrlSafety::UnsafeUrl, '安全でない接続先です.')
|
|
||||||
allow(Rails.logger).to receive(:info)
|
|
||||||
|
|
||||||
result = described_class.new.preview_rows(rows: [
|
|
||||||
row(source_row: 1, url: 'https://example.com/redirects-to-private')
|
|
||||||
]).first
|
|
||||||
|
|
||||||
expect(result.fetch(:validation_errors)).to include(
|
|
||||||
url: ['安全でない接続先です.']
|
|
||||||
)
|
|
||||||
end
|
|
||||||
|
|
||||||
it 'validates parent IDs from the preloaded set' do
|
|
||||||
parent = create(:post)
|
|
||||||
|
|
||||||
results = described_class.new.preview_rows(
|
|
||||||
rows: [
|
|
||||||
row(source_row: 1, url: 'https://example.com/valid-parent',
|
|
||||||
attributes: { parent_post_ids: parent.id.to_s }),
|
|
||||||
row(source_row: 2, url: 'https://example.com/missing-parent',
|
|
||||||
attributes: { parent_post_ids: '999999' })
|
|
||||||
],
|
|
||||||
fetch_metadata: false
|
|
||||||
)
|
|
||||||
|
|
||||||
expect(results[0].fetch(:validation_errors)).not_to have_key(:parent_post_ids)
|
|
||||||
expect(results[1].fetch(:validation_errors)).to include(
|
|
||||||
parent_post_ids: ['存在しない親投稿 Id. があります.']
|
|
||||||
)
|
|
||||||
end
|
|
||||||
end
|
|
||||||
end
|
|
||||||
@@ -1,77 +0,0 @@
|
|||||||
require 'rails_helper'
|
|
||||||
|
|
||||||
RSpec.describe PostImportRowNormaliser do
|
|
||||||
def valid_row(overrides = { })
|
|
||||||
{
|
|
||||||
sourceRow: '1',
|
|
||||||
url: 'https://example.com/post',
|
|
||||||
metadataUrl: 'https://example.com/post',
|
|
||||||
attributes: { title: 'title', duration: 1_000 },
|
|
||||||
provenance: { url: 'manual', title: 'automatic' },
|
|
||||||
tagSources: { automatic: 'tag', manual: '' }
|
|
||||||
}.deep_merge(overrides)
|
|
||||||
end
|
|
||||||
|
|
||||||
describe '.normalise!' do
|
|
||||||
it 'normalises aliases and source rows into a permitted plain hash' do
|
|
||||||
row = ActionController::Parameters.new(valid_row)
|
|
||||||
|
|
||||||
expect(described_class.normalise!([row])).to eq([
|
|
||||||
{
|
|
||||||
'source_row' => 1,
|
|
||||||
'url' => 'https://example.com/post',
|
|
||||||
'metadata_url' => 'https://example.com/post',
|
|
||||||
'attributes' => { 'title' => 'title', 'duration' => 1_000 },
|
|
||||||
'provenance' => { 'url' => 'manual', 'title' => 'automatic' },
|
|
||||||
'tag_sources' => { 'automatic' => 'tag', 'manual' => '' }
|
|
||||||
}
|
|
||||||
])
|
|
||||||
end
|
|
||||||
|
|
||||||
it 'normalises source rows before checking duplicates' do
|
|
||||||
rows = [valid_row, valid_row(sourceRow: 1, url: 'https://example.com/other')]
|
|
||||||
|
|
||||||
expect { described_class.normalise!(rows) }
|
|
||||||
.to raise_error(ArgumentError, '元行番号が重複しています.')
|
|
||||||
end
|
|
||||||
|
|
||||||
it 'rejects non-array batches and non-hash rows' do
|
|
||||||
expect { described_class.normalise!({}) }
|
|
||||||
.to raise_error(ArgumentError, '取込行の形式が不正です.')
|
|
||||||
expect { described_class.normalise!(['row']) }
|
|
||||||
.to raise_error(ArgumentError, '取込行の形式が不正です.')
|
|
||||||
end
|
|
||||||
|
|
||||||
it 'rejects unknown attributes and invalid field types' do
|
|
||||||
expect { described_class.normalise!([valid_row(attributes: { unknown: 'x' })]) }
|
|
||||||
.to raise_error(ArgumentError, '取込項目が不正です.')
|
|
||||||
expect { described_class.normalise!([valid_row(attributes: { title: [] })]) }
|
|
||||||
.to raise_error(ArgumentError, '取込項目の型が不正です.')
|
|
||||||
expect { described_class.normalise!([valid_row(attributes: { duration: false })]) }
|
|
||||||
.to raise_error(ArgumentError, '取込項目の型が不正です.')
|
|
||||||
end
|
|
||||||
|
|
||||||
it 'rejects unknown provenance and tag-source values' do
|
|
||||||
expect { described_class.normalise!([valid_row(provenance: { title: 'mapped' })]) }
|
|
||||||
.to raise_error(ArgumentError, '値の由来が不正です.')
|
|
||||||
expect { described_class.normalise!([valid_row(tagSources: { mapped: 'tag' })]) }
|
|
||||||
.to raise_error(ArgumentError, 'タグ由来の形式が不正です.')
|
|
||||||
end
|
|
||||||
|
|
||||||
it 'accepts warning fields only at the validation boundary' do
|
|
||||||
row = valid_row.merge(
|
|
||||||
fieldWarnings: { title: ['取得できませんでした.'] },
|
|
||||||
baseWarnings: ['確認してください.']
|
|
||||||
)
|
|
||||||
|
|
||||||
without_warnings = described_class.normalise!([row]).first
|
|
||||||
with_warnings = described_class.normalise!([row], allow_warning_fields: true).first
|
|
||||||
|
|
||||||
expect(without_warnings).not_to include('field_warnings', 'base_warnings')
|
|
||||||
expect(with_warnings).to include(
|
|
||||||
'field_warnings' => { 'title' => ['取得できませんでした.'] },
|
|
||||||
'base_warnings' => ['確認してください.']
|
|
||||||
)
|
|
||||||
end
|
|
||||||
end
|
|
||||||
end
|
|
||||||
変更されたファイルが多すぎるため,一部のファイルは表示されません さらに表示
新しい課題から参照
ユーザをブロックする