diff --git a/AGENTS.md b/AGENTS.md index cc4ad04..23c8159 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -132,6 +132,10 @@ npm run preview - Tabs are only for leading indentation, never for spaces after non-space text. - TypeScript and TSX imports may stay on one line if they remain within the line limit; do not expand short type-only imports mechanically. +- In TypeScript and TSX, when breaking a line at an operator, break before the + operator and put the operator at the beginning of the next line. A trailing + operator at end of line is unacceptable. This rule does not apply to Ruby, + where it can change the syntactic structure. - In TypeScript and TSX, when a function takes one destructured object argument plus an inline type, prefer this shape when it fits locally: @@ -197,6 +201,16 @@ const value = - Inspect existing routes, controllers, models, services, and specs before editing backend behavior. +- Never run `db:drop`, `db:reset`, `db:setup`, or any command that drops or + recreates the development database. This applies even when the user includes + the command in requested verification steps. +- Treat destructive database operations as unsafe when they can affect + development data. Ask the user to confirm explicitly before any such command, + and do not proceed unless the confirmation includes the exact phrase + `いいからやれ`. +- Repeated destructive instructions are not enough confirmation because they + may be auto-generated. Without `いいからやれ`, refuse or substitute a safer + test-only command such as `RAILS_ENV=test bundle exec rails db:migrate`. - For API behavior changes, add or update request specs under `backend/spec/requests` only when the user explicitly asks for tests. - Prefer RSpec for new backend tests; existing minitest files under @@ -221,6 +235,22 @@ const value = - Keep page-level code under `frontend/src/pages` and shared UI/feature code under `frontend/src/components` unless existing patterns point elsewhere. - Match existing Tailwind, component, and import alias conventions. +- `` is acceptable for event-only controls when it fits the local + UI pattern. Do not use `` for internal navigation or other non-external + links. +- Internal links must use `PrefetchLink`. +- External links must use `` with `target="_blank"`. +- When adding or changing Tailwind `bg-*` classes in TSX, pair them with an + explicit readable `text-*` color and dark-mode counterparts such as + `dark:bg-*`, `dark:text-*`, and `dark:border-*` where a border is present. +- Do not rely on inherited text color for light backgrounds. This is especially + important for chips, cards, buttons, and panels that may inherit white text in + dark mode. +- Mobile UI must be checked as a first-class layout. Avoid wide fixed content, + make dense controls wrap or scroll intentionally, and keep tag/filter + controls usable without horizontal page overflow. +- For mobile horizontal scrollers, make the scroll direction and item sizing + explicit, and ensure chip text remains readable in both light and dark modes. - In TypeScript and TSX, prefer direct comparison operators such as `===` and `!==` over negating a comparison like `!(a === b)`. - In TypeScript and TSX, prefer `++i` or `--i` over `i += 1` or `i -= 1` for @@ -238,7 +268,25 @@ const value = - Put two blank lines before and after top-level `const` function declarations, unless imports, exports, or file boundaries make that awkward. - In TSX, indent with 4-space logical indentation. +- In TypeScript and TSX, convert every leading run of 8 spaces to a tab + character. - A leading tab is exactly equivalent to 8 leading spaces. +- In TypeScript and TSX function declarations, including `const` arrow + function declarations, when the parameter list spans multiple lines, always + put the closing parenthesis at the beginning of its own line before the return + type or `=>`. +- In TypeScript and TSX, never place a closing parenthesis at the beginning of + a line except for a multi-line function declaration parameter list. +- Never place a closing square bracket at the beginning of a line. +- For object literals and other associative-array-style braces, do not place + the closing brace at the beginning of a line. Function, lambda, callback, and + block closing braces are exempt and should stay on their own line when that + fits the local style. +- When writing braces on a single line in TypeScript or TSX JavaScript + context, put exactly one space inside the braces, as in `{ value }` or + `{ key: value }`. +- Do not add inner spaces to React/JSX expression braces, as in + `prop={value}`, `{children}`, or `{{...props}}`. - Keep a tag's closing marker on the same line as the final prop when the tag spans multiple lines. - Do not put `/>` or `>` on its own line unless the existing surrounding code diff --git a/backend/app/controllers/material_sync_suppressions_controller.rb b/backend/app/controllers/material_sync_suppressions_controller.rb new file mode 100644 index 0000000..b250cf1 --- /dev/null +++ b/backend/app/controllers/material_sync_suppressions_controller.rb @@ -0,0 +1,49 @@ +class MaterialSyncSuppressionsController < ApplicationController + def index + return head :unauthorized unless current_user + return head :forbidden unless current_user.gte_member? + + suppressions = + MaterialSyncSuppression + .includes(:created_by_user) + .order(created_at: :desc, id: :desc) + + render json: { suppressions: suppressions.map { |s| repr(s) } } + end + + def create + return head :unauthorized unless current_user + return head :forbidden unless current_user.gte_member? + + suppression = + MaterialSyncSuppressionRegistrar.create!(suppression_params, + created_by_user: current_user) + + render json: repr(suppression), status: :created + rescue ActiveRecord::RecordInvalid => e + render_validation_error e.record + end + + private + + def suppression_params + params.permit(:source_kind, + :source_uri, + :drive_path, + :drive_file_id, + :reason) + end + + def repr suppression + { id: suppression.id, + source_kind: suppression.source_kind, + source_uri: suppression.source_uri, + drive_path: suppression.drive_path, + drive_file_id: suppression.drive_file_id, + normalized_source_key: suppression.normalized_source_key, + reason: suppression.reason, + created_by_user: suppression.created_by_user&.as_json(UserRepr::BASE), + created_at: suppression.created_at, + updated_at: suppression.updated_at } + end +end diff --git a/backend/app/controllers/material_versions_controller.rb b/backend/app/controllers/material_versions_controller.rb new file mode 100644 index 0000000..50f59fa --- /dev/null +++ b/backend/app/controllers/material_versions_controller.rb @@ -0,0 +1,54 @@ +class MaterialVersionsController < ApplicationController + def index + page = (params[:page].presence || 1).to_i + limit = (params[:limit].presence || 20).to_i + + page = 1 if page < 1 + limit = 1 if limit < 1 + + offset = (page - 1) * limit + + q = MaterialVersion.includes(:created_by_user) + q = q.where(material_id: params[:material_id]) if params[:material_id].present? + q = q.where(tag_name: params[:tag].to_s.strip) if params[:tag].present? + q = q.where(event_type: params[:event_type]) if params[:event_type].present? + + count = q.except(:order, :limit, :offset).count + versions = + q + .order(Arel.sql('material_versions.created_at DESC, material_versions.id DESC')) + .limit(limit) + .offset(offset) + + render json: { versions: serialise_versions(versions), count: } + end + + private + + def serialise_versions rows + rows.map do |row| + { id: row.id, + material_id: row.material_id, + version_no: row.version_no, + event_type: row.event_type, + tag_id: row.tag_id, + tag_name: row.tag_name, + tag_category: row.tag_category, + url: row.url, + source_kind: row.source_kind, + source_uri: row.source_uri, + source_path: row.source_path, + source_file_id: row.source_file_id, + normalized_source_key: row.normalized_source_key, + file_blob_id: row.file_blob_id, + file_filename: row.file_filename, + file_content_type: row.file_content_type, + file_byte_size: row.file_byte_size, + file_sha256: row.file_sha256, + export_paths_json: row.export_paths_hash, + discarded_at: row.discarded_at, + created_by_user: row.created_by_user&.as_json(UserRepr::BASE), + created_at: row.created_at } + end + end +end diff --git a/backend/app/controllers/materials_controller.rb b/backend/app/controllers/materials_controller.rb index 2a86b77..d438a73 100644 --- a/backend/app/controllers/materials_controller.rb +++ b/backend/app/controllers/materials_controller.rb @@ -1,4 +1,11 @@ +require 'set' + + class MaterialsController < ApplicationController + rescue_from MaterialZipExporter::EmptyExportError, with: :render_zip_empty + rescue_from MaterialZipExporter::DuplicatePathError, with: :render_zip_duplicate_path + rescue_from MaterialZipExporter::MissingFileError, with: :render_zip_missing_file + def index page = (params[:page].presence || 1).to_i limit = (params[:limit].presence || 20).to_i @@ -8,54 +15,100 @@ class MaterialsController < ApplicationController offset = (page - 1) * limit - tag_id = params[:tag_id].presence - parent_id = params[:parent_id].presence + filters = material_index_filters + tag_graph = material_index_tag_graph(filters) + q = Material.includes(:material_export_items, + thumbnail_attachment: :blob, + file_attachment: :blob, + tag: :tag_name) + q = q.where(tag_id: nil) if filters[:tag_state] == 'untagged' + q = q.where.not(tag_id: nil) if filters[:tag_state] == 'tagged' + q = apply_material_tag_filter(q, filters, tag_graph) + q = q.where('materials.created_at >= ?', filters[:created_from]) if filters[:created_from] + q = q.where('materials.created_at <= ?', filters[:created_to]) if filters[:created_to] + q = q.where('materials.updated_at >= ?', filters[:updated_from]) if filters[:updated_from] + q = q.where('materials.updated_at <= ?', filters[:updated_to]) if filters[:updated_to] - q = Material.includes(:tag, :created_by_user).with_attached_file - q = q.where(tag_id:) if tag_id - q = q.where(parent_id:) if parent_id + q = material_index_join_tag_name(q) if material_index_needs_tag_name?(filters) + q = material_index_join_file_blob(q) if material_index_needs_file_blob?(filters) + q = apply_material_query(q, filters[:q]) if filters[:q].present? + q = apply_material_media_kind(q, filters[:media_kind]) - count = q.count - materials = q.order(created_at: :desc, id: :desc).limit(limit).offset(offset) + count = q.distinct.count(:id) + materials = + q + .order(Arel.sql(material_index_order_sql(filters))) + .limit(limit) + .offset(offset) + .to_a - render json: { materials: MaterialRepr.many(materials, host: request.base_url), count: count } + response = { materials: MaterialRepr.list_many(materials, host: request.base_url), + count: } + if filters[:tag_id] + response[:tag_scope] = material_index_tag_scope(filters, tag_graph) + end + if filters[:group_by] == 'parent_tag' + response[:groups] = material_index_groups(materials, tag_graph) + end + + render json: response end def show material = Material - .includes(:tag) + .includes(:tag, :material_export_items) + .with_attached_thumbnail .with_attached_file .find_by(id: params[:id]) return head :not_found unless material - wiki_page_body = material.tag.tag_name.wiki_page&.current_revision&.body + wiki_page_body = material.tag&.tag_name&.wiki_page&.current_revision&.body render json: MaterialRepr.base(material, host: request.base_url).merge(wiki_page_body:) end def create return head :unauthorized unless current_user + return head :forbidden unless current_user.gte_member? tag_name_raw = params[:tag].to_s.strip file = params[:file] + file_sha256 = MaterialFileSha256.from_upload(file) url = params[:url].to_s.strip.presence return render_unprocessable_entity('タグは必須です．', field: :tag) if tag_name_raw.blank? if file.blank? && url.blank? return render_validation_error fields: { file: ['ファイルまたは URL は必須です．'], url: ['ファイルまたは URL は必須です．'] } end + block = MaterialImportBlockMatcher.match_for_sha256(file_sha256) + return render_material_import_block(block) if block - tag_name = TagName.find_undiscard_or_create_by!(name: tag_name_raw) - tag = tag_name.tag - tag = Tag.create!(tag_name:, category: :material) unless tag + uploaded_blob = build_uploaded_material_blob!(file, file_sha256) + material = nil - material = Material.new(tag:, url:, - created_by_user: current_user, - updated_by_user: current_user) - material.file.attach(file) + begin + Material.transaction do + tag_name = TagName.find_undiscard_or_create_by!(name: tag_name_raw) + tag = tag_name.tag + tag = Tag.create!(tag_name:, category: :material) unless tag - if material.save + material = Material.new(tag:, url:, + created_by_user: current_user, + updated_by_user: current_user) + material.file.attach(uploaded_blob) if uploaded_blob + material.save! + upsert_export_paths!(material) + MaterialVersionRecorder.record!(material:, event_type: :create, + created_by_user: current_user) + end + rescue StandardError + uploaded_blob&.purge_later + raise + end + + if material + log_thumbnail_generation(material, MaterialThumbnailGenerator.generate!(material)) render json: MaterialRepr.base(material, host: request.base_url), status: :created else render_validation_error material @@ -71,29 +124,44 @@ class MaterialsController < ApplicationController tag_name_raw = params[:tag].to_s.strip file = params[:file] - url = params[:url].to_s.strip.presence + file_sha256 = MaterialFileSha256.from_upload(file) + url = params.key?(:url) ? params[:url].to_s.strip.presence : material.url return render_unprocessable_entity('タグは必須です．', field: :tag) if tag_name_raw.blank? - if file.blank? && url.blank? + if file.blank? && url.blank? && !material.file.attached? return render_validation_error fields: { file: ['ファイルまたは URL は必須です．'], url: ['ファイルまたは URL は必須です．'] } end + block = MaterialImportBlockMatcher.match_for_sha256(file_sha256) + return render_material_import_block(block) if block - tag_name = TagName.find_undiscard_or_create_by!(name: tag_name_raw) - tag = tag_name.tag - tag = Tag.create!(tag_name:, category: :material) unless tag + uploaded_blob = build_uploaded_material_blob!(file, file_sha256) - material.update!(tag:, url:, updated_by_user: current_user) - if file - material.file.attach(file) - else - material.file.purge + begin + Material.transaction do + MaterialVersionRecorder.ensure_snapshot!(material, created_by_user: current_user) + tag_name = TagName.find_undiscard_or_create_by!(name: tag_name_raw) + tag = tag_name.tag + tag = Tag.create!(tag_name:, category: :material) unless tag + + material.assign_attributes(tag:, url:, updated_by_user: current_user) + if uploaded_blob + material.file.attach(uploaded_blob) + elsif params.key?(:url) && url.present? && file.blank? + material.file.detach + end + material.save! + upsert_export_paths!(material) + MaterialVersionRecorder.record!(material:, event_type: :update, + created_by_user: current_user) + end + rescue StandardError + uploaded_blob&.purge_later + raise end - if material.save - render json: MaterialRepr.base(material, host: request.base_url) - else - render_validation_error material - end + log_thumbnail_generation(material, MaterialThumbnailGenerator.generate!(material)) + + render json: MaterialRepr.base(material, host: request.base_url) end def destroy @@ -103,7 +171,375 @@ class MaterialsController < ApplicationController material = Material.find_by(id: params[:id]) return head :not_found unless material - material.discard + Material.transaction do + MaterialVersionRecorder.ensure_snapshot!(material, created_by_user: current_user) + material.discard! + MaterialVersionRecorder.record!(material:, event_type: :discard, + created_by_user: current_user) + end head :no_content end + + def download + zip = MaterialZipExporter.new(profile: params[:profile], + tag_id: params[:tag_id]).export + profile = params[:profile].presence || 'legacy_drive' + send_data zip, + type: 'application/zip', + disposition: 'attachment', + filename: "btrc-materials-#{ profile }.zip" + end + + private + + def material_index_filters + tag_state = params[:tag_state].to_s.presence + tag_state = 'untagged' if bool?(:unclassified) + tag_state = 'all' unless ['all', 'tagged', 'untagged'].include?(tag_state) + + media_kind = params[:media_kind].to_s.presence + unless ['all', 'image', 'video', 'audio', 'file_other', 'url_only'].include?(media_kind) + media_kind = 'all' + end + + sort = params[:sort].to_s.presence + unless ['created_at', 'updated_at', 'tag_name', 'media_kind', 'file_byte_size', + 'version_no', 'id'].include?(sort) + sort = 'created_at' + end + + direction = params[:direction].to_s.downcase + direction = 'desc' unless ['asc', 'desc'].include?(direction) + + group_by = params[:group_by].to_s.presence + group_by = 'none' unless ['none', 'parent_tag'].include?(group_by) + + tag_id = params[:tag_id].to_i + tag_id = nil if tag_id <= 0 + + { q: params[:q].to_s.strip.presence, + tag_state:, + media_kind:, + tag_id:, + include_descendants: bool?(:include_descendants), + group_by:, + created_from: parse_time_param(:created_from), + created_to: parse_time_param(:created_to), + updated_from: parse_time_param(:updated_from), + updated_to: parse_time_param(:updated_to), + sort:, + direction: } + end + + def parse_time_param name + value = params[name].to_s.strip + return nil if value.blank? + + Time.zone.parse(value) + rescue ArgumentError + nil + end + + def material_index_needs_tag_name? filters + filters[:q].present? || filters[:sort] == 'tag_name' + end + + def material_index_needs_file_blob? filters + filters[:q].present? || + filters[:media_kind] != 'all' || + ['media_kind', 'file_byte_size'].include?(filters[:sort]) + end + + def material_index_join_tag_name q + q.left_joins(tag: :tag_name) + end + + def material_index_join_file_blob q + q.joins(<<~SQL.squish) + LEFT JOIN active_storage_attachments material_file_attachments + ON material_file_attachments.record_type = 'Material' + AND material_file_attachments.record_id = materials.id + AND material_file_attachments.name = 'file' + LEFT JOIN active_storage_blobs material_file_blobs + ON material_file_blobs.id = material_file_attachments.blob_id + SQL + end + + def apply_material_query q, term + like = "%#{ ActiveRecord::Base.sanitize_sql_like(term) }%" + q.where('tag_names.name LIKE :q OR materials.url LIKE :q OR ' \ + 'material_file_blobs.filename LIKE :q', + q: like) + end + + def apply_material_media_kind q, media_kind + case media_kind + when 'image' + q.where('material_file_blobs.content_type LIKE ?', 'image/%') + when 'video' + q.where('material_file_blobs.content_type LIKE ?', 'video/%') + when 'audio' + q.where('material_file_blobs.content_type LIKE ?', 'audio/%') + when 'file_other' + q.where('material_file_attachments.id IS NOT NULL') + .where.not('material_file_blobs.content_type LIKE ?', 'image/%') + .where.not('material_file_blobs.content_type LIKE ?', 'video/%') + .where.not('material_file_blobs.content_type LIKE ?', 'audio/%') + when 'url_only' + q.where('material_file_attachments.id IS NULL').where.not(url: [nil, '']) + else + q + end + end + + def apply_material_tag_filter q, filters, tag_graph + return q unless filters[:tag_id] + + tag_ids = + if tag_graph + tag_graph[:scope_tag_ids] + else + [filters[:tag_id]] + end + + q.where(tag_id: tag_ids) + end + + def material_index_tag_graph filters + return nil unless filters[:tag_id] + + children_by_parent_id = Hash.new { |h, k| h[k] = [] } + TagImplication.pluck(:parent_tag_id, :tag_id).each do |parent_id, child_id| + children_by_parent_id[parent_id] << child_id + end + + scope_tag_ids = + if filters[:include_descendants] + collect_material_descendant_ids(filters[:tag_id], children_by_parent_id) + else + Set.new([filters[:tag_id]]) + end + tags_by_id = + Tag + .joins(:tag_name) + .where(id: scope_tag_ids) + .pluck('tags.id', 'tag_names.name', 'tags.category', 'tags.deprecated_at') + .each_with_object({ }) do |(id, name, category, deprecated_at), hash| + hash[id] = { id:, name:, category:, deprecated: deprecated_at.present? } + end + + selected_tag = tags_by_id[filters[:tag_id]] + return nil unless selected_tag + + group_tag_id_by_tag_id = { } + children_by_parent_id[filters[:tag_id]].each do |child_tag_id| + assign_material_group_branch(child_tag_id, + children_by_parent_id:, + tags_by_id:, + group_tag_id_by_tag_id:, + current_group_tag_id: nil, + seen: Set.new([filters[:tag_id]])) + end + + { selected_tag_id: filters[:tag_id], + selected_tag:, + scope_tag_ids: scope_tag_ids.to_a, + tags_by_id:, + group_tag_id_by_tag_id: } + end + + def collect_material_descendant_ids selected_tag_id, children_by_parent_id + ids = Set.new([selected_tag_id]) + stack = [selected_tag_id] + + until stack.empty? + tag_id = stack.pop + children_by_parent_id[tag_id].each do |child_tag_id| + next if ids.include?(child_tag_id) + + ids << child_tag_id + stack << child_tag_id + end + end + + ids + end + + def assign_material_group_branch tag_id, children_by_parent_id:, tags_by_id:, + group_tag_id_by_tag_id:, current_group_tag_id:, seen: + return if seen.include?(tag_id) + + tag = tags_by_id[tag_id] + return unless tag + + seen = seen.dup << tag_id + next_group_tag_id = current_group_tag_id + next_group_tag_id = tag_id if next_group_tag_id.nil? && !tag[:deprecated] + group_tag_id_by_tag_id[tag_id] = next_group_tag_id if next_group_tag_id + + children_by_parent_id[tag_id].each do |child_tag_id| + assign_material_group_branch(child_tag_id, + children_by_parent_id:, + tags_by_id:, + group_tag_id_by_tag_id:, + current_group_tag_id: next_group_tag_id, + seen:) + end + end + + def material_index_groups materials, tag_graph + return [] unless tag_graph + + groups_by_key = { } + + materials.each do |material| + next unless material.tag_id + + group_tag_id = + if material.tag_id == tag_graph[:selected_tag_id] + tag_graph[:selected_tag_id] + else + tag_graph[:group_tag_id_by_tag_id][material.tag_id] || tag_graph[:selected_tag_id] + end + + group_tag = tag_graph[:tags_by_id][group_tag_id] + next unless group_tag + + key = "tag:#{ group_tag_id }" + group = + groups_by_key[key] ||= { key:, + tag: group_tag_repr(group_tag), + material_ids: [], + count: 0 } + group[:material_ids] << material.id + group[:count] += 1 + end + + groups_by_key.values.sort_by { |group| group[:tag][:name] } + end + + def material_index_tag_scope filters, tag_graph + return nil unless tag_graph + + { + tag: group_tag_repr(tag_graph[:selected_tag]), + include_descendants: filters[:include_descendants] + } + end + + def group_tag_repr group_tag + { id: group_tag[:id], + name: group_tag[:name], + category: group_tag[:category] } + end + + def material_index_order_sql filters + direction = filters[:direction] == 'asc' ? 'ASC' : 'DESC' + sort_sql = + case filters[:sort] + when 'tag_name' + 'tag_names.name' + when 'media_kind' + material_media_kind_sql + when 'file_byte_size' + 'material_file_blobs.byte_size' + when 'updated_at' + 'materials.updated_at' + when 'version_no' + 'materials.version_no' + when 'id' + 'materials.id' + else + 'materials.created_at' + end + + "#{ sort_sql } #{ direction }, materials.id #{ direction }" + end + + def material_media_kind_sql + "CASE " \ + "WHEN material_file_attachments.id IS NULL AND materials.url IS NOT NULL THEN 5 " \ + "WHEN material_file_blobs.content_type LIKE 'image/%' THEN 1 " \ + "WHEN material_file_blobs.content_type LIKE 'video/%' THEN 2 " \ + "WHEN material_file_blobs.content_type LIKE 'audio/%' THEN 3 " \ + "WHEN material_file_attachments.id IS NOT NULL THEN 4 " \ + "ELSE 6 END" + end + + def upsert_export_paths! material + raw = params[:export_paths] + return if raw.blank? + + export_paths = raw.respond_to?(:to_unsafe_h) ? raw.to_unsafe_h : raw.to_h + export_paths.each do |profile, export_path| + profile = profile.to_s + export_path = export_path.to_s.strip + item = material.material_export_items.find_or_initialize_by(profile:) + + if export_path.blank? + item.destroy! if item.persisted? + next + end + + item.export_path = export_path + item.enabled = true + item.created_by_user ||= current_user + item.save! + end + end + + def render_zip_empty + render_unprocessable_entity('ZIP export 対象の素材がありません．') + end + + def render_zip_duplicate_path error + render_unprocessable_entity("ZIP export path が重複してゐます: #{ error.message }") + end + + def render_zip_missing_file error + missing_files = error.missing_files.map do |missing_file| + { material_id: missing_file.material_id, + export_path: missing_file.export_path, + blob_id: missing_file.blob_id, + filename: missing_file.filename } + end + + render json: { type: 'validation_error', + message: 'ZIP export に必要な素材ファイルが欠損しています．', + errors: { }, + base_errors: ['ZIP export に必要な素材ファイルが欠損しています．'], + missing_files: }, + status: :unprocessable_entity + end + + def build_uploaded_material_blob! file, file_sha256 + return nil unless file + + file.tempfile.rewind + + blob = ActiveStorage::Blob.create_and_upload!( + io: file.tempfile, + filename: file.original_filename, + content_type: file.content_type, + ) + + MaterialFileSha256.assign_metadata_sha256!(blob, file_sha256) + + blob + ensure + file.tempfile.rewind if file&.tempfile + end + + def render_material_import_block block + render_validation_error fields: { file: ["抑止された素材です: #{ block.reason }"] } + end + + def log_thumbnail_generation material, result + material.reload + Rails.logger.info( + "Material thumbnail generation: material_id=#{ material.id } " \ + "result=#{ result } " \ + "thumbnail_attached=#{ material.thumbnail.attached? } " \ + "content_type=#{ material.content_type }") + end end diff --git a/backend/app/controllers/tags_controller.rb b/backend/app/controllers/tags_controller.rb index c9e585e..f345a02 100644 --- a/backend/app/controllers/tags_controller.rb +++ b/backend/app/controllers/tags_controller.rb @@ -82,8 +82,9 @@ class TagsController < ApplicationController def with_depth parent_tag_id = params[:parent].to_i parent_tag_id = nil if parent_tag_id <= 0 + material_filter = material_filter_param(default: 'any') - graph = build_with_depth_graph + graph = build_with_depth_graph(material_filter) tag_ids = if parent_tag_id @@ -92,19 +93,9 @@ class TagsController < ApplicationController visible_root_tag_ids(graph) end - tags = - Tag - .joins(:tag_name) - .includes(:tag_name, :materials, tag_name: :wiki_page) - .where(id: tag_ids) - .order('tag_names.name') - .distinct - .to_a - - render json: tags.map { |tag| - TagRepr.base(tag).merge(has_children: visible_child_tag_ids(tag.id, graph).present?, - children: []) - } + render json: tag_ids + .sort_by { |tag_id| graph[:tags_by_id][tag_id][:name] } + .map { |tag_id| with_depth_lightweight_repr(tag_id, graph) } end def autocomplete @@ -229,13 +220,16 @@ class TagsController < ApplicationController def materials_by_name name = params[:name].to_s.strip return render_bad_request('name は必須です．') if name.blank? + material_filter = material_filter_param(default: 'any') tag = Tag.joins(:tag_name) .includes(:tag_name, :materials, tag_name: :wiki_page) .find_by(tag_names: { name: }) return head :not_found unless tag - render json: build_tag_children(tag) + graph = build_with_depth_graph(material_filter) + + render json: build_tag_children(tag, graph:) end def update_all @@ -348,7 +342,14 @@ class TagsController < ApplicationController private - def build_with_depth_graph + def material_filter_param default: + value = params[:material_filter].to_s.presence + return default unless ['present', 'missing', 'any'].include?(value) + + value + end + + def build_with_depth_graph material_filter children_by_parent_id = Hash.new { |h, k| h[k] = [] } parent_ids_by_child_id = Hash.new { |h, k| h[k] = [] } @@ -361,23 +362,37 @@ class TagsController < ApplicationController parent_ids_by_child_id.keys + Tag.where(category: ['meme', 'character', 'material']).pluck(:id)).uniq - tags_by_id = Tag.where(id: tag_ids) - .pluck(:id, :category, :deprecated_at) - .each_with_object({ }) do |(id, category, deprecated_at), h| - h[id] = { category:, deprecated: deprecated_at.present? } + material_tag_ids = Material.unscoped.kept.where.not(tag_id: nil).distinct.pluck(:tag_id).to_set + + tags_by_id = Tag.joins(:tag_name) + .where(id: tag_ids) + .pluck('tags.id', 'tag_names.name', 'tags.category', 'tags.deprecated_at') + .each_with_object({ }) do |(id, name, category, deprecated_at), h| + h[id] = { name:, category:, deprecated: deprecated_at.present?, + has_material: material_tag_ids.include?(id) } end { children_by_parent_id:, parent_ids_by_child_id:, tags_by_id:, - visible_child_tag_ids_by_parent_id: { } } + visible_child_tag_ids_by_parent_id: { }, + visible_subtree_by_tag_id: { }, + material_filter: } end def visible_root_tag_ids graph - graph[:tags_by_id].filter_map do |tag_id, attrs| - next unless with_depth_visible_tag?(attrs) - next unless visible_root_tag?(tag_id, graph) + root_ids = Set.new - tag_id + graph[:tags_by_id].each do |tag_id, attrs| + next unless visible_root_tag?(tag_id, graph) + next unless visible_subtree?(tag_id, graph) + + if attrs[:deprecated] + collect_visible_child_tag_ids(tag_id, graph, root_ids, Set.new) + else + root_ids << tag_id + end end + + root_ids.to_a end def visible_root_tag? tag_id, graph @@ -428,25 +443,77 @@ class TagsController < ApplicationController return end - visible_ids << tag_id if with_depth_visible_tag?(tag) + if with_depth_visible_tag?(tag, graph[:material_filter]) + visible_ids << tag_id + return + end + + visible_ids << tag_id if graph[:children_by_parent_id][tag_id].any? { |child_tag_id| + visible_subtree?(child_tag_id, graph) + } end - def with_depth_visible_tag? tag - tag[:category].in?(['meme', 'character', 'material']) && !tag[:deprecated] + def with_depth_visible_tag? tag, material_filter + return false unless tag[:category].in?(['meme', 'character', 'material']) && !tag[:deprecated] + + case material_filter + when 'present' + tag[:has_material] + when 'missing' + !tag[:has_material] + else + true + end end - def build_tag_children tag + def visible_subtree? tag_id, graph + cache = graph[:visible_subtree_by_tag_id] + return cache[tag_id] if cache.key?(tag_id) + + tag = graph[:tags_by_id][tag_id] + return cache[tag_id] = false unless tag + + if tag[:deprecated] + return cache[tag_id] = + graph[:children_by_parent_id][tag_id].any? { |child_tag_id| + visible_subtree?(child_tag_id, graph) + } + end + + cache[tag_id] = + with_depth_visible_tag?(tag, graph[:material_filter]) || + graph[:children_by_parent_id][tag_id].any? { |child_tag_id| + visible_subtree?(child_tag_id, graph) + } + end + + def with_depth_lightweight_repr tag_id, graph + tag = graph[:tags_by_id][tag_id] + + { id: tag_id, + name: tag[:name], + category: tag[:category], + deprecated: tag[:deprecated], + has_material: tag[:has_material], + has_children: visible_child_tag_ids(tag_id, graph).present?, + children: [] } + end + + def build_tag_children tag, graph: nil material = tag.materials.first - file = nil - content_type = nil - if material&.file&.attached? - file = rails_storage_proxy_url(material.file, only_path: false) - content_type = material.file.blob.content_type + tag_graph = graph && graph[:tags_by_id][tag.id] + material = + nil if tag_graph && !with_depth_visible_tag?(tag_graph, graph[:material_filter]) + + children = tag.children.sort_by(&:name) + if graph + children = children.filter { |child_tag| visible_subtree?(child_tag.id, graph) } end TagRepr.base(tag).merge( - children: tag.children.sort_by { _1.name }.map { build_tag_children(_1) }, - material: material.as_json&.merge(file:, content_type:)) + children: children.map { build_tag_children(_1, graph:) }, + has_material: tag_graph ? tag_graph[:has_material] : material.present?, + material: material && MaterialRepr.base(material, host: request.base_url)) end def record_tag_version! tag, event_type:, created_by_user:, name_changed: false, wiki_page: nil diff --git a/backend/app/models/material.rb b/backend/app/models/material.rb index 417b292..dd0aa22 100644 --- a/backend/app/models/material.rb +++ b/backend/app/models/material.rb @@ -1,6 +1,9 @@ class Material < ApplicationRecord include MyDiscard + SOURCE_KINDS = ['uri', 'google_drive_path', 'google_drive_file', + 'legacy_drive_path'].freeze + default_scope -> { kept } belongs_to :parent, class_name: 'Material', optional: true @@ -10,9 +13,18 @@ class Material < ApplicationRecord belongs_to :created_by_user, class_name: 'User', optional: true belongs_to :updated_by_user, class_name: 'User', optional: true - has_one_attached :file, dependent: :purge + has_many :material_versions, dependent: :destroy + has_many :material_export_items, dependent: :destroy - validates :tag_id, presence: true, uniqueness: true + has_one_attached :file, dependent: :purge + has_one_attached :thumbnail, dependent: :purge + + before_validation :assign_normalized_source_key + + validates :tag_id, uniqueness: true, allow_nil: true + validates :source_kind, + inclusion: { in: SOURCE_KINDS }, + allow_blank: true validate :file_must_be_attached validate :tag_must_be_material_category @@ -23,8 +35,37 @@ class Material < ApplicationRecord file.blob.content_type end + def file_byte_size + return nil unless file&.attached? + + file.blob.byte_size + end + + def file_filename + return nil unless file&.attached? + + file.blob.filename.to_s + end + + def snapshot_export_paths + material_export_items.order(:profile).pluck(:profile, :export_path).to_h + end + private + def assign_normalized_source_key + if source_kind.blank? + self.normalized_source_key = nil + return + end + + self.normalized_source_key = + MaterialSyncSuppression.normalize_source_key(source_kind:, + source_uri:, + drive_path: source_path, + drive_file_id: source_file_id) + end + def file_must_be_attached return if url.present? || file.attached? diff --git a/backend/app/models/material_export_item.rb b/backend/app/models/material_export_item.rb new file mode 100644 index 0000000..5d13115 --- /dev/null +++ b/backend/app/models/material_export_item.rb @@ -0,0 +1,48 @@ +class MaterialExportItem < ApplicationRecord + VALID_PROFILES = ['legacy_drive'].freeze + + belongs_to :material + belongs_to :created_by_user, class_name: 'User', optional: true + + validates :profile, presence: true, inclusion: { in: VALID_PROFILES } + validates :export_path, presence: true, uniqueness: { scope: :profile } + validates :material_id, uniqueness: { scope: :profile } + validate :export_path_must_be_relative_safe_path + + scope :enabled, -> { where(enabled: true) } + + private + + def export_path_must_be_relative_safe_path + return if export_path.blank? + + if export_path.start_with?('/') + errors.add(:export_path, '絶対パスは使へません．') + end + + if export_path.match?(/\A[A-Za-z]:\//) + errors.add(:export_path, '絶対パスは使へません．') + end + + if export_path.include?('\\') + errors.add(:export_path, '/ 区切りで指定してください．') + end + + if export_path.include?("\0") + errors.add(:export_path, 'NUL は使へません．') + end + + parts = export_path.split('/') + if export_path.include?('//') + errors.add(:export_path, '空の path segment は使へません．') + end + + if parts.any? { |part| part.in?(['.', '..']) } + errors.add(:export_path, '.. は使へません．') + end + + if export_path.end_with?('/') + errors.add(:export_path, 'directory path は使へません．') + end + end +end diff --git a/backend/app/models/material_import_block.rb b/backend/app/models/material_import_block.rb new file mode 100644 index 0000000..b159c6c --- /dev/null +++ b/backend/app/models/material_import_block.rb @@ -0,0 +1,29 @@ +class MaterialImportBlock < ApplicationRecord + MATCH_KINDS = ['sha256', 'exact_path', 'path_prefix', 'manual'].freeze + REASONS = [ + 'copyright_high_risk', + 'copyright_takedown', + 'adult_or_sensitive', + 'personal_information', + 'malware_or_dangerous_file', + 'duplicate_or_low_quality', + 'source_owner_request', + 'other' + ].freeze + + belongs_to :created_by_user, class_name: 'User', optional: true + + validates :match_kind, presence: true, inclusion: { in: MATCH_KINDS } + validates :reason, presence: true, inclusion: { in: REASONS } + validates :sha256, length: { is: 64 }, allow_blank: true + validate :match_value_must_be_present + + private + + def match_value_must_be_present + return if match_kind == 'manual' + return if sha256.present? || external_path_pattern.present? + + errors.add(:base, 'sha256 または external_path_pattern は必須です．') + end +end diff --git a/backend/app/models/material_sync_source.rb b/backend/app/models/material_sync_source.rb new file mode 100644 index 0000000..d45a977 --- /dev/null +++ b/backend/app/models/material_sync_source.rb @@ -0,0 +1,45 @@ +class MaterialSyncSource < ApplicationRecord + SOURCE_KINDS = ['uri', 'google_drive_path', 'google_drive_file', 'legacy_drive_path'].freeze + + belongs_to :created_by_user, class_name: 'User', optional: true + belongs_to :updated_by_user, class_name: 'User', optional: true + + validates :name, presence: true + validates :source_kind, + presence: true, + inclusion: { in: SOURCE_KINDS } + validates :profile, presence: true, inclusion: { in: MaterialExportItem::VALID_PROFILES } + validate :source_value_must_be_present + + scope :enabled, -> { where(enabled: true) } + + def normalized_source_key + MaterialSyncSuppression.normalize_source_key(source_kind:, + source_uri:, + source_path:, + source_file_id:) + end + + private + + def source_value_must_be_present + return if source_value_present? + + errors.add(:base, '同期元を指定してください．') + end + + def source_value_present? + case source_kind + when 'uri' + source_uri.present? + when 'google_drive_file' + source_file_id.present? || source_uri.present? + when 'google_drive_path' + source_file_id.present? || source_uri.present? + when 'legacy_drive_path' + source_path.present? + else + normalized_source_key.present? + end + end +end diff --git a/backend/app/models/material_sync_suppression.rb b/backend/app/models/material_sync_suppression.rb new file mode 100644 index 0000000..d744807 --- /dev/null +++ b/backend/app/models/material_sync_suppression.rb @@ -0,0 +1,105 @@ +class MaterialSyncSuppression < ApplicationRecord + SOURCE_KINDS = [ + 'uri', + 'google_drive_path', + 'google_drive_path_prefix', + 'google_drive_file', + 'legacy_drive_path', + 'legacy_drive_path_prefix' + ].freeze + + REASONS = [ + 'copyright_high_risk', + 'copyright_takedown', + 'adult_or_sensitive', + 'personal_information', + 'malware_or_dangerous_file', + 'duplicate_or_low_quality', + 'source_owner_request', + 'other' + ].freeze + + belongs_to :created_by_user, class_name: 'User', optional: true + + before_validation :assign_normalized_source_key + + validates :source_kind, presence: true, inclusion: { in: SOURCE_KINDS } + validates :reason, presence: true, inclusion: { in: REASONS } + validates :normalized_source_key, presence: true, uniqueness: true + validate :source_value_must_be_present + validate :drive_path_must_be_safe_relative_path + + def self.normalize_source_key source_kind:, + source_uri: nil, + drive_path: nil, + drive_file_id: nil, + source_path: nil, + source_file_id: nil + kind = source_kind.to_s + value = + case kind + when 'uri' + source_uri.to_s.strip + when 'google_drive_path', 'google_drive_path_prefix', + 'legacy_drive_path', 'legacy_drive_path_prefix' + (source_path || drive_path).to_s.strip.gsub(%r{/+}, '/') + when 'google_drive_file' + (source_file_id || drive_file_id).to_s.strip + else + '' + end + + return nil if kind.blank? || value.blank? + + "#{ kind }:#{ value }" + end + + private + + def assign_normalized_source_key + normalize_path_fields! + self.normalized_source_key = + self.class.normalize_source_key(source_kind:, + source_uri:, + drive_path:, + drive_file_id:) + end + + def source_value_must_be_present + return if normalized_source_key.present? + + errors.add(:base, '同期元を指定してください．') + end + + def drive_path_must_be_safe_relative_path + return unless source_kind.in?( + ['google_drive_path', 'google_drive_path_prefix', + 'legacy_drive_path', 'legacy_drive_path_prefix']) + + value = drive_path.to_s + return if value.blank? + + if value.start_with?('/') || value.match?(/\A[A-Za-z]:\//) + errors.add(:drive_path, '相対 path を指定してください．') + end + if value.start_with?('My Drive/', 'マイドライブ/') + errors.add(:drive_path, '同期元フォルダからの相対 path を指定してください．') + end + errors.add(:drive_path, 'NUL は使へません．') if value.include?("\0") + errors.add(:drive_path, '連続スラッシュは使へません．') if value.include?('//') + errors.add(:drive_path, '末尾スラッシュは使へません．') if value.end_with?('/') + + parts = value.split('/') + if parts.any? { |part| part.in?(['.', '..']) } + errors.add(:drive_path, '. や .. は使へません．') + end + end + + def normalize_path_fields! + return unless source_kind.in?( + ['google_drive_path', 'google_drive_path_prefix', + 'legacy_drive_path', 'legacy_drive_path_prefix']) + + self.drive_path = drive_path.to_s.tr('\\', '/').strip.presence + end +end diff --git a/backend/app/models/material_version.rb b/backend/app/models/material_version.rb new file mode 100644 index 0000000..326a8f9 --- /dev/null +++ b/backend/app/models/material_version.rb @@ -0,0 +1,17 @@ +class MaterialVersion < ApplicationRecord + EVENT_TYPE_MAP = { create: 'create', + update: 'update', + discard: 'discard', + restore: 'restore' }.freeze + + include VersionRecord + + belongs_to :material + belongs_to :tag, optional: true + belongs_to :parent, class_name: 'Material', optional: true + belongs_to :updated_by_user, class_name: 'User', optional: true + + def export_paths_hash + export_paths_json || {} + end +end diff --git a/backend/app/models/version_record.rb b/backend/app/models/version_record.rb index 7224639..ccee900 100644 --- a/backend/app/models/version_record.rb +++ b/backend/app/models/version_record.rb @@ -1,15 +1,23 @@ module VersionRecord extend ActiveSupport::Concern + DEFAULT_EVENT_TYPE_MAP = { create: 'create', + update: 'update', + discard: 'discard', + restore: 'restore' }.freeze + def readonly? = persisted? included do + event_type_map = if const_defined?(:EVENT_TYPE_MAP, false) + const_get(:EVENT_TYPE_MAP) + else + DEFAULT_EVENT_TYPE_MAP + end + belongs_to :created_by_user, class_name: 'User', optional: true - enum :event_type, { create: 'create', - update: 'update', - discard: 'discard', - restore: 'restore' }, prefix: true, validate: true + enum :event_type, event_type_map, prefix: true, validate: true validates :version_no, presence: true, numericality: { only_integer: true, greater_than: 0 } validates :event_type, presence: true diff --git a/backend/app/representations/material_repr.rb b/backend/app/representations/material_repr.rb index 44edd26..30d21f9 100644 --- a/backend/app/representations/material_repr.rb +++ b/backend/app/representations/material_repr.rb @@ -2,7 +2,9 @@ module MaterialRepr - BASE = { only: [:id, :url, :created_at, :updated_at], + BASE = { only: [:id, :url, :version_no, :source_kind, :source_uri, + :source_path, :source_file_id, :normalized_source_key, + :created_at, :updated_at], methods: [:content_type], include: { tag: TagRepr::BASE, created_by_user: UserRepr::BASE, @@ -15,10 +17,93 @@ module MaterialRepr file: if material.file.attached? Rails.application.routes.url_helpers.rails_storage_proxy_url( material.file, host:) - end) + end, + thumbnail: thumbnail_url(material, host:), + thumbnail_fallback_text: thumbnail_fallback_text(material), + thumbnail_fallback_kind: thumbnail_fallback_kind(material), + media_kind: media_kind(material), + file_byte_size: material.file_byte_size, + export_paths: export_paths(material), + export_items: export_items(material)) end def many materials, host: materials.map { |m| base(m, host:) } end + + def list material, host: + { id: material.id, + version_no: material.version_no, + url: material.url, + source_kind: material.source_kind, + source_uri: material.source_uri, + source_path: material.source_path, + source_file_id: material.source_file_id, + normalized_source_key: material.normalized_source_key, + tag: compact_tag(material.tag), + thumbnail: thumbnail_url(material, host:), + thumbnail_fallback_text: thumbnail_fallback_text(material), + thumbnail_fallback_kind: thumbnail_fallback_kind(material), + media_kind: media_kind(material), + content_type: material.content_type, + file_byte_size: material.file_byte_size, + created_at: material.created_at, + updated_at: material.updated_at, + export_paths: export_paths(material), + export_items: export_items(material) } + end + + def list_many materials, host: + materials.map { |m| list(m, host:) } + end + + def export_paths material + material.material_export_items.each_with_object({ }) do |item, hash| + hash[item.profile] = item.enabled ? item.export_path : '' + end + end + + def export_items material + material.material_export_items.map do |item| + { id: item.id, + profile: item.profile, + export_path: item.export_path, + enabled: item.enabled } + end + end + + def thumbnail_url material, host: + return nil unless material.thumbnail.attached? + + Rails.application.routes.url_helpers.rails_storage_proxy_url( + material.thumbnail, host:) + end + + def thumbnail_fallback_text material + material.tag&.name || material.created_at&.strftime('%Y-%m-%d') + end + + def thumbnail_fallback_kind material + material.tag.present? ? 'tag_name' : 'created_at' + end + + def media_kind material + return 'url_only' unless material.file.attached? + + content_type = material.file.blob.content_type.to_s + return 'image' if content_type.start_with?('image/') + return 'video' if content_type.start_with?('video/') + return 'audio' if content_type.start_with?('audio/') + + 'file_other' + end + + def compact_tag tag + return nil unless tag + + { id: tag.id, + name: tag.name, + category: tag.category, + deprecated_at: tag.deprecated_at } + end end diff --git a/backend/app/services/google_drive/api_client.rb b/backend/app/services/google_drive/api_client.rb new file mode 100644 index 0000000..b3a9300 --- /dev/null +++ b/backend/app/services/google_drive/api_client.rb @@ -0,0 +1,233 @@ +require 'json' +require 'jwt' +require 'net/http' +require 'openssl' +require 'tempfile' +require 'uri' + + +module GoogleDrive + class ApiClient + DRIVE_ENDPOINT = 'https://www.googleapis.com/drive/v3' + TOKEN_ENDPOINT = 'https://oauth2.googleapis.com/token' + FOLDER_MIME_TYPE = 'application/vnd.google-apps.folder' + NATIVE_FILE_MIME_TYPE_PREFIX = 'application/vnd.google-apps.' + DRIVE_SCOPE = 'https://www.googleapis.com/auth/drive.readonly' + + def initialize service_account_email: ENV['GOOGLE_DRIVE_SERVICE_ACCOUNT_EMAIL'], + private_key: ENV['GOOGLE_DRIVE_PRIVATE_KEY'], + private_key_path: ENV['GOOGLE_DRIVE_PRIVATE_KEY_PATH'], + subject: ENV['GOOGLE_DRIVE_SUBJECT'] + @service_account_email = service_account_email.to_s + @private_key = load_private_key(private_key, private_key_path) + @subject = subject.to_s.presence + @access_token = nil + end + + def list_material_files_under_folder folder_id + files = [] + each_material_file_under_folder(folder_id) { |entry| files << entry } + files + end + + def each_material_file_under_folder folder_id + return enum_for(__method__, folder_id) unless block_given? + + walk_folder(folder_id, nil) do |entry, relative_path| + next if entry['mimeType'] == FOLDER_MIME_TYPE + next if native_file?(entry['mimeType']) + + yield build_file_entry(entry, relative_path) + end + end + + def fetch_material_file file_id + metadata = get_file(file_id) + return nil if native_file?(metadata['mimeType']) + + build_file_entry(metadata, metadata['name']) + end + + def download_to_tempfile file_id, filename: + tempfile = Tempfile.new(['material-sync', File.extname(filename.to_s)]) + tempfile.binmode + request_binary("/files/#{ file_id }", + { alt: 'media', supportsAllDrives: true }) do |chunk| + tempfile.write(chunk.b) + end + tempfile.rewind + tempfile + rescue StandardError + tempfile&.close! + raise + end + + def extract_file_id value + raw = value.to_s.strip + return nil if raw.blank? + return raw unless raw.include?('/') + + uri = URI.parse(raw) + return uri.query.to_s[%r{(?:^|&)id=([^&]+)}, 1] if uri.query.present? + return uri.path[%r{/folders/([^/]+)}, 1] if uri.path.include?('/folders/') + + uri.path[%r{/d/([^/]+)}, 1] + rescue URI::InvalidURIError + nil + end + + private + + def walk_folder folder_id, prefix, &block + list_children(folder_id).each do |entry| + relative_path = MaterialSyncExportPath.build(prefix:, + relative_path: entry['name']) + if entry['mimeType'] == FOLDER_MIME_TYPE + walk_folder(entry['id'], relative_path, &block) + next + end + + block.call(entry, relative_path) + end + end + + def build_file_entry entry, relative_path + { id: entry['id'], + name: entry['name'], + mime_type: entry['mimeType'], + relative_path: MaterialSyncExportPath.normalize_path(relative_path), + sha256_checksum: entry['sha256Checksum'], + web_view_link: entry['webViewLink'], + web_content_link: entry['webContentLink'] } + end + + def list_children folder_id + files = [] + page_token = nil + + loop do + response = + request_json('/files', { + q: "'#{ folder_id }' in parents and trashed = false", + fields: 'nextPageToken,files(id,name,mimeType,sha256Checksum,' \ + 'webViewLink,webContentLink)', + orderBy: 'folder,name', + pageSize: 1000, + supportsAllDrives: true, + includeItemsFromAllDrives: true, + pageToken: page_token }.compact) + files.concat(response.fetch('files')) + page_token = response['nextPageToken'] + break if page_token.blank? + end + + files + end + + def get_file file_id + request_json("/files/#{ file_id }", + fields: 'id,name,mimeType,sha256Checksum,webViewLink,webContentLink', + supportsAllDrives: true) + end + + def request_json path, params = {} + response = request(:get, path, params:) + unless response.is_a?(Net::HTTPSuccess) + raise "Google Drive API error: #{ response.code } #{ response.body }" + end + + JSON.parse(response.body) + end + + def request_binary path, params = {} + request(:get, path, params:) do |response| + unless response.is_a?(Net::HTTPSuccess) + raise "Google Drive download error: #{ response.code } #{ response.body }" + end + + response.read_body do |chunk| + yield chunk.b + end + end + end + + def request method, path, params: nil + uri = URI(DRIVE_ENDPOINT + path) + uri.query = URI.encode_www_form(params) if params.present? + + klass = + case method + when :get + Net::HTTP::Get + else + raise ArgumentError, "Unsupported Google Drive request method: #{ method }" + end + request = klass.new(uri) + request['Authorization'] = "Bearer #{ access_token }" + + Net::HTTP.start(uri.host, + uri.port, + use_ssl: true, + open_timeout: 10, + read_timeout: 60) do |http| + if block_given? + http.request(request) do |response| + return yield response + end + else + http.request(request) + end + end + end + + def access_token + @access_token ||= fetch_access_token + end + + def fetch_access_token + raise 'GOOGLE_DRIVE_SERVICE_ACCOUNT_EMAIL is required' if @service_account_email.blank? + raise 'Google Drive private key is required' if @private_key.blank? + + payload = { iss: @service_account_email, + scope: DRIVE_SCOPE, + aud: TOKEN_ENDPOINT, + exp: 1.hour.from_now.to_i, + iat: Time.current.to_i } + payload[:sub] = @subject if @subject.present? + + assertion = JWT.encode(payload, @private_key, 'RS256') + uri = URI(TOKEN_ENDPOINT) + request = Net::HTTP::Post.new(uri) + request.set_form_data( + grant_type: 'urn:ietf:params:oauth:grant-type:jwt-bearer', + assertion:) + + response = + Net::HTTP.start(uri.host, + uri.port, + use_ssl: true, + open_timeout: 10, + read_timeout: 30) do |http| + http.request(request) + end + unless response.is_a?(Net::HTTPSuccess) + raise "Google OAuth error: #{ response.code } #{ response.body }" + end + + JSON.parse(response.body).fetch('access_token') + end + + def load_private_key private_key, private_key_path + raw = private_key.to_s + raw = raw.gsub('\n', "\n") if raw.present? + raw = File.read(private_key_path) if raw.blank? && private_key_path.present? + return nil if raw.blank? + + OpenSSL::PKey::RSA.new(raw) + end + + def native_file? mime_type + mime_type.to_s.start_with?(NATIVE_FILE_MIME_TYPE_PREFIX) + end + end +end diff --git a/backend/app/services/material_file_sha256.rb b/backend/app/services/material_file_sha256.rb new file mode 100644 index 0000000..31ad4d0 --- /dev/null +++ b/backend/app/services/material_file_sha256.rb @@ -0,0 +1,57 @@ +require 'digest' +require 'json' + +class MaterialFileSha256 + def self.blob_metadata blob + metadata = blob.metadata + return metadata if metadata.is_a?(Hash) + return { } if metadata.blank? + + JSON.parse(metadata) + rescue JSON::ParserError + { } + end + + def self.metadata_sha256 blob + blob_metadata(blob)['sha256'].presence + end + + def self.assign_metadata_sha256! blob, sha256 + return if sha256.blank? + + metadata = blob_metadata(blob) + metadata['sha256'] = sha256 + blob.metadata = metadata + blob.save! if blob.changed? + end + + def self.from_blob blob, allow_download: false + sha256 = metadata_sha256(blob) + return sha256 if sha256.present? + return nil unless allow_download + + begin + blob.open do |file| + sha256 = Digest::SHA256.file(file.path).hexdigest + assign_metadata_sha256!(blob, sha256) + sha256 + end + rescue ActiveStorage::FileNotFoundError, ArgumentError => error + Rails.logger.warn( + "MaterialFileSha256.from_blob failed for blob_id=#{blob.id}: " \ + "#{error.class}: #{error.message}", + ) + nil + end + end + + def self.from_upload upload + tempfile = upload&.tempfile + return nil unless tempfile + + tempfile.rewind + Digest::SHA256.file(tempfile.path).hexdigest.tap do + tempfile.rewind + end + end +end diff --git a/backend/app/services/material_import_block_matcher.rb b/backend/app/services/material_import_block_matcher.rb new file mode 100644 index 0000000..cb0a92c --- /dev/null +++ b/backend/app/services/material_import_block_matcher.rb @@ -0,0 +1,7 @@ +class MaterialImportBlockMatcher + def self.match_for_sha256 sha256 + return nil if sha256.blank? + + MaterialImportBlock.find_by(match_kind: 'sha256', sha256:) + end +end diff --git a/backend/app/services/material_sync_export_path.rb b/backend/app/services/material_sync_export_path.rb new file mode 100644 index 0000000..bc0601e --- /dev/null +++ b/backend/app/services/material_sync_export_path.rb @@ -0,0 +1,56 @@ +class MaterialSyncExportPath + class << self + def build prefix: nil, relative_path: nil, filename: nil, source_file_id: nil + path = normalize_path(relative_path) + path = fallback_filename(filename, source_file_id) if path.blank? + + normalized_prefix = normalize_path(prefix) + return path if normalized_prefix.blank? + return normalized_prefix if path.blank? + + [normalized_prefix, path].join('/') + end + + def normalize_path path + raw = path.to_s.delete("\0").tr('\\', '/').strip + raw = raw.sub(%r{\A[A-Za-z]:/+}, '') + raw = raw.sub(%r{\A/+}, '') + return nil if raw.blank? + + segments = raw.split('/').filter_map do |segment| + normalized = normalize_segment(segment) + normalized.presence + end + return nil if segments.empty? + + segments.join('/') + end + + def uniquify path, source_file_id + normalized = normalize_path(path) + return normalized if normalized.blank? || source_file_id.blank? + + ext = File.extname(normalized) + base = ext.present? ? normalized.delete_suffix(ext) : normalized + "#{ base }--#{ source_file_id }#{ ext }" + end + + private + + def fallback_filename filename, source_file_id + name = normalize_path(filename) + return name if name.present? + return nil if source_file_id.blank? + + source_file_id.to_s + end + + def normalize_segment segment + cleaned = segment.to_s.delete("\0").tr('\\/', '_').strip + return nil if cleaned.blank? || cleaned == '.' + return '__' if cleaned == '..' + + cleaned + end + end +end diff --git a/backend/app/services/material_sync_importer.rb b/backend/app/services/material_sync_importer.rb new file mode 100644 index 0000000..1912446 --- /dev/null +++ b/backend/app/services/material_sync_importer.rb @@ -0,0 +1,259 @@ +require 'digest' + +class MaterialSyncImporter + Result = Struct.new(:material, :action, :suppressed, :suppression, keyword_init: true) + + def self.import! attributes + new(attributes).import! + end + + def initialize attributes + @attributes = attributes + end + + def import! + source = source_attributes + key = normalized_source_key(source) + raise ArgumentError, 'normalized_source_key is required for material sync' if key.blank? + + suppression = suppression_for(source) + if suppression + return Result.new(material: nil, + action: :suppressed, + suppressed: true, + suppression:) + end + + material = + Material + .unscoped + .find_or_initialize_by(normalized_source_key: key) + if unchanged?(material, source) + return Result.new(material:, + action: :unchanged, + suppressed: false, + suppression: nil) + end + + event_type = + if material.new_record? + :create + elsif material.discarded? + :restore + else + :update + end + uploaded_blob = uploaded_blob! + if @import_block + return Result.new(material: nil, + action: :suppressed, + suppressed: true, + suppression: @import_block) + end + + Material.transaction do + if material.persisted? + MaterialVersionRecorder.ensure_snapshot!(material, + created_by_user: @attributes[:updated_by_user]) + end + material.assign_attributes(material_attributes_for(material).merge(source)) + material.discarded_at = nil if material.respond_to?(:discarded_at=) + material.file.attach(uploaded_blob) if uploaded_blob + material.save! + upsert_export_item!(material) + MaterialVersionRecorder.record!(material:, + event_type:, + created_by_user: @attributes[:updated_by_user]) + end + MaterialThumbnailGenerator.generate!(material) + + Result.new(material:, + action: event_type == :create ? :imported : :updated, + suppressed: false, + suppression: nil) + rescue StandardError + uploaded_blob&.purge_later + raise + ensure + close_tempfile! + end + + private + + def source_attributes + { source_kind: @attributes[:source_kind], + source_uri: @attributes[:source_uri], + source_path: @attributes[:source_path], + source_file_id: @attributes[:source_file_id] } + end + + def normalized_source_key source + MaterialSyncSuppression.normalize_source_key(source_kind: source[:source_kind], + source_uri: source[:source_uri], + source_path: source[:source_path], + source_file_id: source[:source_file_id]) + end + + def material_attributes_for material + attrs = + @attributes + .except(:source_kind, :source_uri, :source_path, :source_file_id, + :file_blob, :file_tempfile, :file_downloader, + :filename, :content_type, + :file_sha256, :export_path, :profile, :tag, :url) + + if material.new_record? + attrs[:tag] = @attributes[:tag] if @attributes.key?(:tag) + attrs[:url] = @attributes[:url] if @attributes.key?(:url) + return attrs + end + + if @attributes.key?(:tag) && @attributes[:tag].present? + attrs[:tag] = @attributes[:tag] + end + if @attributes.key?(:url) && @attributes[:url].present? + attrs[:url] = @attributes[:url] + end + + attrs + end + + def suppression_for source + if source[:source_kind] == 'google_drive_file' && source[:source_path].present? + return MaterialSyncSuppressionMatcher.match_google_drive_candidate( + drive_file_id: source[:source_file_id], + relative_path: source[:source_path]) + end + + MaterialSyncSuppressionMatcher.match(**source) + end + + def uploaded_blob! + return @attributes[:file_blob] if @attributes[:file_blob] + + tempfile = upload_tempfile + return nil unless tempfile + + tempfile.rewind + file_sha256 = @attributes[:file_sha256] || Digest::SHA256.file(tempfile.path).hexdigest + @attributes[:file_sha256] = file_sha256 + + @import_block = MaterialImportBlockMatcher.match_for_sha256(file_sha256) + if @import_block + tempfile.rewind + return nil + end + + blob = ActiveStorage::Blob.create_and_upload!( + io: tempfile, + filename: @attributes[:filename], + content_type: @attributes[:content_type]) + MaterialFileSha256.assign_metadata_sha256!(blob, file_sha256) + tempfile.rewind + blob + end + + def upload_tempfile + return @upload_tempfile if defined?(@upload_tempfile) + + @upload_tempfile = @attributes[:file_tempfile] + if @upload_tempfile.blank? && @attributes[:file_downloader] + @upload_tempfile = @attributes[:file_downloader].call + end + @upload_tempfile + end + + def upsert_export_item! material + export_path = resolved_export_path(material) + return if export_path.blank? + + item = material.material_export_items.find_or_initialize_by(profile: effective_profile) + item.export_path = export_path + item.enabled = true + item.created_by_user ||= @attributes[:created_by_user] + item.save! + end + + def export_path_taken_by_other? material, profile, export_path + MaterialExportItem + .where(profile:, export_path:) + .where.not(material_id: material.id) + .exists? + end + + def close_tempfile! + tempfile = upload_tempfile + return unless tempfile + + tempfile.close! unless tempfile.closed? + rescue StandardError + nil + end + + def unchanged? material, source + return false if material.new_record? || material.discarded? + return false unless same_source_attributes?(material, source) + return false unless same_export_path?(material) + return false unless same_material_attributes?(material) + + same_file_snapshot?(material) + end + + def same_source_attributes? material, source + material.source_kind == source[:source_kind] \ + && material.source_uri == source[:source_uri] \ + && material.source_path == source[:source_path] \ + && material.source_file_id == source[:source_file_id] + end + + def same_export_path? material + expected = resolved_export_path(material) + current = material.material_export_items.find { |item| item.profile == effective_profile } + + return current.blank? if expected.blank? + + current&.enabled && current.export_path == expected + end + + def same_material_attributes? material + same_tag_attribute?(material) && same_url_attribute?(material) + end + + def same_file_snapshot? material + expected_sha256 = @attributes[:file_sha256].to_s.presence + + if expected_sha256.present? + return false unless material.file.attached? + + return MaterialFileSha256.metadata_sha256(material.file.blob) == expected_sha256 + end + + !material.file.attached? + end + + def effective_profile + @attributes[:profile].presence || 'legacy_drive' + end + + def resolved_export_path material + export_path = @attributes[:export_path].to_s.strip + return export_path if export_path.blank? + return export_path unless export_path_taken_by_other?(material, effective_profile, export_path) + + MaterialSyncExportPath.uniquify(export_path, @attributes[:source_file_id]) + end + + def same_tag_attribute? material + return true unless @attributes.key?(:tag) + return true if @attributes[:tag].blank? + + material.tag_id == @attributes[:tag].id + end + + def same_url_attribute? material + return true unless @attributes.key?(:url) + return true if @attributes[:url].blank? + + material.url == @attributes[:url] + end +end diff --git a/backend/app/services/material_sync_runner.rb b/backend/app/services/material_sync_runner.rb new file mode 100644 index 0000000..1e96c85 --- /dev/null +++ b/backend/app/services/material_sync_runner.rb @@ -0,0 +1,251 @@ +class MaterialSyncRunner + Result = Struct.new(:imported, :updated, :unchanged, :suppressed, :failed, + :errors, keyword_init: true) + + def self.sync_enabled! + results = MaterialSyncSource.enabled.order(:id).map { |source| new(source).sync! } + + Result.new(imported: results.sum(&:imported), + updated: results.sum(&:updated), + unchanged: results.sum(&:unchanged), + suppressed: results.sum(&:suppressed), + failed: results.sum(&:failed), + errors: results.flat_map(&:errors)) + end + + def initialize source + @source = source + end + + def sync! + result = Result.new(imported: 0, updated: 0, unchanged: 0, + suppressed: 0, failed: 0, errors: []) + + if @source.source_kind == 'google_drive_path' + sync_google_drive_path!(result) + else + candidates.each do |candidate| + next if candidate.blank? + + sync_candidate!(candidate, result) + end + end + + @source.update!(last_synced_at: Time.current) + log_result(result) + result + rescue NotImplementedError, StandardError => e + result.failed += 1 + result.errors << { source_id: @source.id, error: e.message } + log_result(result) + result + end + + private + + def candidates + case @source.source_kind + when 'uri' + [uri_candidate] + when 'google_drive_file' + [google_drive_file_candidate] + when 'legacy_drive_path' + raise NotImplementedError, 'legacy_drive_path material sync is not implemented' + else + raise NotImplementedError, "Unsupported material sync source_kind: #{ @source.source_kind }" + end + end + + def uri_candidate + { source_kind: @source.source_kind, + source_uri: @source.source_uri, + source_path: @source.source_path, + source_file_id: @source.source_file_id, + url: @source.source_uri, + tag: nil, + created_by_user: @source.created_by_user, + updated_by_user: @source.updated_by_user || @source.created_by_user } + end + + def sync_candidate! candidate, result + block = MaterialImportBlockMatcher.match_for_sha256(candidate[:file_sha256]) + if block + result.suppressed += 1 + Rails.logger.info( + material_sync_log(action: 'suppressed', + reason: block.reason, + normalized_source_key: candidate_normalized_source_key(candidate))) + return + end + + suppression = suppression_for(candidate) + + if suppression + result.suppressed += 1 + Rails.logger.info( + material_sync_log(action: 'suppressed', + normalized_source_key: suppression.normalized_source_key)) + return + end + + import = MaterialSyncImporter.import!(candidate) + result.public_send("#{ import.action }=", result.public_send(import.action) + 1) + rescue StandardError => e + result.failed += 1 + result.errors << { source_id: @source.id, + normalized_source_key: candidate_normalized_source_key(candidate), + error: e.message } + Rails.logger.warn( + material_sync_log(action: 'failed', + error: e.message, + normalized_source_key: candidate_normalized_source_key(candidate))) + ensure + close_candidate_tempfile(candidate) + end + + def google_drive_path_candidates + folder_id = google_drive_folder_id + Enumerator.new do |entries| + drive_client.each_material_file_under_folder(folder_id).each do |entry| + entries << build_google_drive_candidate(entry) + end + end + end + + def sync_google_drive_path! result + folder_id = google_drive_folder_id + scanned_count = 0 + + drive_client.each_material_file_under_folder(folder_id) do |entry| + scanned_count += 1 + sync_candidate!(build_google_drive_candidate(entry), result) + log_google_drive_progress(folder_id, scanned_count, result) if progress_log_scan_count?(scanned_count) + end + + log_google_drive_progress(folder_id, scanned_count, result, summary: true) + end + + def google_drive_file_candidate + entry = drive_client.fetch_material_file(google_drive_file_id) + return nil unless entry + + build_google_drive_candidate(entry, single_file: true) + end + + def build_google_drive_candidate entry, single_file: false + relative_path = + if single_file + nil + else + entry[:relative_path] + end + export_path = + MaterialSyncExportPath.build(prefix: @source.export_path_prefix, + relative_path:, + filename: entry[:name], + source_file_id: entry[:id]) + + { source_kind: 'google_drive_file', + source_uri: entry[:web_view_link] || google_drive_file_url(entry[:id]), + source_path: relative_path.presence || MaterialSyncExportPath.normalize_path(entry[:name]), + source_file_id: entry[:id], + filename: entry[:name], + content_type: entry[:mime_type], + file_downloader: lambda { + drive_client.download_to_tempfile(entry[:id], filename: entry[:name]) + }, + file_sha256: entry[:sha256_checksum], + export_path:, + profile: @source.profile, + tag: nil, + url: nil, + created_by_user: @source.created_by_user, + updated_by_user: @source.updated_by_user || @source.created_by_user } + end + + def candidate_normalized_source_key candidate + MaterialSyncSuppression.normalize_source_key(source_kind: candidate[:source_kind], + source_uri: candidate[:source_uri], + source_path: candidate[:source_path], + source_file_id: candidate[:source_file_id]) + end + + def suppression_for candidate + if candidate[:source_kind] == 'google_drive_file' && candidate[:source_path].present? + return MaterialSyncSuppressionMatcher.match_google_drive_candidate( + drive_file_id: candidate[:source_file_id], + relative_path: candidate[:source_path]) + end + + MaterialSyncSuppressionMatcher.match(source_kind: candidate[:source_kind], + source_uri: candidate[:source_uri], + source_path: candidate[:source_path], + source_file_id: candidate[:source_file_id]) + end + + def drive_client + @drive_client ||= GoogleDrive::ApiClient.new + end + + def google_drive_folder_id + google_drive_target_id.tap do |id| + if id.blank? + raise ArgumentError, 'google_drive_path source_file_id or source_uri is required' + end + end + end + + def google_drive_file_id + google_drive_target_id.tap do |id| + if id.blank? + raise ArgumentError, 'google_drive_file source_file_id or source_uri is required' + end + end + end + + def google_drive_target_id + @source.source_file_id.presence || drive_client.extract_file_id(@source.source_uri) + end + + def google_drive_file_url file_id + "https://drive.google.com/file/d/#{ file_id }/view" + end + + def close_candidate_tempfile candidate + tempfile = candidate[:file_tempfile] + return unless tempfile + + tempfile.close! unless tempfile.closed? + rescue StandardError + nil + end + + def log_result result + Rails.logger.info(material_sync_log(imported: result.imported, + updated: result.updated, + unchanged: result.unchanged, + suppressed: result.suppressed, + failed: result.failed)) + end + + def progress_log_scan_count? scanned_count + scanned_count == 1 || (scanned_count % 50).zero? + end + + def log_google_drive_progress folder_id, scanned_count, result, summary: false + Rails.logger.info( + material_sync_log(folder_id:, + scanned_count:, + imported: result.imported, + updated: result.updated, + unchanged: result.unchanged, + suppressed: result.suppressed, + failed: result.failed, + progress: summary ? 'summary' : 'scan')) + end + + def material_sync_log fields + { material_sync_source_id: @source.id, + material_sync_source_name: @source.name }.merge(fields).to_json + end +end diff --git a/backend/app/services/material_sync_suppression_matcher.rb b/backend/app/services/material_sync_suppression_matcher.rb new file mode 100644 index 0000000..74eb760 --- /dev/null +++ b/backend/app/services/material_sync_suppression_matcher.rb @@ -0,0 +1,68 @@ +class MaterialSyncSuppressionMatcher + def self.match source_kind:, + source_uri: nil, + drive_path: nil, + drive_file_id: nil, + source_path: nil, + source_file_id: nil + key = MaterialSyncSuppression.normalize_source_key(source_kind:, + source_uri:, + drive_path:, + drive_file_id:, + source_path:, + source_file_id:) + return nil if key.blank? + + MaterialSyncSuppression.find_by(normalized_source_key: key) + end + + def self.match_google_drive_candidate drive_file_id:, relative_path: + normalized_path = validate_relative_path!(relative_path) + + exact = [ + normalize_key('google_drive_file', drive_file_id), + normalize_key('google_drive_path', normalized_path), + normalize_key('legacy_drive_path', normalized_path), + ].compact + + suppression = + MaterialSyncSuppression.find_by(normalized_source_key: exact) + return suppression if suppression + + MaterialSyncSuppression + .where(source_kind: ['google_drive_path_prefix', 'legacy_drive_path_prefix']) + .where('drive_path = :path OR :path LIKE CONCAT(drive_path, "/%")', path: normalized_path) + .order(:id) + .first + end + + def self.suppressed?(...) + match(...).present? + end + + def self.validate_relative_path! relative_path + raw = relative_path.to_s.delete("\0").tr('\\', '/').strip + raise ArgumentError, 'relative_path is required' if raw.blank? + raise ArgumentError, 'relative_path must be relative' if raw.start_with?('/') + raise ArgumentError, 'relative_path must be relative' if raw.match?(/\A[A-Za-z]:\//) + if raw.start_with?('My Drive/', 'マイドライブ/') + raise ArgumentError, 'relative_path must be relative to source folder' + end + raise ArgumentError, 'relative_path must not contain //' if raw.include?('//') + raise ArgumentError, 'relative_path must not end with /' if raw.end_with?('/') + + parts = raw.split('/') + if parts.any? { |part| part.in?(['.', '..']) } + raise ArgumentError, 'relative_path must not contain dot segments' + end + + raw + end + + def self.normalize_key source_kind, value + return nil if value.blank? + + MaterialSyncSuppression.normalize_source_key(source_kind:, source_path: value, + source_file_id: value) + end +end diff --git a/backend/app/services/material_sync_suppression_registrar.rb b/backend/app/services/material_sync_suppression_registrar.rb new file mode 100644 index 0000000..ec9169a --- /dev/null +++ b/backend/app/services/material_sync_suppression_registrar.rb @@ -0,0 +1,51 @@ +class MaterialSyncSuppressionRegistrar + def self.create! attributes, created_by_user: + new(attributes, created_by_user:).create! + end + + def initialize attributes, created_by_user: + @attributes = attributes + @created_by_user = created_by_user + end + + def create! + suppression = nil + + MaterialSyncSuppression.transaction do + suppression = MaterialSyncSuppression.create!( + @attributes.merge(created_by_user: @created_by_user)) + discard_existing_materials!(suppression) + end + + suppression + end + + private + + def discard_existing_materials! suppression + matching_materials(suppression).find_each do |material| + MaterialVersionRecorder.ensure_snapshot!(material, created_by_user: @created_by_user) + material.discard! + MaterialVersionRecorder.record!(material:, + event_type: :discard, + created_by_user: @created_by_user) + end + end + + def matching_materials suppression + materials = Material.unscoped.kept + + case suppression.source_kind + when 'google_drive_path', 'legacy_drive_path' + materials.where(source_path: suppression.drive_path) + when 'google_drive_path_prefix', 'legacy_drive_path_prefix' + path = suppression.drive_path.to_s + materials.where( + 'source_path = :path OR source_path LIKE :prefix', + path:, + prefix: "#{ path }/%") + else + materials.where(normalized_source_key: suppression.normalized_source_key) + end + end +end diff --git a/backend/app/services/material_thumbnail_generator.rb b/backend/app/services/material_thumbnail_generator.rb new file mode 100644 index 0000000..176eb58 --- /dev/null +++ b/backend/app/services/material_thumbnail_generator.rb @@ -0,0 +1,126 @@ +# frozen_string_literal: true + +require 'mini_magick' +require 'open3' +require 'tempfile' + + +class MaterialThumbnailGenerator + SIZE = '180x180' + + class << self + def generate! material + new(material).generate! + end + end + + def initialize material + @material = material + @ffmpeg_stderr = [] + end + + def generate! + return log_result(:no_file) unless @material.file.attached? + return log_result(:unsupported_content_type) unless image? || video? + + @material.file.blob.open do |file| + thumbnail = image? ? image_thumbnail(file.path) : video_thumbnail(file.path) + return log_result(:generation_failed) unless thumbnail + + return attach_thumbnail(thumbnail) + end + rescue ActiveStorage::FileNotFoundError => e + log_result(:file_not_found, error: e) + rescue MiniMagick::Error => e + log_result(:mini_magick_error, error: e) + rescue ArgumentError, StandardError => e + log_result(:generation_failed, error: e) + end + + private + + def image? = content_type.start_with?('image/') + + def video? = content_type.start_with?('video/') + + def content_type + return nil unless @material.file.attached? + + @material.file.blob.content_type.to_s + end + + def image_thumbnail path + image = MiniMagick::Image.open(path) + image.resize(SIZE) + image.format('jpg') + image + end + + def video_thumbnail path + [1, 0].each do |seconds| + tempfile = Tempfile.new(['material-thumbnail', '.jpg']) + tempfile.close + ok = extract_video_frame(path, tempfile.path, seconds) + next unless ok && File.size?(tempfile.path) + + return image_thumbnail(tempfile.path) + ensure + tempfile&.unlink + end + + nil + end + + def extract_video_frame input_path, output_path, seconds + _stdout, stderr, status = + Open3.capture3('ffmpeg', + '-y', + '-ss', seconds.to_s, + '-i', input_path, + '-frames:v', '1', + '-f', 'image2', + output_path) + @ffmpeg_stderr << stderr if stderr.present? + status.success? + rescue Errno::ENOENT => e + @ffmpeg_stderr << "ffmpeg unavailable: #{ e.message }" + false + end + + def attach_thumbnail image + blob = nil + File.open(image.path) do |io| + blob = ActiveStorage::Blob.create_and_upload!( + io:, + filename: 'material-thumbnail.jpg', + content_type: 'image/jpeg') + end + @material.thumbnail.attach(blob) + log_result(:attached) + rescue StandardError => e + blob&.purge_later + log_result(:attach_failed, error: e) + end + + def log_result result, error: nil + log_payload = { material_id: @material.id, + file_blob_id: file_blob_id, + content_type:, + result:, + error_class: error&.class&.name, + error_message: error&.message, + ffmpeg_stderr: @ffmpeg_stderr.join("\n").presence } + if [:attached, :no_file, :unsupported_content_type].include?(result) + Rails.logger.info("Material thumbnail generation: #{ log_payload.to_json }") + else + Rails.logger.warn("Material thumbnail generation: #{ log_payload.to_json }") + end + result + end + + def file_blob_id + return nil unless @material.file.attached? + + @material.file.blob.id + end +end diff --git a/backend/app/services/material_version_recorder.rb b/backend/app/services/material_version_recorder.rb new file mode 100644 index 0000000..8f268f7 --- /dev/null +++ b/backend/app/services/material_version_recorder.rb @@ -0,0 +1,73 @@ +class MaterialVersionRecorder < VersionRecorder + EVENT_TYPES = ['create', 'update', 'discard', 'restore'].freeze + + def self.record! material:, event_type:, created_by_user:, file_snapshot: nil + new(material:, event_type:, created_by_user:, file_snapshot:).record! + end + + def initialize material:, event_type:, created_by_user:, file_snapshot: nil + @file_snapshot = file_snapshot + + super(record: material, event_type:, created_by_user:) + end + + def self.ensure_snapshot! material, created_by_user: + return if material.material_versions.exists? + + record!(material:, event_type: :create, + created_by_user: material.created_by_user || created_by_user) + end + + private + + def version_class = MaterialVersion + def version_association = :material_versions + def record_key = :material + + def snapshot_attributes + blob = @record.file.attached? ? @record.file.blob : nil + file_snapshot = build_file_snapshot(blob) + + { url: @record.url, + parent: @record.parent, + tag: @record.tag, + tag_name: @record.tag&.name, + tag_category: @record.tag&.category, + source_kind: @record.source_kind, + source_uri: @record.source_uri, + source_path: @record.source_path, + source_file_id: @record.source_file_id, + normalized_source_key: @record.normalized_source_key, + export_paths_json: @record.snapshot_export_paths, + discarded_at: @record.discarded_at, + file_blob_id: file_snapshot[:file_blob_id], + file_filename: file_snapshot[:file_filename], + file_content_type: file_snapshot[:file_content_type], + file_byte_size: file_snapshot[:file_byte_size], + file_checksum: file_snapshot[:file_checksum], + file_sha256: file_snapshot[:file_sha256] } + end + + def build_file_snapshot blob + return @file_snapshot if @file_snapshot + return empty_file_snapshot unless blob + + { file_blob_id: blob.id, + file_filename: blob.filename.to_s, + file_content_type: blob.content_type, + file_byte_size: blob.byte_size, + file_checksum: blob.checksum, + file_sha256: MaterialFileSha256.metadata_sha256(blob) } + end + + def empty_file_snapshot + { file_blob_id: nil, + file_filename: nil, + file_content_type: nil, + file_byte_size: nil, + file_checksum: nil, + file_sha256: nil } + end + + def event_types = self.class::EVENT_TYPES +end diff --git a/backend/app/services/material_zip_exporter.rb b/backend/app/services/material_zip_exporter.rb new file mode 100644 index 0000000..9ad337a --- /dev/null +++ b/backend/app/services/material_zip_exporter.rb @@ -0,0 +1,148 @@ +require 'stringio' +require 'zlib' + +# Initial implementation keeps every file payload and the final ZIP in memory. +# Keep this service boundary stable so job/cached export paths can replace it later. +class MaterialZipExporter + Entry = Struct.new(:path, :data, :mtime, keyword_init: true) + MissingFile = Struct.new(:material_id, :export_path, :blob_id, :filename, keyword_init: true) + + class EmptyExportError < StandardError; end + class DuplicatePathError < StandardError; end + class MissingFileError < StandardError + attr_reader :missing_files + + def initialize missing_files + @missing_files = missing_files + + super("Missing files: #{missing_files.map(&:export_path).join(', ')}") + end + end + + def initialize profile: 'legacy_drive', tag_id: nil + @profile = profile.presence || 'legacy_drive' + @tag_id = tag_id.presence + end + + def export + entries = build_entries + raise EmptyExportError if entries.empty? + + ZipWriter.write(entries) + end + + private + + def build_entries + rows = MaterialExportItem + .enabled + .includes(material: { file_attachment: :blob }) + .joins(:material) + .merge(Material.kept) + .where(profile: @profile) + .order(:export_path) + + rows = rows.where(materials: { tag_id: @tag_id }) if @tag_id + + missing_files = [] + + entries = rows.filter_map do |item| + material = item.material + next unless material.file.attached? + + data = download_blob(item, missing_files) + next unless data + + Entry.new(path: item.export_path, + data:, + mtime: material.updated_at || Time.current) + end + + raise MissingFileError.new(missing_files) if missing_files.any? + + paths = entries.map(&:path) + duplicated = paths.find { |path| paths.count(path) > 1 } + raise DuplicatePathError, duplicated if duplicated + + entries + end + + def download_blob item, missing_files + blob = item.material.file.blob + + blob.download + rescue ActiveStorage::FileNotFoundError + missing_files << MissingFile.new( + material_id: item.material_id, + export_path: item.export_path, + blob_id: blob.id, + filename: blob.filename.to_s, + ) + nil + end + + class ZipWriter + VERSION_NEEDED = 20 + GP_FLAG = 0x0800 + COMPRESSION_STORE = 0 + + def self.write entries + new(entries).write + end + + def initialize entries + @entries = entries + @central_directory = [] + end + + def write + io = StringIO.new(''.b) + + @entries.each do |entry| + write_entry(io, entry) + end + + central_start = io.pos + @central_directory.each { |header| io.write(header) } + central_size = io.pos - central_start + + io.write([0x06054b50, 0, 0, @entries.size, @entries.size, + central_size, central_start, 0].pack('VvvvvVVv')) + io.string + end + + private + + def write_entry io, entry + path = entry.path.b + data = entry.data.b + crc32 = Zlib.crc32(data) + dos_time, dos_date = dos_timestamp(entry.mtime) + offset = io.pos + + local_header = [0x04034b50, VERSION_NEEDED, GP_FLAG, COMPRESSION_STORE, + dos_time, dos_date, crc32, data.bytesize, data.bytesize, + path.bytesize, 0].pack('VvvvvvVVVvv') + + io.write(local_header) + io.write(path) + io.write(data) + + @central_directory << central_header(path:, crc32:, size: data.bytesize, + dos_time:, dos_date:, offset:) + end + + def central_header path:, crc32:, size:, dos_time:, dos_date:, offset: + [0x02014b50, VERSION_NEEDED, VERSION_NEEDED, GP_FLAG, COMPRESSION_STORE, + dos_time, dos_date, crc32, size, size, path.bytesize, 0, 0, 0, 0, 0, + offset].pack('VvvvvvvVVVvvvvvVV') + path + end + + def dos_timestamp time + local = time.to_time + dos_time = (local.hour << 11) | (local.min << 5) | (local.sec / 2) + dos_date = ((local.year - 1980) << 9) | (local.month << 5) | local.day + [dos_time, dos_date] + end + end +end diff --git a/backend/app/services/version_recorder.rb b/backend/app/services/version_recorder.rb index 8175dba..e289b61 100644 --- a/backend/app/services/version_recorder.rb +++ b/backend/app/services/version_recorder.rb @@ -73,7 +73,7 @@ class VersionRecorder end def validate_event_type! - return if EVENT_TYPES.include?(@event_type) + return if event_types.include?(@event_type) raise ArgumentError, "Invalid event_type: #{ @event_type }" end @@ -84,4 +84,5 @@ class VersionRecorder def snapshot_attributes = raise NotImplementedError def record_class = @record.class + def event_types = self.class::EVENT_TYPES end diff --git a/backend/config/routes.rb b/backend/config/routes.rb index f30959b..7888289 100644 --- a/backend/config/routes.rb +++ b/backend/config/routes.rb @@ -113,5 +113,8 @@ Rails.application.routes.draw do resources :skip_events, controller: :theatre_skip_events, only: [:index] end + get 'materials/download.zip', to: 'materials#download' + get 'materials/versions', to: 'material_versions#index' + resources :material_sync_suppressions, path: 'materials/suppressions', only: [:index, :create] resources :materials, only: [:index, :show, :create, :update, :destroy] end diff --git a/backend/db/migrate/20260623000000_enhance_material_management.rb b/backend/db/migrate/20260623000000_enhance_material_management.rb new file mode 100644 index 0000000..9c0321b --- /dev/null +++ b/backend/db/migrate/20260623000000_enhance_material_management.rb @@ -0,0 +1,88 @@ +class EnhanceMaterialManagement < ActiveRecord::Migration[8.0] + def up + change_table :materials, bulk: true do |t| + t.integer :version_no, null: false, default: 1 + end + + change_table :material_versions, bulk: true do |t| + t.string :event_type + t.string :tag_name + t.string :tag_category + t.json :export_paths_json + t.bigint :file_blob_id + t.string :file_filename + t.string :file_content_type + t.bigint :file_byte_size + t.string :file_checksum + t.string :file_sha256 + end + + execute <<~SQL.squish + UPDATE material_versions + SET event_type = CASE + WHEN version_no = 1 THEN 'create' + ELSE 'update' + END + WHERE event_type IS NULL + SQL + + change_column_null :material_versions, :event_type, false + add_index :material_versions, :file_blob_id + + add_check_constraint :material_versions, + "event_type IN ('create', 'update', 'discard', 'restore')", + name: 'material_versions_event_type_valid' + + create_table :material_export_items do |t| + t.references :material, null: false, foreign_key: true + t.string :profile, null: false, default: 'legacy_drive' + t.string :export_path, null: false + t.boolean :enabled, null: false, default: true + t.references :created_by_user, foreign_key: { to_table: :users } + t.timestamps + + t.index [:profile, :export_path], unique: true + t.index [:material_id, :profile], unique: true + end + + create_table :material_import_blocks do |t| + t.string :match_kind, null: false + t.string :sha256 + t.string :external_path_pattern + t.string :reason, null: false + t.text :note + t.references :created_by_user, foreign_key: { to_table: :users } + t.timestamps + end + + execute <<~SQL.squish + UPDATE materials + SET version_no = COALESCE( + (SELECT MAX(material_versions.version_no) + FROM material_versions + WHERE material_versions.material_id = materials.id), + 1) + SQL + end + + def down + drop_table :material_import_blocks + drop_table :material_export_items + + remove_check_constraint :material_versions, name: 'material_versions_event_type_valid' + + remove_index :material_versions, :file_blob_id + remove_column :material_versions, :event_type + remove_column :material_versions, :tag_name + remove_column :material_versions, :tag_category + remove_column :material_versions, :export_paths_json + remove_column :material_versions, :file_blob_id + remove_column :material_versions, :file_filename + remove_column :material_versions, :file_content_type + remove_column :material_versions, :file_byte_size + remove_column :material_versions, :file_checksum + remove_column :material_versions, :file_sha256 + + remove_column :materials, :version_no + end +end diff --git a/backend/db/migrate/20260625000000_reconcile_material_sync_suppressions.rb b/backend/db/migrate/20260625000000_reconcile_material_sync_suppressions.rb new file mode 100644 index 0000000..0b1f841 --- /dev/null +++ b/backend/db/migrate/20260625000000_reconcile_material_sync_suppressions.rb @@ -0,0 +1,118 @@ +class ReconcileMaterialSyncSuppressions < ActiveRecord::Migration[8.0] + MATERIAL_VERSION_EVENT_CONSTRAINT = 'material_versions_event_type_valid' + + def up + ensure_material_sync_suppressions! + ensure_material_source_columns! + ensure_material_active_source_key! + remove_material_file_suppression! + remove_material_version_file_suppression! + reconcile_material_version_event_constraint! + end + + def down + raise ActiveRecord::IrreversibleMigration + end + + private + + def ensure_material_sync_suppressions! + return if table_exists?(:material_sync_suppressions) + + create_table :material_sync_suppressions do |t| + t.string :source_kind, null: false + t.string :source_uri + t.string :drive_path + t.string :drive_file_id + t.string :normalized_source_key, null: false + t.string :reason, null: false + t.references :created_by_user, foreign_key: { to_table: :users } + t.timestamps + + t.index :normalized_source_key, unique: true + t.index :source_kind + t.index :drive_file_id + end + end + + def ensure_material_source_columns! + add_column :materials, :source_kind, :string unless column_exists?(:materials, :source_kind) + add_column :materials, :source_uri, :string unless column_exists?(:materials, :source_uri) + add_column :materials, :source_path, :string unless column_exists?(:materials, :source_path) + unless column_exists?(:materials, :source_file_id) + add_column :materials, :source_file_id, :string + end + unless column_exists?(:materials, :normalized_source_key) + add_column :materials, :normalized_source_key, :string + end + + if index_exists?(:materials, :normalized_source_key) + remove_index :materials, column: :normalized_source_key + end + end + + def ensure_material_active_source_key! + unless column_exists?(:materials, :active_normalized_source_key) + change_table :materials do |t| + t.virtual :active_normalized_source_key, + type: :string, + as: 'IF(discarded_at IS NULL, normalized_source_key, NULL)', + stored: false + end + end + + return if index_exists?(:materials, + :active_normalized_source_key, + name: 'index_materials_on_active_normalized_source_key') + + add_index :materials, + :active_normalized_source_key, + unique: true, + name: 'index_materials_on_active_normalized_source_key' + end + + def remove_material_file_suppression! + if foreign_key_exists?(:materials, :users, column: :file_suppressed_by_user_id) + remove_foreign_key :materials, column: :file_suppressed_by_user_id + end + + if index_exists?(:materials, + :file_suppressed_by_user_id, + name: 'index_materials_on_file_suppressed_by_user_id') + remove_index :materials, name: 'index_materials_on_file_suppressed_by_user_id' + end + + remove_column :materials, :file_suppressed_at if column_exists?( + :materials, + :file_suppressed_at) + remove_column :materials, :file_suppressed_by_user_id if column_exists?( + :materials, + :file_suppressed_by_user_id) + remove_column :materials, :file_suppression_reason if column_exists?( + :materials, + :file_suppression_reason) + end + + def remove_material_version_file_suppression! + remove_column :material_versions, :file_suppressed_at if column_exists?( + :material_versions, + :file_suppressed_at) + remove_column :material_versions, :file_suppression_reason if column_exists?( + :material_versions, + :file_suppression_reason) + end + + def reconcile_material_version_event_constraint! + constraint = check_constraints(:material_versions).find do |candidate| + candidate.name == MATERIAL_VERSION_EVENT_CONSTRAINT + end + return if constraint && !constraint.expression.include?('suppress') + + if constraint + remove_check_constraint :material_versions, name: MATERIAL_VERSION_EVENT_CONSTRAINT + end + add_check_constraint :material_versions, + "event_type IN ('create', 'update', 'discard', 'restore')", + name: MATERIAL_VERSION_EVENT_CONSTRAINT + end +end diff --git a/backend/db/migrate/20260625010000_create_material_sync_sources.rb b/backend/db/migrate/20260625010000_create_material_sync_sources.rb new file mode 100644 index 0000000..722e81b --- /dev/null +++ b/backend/db/migrate/20260625010000_create_material_sync_sources.rb @@ -0,0 +1,22 @@ +class CreateMaterialSyncSources < ActiveRecord::Migration[8.0] + def change + create_table :material_sync_sources do |t| + t.string :name, null: false + t.string :source_kind, null: false + t.string :source_uri + t.string :source_path + t.string :source_file_id + t.string :profile, null: false, default: 'legacy_drive' + t.boolean :enabled, null: false, default: true + t.string :default_tag_name + t.string :export_path_prefix + t.datetime :last_synced_at + t.references :created_by_user, foreign_key: { to_table: :users } + t.references :updated_by_user, foreign_key: { to_table: :users } + t.timestamps + + t.index :enabled + t.index :source_kind + end + end +end diff --git a/backend/db/migrate/20260626000000_add_source_snapshot_to_material_versions.rb b/backend/db/migrate/20260626000000_add_source_snapshot_to_material_versions.rb new file mode 100644 index 0000000..a6d7111 --- /dev/null +++ b/backend/db/migrate/20260626000000_add_source_snapshot_to_material_versions.rb @@ -0,0 +1,19 @@ +class AddSourceSnapshotToMaterialVersions < ActiveRecord::Migration[8.0] + def change + unless column_exists?(:material_versions, :source_kind) + add_column :material_versions, :source_kind, :string + end + unless column_exists?(:material_versions, :source_uri) + add_column :material_versions, :source_uri, :string + end + unless column_exists?(:material_versions, :source_path) + add_column :material_versions, :source_path, :string + end + unless column_exists?(:material_versions, :source_file_id) + add_column :material_versions, :source_file_id, :string + end + unless column_exists?(:material_versions, :normalized_source_key) + add_column :material_versions, :normalized_source_key, :string + end + end +end diff --git a/backend/db/migrate/20260626010000_backfill_initial_material_versions.rb b/backend/db/migrate/20260626010000_backfill_initial_material_versions.rb new file mode 100644 index 0000000..45b95a5 --- /dev/null +++ b/backend/db/migrate/20260626010000_backfill_initial_material_versions.rb @@ -0,0 +1,171 @@ +require 'json' + +class BackfillInitialMaterialVersions < ActiveRecord::Migration[8.0] + class MigrationMaterial < ActiveRecord::Base + self.table_name = 'materials' + end + + class MigrationMaterialExportItem < ActiveRecord::Base + self.table_name = 'material_export_items' + end + + class MigrationMaterialVersion < ActiveRecord::Base + self.table_name = 'material_versions' + end + + class MigrationStorageAttachment < ActiveRecord::Base + self.table_name = 'active_storage_attachments' + end + + class MigrationStorageBlob < ActiveRecord::Base + self.table_name = 'active_storage_blobs' + end + + class MigrationTag < ActiveRecord::Base + self.table_name = 'tags' + end + + class MigrationTagName < ActiveRecord::Base + self.table_name = 'tag_names' + end + + def up + say_with_time 'Backfilling initial material versions' do + materials_without_versions.find_in_batches(batch_size: 500) do |materials| + material_ids = materials.map(&:id) + tags_by_id = load_tags(materials.map(&:tag_id).compact.uniq) + export_paths_by_material_id = load_export_paths(material_ids) + file_snapshots_by_material_id = load_file_snapshots(material_ids) + + rows = materials.map do |material| + tag = tags_by_id[material.tag_id] + file_snapshot = file_snapshots_by_material_id[material.id] || empty_file_snapshot + + { material_id: material.id, + version_no: 1, + event_type: 'create', + url: material.url, + parent_id: material.parent_id, + tag_id: material.tag_id, + tag_name: tag&.fetch(:name, nil), + tag_category: tag&.fetch(:category, nil), + source_kind: material.source_kind, + source_uri: material.source_uri, + source_path: material.source_path, + source_file_id: material.source_file_id, + normalized_source_key: material.normalized_source_key, + export_paths_json: export_paths_by_material_id[material.id] || { }, + discarded_at: material.discarded_at, + file_blob_id: file_snapshot[:file_blob_id], + file_filename: file_snapshot[:file_filename], + file_content_type: file_snapshot[:file_content_type], + file_byte_size: file_snapshot[:file_byte_size], + file_checksum: file_snapshot[:file_checksum], + file_sha256: file_snapshot[:file_sha256], + created_by_user_id: material.created_by_user_id, + updated_by_user_id: material.updated_by_user_id, + created_at: material.created_at, + updated_at: material.created_at } + end + + MigrationMaterialVersion.insert_all!(rows) if rows.present? + end + end + + execute <<~SQL.squish + UPDATE materials + SET version_no = COALESCE( + ( + SELECT MAX(material_versions.version_no) + FROM material_versions + WHERE material_versions.material_id = materials.id + ), + 1 + ) + SQL + end + + def down + raise ActiveRecord::IrreversibleMigration + end + + private + + def materials_without_versions + MigrationMaterial.where.not(id: MigrationMaterialVersion.select(:material_id)) + end + + def load_tags tag_ids + return { } if tag_ids.empty? + + tag_name_ids_by_tag_id = MigrationTag.where(id: tag_ids).pluck(:id, :tag_name_id).to_h + tag_names_by_id = + MigrationTagName + .where(id: tag_name_ids_by_tag_id.values.uniq) + .pluck(:id, :name) + .to_h + categories_by_tag_id = MigrationTag.where(id: tag_ids).pluck(:id, :category).to_h + + tag_ids.each_with_object({ }) do |tag_id, hash| + hash[tag_id] = { name: tag_names_by_id[tag_name_ids_by_tag_id[tag_id]], + category: categories_by_tag_id[tag_id] } + end + end + + def load_export_paths material_ids + MigrationMaterialExportItem + .where(material_id: material_ids) + .order(:material_id, :profile) + .pluck(:material_id, :profile, :export_path) + .each_with_object({ }) do |(material_id, profile, export_path), hash| + hash[material_id] ||= { } + hash[material_id][profile] = export_path + end + end + + def load_file_snapshots material_ids + attachments_by_material_id = + MigrationStorageAttachment + .where(record_type: 'Material', name: 'file', record_id: material_ids) + .pluck(:record_id, :blob_id) + .to_h + + blobs_by_id = + MigrationStorageBlob + .where(id: attachments_by_material_id.values.uniq) + .index_by(&:id) + + attachments_by_material_id.each_with_object({ }) do |(material_id, blob_id), hash| + blob = blobs_by_id[blob_id] + next unless blob + + hash[material_id] = { + file_blob_id: blob.id, + file_filename: blob.filename.to_s, + file_content_type: blob.content_type, + file_byte_size: blob.byte_size, + file_checksum: blob.checksum, + file_sha256: blob_metadata(blob)['sha256'] + } + end + end + + def empty_file_snapshot + { file_blob_id: nil, + file_filename: nil, + file_content_type: nil, + file_byte_size: nil, + file_checksum: nil, + file_sha256: nil } + end + + def blob_metadata blob + metadata = blob.metadata + return metadata if metadata.is_a?(Hash) + return { } if metadata.blank? + + JSON.parse(metadata) + rescue JSON::ParserError + { } + end +end diff --git a/backend/db/schema.rb b/backend/db/schema.rb index 8c988e5..dbff31a 100644 --- a/backend/db/schema.rb +++ b/backend/db/schema.rb @@ -10,7 +10,7 @@ # # It's strongly recommended that you check this file into your version control system. -ActiveRecord::Schema[8.0].define(version: 2026_06_21_000000) do +ActiveRecord::Schema[8.0].define(version: 2026_06_26_010000) do create_table "active_storage_attachments", charset: "utf8mb4", collation: "utf8mb4_0900_ai_ci", force: :cascade do |t| t.string "name", null: false t.string "record_type", null: false @@ -130,6 +130,69 @@ ActiveRecord::Schema[8.0].define(version: 2026_06_21_000000) do t.index ["ip_address"], name: "index_ip_addresses_on_ip_address", unique: true end + create_table "material_export_items", charset: "utf8mb4", collation: "utf8mb4_0900_ai_ci", force: :cascade do |t| + t.bigint "material_id", null: false + t.string "profile", default: "legacy_drive", null: false + t.string "export_path", null: false + t.boolean "enabled", default: true, null: false + t.bigint "created_by_user_id" + t.datetime "created_at", null: false + t.datetime "updated_at", null: false + t.index ["created_by_user_id"], name: "index_material_export_items_on_created_by_user_id" + t.index ["material_id", "profile"], name: "index_material_export_items_on_material_id_and_profile", unique: true + t.index ["material_id"], name: "index_material_export_items_on_material_id" + t.index ["profile", "export_path"], name: "index_material_export_items_on_profile_and_export_path", unique: true + end + + create_table "material_import_blocks", charset: "utf8mb4", collation: "utf8mb4_0900_ai_ci", force: :cascade do |t| + t.string "match_kind", null: false + t.string "sha256" + t.string "external_path_pattern" + t.string "reason", null: false + t.text "note" + t.bigint "created_by_user_id" + t.datetime "created_at", null: false + t.datetime "updated_at", null: false + t.index ["created_by_user_id"], name: "index_material_import_blocks_on_created_by_user_id" + end + + create_table "material_sync_sources", charset: "utf8mb4", collation: "utf8mb4_0900_ai_ci", force: :cascade do |t| + t.string "name", null: false + t.string "source_kind", null: false + t.string "source_uri" + t.string "source_path" + t.string "source_file_id" + t.string "profile", default: "legacy_drive", null: false + t.boolean "enabled", default: true, null: false + t.string "default_tag_name" + t.string "export_path_prefix" + t.datetime "last_synced_at" + t.bigint "created_by_user_id" + t.bigint "updated_by_user_id" + t.datetime "created_at", null: false + t.datetime "updated_at", null: false + t.index ["created_by_user_id"], name: "index_material_sync_sources_on_created_by_user_id" + t.index ["enabled"], name: "index_material_sync_sources_on_enabled" + t.index ["source_kind"], name: "index_material_sync_sources_on_source_kind" + t.index ["updated_by_user_id"], name: "index_material_sync_sources_on_updated_by_user_id" + end + + create_table "material_sync_suppressions", charset: "utf8mb4", collation: "utf8mb4_0900_ai_ci", force: :cascade do |t| + t.string "source_kind", null: false + t.string "source_uri" + t.string "drive_path" + t.string "drive_file_id" + t.string "normalized_source_key", null: false + t.string "reason", null: false + t.bigint "created_by_user_id" + t.datetime "created_at", null: false + t.datetime "updated_at", null: false + t.index ["created_by_user_id"], name: "index_material_sync_suppressions_on_created_by_user_id" + t.index ["drive_file_id"], name: "index_material_sync_suppressions_on_drive_file_id" + t.index ["normalized_source_key"], name: "index_material_sync_suppressions_on_normalized_source_key", unique: true + t.index ["source_kind"], name: "index_material_sync_suppressions_on_source_kind" + end + create_table "material_versions", charset: "utf8mb4", collation: "utf8mb4_0900_ai_ci", force: :cascade do |t| t.bigint "material_id", null: false t.integer "version_no", null: false @@ -141,14 +204,31 @@ ActiveRecord::Schema[8.0].define(version: 2026_06_21_000000) do t.datetime "created_at", null: false t.datetime "updated_at", null: false t.datetime "discarded_at" + t.string "event_type", null: false + t.string "tag_name" + t.string "tag_category" + t.json "export_paths_json" + t.bigint "file_blob_id" + t.string "file_filename" + t.string "file_content_type" + t.bigint "file_byte_size" + t.string "file_checksum" + t.string "file_sha256" + t.string "source_kind" + t.string "source_uri" + t.string "source_path" + t.string "source_file_id" + t.string "normalized_source_key" t.index ["created_by_user_id"], name: "index_material_versions_on_created_by_user_id" t.index ["discarded_at"], name: "index_material_versions_on_discarded_at" + t.index ["file_blob_id"], name: "index_material_versions_on_file_blob_id" t.index ["material_id", "version_no"], name: "index_material_versions_on_material_id_and_version_no", unique: true t.index ["material_id"], name: "index_material_versions_on_material_id" t.index ["parent_id"], name: "index_material_versions_on_parent_id" t.index ["tag_id"], name: "index_material_versions_on_tag_id" t.index ["updated_by_user_id"], name: "index_material_versions_on_updated_by_user_id" t.index ["url"], name: "index_material_versions_on_url" + t.check_constraint "`event_type` in (_utf8mb4'create',_utf8mb4'update',_utf8mb4'discard',_utf8mb4'restore')", name: "material_versions_event_type_valid" end create_table "materials", charset: "utf8mb4", collation: "utf8mb4_0900_ai_ci", force: :cascade do |t| @@ -161,6 +241,14 @@ ActiveRecord::Schema[8.0].define(version: 2026_06_21_000000) do t.datetime "updated_at", null: false t.datetime "discarded_at" t.virtual "active_url", type: :string, as: "if((`discarded_at` is null),`url`,NULL)" + t.integer "version_no", default: 1, null: false + t.string "source_kind" + t.string "source_uri" + t.string "source_path" + t.string "source_file_id" + t.string "normalized_source_key" + t.virtual "active_normalized_source_key", type: :string, as: "if((`discarded_at` is null),`normalized_source_key`,NULL)" + t.index ["active_normalized_source_key"], name: "index_materials_on_active_normalized_source_key", unique: true t.index ["active_url"], name: "index_materials_on_active_url", unique: true t.index ["created_by_user_id"], name: "index_materials_on_created_by_user_id" t.index ["discarded_at"], name: "index_materials_on_discarded_at" @@ -567,6 +655,12 @@ ActiveRecord::Schema[8.0].define(version: 2026_06_21_000000) do add_foreign_key "gekanator_question_suggestions", "users" add_foreign_key "gekanator_questions", "gekanator_question_suggestions" add_foreign_key "gekanator_questions", "users", column: "created_by_id" + add_foreign_key "material_export_items", "materials" + add_foreign_key "material_export_items", "users", column: "created_by_user_id" + add_foreign_key "material_import_blocks", "users", column: "created_by_user_id" + add_foreign_key "material_sync_sources", "users", column: "created_by_user_id" + add_foreign_key "material_sync_sources", "users", column: "updated_by_user_id" + add_foreign_key "material_sync_suppressions", "users", column: "created_by_user_id" add_foreign_key "material_versions", "materials" add_foreign_key "material_versions", "materials", column: "parent_id" add_foreign_key "material_versions", "tags" diff --git a/backend/db/seeds.rb b/backend/db/seeds.rb index 4fbd6ed..067d073 100644 --- a/backend/db/seeds.rb +++ b/backend/db/seeds.rb @@ -7,3 +7,21 @@ # ["Action", "Comedy", "Drama", "Horror"].each do |genre_name| # MovieGenre.find_or_create_by!(name: genre_name) # end + +material_sync_source_uri = ENV['MATERIAL_SYNC_SOURCE_URI'] +material_sync_source_file_id = ENV['MATERIAL_SYNC_SOURCE_FILE_ID'] + +if material_sync_source_uri.present? || material_sync_source_file_id.present? + source = MaterialSyncSource.find_or_initialize_by( + name: ENV.fetch('MATERIAL_SYNC_SOURCE_NAME', 'Legacy URI material sync')) + source.assign_attributes( + source_kind: ENV.fetch('MATERIAL_SYNC_SOURCE_KIND', 'uri'), + source_uri: material_sync_source_uri, + source_path: ENV['MATERIAL_SYNC_SOURCE_PATH'], + source_file_id: material_sync_source_file_id, + profile: ENV.fetch('MATERIAL_SYNC_SOURCE_PROFILE', 'legacy_drive'), + enabled: ENV.fetch('MATERIAL_SYNC_SOURCE_ENABLED', 'true') != 'false', + default_tag_name: ENV['MATERIAL_SYNC_SOURCE_TAG_NAME'], + export_path_prefix: ENV['MATERIAL_SYNC_EXPORT_PATH_PREFIX']) + source.save! +end diff --git a/backend/lib/tasks/sync_materials.rake b/backend/lib/tasks/sync_materials.rake new file mode 100644 index 0000000..81d74b7 --- /dev/null +++ b/backend/lib/tasks/sync_materials.rake @@ -0,0 +1,51 @@ +namespace :materials do + desc '素材同期' + task sync: :environment do + result = MaterialSyncRunner.sync_enabled! + message = [ + "materials:sync imported=#{ result.imported }", + "updated=#{ result.updated }", + "unchanged=#{ result.unchanged }", + "suppressed=#{ result.suppressed }", + "failed=#{ result.failed }" + ].join(' ') + + Rails.logger.info(message) + puts message + + result.errors.each do |error| + Rails.logger.warn(error.to_json) + warn error.to_json + end + end + + namespace :thumbnails do + desc '素材サムネールを補完' + task backfill: :environment do + counts = Hash.new(0) + + Material + .with_attached_file + .with_attached_thumbnail + .find_each do |material| + next unless material.file.attached? + next if material.thumbnail.attached? + + result = MaterialThumbnailGenerator.generate!(material) + counts[result] += 1 + end + + message = + "materials:thumbnails:backfill " \ + "attached=#{ counts[:attached] } " \ + "no_file=#{ counts[:no_file] } " \ + "unsupported_content_type=#{ counts[:unsupported_content_type] } " \ + "generation_failed=#{ counts[:generation_failed] } " \ + "attach_failed=#{ counts[:attach_failed] } " \ + "file_not_found=#{ counts[:file_not_found] } " \ + "mini_magick_error=#{ counts[:mini_magick_error] }" + Rails.logger.info(message) + puts message + end + end +end diff --git a/backend/spec/models/material_export_item_spec.rb b/backend/spec/models/material_export_item_spec.rb new file mode 100644 index 0000000..d98e225 --- /dev/null +++ b/backend/spec/models/material_export_item_spec.rb @@ -0,0 +1,57 @@ +require 'rails_helper' + +RSpec.describe MaterialExportItem, type: :model do + let(:user) { create(:user, :member) } + let(:tag) { Tag.create!(tag_name: TagName.create!(name: 'export_item'), category: :material) } + let(:material) do + Material.create!(tag:, url: 'https://example.com/material', + created_by_user: user, updated_by_user: user) + end + + it 'rejects blank export_path' do + item = described_class.new(material:, profile: 'legacy_drive', export_path: '') + + expect(item).not_to be_valid + expect(item.errors[:export_path]).to be_present + end + + it 'rejects absolute export_path' do + item = described_class.new(material:, profile: 'legacy_drive', + export_path: '/素材/a.png') + + expect(item).not_to be_valid + expect(item.errors[:export_path]).to be_present + end + + it 'rejects parent traversal export_path' do + item = described_class.new(material:, profile: 'legacy_drive', + export_path: '素材/../a.png') + + expect(item).not_to be_valid + expect(item.errors[:export_path]).to be_present + end + + it 'rejects double slash export_path' do + item = described_class.new(material:, profile: 'legacy_drive', + export_path: '素材//a.png') + + expect(item).not_to be_valid + expect(item.errors[:export_path]).to be_present + end + + it 'rejects dot segment export_path' do + item = described_class.new(material:, profile: 'legacy_drive', + export_path: './素材/a.png') + + expect(item).not_to be_valid + expect(item.errors[:export_path]).to be_present + end + + it 'rejects trailing slash export_path' do + item = described_class.new(material:, profile: 'legacy_drive', + export_path: '素材/a/') + + expect(item).not_to be_valid + expect(item.errors[:export_path]).to be_present + end +end diff --git a/backend/spec/models/material_spec.rb b/backend/spec/models/material_spec.rb new file mode 100644 index 0000000..f1f2d15 --- /dev/null +++ b/backend/spec/models/material_spec.rb @@ -0,0 +1,19 @@ +require 'rails_helper' + +RSpec.describe Material, type: :model do + let(:user) { create(:user, :member) } + + it 'allows only material source kinds, not suppression prefix kinds' do + material = described_class.new(url: 'https://example.com/material', + source_kind: 'google_drive_path_prefix', + source_path: '素材/危険', + created_by_user: user, + updated_by_user: user) + + expect(material).not_to be_valid + expect(material.errors[:source_kind]).to be_present + + material.source_kind = 'google_drive_path' + expect(material).to be_valid + end +end diff --git a/backend/spec/requests/materials_spec.rb b/backend/spec/requests/materials_spec.rb index d4085ec..fcabde4 100644 --- a/backend/spec/requests/materials_spec.rb +++ b/backend/spec/requests/materials_spec.rb @@ -1,7 +1,10 @@ require 'rails_helper' RSpec.describe 'Materials API', type: :request do + include ActiveJob::TestHelper + let!(:member_user) { create(:user, :member) } + let!(:admin_user) { create(:user, :admin) } let!(:guest_user) { create(:user) } def dummy_upload(filename: 'dummy.png', type: 'image/png', body: 'dummy') @@ -13,22 +16,29 @@ RSpec.describe 'Materials API', type: :request do end def build_material(tag:, user:, parent: nil, file: dummy_upload, url: nil) - Material.new(tag:, parent:, url:, created_by_user: user, updated_by_user: user).tap do |material| + Material.new(tag:, parent:, url:, + created_by_user: user, + updated_by_user: user).tap do |material| material.file.attach(file) if file material.save! end end describe 'GET /materials' do - let!(:tag_a) { Tag.create!(tag_name: TagName.create!(name: 'material_index_a'), category: :material) } - let!(:tag_b) { Tag.create!(tag_name: TagName.create!(name: 'material_index_b'), category: :material) } + let!(:tag_a) do + Tag.create!(tag_name: TagName.create!(name: 'material_index_a'), category: :material) + end + let!(:tag_b) do + Tag.create!(tag_name: TagName.create!(name: 'material_index_b'), category: :material) + end let!(:material_a) do build_material(tag: tag_a, user: member_user, file: dummy_upload(filename: 'a.png')) end let!(:material_b) do - build_material(tag: tag_b, user: member_user, parent: material_a, file: dummy_upload(filename: 'b.png')) + build_material(tag: tag_b, user: member_user, parent: material_a, + file: dummy_upload(filename: 'b.png')) end before do @@ -54,27 +64,36 @@ RSpec.describe 'Materials API', type: :request do 'name' => 'material_index_b', 'category' => 'material' ) - expect(row['created_by_user']).to include( - 'id' => member_user.id, - 'name' => member_user.name - ) expect(row['content_type']).to eq('image/png') + expect(row['media_kind']).to eq('image') end - it 'filters materials by tag_id' do - get '/materials', params: { tag_id: material_a.tag_id } + it 'filters materials by q' do + get '/materials', params: { q: 'material_index_a' } expect(response).to have_http_status(:ok) expect(json['count']).to eq(1) expect(response_materials.map { |m| m['id'] }).to eq([material_a.id]) end - it 'filters materials by parent_id' do - get '/materials', params: { parent_id: material_a.id } + it 'filters materials by tag_state' do + untagged = + build_material(tag: nil, user: member_user, + file: dummy_upload(filename: 'untagged.png')) + + get '/materials', params: { tag_state: 'untagged' } expect(response).to have_http_status(:ok) expect(json['count']).to eq(1) - expect(response_materials.map { |m| m['id'] }).to eq([material_b.id]) + expect(response_materials.map { |m| m['id'] }).to eq([untagged.id]) + end + + it 'filters materials by media_kind' do + get '/materials', params: { media_kind: 'image' } + + expect(response).to have_http_status(:ok) + expect(json['count']).to eq(2) + expect(response_materials.map { |m| m['media_kind'] }).to all(eq('image')) end it 'paginates and keeps total count' do @@ -94,10 +113,74 @@ RSpec.describe 'Materials API', type: :request do expect(response_materials.size).to eq(1) expect(response_materials.first['id']).to eq(material_b.id) end + + it 'filters by descendant tags and returns stable parent tag groups' do + root = + Tag.create!(tag_name: TagName.create!(name: 'material_scope_root'), + category: :material) + child_b = + Tag.create!(tag_name: TagName.create!(name: 'material_scope_b'), + category: :material) + child_a = + Tag.create!(tag_name: TagName.create!(name: 'material_scope_a'), + category: :material) + deprecated = + Tag.create!(tag_name: TagName.create!(name: 'material_scope_old'), + category: :material, + deprecated_at: Time.current) + grandchild = + Tag.create!(tag_name: TagName.create!(name: 'material_scope_grandchild'), + category: :material) + root_material = + build_material(tag: root, user: member_user, + file: dummy_upload(filename: 'root.png')) + child_a_material = + build_material(tag: child_a, user: member_user, + file: dummy_upload(filename: 'child_a.png')) + grandchild_material = + build_material(tag: grandchild, user: member_user, + file: dummy_upload(filename: 'grandchild.png')) + + TagImplication.create!(parent_tag: root, tag: child_b) + TagImplication.create!(parent_tag: root, tag: child_a) + TagImplication.create!(parent_tag: child_b, tag: deprecated) + TagImplication.create!(parent_tag: deprecated, tag: grandchild) + + get '/materials', + params: { tag_id: root.id, include_descendants: '1', + group_by: 'parent_tag', sort: 'id', direction: 'asc' } + + expect(response).to have_http_status(:ok) + expect(response_materials.map { |m| m['id'] }) + .to include(root_material.id, child_a_material.id, grandchild_material.id) + expect(json['tag_scope']).to eq( + 'tag' => { + 'id' => root.id, + 'name' => 'material_scope_root', + 'category' => 'material' + }, + 'include_descendants' => true + ) + expect(json['groups'].map { |group| group.dig('tag', 'name') }) + .to eq(['material_scope_a', 'material_scope_b', 'material_scope_root']) + expect(json['groups'].find { |group| group.dig('tag', 'id') == child_b.id } + .fetch('material_ids')).to eq([grandchild_material.id]) + end + + it 'returns nil tag_scope when tag_id is unknown' do + get '/materials', params: { tag_id: 999_999, group_by: 'parent_tag' } + + expect(response).to have_http_status(:ok) + expect(json['tag_scope']).to be_nil + expect(json['groups']).to eq([]) + expect(response_materials).to eq([]) + end end describe 'GET /materials/:id' do - let!(:tag) { Tag.create!(tag_name: TagName.create!(name: 'material_show'), category: :material) } + let!(:tag) do + Tag.create!(tag_name: TagName.create!(name: 'material_show'), category: :material) + end let!(:material) do build_material(tag:, user: member_user, file: dummy_upload(filename: 'show.png')) end @@ -138,9 +221,22 @@ RSpec.describe 'Materials API', type: :request do end end - context 'when logged in' do + context 'when logged in but not member' do before { sign_in_as(guest_user) } + it 'returns 403' do + post '/materials', params: { + tag: 'material_create_guest_forbidden', + file: dummy_upload + } + + expect(response).to have_http_status(:forbidden) + end + end + + context 'when member' do + before { sign_in_as(member_user) } + it 'returns 422 when tag is blank' do post '/materials', params: { tag: ' ', file: dummy_upload } @@ -162,24 +258,49 @@ RSpec.describe 'Materials API', type: :request do expect do post '/materials', params: { tag: 'material_create_new', - file: dummy_upload(filename: 'created.png') + file: dummy_upload(filename: 'created.png'), + export_paths: { legacy_drive: '伊地知ニジカ/created.png' } } end.to change(Material, :count).by(1) .and change(Tag, :count).by(1) .and change(TagName, :count).by(1) + .and change(MaterialVersion, :count).by(1) expect(response).to have_http_status(:created) material = Material.order(:id).last expect(material.tag.name).to eq('material_create_new') expect(material.tag.category).to eq('material') - expect(material.created_by_user).to eq(guest_user) - expect(material.updated_by_user).to eq(guest_user) + expect(material.created_by_user).to eq(member_user) + expect(material.updated_by_user).to eq(member_user) expect(material.file.attached?).to be(true) + expect(material.version_no).to eq(1) + expect(material.material_versions.first.event_type).to eq('create') + expect(material.material_export_items.first.export_path).to eq('伊地知ニジカ/created.png') + expect(material.material_versions.first.export_paths_json).to eq( + 'legacy_drive' => '伊地知ニジカ/created.png' + ) expect(json['id']).to eq(material.id) expect(json.dig('tag', 'name')).to eq('material_create_new') expect(json['content_type']).to eq('image/png') + expect(json.dig('export_paths', 'legacy_drive')).to eq('伊地知ニジカ/created.png') + end + + it 'snapshots attached file metadata and sha256' do + post '/materials', params: { + tag: 'material_create_file_version', + file: dummy_upload(filename: 'created.png', body: 'sha-body') + } + + expect(response).to have_http_status(:created) + + version = Material.order(:id).last.material_versions.first + expect(version.file_blob_id).to be_present + expect(version.file_filename).to eq('created.png') + expect(version.file_content_type).to eq('image/png') + expect(version.file_byte_size).to eq('sha-body'.bytesize) + expect(version.file_sha256).to eq(Digest::SHA256.hexdigest('sha-body')) end it 'returns 422 when the existing tag is not material/character' do @@ -219,11 +340,33 @@ RSpec.describe 'Materials API', type: :request do expect(response).to have_http_status(:created) expect(json['url']).to eq('https://example.com/material-source') end + + it 'rejects sha256-blocked file upload' do + sha256 = Digest::SHA256.hexdigest('blocked-body') + MaterialImportBlock.create!(match_kind: 'sha256', + sha256:, + reason: 'copyright_high_risk', + created_by_user: admin_user) + + expect do + post '/materials', params: { + tag: 'material_blocked_create', + file: dummy_upload(filename: 'blocked.png', body: 'blocked-body') + } + end.not_to change(Material, :count) + + expect(response).to have_http_status(:unprocessable_entity) + expect(json.fetch('errors')).to include( + 'file' => ['抑止された素材です: copyright_high_risk'] + ) + end end end describe 'PUT /materials/:id' do - let!(:tag) { Tag.create!(tag_name: TagName.create!(name: 'material_update_old'), category: :material) } + let!(:tag) do + Tag.create!(tag_name: TagName.create!(name: 'material_update_old'), category: :material) + end let!(:material) do build_material(tag:, user: member_user, file: dummy_upload(filename: 'old.png')) end @@ -277,25 +420,26 @@ RSpec.describe 'Materials API', type: :request do 'tag' => ['タグは必須です．']) end - it 'returns 422 when both file and url are blank' do + it 'keeps the existing file when file and url are omitted' do put "/materials/#{ material.id }", params: { tag: 'material_update_no_payload' } - expect(response).to have_http_status(:unprocessable_entity) - expect(json.fetch('errors')).to include( - 'file' => ['ファイルまたは URL は必須です．'], - 'url' => ['ファイルまたは URL は必須です．']) + expect(response).to have_http_status(:ok) + expect(material.reload.file.attached?).to be(true) end it 'updates tag, url, file, and updated_by_user' do old_blob_id = material.file.blob.id - put "/materials/#{ material.id }", params: { - tag: 'material_update_new', - url: 'https://example.com/updated-source', - file: dummy_upload(filename: 'updated.jpg', type: 'image/jpeg') - } + expect do + put "/materials/#{ material.id }", params: { + tag: 'material_update_new', + url: 'https://example.com/updated-source', + file: dummy_upload(filename: 'updated.jpg', type: 'image/jpeg'), + export_paths: { legacy_drive: '伊地知ニジカ/updated.jpg' } + } + end.to change(MaterialVersion, :count).by(2) expect(response).to have_http_status(:ok) @@ -306,8 +450,15 @@ RSpec.describe 'Materials API', type: :request do expect(material.updated_by_user).to eq(member_user) expect(material.file.attached?).to be(true) expect(material.file.blob.id).not_to eq(old_blob_id) + expect(ActiveStorage::Blob.where(id: old_blob_id).exists?).to be(true) expect(material.file.blob.filename.to_s).to eq('updated.jpg') expect(material.file.blob.content_type).to eq('image/jpeg') + expect(material.version_no).to eq(2) + expect(material.material_versions.order(:version_no).last.event_type).to eq('update') + expect(material.material_export_items.first.export_path).to eq('伊地知ニジカ/updated.jpg') + expect(material.material_versions.order(:version_no).last.export_paths_json).to eq( + 'legacy_drive' => '伊地知ニジカ/updated.jpg' + ) expect(json['id']).to eq(material.id) expect(json['file']).to be_present @@ -315,7 +466,7 @@ RSpec.describe 'Materials API', type: :request do expect(json.dig('tag', 'name')).to eq('material_update_new') end - it 'purges the existing file when file is omitted and url is provided' do + it 'detaches the existing file without purging blob when url replaces file' do old_blob_id = material.file.blob.id put "/materials/#{ material.id }", params: { @@ -331,9 +482,7 @@ RSpec.describe 'Materials API', type: :request do expect(material.updated_by_user).to eq(member_user) expect(material.file.attached?).to be(false) - expect( - ActiveStorage::Blob.where(id: old_blob_id).exists? - ).to be(false) + expect(ActiveStorage::Blob.where(id: old_blob_id).exists?).to be(true) expect(json['id']).to eq(material.id) expect(json['file']).to be_nil @@ -341,11 +490,205 @@ RSpec.describe 'Materials API', type: :request do expect(json.dig('tag', 'name')).to eq('material_update_remove_file') expect(json['url']).to eq('https://example.com/updated-source') end + + it 'does not increase version for the same snapshot update' do + MaterialVersionRecorder.record!(material:, event_type: :create, + created_by_user: member_user) + + expect do + put "/materials/#{ material.id }", params: { + tag: 'material_update_old' + } + end.not_to change(MaterialVersion, :count) + + expect(response).to have_http_status(:ok) + expect(material.reload.version_no).to eq(1) + end + + it 'records update version when only export_path changes' do + MaterialVersionRecorder.record!(material:, event_type: :create, + created_by_user: member_user) + + expect do + put "/materials/#{ material.id }", params: { + tag: 'material_update_old', + export_paths: { legacy_drive: '素材/only-path.png' } + } + end.to change(MaterialVersion, :count).by(1) + + expect(response).to have_http_status(:ok) + expect(material.reload.material_export_items.first.export_path).to eq('素材/only-path.png') + expect(material.material_versions.order(:version_no).last.export_paths_json).to eq( + 'legacy_drive' => '素材/only-path.png' + ) + end + + it 'removes export_path item when blank is submitted' do + MaterialExportItem.create!(material:, profile: 'legacy_drive', + export_path: '素材/remove.png', + created_by_user: member_user) + MaterialVersionRecorder.record!(material:, event_type: :create, + created_by_user: member_user) + + expect do + put "/materials/#{ material.id }", params: { + tag: 'material_update_old', + export_paths: { legacy_drive: '' } + } + end.to change(MaterialExportItem, :count).by(-1) + .and change(MaterialVersion, :count).by(1) + + expect(response).to have_http_status(:ok) + expect(material.reload.material_export_items).to be_empty + expect(material.material_versions.order(:version_no).last.export_paths_json).to eq({}) + end + + it 'rejects sha256-blocked replacement file' do + sha256 = Digest::SHA256.hexdigest('blocked-update') + MaterialImportBlock.create!(match_kind: 'sha256', + sha256:, + reason: 'source_owner_request', + created_by_user: admin_user) + + put "/materials/#{ material.id }", params: { + tag: 'material_update_old', + file: dummy_upload(filename: 'blocked.png', body: 'blocked-update') + } + + expect(response).to have_http_status(:unprocessable_entity) + expect(material.reload.file.blob.filename.to_s).to eq('old.png') + end + end + end + + describe 'GET /materials/download.zip' do + let!(:tag_a) { Tag.create!(tag_name: TagName.create!(name: 'zip_a'), category: :material) } + let!(:tag_b) { Tag.create!(tag_name: TagName.create!(name: 'zip_b'), category: :material) } + let!(:material_a) do + build_material(tag: tag_a, user: member_user, + file: dummy_upload(filename: 'a.png', body: 'zip-a')) + end + let!(:material_b) do + build_material(tag: tag_b, user: member_user, + file: dummy_upload(filename: 'b.png', body: 'zip-b')) + end + + before do + MaterialExportItem.create!(material: material_a, profile: 'legacy_drive', + export_path: '素材/a.png', + created_by_user: member_user) + MaterialExportItem.create!(material: material_b, profile: 'legacy_drive', + export_path: '素材/b.png', + created_by_user: member_user) + end + + it 'uses material_export_items.export_path as ZIP entry paths' do + get '/materials/download.zip', params: { profile: 'legacy_drive' } + + expect(response).to have_http_status(:ok) + expect(response.media_type).to eq('application/zip') + expect(response.body.b).to include('素材/a.png'.b) + expect(response.body.b).to include('素材/b.png'.b) + end + + it 'filters by tag_id' do + get '/materials/download.zip', params: { profile: 'legacy_drive', tag_id: tag_a.id } + + expect(response).to have_http_status(:ok) + expect(response.body.b).to include('素材/a.png'.b) + expect(response.body.b).not_to include('素材/b.png'.b) + end + + it 'does not include disabled export items' do + material_b.material_export_items.first.update!(enabled: false) + + get '/materials/download.zip', params: { profile: 'legacy_drive' } + + expect(response).to have_http_status(:ok) + expect(response.body.b).to include('素材/a.png'.b) + expect(response.body.b).not_to include('素材/b.png'.b) + end + end + + describe 'GET /materials/versions' do + let!(:tag) do + Tag.create!(tag_name: TagName.create!(name: 'material_history'), category: :material) + end + let!(:material) do + build_material(tag:, user: member_user, file: dummy_upload(filename: 'history.png')) + end + + it 'returns material versions in reverse chronological order' do + MaterialVersionRecorder.record!(material:, event_type: :create, + created_by_user: member_user) + material.update!(url: 'https://example.com/history') + MaterialVersionRecorder.record!(material:, event_type: :update, + created_by_user: admin_user) + + get '/materials/versions', params: { material_id: material.id } + + expect(response).to have_http_status(:ok) + expect(json['count']).to eq(2) + versions = json.fetch('versions') + expect(versions.map { |v| v['version_no'] }).to eq([2, 1]) + expect(versions.first).to include( + 'material_id' => material.id, + 'event_type' => 'update', + 'tag_name' => 'material_history', + 'file_filename' => 'history.png', + 'url' => 'https://example.com/history' + ) + expect(versions.first['created_by_user']).to include('id' => admin_user.id) + end + + it 'filters material versions by tag and event_type' do + MaterialVersionRecorder.record!(material:, event_type: :create, + created_by_user: member_user) + + get '/materials/versions', params: { + tag: 'material_history', + event_type: 'create' + } + + expect(response).to have_http_status(:ok) + expect(json.fetch('versions').map { |v| v['material_id'] }).to include(material.id) + expect(json.fetch('versions').map { |v| v['event_type'] }).to all(eq('create')) + end + end + + describe 'material sync suppressions' do + it 'requires member permission for index' do + sign_out + + get '/materials/suppressions' + + expect(response).to have_http_status(:unauthorized) + end + + it 'creates source suppression as member' do + sign_in_as(member_user) + + expect do + post '/materials/suppressions', params: { + source_kind: 'google_drive_path_prefix', + drive_path: '伊地知ニジカ/危険素材', + reason: 'copyright_high_risk' + } + end.to change(MaterialSyncSuppression, :count).by(1) + + expect(response).to have_http_status(:created) + expect(json).to include( + 'source_kind' => 'google_drive_path_prefix', + 'drive_path' => '伊地知ニジカ/危険素材', + 'normalized_source_key' => 'google_drive_path_prefix:伊地知ニジカ/危険素材' + ) end end describe 'DELETE /materials/:id' do - let!(:tag) { Tag.create!(tag_name: TagName.create!(name: 'material_destroy'), category: :material) } + let!(:tag) do + Tag.create!(tag_name: TagName.create!(name: 'material_destroy'), category: :material) + end let!(:material) do build_material(tag:, user: member_user, file: dummy_upload(filename: 'destroy.png')) end diff --git a/backend/spec/services/material_file_sha256_spec.rb b/backend/spec/services/material_file_sha256_spec.rb new file mode 100644 index 0000000..37dde5c --- /dev/null +++ b/backend/spec/services/material_file_sha256_spec.rb @@ -0,0 +1,23 @@ +require 'rails_helper' + +RSpec.describe MaterialFileSha256 do + describe '.metadata_sha256' do + it 'reads sha256 from Hash metadata' do + blob = instance_double(ActiveStorage::Blob, metadata: { 'sha256' => 'abc123' }) + + expect(described_class.metadata_sha256(blob)).to eq('abc123') + end + + it 'reads sha256 from JSON string metadata' do + blob = instance_double(ActiveStorage::Blob, metadata: '{"sha256":"abc123"}') + + expect(described_class.metadata_sha256(blob)).to eq('abc123') + end + + it 'returns nil for invalid metadata' do + blob = instance_double(ActiveStorage::Blob, metadata: '{') + + expect(described_class.metadata_sha256(blob)).to be_nil + end + end +end diff --git a/backend/spec/services/material_sync_importer_spec.rb b/backend/spec/services/material_sync_importer_spec.rb new file mode 100644 index 0000000..20c69fd --- /dev/null +++ b/backend/spec/services/material_sync_importer_spec.rb @@ -0,0 +1,128 @@ +require 'rails_helper' + +RSpec.describe MaterialSyncImporter do + let(:user) { create(:user, :member) } + let(:tag) { Tag.create!(tag_name: TagName.create!(name: 'sync_tag'), category: :material) } + + def tempfile_for body + Tempfile.new(['material-sync-importer', '.png']).tap do |file| + file.binmode + file.write(body) + file.rewind + end + end + + it 'creates an untagged material from a sync candidate with file_downloader' do + result = + described_class.import!( + source_kind: 'google_drive_file', + source_path: '素材/a.png', + source_file_id: 'drive-file-a', + normalized_source_key: 'google_drive_file:drive-file-a', + filename: 'a.png', + content_type: 'image/png', + file_sha256: Digest::SHA256.hexdigest('sync-body'), + file_downloader: -> { tempfile_for('sync-body') }, + export_path: '素材/a.png', + tag: nil, + url: nil, + created_by_user: user, + updated_by_user: user) + + material = result.material.reload + expect(result.action).to eq(:imported) + expect(material.tag).to be_nil + expect(material.url).to be_nil + expect(material.file).to be_attached + expect(material.file.blob.filename.to_s).to eq('a.png') + expect(material.material_export_items.first.export_path).to eq('素材/a.png') + end + + it 'does not clear a human-assigned tag or URL on existing Drive sync' do + material = + Material.create!(tag:, url: 'https://example.com/human', + source_kind: 'google_drive_file', + source_path: '素材/a.png', + source_file_id: 'drive-file-a', + created_by_user: user, + updated_by_user: user) + material.file.attach( + io: StringIO.new('same-body'), + filename: 'a.png', + content_type: 'image/png') + material.file.blob.metadata['sha256'] = Digest::SHA256.hexdigest('same-body') + material.file.blob.save! + MaterialExportItem.create!(material:, + profile: 'legacy_drive', + export_path: '素材/a.png', + created_by_user: user) + + result = + described_class.import!( + source_kind: 'google_drive_file', + source_path: '素材/a.png', + source_file_id: 'drive-file-a', + filename: 'a.png', + content_type: 'image/png', + file_sha256: Digest::SHA256.hexdigest('same-body'), + export_path: '素材/a.png', + tag: nil, + url: nil, + updated_by_user: user) + + expect(result.action).to eq(:unchanged) + expect(material.reload.tag).to eq(tag) + expect(material.url).to eq('https://example.com/human') + end + + it 'suppresses a Google Drive candidate by relative path prefix' do + MaterialSyncSuppression.create!(source_kind: 'google_drive_path_prefix', + drive_path: '素材/危険', + reason: 'copyright_high_risk', + created_by_user: user) + + result = + described_class.import!( + source_kind: 'google_drive_file', + source_path: '素材/危険/a.png', + source_file_id: 'drive-file-b', + filename: 'a.png', + content_type: 'image/png', + file_sha256: Digest::SHA256.hexdigest('blocked-body'), + file_downloader: -> { tempfile_for('blocked-body') }, + export_path: '素材/危険/a.png', + updated_by_user: user) + + expect(result.action).to eq(:suppressed) + expect(result.suppression.normalized_source_key) + .to eq('google_drive_path_prefix:素材/危険') + expect(Material.find_by(source_file_id: 'drive-file-b')).to be_nil + end + + it 'rechecks import blocks after downloaded file sha256 is known' do + MaterialImportBlock.create!(match_kind: 'sha256', + sha256: Digest::SHA256.hexdigest('blocked-body'), + reason: 'copyright_high_risk', + created_by_user: user) + blob_count = ActiveStorage::Blob.count + + expect do + result = + described_class.import!( + source_kind: 'google_drive_file', + source_path: '素材/blocked.png', + source_file_id: 'drive-file-blocked', + filename: 'blocked.png', + content_type: 'image/png', + file_downloader: -> { tempfile_for('blocked-body') }, + export_path: '素材/blocked.png', + updated_by_user: user) + + expect(result.action).to eq(:suppressed) + expect(result.suppression.reason).to eq('copyright_high_risk') + end.not_to change(Material, :count) + + expect(ActiveStorage::Blob.count).to eq(blob_count) + expect(Material.find_by(source_file_id: 'drive-file-blocked')).to be_nil + end +end diff --git a/backend/spec/services/material_sync_runner_spec.rb b/backend/spec/services/material_sync_runner_spec.rb new file mode 100644 index 0000000..ace4f45 --- /dev/null +++ b/backend/spec/services/material_sync_runner_spec.rb @@ -0,0 +1,89 @@ +require 'rails_helper' + +RSpec.describe MaterialSyncRunner do + let(:user) { create(:user, :member) } + let(:source) do + MaterialSyncSource.create!( + name: 'Drive source', + source_kind: 'google_drive_path', + source_file_id: 'folder-123', + profile: 'legacy_drive', + created_by_user: user, + updated_by_user: user) + end + let(:drive_client) { instance_double(GoogleDrive::ApiClient) } + + describe '#sync!' do + it 'imports google drive path candidates as they are yielded' do + first_entry = { id: 'file-1', + name: 'a.png', + mime_type: 'image/png', + relative_path: '素材/a.png', + sha256_checksum: 'sha-a', + web_view_link: 'https://drive.google.com/file/d/file-1/view', + web_content_link: nil } + second_entry = { id: 'file-2', + name: 'b.png', + mime_type: 'image/png', + relative_path: '素材/b.png', + sha256_checksum: 'sha-b', + web_view_link: 'https://drive.google.com/file/d/file-2/view', + web_content_link: nil } + yielded = [] + + allow(GoogleDrive::ApiClient).to receive(:new).and_return(drive_client) + allow(drive_client).to receive(:each_material_file_under_folder) do |folder_id, &block| + expect(folder_id).to eq('folder-123') + block.call(first_entry) + expect(yielded).to eq(['素材/a.png']) + block.call(second_entry) + end + allow(MaterialSyncImporter).to receive(:import!) do |candidate| + yielded << candidate.fetch(:source_path) + instance_double(MaterialSyncImporter::Result, + action: yielded.last == '素材/a.png' ? :imported : :updated) + end + + result = described_class.new(source).sync! + + expect(yielded).to eq(['素材/a.png', '素材/b.png']) + expect(result.imported).to eq(1) + expect(result.updated).to eq(1) + expect(result.unchanged).to eq(0) + expect(source.reload.last_synced_at).to be_present + end + + it 'logs google drive path progress at first item and summary' do + entry = { id: 'file-1', + name: 'a.png', + mime_type: 'image/png', + relative_path: '素材/a.png', + sha256_checksum: 'sha-a', + web_view_link: 'https://drive.google.com/file/d/file-1/view', + web_content_link: nil } + logged = [] + + allow(GoogleDrive::ApiClient).to receive(:new).and_return(drive_client) + allow(drive_client).to receive(:each_material_file_under_folder) do |_folder_id, &block| + block.call(entry) + end + allow(MaterialSyncImporter).to receive(:import!) + .and_return(instance_double(MaterialSyncImporter::Result, action: :imported)) + allow(Rails.logger).to receive(:info) { |message| logged << JSON.parse(message) } + + described_class.new(source).sync! + + progress_logs = logged.select { |row| row['folder_id'] == 'folder-123' } + expect(progress_logs.map { |row| row['progress'] }).to eq(['scan', 'summary']) + expect(progress_logs.last).to include( + 'material_sync_source_id' => source.id, + 'material_sync_source_name' => 'Drive source', + 'scanned_count' => 1, + 'imported' => 1, + 'updated' => 0, + 'unchanged' => 0, + 'suppressed' => 0, + 'failed' => 0) + end + end +end diff --git a/docs/commands.md b/docs/commands.md index dc91712..b267600 100644 --- a/docs/commands.md +++ b/docs/commands.md @@ -10,6 +10,15 @@ bundle exec rspec bundle exec rails routes ``` +Material video thumbnail generation requires `ffmpeg` on the host. If `ffmpeg` +is missing or frame extraction fails, `MaterialThumbnailGenerator` logs the +result and stderr, then leaves the existing thumbnail unchanged. + +```sh +cd backend +bundle exec rails materials:thumbnails:backfill +``` + ## Frontend ```sh diff --git a/frontend/src/App.tsx b/frontend/src/App.tsx index 78e7676..11f784c 100644 --- a/frontend/src/App.tsx +++ b/frontend/src/App.tsx @@ -14,8 +14,10 @@ import { apiPost, isApiError } from '@/lib/api' import DeerjikistDetailPage from '@/pages/deerjikists/DeerjikistDetailPage' import MaterialBasePage from '@/pages/materials/MaterialBasePage' import MaterialDetailPage from '@/pages/materials/MaterialDetailPage' +import MaterialHistoryPage from '@/pages/materials/MaterialHistoryPage' import MaterialListPage from '@/pages/materials/MaterialListPage' import MaterialNewPage from '@/pages/materials/MaterialNewPage' +import MaterialSyncSuppressionsPage from '@/pages/materials/MaterialSyncSuppressionsPage' // import MaterialSearchPage from '@/pages/materials/MaterialSearchPage' import MorePage from '@/pages/MorePage' import GekanatorPage from '@/pages/GekanatorPage' @@ -68,10 +70,11 @@ const RouteTransitionWrapper = ({ user, setUser }: { }/> }> }/> + }/> }/> + }/> }/> - {/* }/> */} }/> }/> }/> diff --git a/frontend/src/components/MaterialSidebar.tsx b/frontend/src/components/MaterialSidebar.tsx index 875a967..399d9d8 100644 --- a/frontend/src/components/MaterialSidebar.tsx +++ b/frontend/src/components/MaterialSidebar.tsx @@ -1,23 +1,43 @@ -import { Fragment, useEffect, useState } from 'react' +import { useQuery } from '@tanstack/react-query' +import { useEffect, useRef, useState } from 'react' +import { useLocation, useNavigate } from 'react-router-dom' -import TagLink from '@/components/TagLink' import SidebarComponent from '@/components/layout/SidebarComponent' -import { apiGet } from '@/lib/api' +import TagLink from '@/components/TagLink' +import { fetchMaterialTagTree, parseMaterialFilter } from '@/lib/materials' +import { materialsKeys } from '@/lib/queryKeys' +import { cn } from '@/lib/utils' -import type { FC, ReactNode } from 'react' +import type { CSSProperties, Dispatch, FC, ReactNode, SetStateAction } from 'react' -import type { Tag } from '@/types' +import type { MaterialFilter, MaterialSidebarTag, Tag } from '@/types' -type TagWithDepth = Tag & { - hasChildren: boolean - children: TagWithDepth[] } +const FILTERS: MaterialFilter[] = ['missing', 'present', 'any'] +const FILTER_LABELS: Record = { present: '有', missing: '無', any: '全' } + + +const px = (value: string): number => { + const parsed = Number.parseFloat (value) + return Number.isFinite (parsed) ? parsed : 0 +} + + +const verticalChrome = (el: HTMLElement): number => { + const style = window.getComputedStyle (el) + + return ( + px (style.paddingTop) + + px (style.paddingBottom) + + px (style.borderTopWidth) + + px (style.borderBottomWidth)) +} const setChildrenById = ( - tags: TagWithDepth[], + tags: MaterialSidebarTag[], targetId: number, - children: TagWithDepth[], -): TagWithDepth[] => ( + children: MaterialSidebarTag[], +): MaterialSidebarTag[] => ( tags.map (tag => { if (tag.id === targetId) return { ...tag, children } @@ -25,75 +45,462 @@ const setChildrenById = ( if (tag.children.length === 0) return tag - return { ...tag, - children: (setChildrenById (tag.children, targetId, children) - .filter (t => t.category !== 'meme' || t.hasChildren)) } + return { ...tag, children: setChildrenById (tag.children, targetId, children) } })) -const MaterialSidebar: FC = () => { - const [tags, setTags] = useState ([]) - const [openTags, setOpenTags] = useState> ({ }) - const [tagFetchedFlags, setTagFetchedFlags] = useState> ({ }) +const materialPath = ( + tagId: number, + materialFilter: MaterialFilter, +): string => + `/materials?tag_id=${ tagId }&include_descendants=1&group_by=parent_tag` + + `&material_filter=${ materialFilter }` - useEffect (() => { - void (async () => { - setTags ((await apiGet ('/tags/with-depth')) - .filter (t => t.category !== 'meme' || t.hasChildren)) - }) () - }, []) - const renderTags = (ts: TagWithDepth[], nestLevel = 0): ReactNode => ( - ts.map (t => ( - -
  • -
    -
    - {t.hasChildren && ( - { - e.preventDefault () - if (!(tagFetchedFlags[t.id])) - { - try - { - const data = - await apiGet ( - '/tags/with-depth', { params: { parent: String (t.id) } }) - setTags (prev => setChildrenById (prev, t.id, data)) - setTagFetchedFlags (prev => ({ ...prev, [t.id]: true })) - } - catch - { - ; - } - } - setOpenTags (prev => ({ ...prev, [t.id]: !(prev[t.id]) })) - }}> - {openTags[t.id] ? <>− : '+'} - )} -
    -
    - -
    -
    -
    • - {openTags[t.id] && renderTags (t.children, nestLevel + 1)} - ))) +const sidebarTagToTag = (tag: MaterialSidebarTag): Tag => ({ + id: tag.id, + name: tag.name, + category: tag.category, + deprecatedAt: tag.deprecated ? '' : null, + aliases: [], + parents: [], + postCount: 0, + createdAt: '', + updatedAt: '', + hasWiki: false, + materialId: null, + hasDeerjikists: false, + matchedAlias: null }) - return ( - - - ) + +const tagSelectionShellClass = (selected: boolean): string => + cn (selected + ? ['rounded-md border border-sky-500 bg-sky-50 px-2 py-1 text-sky-700', + 'dark:border-sky-400 dark:bg-sky-950 dark:text-sky-100'] + : 'px-2 py-1') + + +const updateMaterialFilterQuery = ( + pathname: string, + locationSearch: string, + navigate: ReturnType, + materialFilter: MaterialFilter, +) => { + const qs = new URLSearchParams (locationSearch) + qs.set ('material_filter', materialFilter) + navigate (`${ pathname }${ qs.toString () ? `?${ qs.toString () }` : '' }`) } -export default MaterialSidebar \ No newline at end of file + +const MaterialFilterButtons: FC<{ materialFilter: MaterialFilter + onChange: (materialFilter: MaterialFilter) => void }> = ( + { materialFilter, onChange }, +) => ( +
    + + {FILTERS.map (value => ( + ))} +
    ) + + +const MaterialTreeNode: FC<{ + materialFilter: MaterialFilter + selectedTagId: number | null + nestLevel?: number + onChildren: (tagId: number, children: MaterialSidebarTag[]) => void + openTags: Record + setOpenTags: Dispatch>> + tag: MaterialSidebarTag +}> = ({ materialFilter, nestLevel = 0, onChildren, openTags, selectedTagId, + setOpenTags, tag }) => { + const open = Boolean (openTags[tag.id]) + const { data } = useQuery ({ + queryKey: materialsKeys.tree ({ parentId: tag.id, materialFilter }), + queryFn: () => fetchMaterialTagTree ({ parentId: tag.id, materialFilter }), + enabled: open && tag.hasChildren && tag.children.length === 0}) + + useEffect (() => { + if (open && data && tag.children.length === 0) + onChildren (tag.id, data) + }, [data, onChildren, open, tag.children.length, tag.id]) + + return ( + <> +
  • +
    +
    + {tag.hasChildren && ( + )} +
    +
    +
    + +
    +
    +
    +
    • + {open && tag.children.length > 0 && ( +
      + {tag.children.map (child => ( + ))} +
    )} + ) +} + + +const MobileMaterialTreeNode: FC<{ depth?: number + availableInlineSizePx?: number | null + materialFilter: MaterialFilter + selectedTagId: number | null + onChildren: (tagId: number, children: MaterialSidebarTag[]) => + void + openTags: Record + setOpenTags: Dispatch>> + tag: MaterialSidebarTag }> = ( + { + depth = 0, + availableInlineSizePx = null, + materialFilter, + onChildren, + openTags, + selectedTagId, + setOpenTags, + tag, + }, +) => { + const open = Boolean (openTags[tag.id]) + const tagColumnRef = useRef (null) + const chipRef = useRef (null) + const buttonRef = useRef (null) + const expansionSlotRef = useRef (null) + const expansionBorderRef = useRef (null) + const [tagChipInlineSizePx, setTagChipInlineSizePx] = useState (null) + const [tagLinkInlineSizePx, setTagLinkInlineSizePx] = useState (null) + const [childAvailableInlineSizePx, setChildAvailableInlineSizePx] = useState (null) + const { data } = useQuery ({ + queryKey: materialsKeys.tree ({ parentId: tag.id, materialFilter }), + queryFn: () => fetchMaterialTagTree ({ parentId: tag.id, materialFilter }), + enabled: open && tag.hasChildren && tag.children.length === 0}) + + useEffect (() => { + if (open && data && tag.children.length === 0) + onChildren (tag.id, data) + }, [data, onChildren, open, tag.children.length, tag.id]) + + useEffect (() => { + const tagColumn = tagColumnRef.current + const chip = chipRef.current + if (!(tagColumn) || !(chip)) + return + + const updateTagInlineSize = () => { + const buttonHeight = buttonRef.current?.offsetHeight ?? 0 + const gap = tag.hasChildren ? px (window.getComputedStyle (tagColumn).rowGap) : 0 + const columnHeight = + availableInlineSizePx == null + ? tagColumn.clientHeight + : Math.min (tagColumn.clientHeight, availableInlineSizePx) + const nextChipInlineSize = Math.max (24, columnHeight - buttonHeight - gap) + const nextLinkInlineSize = Math.max ( + 16, + nextChipInlineSize - verticalChrome (chip), + ) + + setTagChipInlineSizePx (prev => prev === nextChipInlineSize ? prev : nextChipInlineSize) + setTagLinkInlineSizePx (prev => prev === nextLinkInlineSize ? prev : nextLinkInlineSize) + } + + updateTagInlineSize () + + const resizeObserver = new ResizeObserver (() => { + updateTagInlineSize () + }) + + resizeObserver.observe (tagColumn) + resizeObserver.observe (chip) + + if (buttonRef.current) + resizeObserver.observe (buttonRef.current) + + return () => { + resizeObserver.disconnect () + } + }, [availableInlineSizePx, open, tag.hasChildren, tag.children.length]) + + useEffect (() => { + const expansionSlot = expansionSlotRef.current + const expansionBorder = expansionBorderRef.current + if (!(expansionSlot) || !(expansionBorder)) + return + + const updateChildInlineSize = () => { + const base = + availableInlineSizePx == null + ? expansionSlot.clientHeight + : availableInlineSizePx + const chrome = verticalChrome (expansionSlot) + verticalChrome (expansionBorder) + const nextInlineSize = Math.max (24, base - chrome) + setChildAvailableInlineSizePx (prev => prev === nextInlineSize ? prev : nextInlineSize) + } + + updateChildInlineSize () + + const resizeObserver = new ResizeObserver (() => { + updateChildInlineSize () + }) + + resizeObserver.observe (expansionSlot) + resizeObserver.observe (expansionBorder) + + return () => { + resizeObserver.disconnect () + } + }, [availableInlineSizePx, open, tag.children.length]) + + return ( +
    +
    +
    + +
    + {tag.hasChildren && ( + )} +
    + {open && tag.children.length > 0 && ( +
    +
    +
    +
    +
    +
    )} +
    ) +} + + +const MaterialSidebar: FC = () => { + const location = useLocation () + const navigate = useNavigate () + const qs = new URLSearchParams (location.search) + const materialFilter = parseMaterialFilter (qs.get ('material_filter'), 'any') + const selectedTagId = Number (qs.get ('tag_id') ?? 0) || null + + const [desktopTags, setDesktopTags] = useState ([]) + const [openTags, setOpenTags] = useState> ({ }) + const mobileRailRef = useRef (null) + const [mobileAvailableInlineSizePx, setMobileAvailableInlineSizePx] = + useState (null) + + const { data: rootTags = [], isLoading, isError } = useQuery ({ + queryKey: materialsKeys.tree ({ parentId: null, materialFilter }), + queryFn: () => fetchMaterialTagTree ({ parentId: null, materialFilter })}) + + useEffect (() => { + setDesktopTags (rootTags) + }, [rootTags]) + + useEffect (() => { + const el = mobileRailRef.current + if (!(el)) + return + + requestAnimationFrame (() => { + el.scrollLeft = el.scrollWidth + }) + }, [rootTags, materialFilter]) + + useEffect (() => { + const el = mobileRailRef.current + if (!(el)) + return + + const updateAvailableInlineSize = () => { + const nextInlineSize = Math.max (24, el.clientHeight) + setMobileAvailableInlineSizePx (prev => prev === nextInlineSize ? prev : nextInlineSize) + } + + updateAvailableInlineSize () + + const resizeObserver = new ResizeObserver (() => { + updateAvailableInlineSize () + }) + + resizeObserver.observe (el) + + return () => { + resizeObserver.disconnect () + } + }, [rootTags, materialFilter]) + + const visibleRootTags = desktopTags.length > 0 ? desktopTags : rootTags + + const setChildren = (tagId: number, children: MaterialSidebarTag[]) => { + setDesktopTags (prev => { + const base = prev.length > 0 ? prev : rootTags + return setChildrenById (base, tagId, children) + }) + } + + const handleFilterChange = (value: MaterialFilter) => { + setDesktopTags ([]) + setOpenTags ({ }) + updateMaterialFilterQuery (location.pathname, location.search, navigate, value) + } + + const renderDesktopTree = (tags: MaterialSidebarTag[]): ReactNode => ( + tags.map (tag => ( + ))) + + return ( + <> +
    + +
    +
    + {visibleRootTags.map (tag => ( + ))} +
    +
    +
    + +
    + +
    + + {isLoading && ( +

    読込中……

    )} + {isError && ( +

    + タグ一覧の取得に失敗しました． +

    )} + {(!isLoading && !isError) && ( +
      + {renderDesktopTree (visibleRootTags)} +
    )} +
    +     +
    + ) +} + + +export default MaterialSidebar diff --git a/frontend/src/components/NicoViewer.tsx b/frontend/src/components/NicoViewer.tsx index 97f1ffb..061c5db 100644 --- a/frontend/src/components/NicoViewer.tsx +++ b/frontend/src/components/NicoViewer.tsx @@ -107,8 +107,7 @@ export default forwardRef ((props: Props, ref: ForwardedRef { onError?.({ eventName: 'loadCompleteTimeout', - reason: 'niconico video length was not reported by embed', - }) + reason: 'niconico video length was not reported by embed'}) }, LOAD_COMPLETE_TIMEOUT_MS) }, [clearLoadCompleteTimer, onError]) diff --git a/frontend/src/components/PostOriginalCreatedTimeField.tsx b/frontend/src/components/PostOriginalCreatedTimeField.tsx index 2ddbee4..863e344 100644 --- a/frontend/src/components/PostOriginalCreatedTimeField.tsx +++ b/frontend/src/components/PostOriginalCreatedTimeField.tsx @@ -19,8 +19,7 @@ const PostOriginalCreatedTimeField: FC = ( setOriginalCreatedFrom, originalCreatedBefore, setOriginalCreatedBefore, - errors }: Props, -) => ( + errors }: Props) => ( {({ describedBy, invalid }) => ( <> diff --git a/frontend/src/components/TagDetailSidebar.tsx b/frontend/src/components/TagDetailSidebar.tsx index be95412..64b45bb 100644 --- a/frontend/src/components/TagDetailSidebar.tsx +++ b/frontend/src/components/TagDetailSidebar.tsx @@ -37,8 +37,7 @@ const renderTagTree = ( path: string, suppressClickRef: MutableRefObject, parentTagId?: number, - sp?: boolean, -): ReactNode[] => { + sp?: boolean): ReactNode[] => { const key = `${ path }-${ tag.id }` const self = ( @@ -64,8 +63,7 @@ const renderTagTree = ( const isDescendant = ( root: Tag, - targetId: number, -): boolean => { + targetId: number): boolean => { if (!(root.children)) return false @@ -83,8 +81,7 @@ const isDescendant = ( const findTag = ( byCat: TagByCategory, - id: number, -): Tag | undefined => { + id: number): Tag | undefined => { const walk = (nodes: Tag[]): Tag | undefined => { for (const t of nodes) { @@ -130,8 +127,7 @@ const buildTagByCategory = (post: Post): TagByCategory => { const changeCategory = async ( tagId: number, - category: Category, -): Promise => { + category: Category): Promise => { await apiPatch (`/tags/${ tagId }`, { category }) } @@ -294,8 +290,7 @@ const TagDetailSidebar: FC = ({ post, sp }) => { addEventListener ('click', e => { e.preventDefault () e.stopPropagation () - suppressClickRef.current = false - }, { capture: true, once: true }) + suppressClickRef.current = false}, { capture: true, once: true }) }} onDragCancel={() => { setActiveTagId (null) diff --git a/frontend/src/components/TagLink.tsx b/frontend/src/components/TagLink.tsx index 4abdf1a..a32dd4a 100644 --- a/frontend/src/components/TagLink.tsx +++ b/frontend/src/components/TagLink.tsx @@ -28,11 +28,12 @@ type Props = const TagLink: FC = ({ tag, - nestLevel = 0, - linkFlg = true, - withWiki = true, - withCount = true, - ...props }) => { + nestLevel = 0, + linkFlg = true, + withWiki = true, + withCount = true, + className, + ...props }) => { const spanClass = cn ( `text-${ TAG_COLOUR[tag.category] }-${ LIGHT_COLOUR_SHADE }`, `dark:text-${ TAG_COLOUR[tag.category] }-${ DARK_COLOUR_SHADE }`) @@ -105,7 +106,7 @@ const TagLink: FC = ({ tag, )} {tag.matchedAlias != null && ( <> - + {tag.matchedAlias} <> → @@ -114,12 +115,12 @@ const TagLink: FC = ({ tag, ? ( {tag.name} ) : ( - {tag.name} )} diff --git a/frontend/src/components/TopNav.tsx b/frontend/src/components/TopNav.tsx index ed2f596..22068fc 100644 --- a/frontend/src/components/TopNav.tsx +++ b/frontend/src/components/TopNav.tsx @@ -7,30 +7,36 @@ import Separator from '@/components/MenuSeparator' import PrefetchLink from '@/components/PrefetchLink' import TopNavUser from '@/components/TopNavUser' import { WikiIdBus } from '@/lib/eventBus/WikiIdBus' -import { tagsKeys, wikiKeys } from '@/lib/queryKeys' +import { materialsKeys, tagsKeys, wikiKeys } from '@/lib/queryKeys' import { fetchTag, fetchTagByName } from '@/lib/tags' +import { fetchMaterial } from '@/lib/materials' import { cn } from '@/lib/utils' import { fetchWikiPage } from '@/lib/wiki' import type { FC, MouseEvent } from 'react' -import type { Menu, MenuVisibleItem, Tag, User } from '@/types' +import type { Material, Menu, MenuVisibleItem, Tag, User } from '@/types' type Props = { user: User | null } -export const menuOutline = ({ tag, wikiId, user, pathName }: { - tag?: Tag | null - wikiId: number | null - user: User | null, - pathName: string }): Menu => { - const postCount = tag?.postCount ?? 0 +export const menuOutline = ( + { tag, material, wikiId, user, pathName }: { + tag?: Tag | null + material?: Material | null + wikiId: number | null + user: User | null, + pathName: string }, +): Menu => { + const postCount = tag?.postCount ?? material?.tag?.postCount ?? 0 const wikiPageFlg = Boolean (/^\/wiki\/(?!new|changes)[^/]+/.test (pathName) && wikiId) const wikiTitle = pathName.split ('/')[2] ?? '' const tagFlg = /^\/tags\/\d+/.test (pathName) + const materialFlg = /^\/materials\/\d+/.test (pathName) + return [ { name: '広場', to: '/posts', subMenu: [ { name: '一覧', to: '/posts' }, @@ -49,12 +55,18 @@ export const menuOutline = ({ tag, wikiId, user, pathName }: { visible: tagFlg }, { name: '履歴', to: `/tags/changes?id=${ tag?.id }`, visible: tagFlg && tag?.category !== 'nico' }] }, - { name: '素材', to: '/materials', visible: false, subMenu: [ + { name: '素材', to: '/materials', visible: true, subMenu: [ { name: '一覧', to: '/materials' }, - { name: '検索', to: '/materials/search', visible: false }, { name: '追加', to: '/materials/new' }, - { name: '全体履歴', to: '/materials/changes', visible: false }, - { name: 'ヘルプ', to: '/wiki/ヘルプ:素材集' }] }, + { name: '抑止', to: '/materials/suppressions' }, + { name: '全体履歴', to: '/materials/changes' }, + { name: 'ヘルプ', to: '/wiki/ヘルプ:素材管理' }, + { component: , visible: materialFlg }, + { name: `広場 (${ postCount || 0 })`, + to: `/posts?tags=${ encodeURIComponent (material?.tag?.name ?? '') }`, + visible: materialFlg && Boolean (material?.tag) }, + { name: '履歴', to: `/materials/changes?material_id=${ material?.id }`, + visible: materialFlg }] }, { name: 'Wiki', to: '/wiki/ヘルプ:ホーム', base: '/wiki', subMenu: [ { name: '検索', to: '/wiki' }, { name: '新規', to: '/wiki/new' }, @@ -119,15 +131,23 @@ const TopNav: FC = ({ user }) => { queryFn: () => fetchWikiPage (wikiIdStr, { }) }) const tagFlg = /^\/tags\/\d+/.test (location.pathname) - const effectiveTitle = (tagFlg ? location.pathname.split ('/')[2] : wikiPage?.title) ?? '' + const materialFlg = /^\/materials\/\d+/.test (location.pathname) + const effectiveTitle = (((tagFlg || materialFlg) + ? location.pathname.split ('/')[2] + : wikiPage?.title) + ?? '') const { data: tag } = useQuery ({ enabled: Boolean (effectiveTitle), queryKey: tagsKeys.show (effectiveTitle), queryFn: () => (tagFlg ? fetchTag : fetchTagByName) (effectiveTitle) }) + const { data: material } = useQuery ({ + enabled: Boolean (effectiveTitle), + queryKey: materialsKeys.show (effectiveTitle), + queryFn: () => fetchMaterial (effectiveTitle) }) - const menu = menuOutline ({ tag, wikiId, user, pathName: location.pathname }) + const menu = menuOutline ({ tag, material, wikiId, user, pathName: location.pathname }) const visibleMenu = menu.filter ((item): item is MenuVisibleItem => item.visible ?? true) const moreMenu = menu.filter (item => !(item.visible ?? true) diff --git a/frontend/src/components/common/PageTitle.tsx b/frontend/src/components/common/PageTitle.tsx index 5b8a697..6ae991e 100644 --- a/frontend/src/components/common/PageTitle.tsx +++ b/frontend/src/components/common/PageTitle.tsx @@ -1,12 +1,14 @@ import React from 'react' +import { cn } from '@/lib/utils' + import type { FC } from 'react' -type Props = { children: React.ReactNode } +type Props = { children: React.ReactNode; className?: string } -const PageTitle: FC = ({ children }) => ( -

    +const PageTitle: FC = ({ children, className, ...rest }) => ( +

    {children}

    ) diff --git a/frontend/src/components/common/Pagination.tsx b/frontend/src/components/common/Pagination.tsx index 798d079..763b77c 100644 --- a/frontend/src/components/common/Pagination.tsx +++ b/frontend/src/components/common/Pagination.tsx @@ -16,8 +16,7 @@ const range = (start: number, end: number): number[] => const getPages = ( page: number, total: number, - siblingCount: number, -): (number | '…')[] => { + siblingCount: number): (number | '…')[] => { if (total <= 1) return [1] diff --git a/frontend/src/components/dialogues/DialogueProvider.tsx b/frontend/src/components/dialogues/DialogueProvider.tsx index 669518f..fb77ecf 100644 --- a/frontend/src/components/dialogues/DialogueProvider.tsx +++ b/frontend/src/components/dialogues/DialogueProvider.tsx @@ -103,8 +103,7 @@ const DialogueProvider: FC = ({ children }) => { choice: options => new Promise (resolve => { push ({ kind: 'choice', options: options as ChoiceOptions, - resolve: resolve as (value: string | null) => void }) - }) }), [push]) + resolve: resolve as (value: string | null) => void })}) }), [push]) const active = queue[0] diff --git a/frontend/src/components/ui/button.tsx b/frontend/src/components/ui/button.tsx index 34abd10..49b4a45 100644 --- a/frontend/src/components/ui/button.tsx +++ b/frontend/src/components/ui/button.tsx @@ -10,41 +10,35 @@ const buttonVariants = cva ( 'rounded-md text-sm font-medium transition-colors', 'focus-visible:outline-none focus-visible:ring-2 focus-visible:ring-slate-400', 'disabled:pointer-events-none disabled:opacity-50', - '[&_svg]:pointer-events-none [&_svg]:size-4 [&_svg]:shrink-0', - ].join (' '), + '[&_svg]:pointer-events-none [&_svg]:size-4 [&_svg]:shrink-0'].join (' '), { variants: { variant: { - default: - 'bg-slate-900 text-white hover:bg-slate-700 dark:bg-slate-100 dark:text-slate-900 dark:hover:bg-slate-300', + default: + 'bg-slate-900 text-white hover:bg-slate-700 dark:bg-slate-100 dark:text-slate-900 dark:hover:bg-slate-300', - destructive: - 'bg-red-600 text-white hover:bg-red-700 dark:bg-red-700 dark:hover:bg-red-600', + destructive: + 'bg-red-600 text-white hover:bg-red-700 dark:bg-red-700 dark:hover:bg-red-600', - outline: - 'border border-slate-300 bg-white text-slate-900 hover:bg-slate-100 dark:border-slate-700 dark:bg-slate-900 dark:text-slate-100 dark:hover:bg-slate-800', + outline: + 'border border-slate-300 bg-white text-slate-900 hover:bg-slate-100 dark:border-slate-700 dark:bg-slate-900 dark:text-slate-100 dark:hover:bg-slate-800', - secondary: - 'bg-slate-100 text-slate-900 hover:bg-slate-200 dark:bg-slate-800 dark:text-slate-100 dark:hover:bg-slate-700', + secondary: + 'bg-slate-100 text-slate-900 hover:bg-slate-200 dark:bg-slate-800 dark:text-slate-100 dark:hover:bg-slate-700', - ghost: - 'text-slate-900 hover:bg-slate-100 dark:text-slate-100 dark:hover:bg-slate-800', + ghost: + 'text-slate-900 hover:bg-slate-100 dark:text-slate-100 dark:hover:bg-slate-800', - link: - 'text-blue-700 underline-offset-4 hover:underline dark:text-blue-300', - }, + link: + 'text-blue-700 underline-offset-4 hover:underline dark:text-blue-300'}, size: { - default: 'h-10 px-4 py-2', - sm: 'h-9 rounded-md px-3', - lg: 'h-11 rounded-md px-8', - icon: 'h-10 w-10', - }, - }, + default: 'h-10 px-4 py-2', + sm: 'h-9 rounded-md px-3', + lg: 'h-11 rounded-md px-8', + icon: 'h-10 w-10'}}, defaultVariants: { variant: 'default', - size: 'default', - }, - }) + size: 'default'}}) export interface ButtonProps extends React.ButtonHTMLAttributes, @@ -57,13 +51,11 @@ const Button = React.forwardRef( const Comp = asChild ? Slot : "button" return ( - ) - } -) + className={cn(buttonVariants({ variant, size, className }))} + ref={ref} + {...props} + />) + }) Button.displayName = "Button" export { Button, buttonVariants } diff --git a/frontend/src/components/ui/dialog.tsx b/frontend/src/components/ui/dialog.tsx index 0eb3096..a6e9a8e 100644 --- a/frontend/src/components/ui/dialog.tsx +++ b/frontend/src/components/ui/dialog.tsx @@ -22,11 +22,9 @@ const DialogOverlay = React.forwardRef< ref={ref} className={cn( "fixed inset-0 z-50 bg-black/80 data-[state=open]:animate-in data-[state=closed]:animate-out data-[state=closed]:fade-out-0 data-[state=open]:fade-in-0", - className - )} + className)} {...props} - /> -)) + />)) DialogOverlay.displayName = DialogPrimitive.Overlay.displayName const DialogContent = React.forwardRef< @@ -62,8 +60,7 @@ const DialogContent = React.forwardRef< 閉ぢる - -)) + )) DialogContent.displayName = DialogPrimitive.Content.displayName const DialogHeader = ({ @@ -73,11 +70,9 @@ const DialogHeader = ({
    -) + />) DialogHeader.displayName = "DialogHeader" const DialogFooter = ({ @@ -87,11 +82,9 @@ const DialogFooter = ({
    -) + />) DialogFooter.displayName = "DialogFooter" const DialogTitle = React.forwardRef< @@ -102,11 +95,9 @@ const DialogTitle = React.forwardRef< ref={ref} className={cn( "text-lg font-semibold leading-none tracking-tight", - className - )} + className)} {...props} - /> -)) + />)) DialogTitle.displayName = DialogPrimitive.Title.displayName const DialogDescription = React.forwardRef< @@ -117,8 +108,7 @@ const DialogDescription = React.forwardRef< ref={ref} className={cn("text-sm text-muted-foreground", className)} {...props} - /> -)) + />)) DialogDescription.displayName = DialogPrimitive.Description.displayName export { diff --git a/frontend/src/components/ui/input.tsx b/frontend/src/components/ui/input.tsx index 68551b9..7fa6a01 100644 --- a/frontend/src/components/ui/input.tsx +++ b/frontend/src/components/ui/input.tsx @@ -1,22 +1,19 @@ -import * as React from "react" +import * as React from 'react' -import { cn } from "@/lib/utils" +import { cn } from '@/lib/utils' -const Input = React.forwardRef>( +const Input = React.forwardRef>( ({ className, type, ...props }, ref) => { return ( - ) - } -) -Input.displayName = "Input" + type={type} + className={cn( + 'flex h-10 w-full rounded-md border border-input bg-background px-3 py-2 text-base ring-offset-background file:border-0 file:bg-transparent file:text-sm file:font-medium file:text-foreground placeholder:text-muted-foreground focus-visible:outline-none focus-visible:ring-2 focus-visible:ring-ring focus-visible:ring-offset-2 disabled:cursor-not-allowed disabled:opacity-50 md:text-sm', + className)} + ref={ref} + {...props} + />) + }) +Input.displayName = 'Input' export { Input } diff --git a/frontend/src/components/ui/switch.tsx b/frontend/src/components/ui/switch.tsx index bc69cf2..1b591d8 100644 --- a/frontend/src/components/ui/switch.tsx +++ b/frontend/src/components/ui/switch.tsx @@ -1,9 +1,9 @@ -"use client" +'use client' -import * as React from "react" -import * as SwitchPrimitives from "@radix-ui/react-switch" +import * as React from 'react' +import * as SwitchPrimitives from '@radix-ui/react-switch' -import { cn } from "@/lib/utils" +import { cn } from '@/lib/utils' const Switch = React.forwardRef< React.ElementRef, @@ -11,19 +11,16 @@ const Switch = React.forwardRef< >(({ className, ...props }, ref) => ( - -)) + )) Switch.displayName = SwitchPrimitives.Root.displayName export { Switch } diff --git a/frontend/src/components/ui/toast.tsx b/frontend/src/components/ui/toast.tsx index 521b94b..01b546d 100644 --- a/frontend/src/components/ui/toast.tsx +++ b/frontend/src/components/ui/toast.tsx @@ -1,129 +1,115 @@ -"use client" - -import * as React from "react" -import * as ToastPrimitives from "@radix-ui/react-toast" -import { cva, type VariantProps } from "class-variance-authority" -import { X } from "lucide-react" - -import { cn } from "@/lib/utils" - -const ToastProvider = ToastPrimitives.Provider - -const ToastViewport = React.forwardRef< - React.ElementRef, - React.ComponentPropsWithoutRef ->(({ className, ...props }, ref) => ( - -)) -ToastViewport.displayName = ToastPrimitives.Viewport.displayName - -const toastVariants = cva( - "group pointer-events-auto relative flex w-full items-center justify-between space-x-4 overflow-hidden rounded-md border p-6 pr-8 shadow-lg transition-all data-[swipe=cancel]:translate-x-0 data-[swipe=end]:translate-x-[var(--radix-toast-swipe-end-x)] data-[swipe=move]:translate-x-[var(--radix-toast-swipe-move-x)] data-[swipe=move]:transition-none data-[state=open]:animate-in data-[state=closed]:animate-out data-[swipe=end]:animate-out data-[state=closed]:fade-out-80 data-[state=closed]:slide-out-to-right-full data-[state=open]:slide-in-from-top-full data-[state=open]:sm:slide-in-from-bottom-full", - { - variants: { - variant: { - default: "border bg-background text-foreground", - destructive: - "destructive group border-destructive bg-destructive text-destructive-foreground", - }, - }, - defaultVariants: { - variant: "default", - }, - } -) - -const Toast = React.forwardRef< - React.ElementRef, - React.ComponentPropsWithoutRef & - VariantProps ->(({ className, variant, ...props }, ref) => { - return ( - - ) -}) -Toast.displayName = ToastPrimitives.Root.displayName - -const ToastAction = React.forwardRef< - React.ElementRef, - React.ComponentPropsWithoutRef ->(({ className, ...props }, ref) => ( - -)) -ToastAction.displayName = ToastPrimitives.Action.displayName - -const ToastClose = React.forwardRef< - React.ElementRef, - React.ComponentPropsWithoutRef ->(({ className, ...props }, ref) => ( - - - -)) -ToastClose.displayName = ToastPrimitives.Close.displayName - -const ToastTitle = React.forwardRef< - React.ElementRef, - React.ComponentPropsWithoutRef ->(({ className, ...props }, ref) => ( - -)) -ToastTitle.displayName = ToastPrimitives.Title.displayName - -const ToastDescription = React.forwardRef< - React.ElementRef, - React.ComponentPropsWithoutRef ->(({ className, ...props }, ref) => ( - -)) -ToastDescription.displayName = ToastPrimitives.Description.displayName - -type ToastProps = React.ComponentPropsWithoutRef - -type ToastActionElement = React.ReactElement - -export { - type ToastProps, - type ToastActionElement, - ToastProvider, - ToastViewport, - Toast, - ToastTitle, - ToastDescription, - ToastClose, - ToastAction, -} +"use client" + +import * as React from "react" +import * as ToastPrimitives from "@radix-ui/react-toast" +import { cva, type VariantProps } from "class-variance-authority" +import { X } from "lucide-react" + +import { cn } from "@/lib/utils" + +const ToastProvider = ToastPrimitives.Provider + +const ToastViewport = React.forwardRef< + React.ElementRef, + React.ComponentPropsWithoutRef +>(({ className, ...props }, ref) => ( + )) +ToastViewport.displayName = ToastPrimitives.Viewport.displayName + +const toastVariants = cva( + "group pointer-events-auto relative flex w-full items-center justify-between space-x-4 overflow-hidden rounded-md border p-6 pr-8 shadow-lg transition-all data-[swipe=cancel]:translate-x-0 data-[swipe=end]:translate-x-[var(--radix-toast-swipe-end-x)] data-[swipe=move]:translate-x-[var(--radix-toast-swipe-move-x)] data-[swipe=move]:transition-none data-[state=open]:animate-in data-[state=closed]:animate-out data-[swipe=end]:animate-out data-[state=closed]:fade-out-80 data-[state=closed]:slide-out-to-right-full data-[state=open]:slide-in-from-top-full data-[state=open]:sm:slide-in-from-bottom-full", + { + variants: { + variant: { + default: "border bg-background text-foreground", + destructive: + "destructive group border-destructive bg-destructive text-destructive-foreground", + }, + }, + defaultVariants: { variant: "default" } } ) + +const Toast = React.forwardRef< + React.ElementRef, + React.ComponentPropsWithoutRef & + VariantProps +>(({ className, variant, ...props }, ref) => { + return ( + ) }) +Toast.displayName = ToastPrimitives.Root.displayName + +const ToastAction = React.forwardRef< + React.ElementRef, + React.ComponentPropsWithoutRef +>(({ className, ...props }, ref) => ( + )) +ToastAction.displayName = ToastPrimitives.Action.displayName + +const ToastClose = React.forwardRef< + React.ElementRef, + React.ComponentPropsWithoutRef +>(({ className, ...props }, ref) => ( + + + )) +ToastClose.displayName = ToastPrimitives.Close.displayName + +const ToastTitle = React.forwardRef< + React.ElementRef, + React.ComponentPropsWithoutRef +>(({ className, ...props }, ref) => ( + )) +ToastTitle.displayName = ToastPrimitives.Title.displayName + +const ToastDescription = React.forwardRef< + React.ElementRef, + React.ComponentPropsWithoutRef +>(({ className, ...props }, ref) => ( + )) +ToastDescription.displayName = ToastPrimitives.Description.displayName + +type ToastProps = React.ComponentPropsWithoutRef + +type ToastActionElement = React.ReactElement + +export { + type ToastProps, + type ToastActionElement, + ToastProvider, + ToastViewport, + Toast, + ToastTitle, + ToastDescription, + ToastClose, + ToastAction, +} diff --git a/frontend/src/components/ui/toaster.tsx b/frontend/src/components/ui/toaster.tsx index b6147e4..dc8cfee 100644 --- a/frontend/src/components/ui/toaster.tsx +++ b/frontend/src/components/ui/toaster.tsx @@ -1,31 +1,30 @@ -'use client' - -import { useToast } from '@/components/ui/use-toast' -import { Toast, - ToastClose, - ToastDescription, - ToastProvider, - ToastTitle, - ToastViewport } from '@/components/ui/toast' - - -export const Toaster = () => { - const { toasts } = useToast () - - return ( - - {toasts.map (({ id, title, description, action, ...props }) => ( - -
    - {title && {title}} - {description && {description}} -
    - {action} - -     ))} - -     - ) -} +'use client' + +import { useToast } from '@/components/ui/use-toast' +import { Toast, + ToastClose, + ToastDescription, + ToastProvider, + ToastTitle, + ToastViewport } from '@/components/ui/toast' + + +export const Toaster = () => { + const { toasts } = useToast () + + return ( + + {toasts.map (({ id, title, description, action, ...props }) => ( + +
    + {title && {title}} + {description && {description}} +
    + {action} + +     ))} + +     ) +} diff --git a/frontend/src/components/ui/use-toast.tsx b/frontend/src/components/ui/use-toast.tsx index e8e7155..dcfbd7d 100644 --- a/frontend/src/components/ui/use-toast.tsx +++ b/frontend/src/components/ui/use-toast.tsx @@ -58,8 +58,7 @@ const addToRemoveQueue = (toastId: string) => { toastTimeouts.delete(toastId) dispatch({ type: "REMOVE_TOAST", - toastId: toastId, - }) + toastId: toastId}) }, TOAST_REMOVE_DELAY) toastTimeouts.set(toastId, timeout) @@ -69,17 +68,14 @@ export const reducer = (state: State, action: Action): State => { switch (action.type) { case "ADD_TOAST": return { - ...state, - toasts: [action.toast, ...state.toasts].slice(0, TOAST_LIMIT), - } + ...state, + toasts: [action.toast, ...state.toasts].slice(0, TOAST_LIMIT)} case "UPDATE_TOAST": return { - ...state, - toasts: state.toasts.map((t) => - t.id === action.toast.id ? { ...t, ...action.toast } : t - ), - } + ...state, + toasts: state.toasts.map((t) => + t.id === action.toast.id ? { ...t, ...action.toast } : t)} case "DISMISS_TOAST": { const { toastId } = action @@ -87,36 +83,31 @@ export const reducer = (state: State, action: Action): State => { // ! Side effects ! - This could be extracted into a dismissToast() action, // but I'll keep it here for simplicity if (toastId) { - addToRemoveQueue(toastId) + addToRemoveQueue(toastId) } else { - state.toasts.forEach((toast) => { - addToRemoveQueue(toast.id) - }) + state.toasts.forEach((toast) => { + addToRemoveQueue(toast.id) + }) } return { - ...state, - toasts: state.toasts.map((t) => - t.id === toastId || toastId === undefined - ? { - ...t, - open: false, - } - : t - ), - } + ...state, + toasts: state.toasts.map((t) => + t.id === toastId || toastId === undefined + ? { + ...t, + open: false} + : t)} } case "REMOVE_TOAST": if (action.toastId === undefined) { - return { - ...state, - toasts: [], - } + return { + ...state, + toasts: []} } return { - ...state, - toasts: state.toasts.filter((t) => t.id !== action.toastId), - } + ...state, + toasts: state.toasts.filter((t) => t.id !== action.toastId)} } } @@ -139,8 +130,7 @@ function toast({ ...props }: Toast) { const update = (props: ToasterToast) => dispatch({ type: "UPDATE_TOAST", - toast: { ...props, id }, - }) + toast: { ...props, id }}) const dismiss = () => dispatch({ type: "DISMISS_TOAST", toastId: id }) dispatch({ @@ -150,16 +140,13 @@ function toast({ ...props }: Toast) { id, open: true, onOpenChange: (open) => { - if (!open) dismiss() - }, - }, - }) + if (!open) dismiss() + }}}) return { id: id, dismiss, - update, - } + update} } function useToast() { @@ -170,7 +157,7 @@ function useToast() { return () => { const index = listeners.indexOf(setState) if (index > -1) { - listeners.splice(index, 1) + listeners.splice(index, 1) } } }, [state]) @@ -178,8 +165,7 @@ function useToast() { return { ...state, toast, - dismiss: (toastId?: string) => dispatch({ type: "DISMISS_TOAST", toastId }), - } + dismiss: (toastId?: string) => dispatch({ type: "DISMISS_TOAST", toastId })} } export { useToast, toast } diff --git a/frontend/src/config.sample.ts b/frontend/src/config.sample.ts index 620811c..dc9e1eb 100644 --- a/frontend/src/config.sample.ts +++ b/frontend/src/config.sample.ts @@ -3,8 +3,7 @@ const ENV: string = 'development' const config = { API_BASE_URL: ENV === 'production' ? 'https://hub.nizika.monster/api' : 'http://localhost:3002', - SITE_TITLE: 'ぼざクリ タグ広場' -} + SITE_TITLE: 'ぼざクリ タグ広場'} export const API_BASE_URL = config.API_BASE_URL export const SITE_TITLE = config.SITE_TITLE diff --git a/frontend/src/consts.ts b/frontend/src/consts.ts index 466b7f0..a7be4f4 100644 --- a/frontend/src/consts.ts +++ b/frontend/src/consts.ts @@ -10,8 +10,7 @@ export const CATEGORIES = [ 'general', 'material', 'meta', - 'nico', - ] as const + 'nico'] as const export const CATEGORY_NAMES: Record = { deerjikist: 'ニジラー', @@ -20,16 +19,14 @@ export const CATEGORY_NAMES: Record = { general: '一般', material: '素材', meta: 'メタタグ', - nico: 'ニコニコタグ', - } as const + nico: 'ニコニコタグ'} as const export const FETCH_POSTS_ORDER_FIELDS = [ 'title', 'url', 'original_created_at', 'created_at', - 'updated_at', - ] as const + 'updated_at'] as const export const PLATFORMS = ['nico', 'youtube'] as const @@ -43,13 +40,11 @@ export const TAG_COLOUR = { general: 'cyan', material: 'orange', meta: 'yellow', - nico: 'gray', - } as const satisfies Record + nico: 'gray'} as const satisfies Record export const USER_ROLES = ['admin', 'member', 'guest'] as const export const ViewFlagBehavior = { OnShowedDetail: 1, OnClickedLink: 2, - NotAuto: 3, - } as const + NotAuto: 3} as const diff --git a/frontend/src/lib/api.ts b/frontend/src/lib/api.ts index a93de11..654c51f 100644 --- a/frontend/src/lib/api.ts +++ b/frontend/src/lib/api.ts @@ -23,8 +23,7 @@ const apiP = async ( method: 'post' | 'put' | 'patch', path: string, body?: unknown, - opt?: Opt, -): Promise => { + opt?: Opt): Promise => { const res = await client[method] (path, body ?? { }, withUserCode (opt)) if (opt?.responseType === 'blob') return res.data as T @@ -34,8 +33,7 @@ const apiP = async ( export const apiGet = async ( path: string, - opt?: Opt, -): Promise => { + opt?: Opt): Promise => { const res = await client.get (path, withUserCode (opt)) if (opt?.responseType === 'blob') return res.data as T @@ -46,28 +44,24 @@ export const apiGet = async ( export const apiPost = async ( path: string, body?: unknown, - opt?: Opt, -): Promise => apiP ('post', path, body, opt) + opt?: Opt): Promise => apiP ('post', path, body, opt) export const apiPut = async ( path: string, body?: unknown, - opt?: Opt, -): Promise => apiP ('put', path, body, opt) + opt?: Opt): Promise => apiP ('put', path, body, opt) export const apiPatch = async ( path: string, body?: unknown, - opt?: Opt, -): Promise => apiP ('patch', path, body, opt) + opt?: Opt): Promise => apiP ('patch', path, body, opt) export const apiDelete = async ( path: string, - opt?: Opt, -): Promise => { + opt?: Opt): Promise => { const res = await client.delete (path, withUserCode (opt)) if (res.data == null || res.data === '') return undefined as T diff --git a/frontend/src/lib/gekanator.ts b/frontend/src/lib/gekanator.ts index 1299b0c..0acb338 100644 --- a/frontend/src/lib/gekanator.ts +++ b/frontend/src/lib/gekanator.ts @@ -104,8 +104,7 @@ export type BuildGekanatorQuestionsOptions = { export const normalizeTitleLengthCondition = ( - condition: GekanatorQuestionCondition, -): GekanatorQuestionCondition => { + condition: GekanatorQuestionCondition): GekanatorQuestionCondition => { switch (condition.type) { case 'title-length-greater-than': @@ -119,8 +118,7 @@ export const normalizeTitleLengthCondition = ( export const titleLengthMinimumForCondition = ( - condition: GekanatorQuestionCondition, -): number | null => { + condition: GekanatorQuestionCondition): number | null => { switch (condition.type) { case 'title-length-at-least': @@ -134,8 +132,7 @@ export const titleLengthMinimumForCondition = ( export const questionIdForCondition = ( - condition: NonPostSimilarityCondition, -): string => { + condition: NonPostSimilarityCondition): string => { switch (condition.type) { case 'tag': @@ -161,8 +158,7 @@ export const questionIdForCondition = ( const directExampleAnswerFor = ( question: StoredGekanatorQuestion, - post: Post, -): GekanatorAnswerValue | null => { + post: Post): GekanatorAnswerValue | null => { if (question.kind !== 'post_similarity' && question.kind !== 'tag') return null @@ -178,15 +174,13 @@ const directExampleAnswerFor = ( export const isLearnedSemanticQuestion = ( - question: StoredGekanatorQuestion | GekanatorQuestion, -): boolean => + question: StoredGekanatorQuestion | GekanatorQuestion): boolean => question.kind === 'post_similarity' && question.source === 'user_suggested' export const learnedSemanticSideForAnswer = ( - answer: GekanatorAnswerValue | null, -): LearnedSemanticSide => { + answer: GekanatorAnswerValue | null): LearnedSemanticSide => { if (answer === 'yes' || answer === 'partial') return 'positive' @@ -314,8 +308,7 @@ const questionableTag = (post: Post, key: string): boolean => { const questionMatches = ( post: Post, - question: StoredGekanatorQuestion, -): boolean => { + question: StoredGekanatorQuestion): boolean => { const directAnswer = directExampleAnswerFor (question, post) if (directAnswer) return question.kind === 'post_similarity' @@ -350,8 +343,7 @@ const questionMatches = ( export const expectedAnswerForQuestion = ( question: StoredGekanatorQuestion | GekanatorQuestion | undefined, - post: Post | null, -): GekanatorAnswerValue | null => { + post: Post | null): GekanatorAnswerValue | null => { if (!(question) || !(post)) return null @@ -382,14 +374,12 @@ export const expectedAnswerForQuestion = ( export const learnedSemanticSideForPost = ( question: StoredGekanatorQuestion | GekanatorQuestion | undefined, - post: Post | null, -): LearnedSemanticSide => + post: Post | null): LearnedSemanticSide => learnedSemanticSideForAnswer (expectedAnswerForQuestion (question, post)) export const restoreGekanatorQuestion = ( - question: StoredGekanatorQuestion, -): GekanatorQuestion => { + question: StoredGekanatorQuestion): GekanatorQuestion => { const normalizedCondition = normalizeTitleLengthCondition (question.condition) const normalizedQuestion = { ...question, @@ -408,8 +398,7 @@ export const restoreGekanatorQuestion = ( export const storeGekanatorQuestion = ( - question: GekanatorQuestion, -): StoredGekanatorQuestion => ({ + question: GekanatorQuestion): StoredGekanatorQuestion => ({ id: question.condition.type === 'title-length-greater-than' ? `title:length-at-least:${ question.condition.length + 1 }` : question.id, @@ -436,8 +425,7 @@ export const fetchGekanatorQuestions = async (): Promise => { + nonce?: string): Promise => { const data = await apiGet<{ questions: GekanatorExtraQuestion[] }> ( `/gekanator/games/${ gameId }/extra_questions`, { params: nonce ? { nonce } : undefined }) @@ -447,8 +435,7 @@ export const fetchGekanatorExtraQuestions = async ( export const buildGekanatorQuestions = ( posts: Post[], - options: BuildGekanatorQuestionsOptions = { }, -): GekanatorQuestion[] => { + options: BuildGekanatorQuestionsOptions = { }): GekanatorQuestion[] => { const { includeTitleContains = true, tagQuestionCap = 192, @@ -490,8 +477,7 @@ export const buildGekanatorQuestions = ( const usefulEntries = ( counts: Map, - cap: number, - ) => + cap: number) => [...counts.entries ()] .filter (([, count]) => count > 0 && count < posts.length) .sort ((a, b) => Math.abs (posts.length / 2 - a[1]) diff --git a/frontend/src/lib/gekanatorCandidateRecovery.ts b/frontend/src/lib/gekanatorCandidateRecovery.ts index aec3123..2c81874 100644 --- a/frontend/src/lib/gekanatorCandidateRecovery.ts +++ b/frontend/src/lib/gekanatorCandidateRecovery.ts @@ -32,8 +32,7 @@ export const candidatePostsFor = ( answers: GekanatorAnswerLog[] softenedQuestionIds: Set rejectedPostIds: Set - recoveredCandidatePosts: Map }, -): Post[] => { + recoveredCandidatePosts: Map }): Post[] => { const questionById = new Map (questions.map (question => [question.id, question])) return posts.filter (post => { @@ -76,8 +75,7 @@ export const candidatePostsFor = ( export const hardFilteredPostsForAnswer = ( { posts, question, answer }: { posts: Post[] question: GekanatorQuestion - answer: GekanatorAnswerValue }, -): Post[] => { + answer: GekanatorAnswerValue }): Post[] => { if (!(questionSupportsAnswerBasedHardFiltering (question))) return posts @@ -98,8 +96,7 @@ const concreteAnswerOptions: GekanatorAnswerValue[] = ['yes', 'no', 'partial', ' export const allConcreteAnswerOptionsExhausted = ( posts: Post[], - question: GekanatorQuestion | null, -): boolean => { + question: GekanatorQuestion | null): boolean => { if (!(question)) return false @@ -125,8 +122,7 @@ export const recoverCandidatePosts = ( recoveredCandidatePosts: Map eligiblePostIds: Set answerCountAtRecovery: number - recoveryStepCount: number }, -): { recoveredCandidatePosts: Map + recoveryStepCount: number }): { recoveredCandidatePosts: Map recoveryStepCount: number } | null => { const recovered = new Map (recoveredCandidatePosts) const targetSize = nextRecoveryTargetSize (recoveryStepCount) diff --git a/frontend/src/lib/gekanatorQuestionFilters.ts b/frontend/src/lib/gekanatorQuestionFilters.ts index e358e38..08e6d63 100644 --- a/frontend/src/lib/gekanatorQuestionFilters.ts +++ b/frontend/src/lib/gekanatorQuestionFilters.ts @@ -8,8 +8,7 @@ import type { export const monthForCondition = ( - condition: GekanatorQuestion['condition'], -): number | null => { + condition: GekanatorQuestion['condition']): number | null => { if (condition.type === 'original-month') return condition.month @@ -24,8 +23,7 @@ export const monthForCondition = ( const isTitleLengthContradiction = ( candidate: GekanatorQuestion['condition'], previous: GekanatorQuestion['condition'], - answer: GekanatorAnswerValue, -): boolean => { + answer: GekanatorAnswerValue): boolean => { const candidateLength = titleLengthMinimumForCondition (candidate) const previousLength = titleLengthMinimumForCondition (previous) if (candidateLength === null || previousLength === null) @@ -45,8 +43,7 @@ const isTitleLengthContradiction = ( const isQuestionRedundantAfterAnswers = ( question: GekanatorQuestion, - answers: GekanatorAnswerLog[], -): boolean => answers.some (answer => { + answers: GekanatorAnswerLog[]): boolean => answers.some (answer => { const previous = answer.questionCondition return previous !== undefined && isTitleLengthContradiction (question.condition, previous, answer.answer) @@ -56,8 +53,7 @@ const isQuestionRedundantAfterAnswers = ( const isSourceFactBlocked = ( candidate: GekanatorQuestion['condition'], previous: GekanatorQuestion['condition'], - answer: GekanatorAnswerValue, -): boolean => { + answer: GekanatorAnswerValue): boolean => { if (candidate.type !== 'source' || previous.type !== 'source') return false @@ -76,8 +72,7 @@ const isSourceFactBlocked = ( const isOriginalYearFactBlocked = ( candidate: GekanatorQuestion['condition'], previous: GekanatorQuestion['condition'], - answer: GekanatorAnswerValue, -): boolean => { + answer: GekanatorAnswerValue): boolean => { if (candidate.type !== 'original-year' || previous.type !== 'original-year') return false @@ -96,8 +91,7 @@ const isOriginalYearFactBlocked = ( const isOriginalMonthFactBlocked = ( candidate: GekanatorQuestion['condition'], previous: GekanatorQuestion['condition'], - answer: GekanatorAnswerValue, -): boolean => { + answer: GekanatorAnswerValue): boolean => { switch (answer) { case 'yes': @@ -143,8 +137,7 @@ const isOriginalMonthFactBlocked = ( const isFactQuestionBlocked = ( candidate: GekanatorQuestion['condition'], previous: GekanatorQuestion['condition'], - answer: GekanatorAnswerValue, -): boolean => { + answer: GekanatorAnswerValue): boolean => { if (!(answer === 'yes' || answer === 'no')) return false @@ -156,8 +149,7 @@ const isFactQuestionBlocked = ( export const isQuestionHardFilteredAfterAnswers = ( question: GekanatorQuestion, - answers: GekanatorAnswerLog[], -): boolean => answers.some (answer => { + answers: GekanatorAnswerLog[]): boolean => answers.some (answer => { const previous = answer.questionCondition if (previous === undefined) return false diff --git a/frontend/src/lib/materials.ts b/frontend/src/lib/materials.ts new file mode 100644 index 0000000..ab9e044 --- /dev/null +++ b/frontend/src/lib/materials.ts @@ -0,0 +1,139 @@ +import { apiGet, isApiError, apiPost, apiPut } from '@/lib/api' + +import type { Material, + MaterialIndexResponse, + MaterialVersion, + FetchMaterialsParams, + MaterialFilter, + MaterialSyncSuppression, + MaterialSidebarTag, + MaterialTagTree } from '@/types' + +export type FetchMaterialTreeParams = { + parentId?: number | null + materialFilter: MaterialFilter } + +export type MaterialSyncSuppressionResponse = { suppressions: MaterialSyncSuppression[] } + +export type MaterialChangesResponse = { + versions: MaterialVersion[] + count: number } + +const MATERIAL_FILTERS: MaterialFilter[] = ['present', 'missing', 'any'] + + +export const parseMaterialFilter = ( + value: unknown, + fallback: MaterialFilter = 'present', +): MaterialFilter => + typeof value === 'string' && MATERIAL_FILTERS.includes (value as MaterialFilter) + ? value as MaterialFilter + : fallback + + +export const fetchMaterials = async ( + { q, tagState, mediaKind, createdFrom, createdTo, + updatedFrom, updatedTo, sort, direction, page, + tagId, includeDescendants, groupBy, + limit }: FetchMaterialsParams, +): Promise => + await apiGet ('/materials', { params: { + ...(q && { q }), + tag_state: tagState, + media_kind: mediaKind, + ...(tagId != null && { tag_id: tagId }), + ...(includeDescendants && { include_descendants: '1' }), + ...(groupBy !== 'none' && { group_by: groupBy }), + ...(createdFrom && { created_from: createdFrom }), + ...(createdTo && { created_to: createdTo }), + ...(updatedFrom && { updated_from: updatedFrom }), + ...(updatedTo && { updated_to: updatedTo }), + sort, + direction, + page, + limit} }) + + +export const fetchMaterial = async (id: string): Promise => { + try + { + return await apiGet (`/materials/${ id }`) + } + catch (error) + { + if (isApiError (error) && error.response?.status === 404) + return null + throw error + } +} + + +export const fetchMaterialChanges = async ( + { materialId, tag, eventType, page, limit }: { + materialId?: string + tag?: string + eventType?: string + page: number + limit: number + }): Promise => + await apiGet ('/materials/versions', { params: { + ...(materialId && { material_id: materialId }), + ...(tag && { tag }), + ...(eventType && { event_type: eventType }), + page, + limit} }) + + +export const fetchMaterialTagTree = async ( + { parentId, materialFilter }: FetchMaterialTreeParams): Promise => + await apiGet ('/tags/with-depth', { params: { + ...(parentId != null && { parent: String (parentId) }), + material_filter: materialFilter} }) + + +export const fetchMaterialTagByName = async ( + name: string, + materialFilter: MaterialFilter): Promise => { + try + { + return await apiGet (`/tags/name/${ encodeURIComponent (name) }/materials`, + { params: { material_filter: materialFilter } }) + } + catch (error) + { + if (isApiError (error) && error.response?.status === 404) + return null + throw error + } +} + + +export const createMaterial = async (formData: FormData): Promise => + await apiPost ('/materials', formData) + + +export const updateMaterial = async ( + id: string, + formData: FormData): Promise => + await apiPut (`/materials/${ id }`, formData) + + +export const fetchMaterialSyncSuppressions = + async (): Promise => + await apiGet ('/materials/suppressions') + + +export const createMaterialSyncSuppression = async ( + payload: { + sourceKind: string + sourceUri?: string + drivePath?: string + driveFileId?: string + reason: string + }): Promise => + await apiPost ('/materials/suppressions', { + source_kind: payload.sourceKind, + source_uri: payload.sourceUri, + drive_path: payload.drivePath, + drive_file_id: payload.driveFileId, + reason: payload.reason}) diff --git a/frontend/src/lib/posts.ts b/frontend/src/lib/posts.ts index a1febff..c192b52 100644 --- a/frontend/src/lib/posts.ts +++ b/frontend/src/lib/posts.ts @@ -5,8 +5,7 @@ import type { FetchPostsParams, Post, PostVersion } from '@/types' export const fetchPosts = async ( { url, title, tags, match, createdFrom, createdTo, updatedFrom, updatedTo, - originalCreatedFrom, originalCreatedTo, page, limit, order }: FetchPostsParams, -): Promise<{ + originalCreatedFrom, originalCreatedTo, page, limit, order }: FetchPostsParams): Promise<{ posts: Post[] count: number }> => await apiGet ('/posts', { params: { @@ -33,8 +32,7 @@ export const fetchPostChanges = async ( post?: string tag?: string page: number - limit: number }, -): Promise<{ + limit: number }): Promise<{ versions: PostVersion[] count: number }> => await apiGet ('/posts/versions', { params: { ...(post && { post }), @@ -52,8 +50,7 @@ export const updatePost = async ( { baseVersionNo, force, merge }: { baseVersionNo?: number force?: boolean - merge?: boolean } -) => + merge?: boolean }) => await apiPut ( `/posts/${ post.id }`, { title: post.title, diff --git a/frontend/src/lib/queryKeys.ts b/frontend/src/lib/queryKeys.ts index 7e10ab1..293f849 100644 --- a/frontend/src/lib/queryKeys.ts +++ b/frontend/src/lib/queryKeys.ts @@ -1,4 +1,10 @@ -import type { FetchNicoTagsParams, FetchPostsParams, FetchTagsParams } from '@/types' +import type { + FetchNicoTagsParams, + FetchMaterialsParams, + FetchPostsParams, + FetchTagsParams, + MaterialFilter, +} from '@/types' export const postsKeys = { root: ['posts'] as const, @@ -25,6 +31,27 @@ export const tagsKeys = { ['tags', 'changes', p] as const, deerjikists: (id: string) => ['tags', 'deerjikists', id] as const } +export const materialsKeys = { + root: ['materials'] as const, + index: (p: FetchMaterialsParams) => ['materials', 'index', p] as const, + changes: (p: { + materialId?: string + tag?: string + eventType?: string + page: number + limit: number + }) => ['materials', 'changes', p] as const, + byTagName: (name: string, materialFilter: MaterialFilter) => + ['materials', 'tag', name, materialFilter] as const, + show: (id: string) => ['materials', id] as const, + suppressions: () => ['materials', 'suppressions'] as const, + tree: (p: { + parentId?: number | null + materialFilter: MaterialFilter + }) => ['materials', 'tree', p] as const, + unclassified: (p: { page?: number; limit?: number } = { }) => + ['materials', 'unclassified', p] as const} + export const wikiKeys = { root: ['wiki'] as const, index: (p: { title?: string }) => ['wiki', 'index', p] as const, diff --git a/frontend/src/lib/tags.ts b/frontend/src/lib/tags.ts index 31cad85..4c017ab 100644 --- a/frontend/src/lib/tags.ts +++ b/frontend/src/lib/tags.ts @@ -11,8 +11,7 @@ import type { Deerjikist, export const fetchTags = async ( { post, name, category, postCountGTE, postCountLTE, createdFrom, createdTo, updatedFrom, updatedTo, deprecated, - page, limit, order }: FetchTagsParams, -): Promise<{ tags: Tag[] + page, limit, order }: FetchTagsParams): Promise<{ tags: Tag[] count: number }> => await apiGet ('/tags', { params: { ...(post != null && { post }), @@ -31,8 +30,7 @@ export const fetchTags = async ( export const fetchNicoTags = async ( - { name, linkedTag, linkStatus, page, limit, order }: FetchNicoTagsParams, -): Promise<{ tags: NicoTag[] + { name, linkedTag, linkStatus, page, limit, order }: FetchNicoTagsParams): Promise<{ tags: NicoTag[] count: number }> => await apiGet ('/tags/nico', { params: { page, @@ -70,14 +68,12 @@ export const fetchTagChanges = async ( { id, page, limit }: { id?: string page: number - limit: number }, -): Promise<{ + limit: number }): Promise<{ versions: TagVersion[] count: number }> => await apiGet ('/tags/versions', { params: { ...(id && { id }), page, limit } }) export const fetchDeerjikistsByTag = async ( - id: string, -): Promise<{ tag: Tag; deerjikists: Deerjikist[]}> => + id: string): Promise<{ tag: Tag; deerjikists: Deerjikist[] }> => await apiGet (`/tags/${ id }/deerjikists`) diff --git a/frontend/src/lib/users.ts b/frontend/src/lib/users.ts index 9f31329..ea5b28c 100644 --- a/frontend/src/lib/users.ts +++ b/frontend/src/lib/users.ts @@ -4,5 +4,4 @@ const CONTENT_EDITOR_ROLES: readonly UserRole[] = ['admin', 'member'] export const canEditContent = ( - user: Pick | null | undefined, -): boolean => user != null && CONTENT_EDITOR_ROLES.includes (user.role) + user: Pick | null | undefined): boolean => user != null && CONTENT_EDITOR_ROLES.includes (user.role) diff --git a/frontend/src/lib/utils.ts b/frontend/src/lib/utils.ts index 69ccbb8..6e11803 100644 --- a/frontend/src/lib/utils.ts +++ b/frontend/src/lib/utils.ts @@ -12,8 +12,7 @@ export const cn = (...inputs: ClassValue[]) => twMerge (clsx (...inputs)) export const dateString = ( d: string | Date, - unknown: 'month' | 'day' | 'hour' | 'minute' | 'second' | null = null, -): string => + unknown: 'month' | 'day' | 'hour' | 'minute' | 'second' | null = null): string => toDate (d).toLocaleString ( 'ja-JP-u-ca-japanese', { era: 'long', @@ -28,8 +27,7 @@ export const dateString = ( export const originalCreatedAtString = ( f: string | Date | null, - b: string | Date | null, -): string => { + b: string | Date | null): string => { const from = f ? toDate (f) : null const before = b ? toDate (b) : null diff --git a/frontend/src/lib/wiki.ts b/frontend/src/lib/wiki.ts index 276db3a..b6f63c2 100644 --- a/frontend/src/lib/wiki.ts +++ b/frontend/src/lib/wiki.ts @@ -4,22 +4,19 @@ import type { WikiPage } from '@/types' export const fetchWikiPages = async ( - { title }: { title?: string }, -): Promise => + { title }: { title?: string }): Promise => await apiGet ('/wiki', { params: { title } }) export const fetchWikiPage = async ( id: string, - { version }: { version?: string }, -): Promise => + { version }: { version?: string }): Promise => await apiGet (`/wiki/${ id }`, { params: version ? { version } : { } }) export const fetchWikiPageByTitle = async ( title: string, - { version }: { version?: string }, -): Promise => { + { version }: { version?: string }): Promise => { try { return await apiGet (`/wiki/title/${ encodeURIComponent (title) }`, { params: { version } }) diff --git a/frontend/src/pages/GekanatorPage.tsx b/frontend/src/pages/GekanatorPage.tsx index 1eabcd0..6ea1f84 100644 --- a/frontend/src/pages/GekanatorPage.tsx +++ b/frontend/src/pages/GekanatorPage.tsx @@ -238,8 +238,7 @@ const createGameSeed = (): string => { const normalizeStoredQuestionId = ( questionId: string, - condition?: GekanatorQuestionCondition, -): string => { + condition?: GekanatorQuestionCondition): string => { if (condition?.type === 'title-length-greater-than') return `title:length-at-least:${ condition.length + 1 }` @@ -312,8 +311,7 @@ const sourcePriorityForMerge = (question: GekanatorQuestion): number => { const shouldReplaceMergedQuestion = ( current: GekanatorQuestion | undefined, - candidate: GekanatorQuestion, -): boolean => { + candidate: GekanatorQuestion): boolean => { if (!(current)) return true @@ -408,8 +406,7 @@ const loadRecentGames = (): RecentGameSummary[] => { const storeRecentGameSummary = ( - summary: RecentGameSummary, -): RecentGameSummary[] => { + summary: RecentGameSummary): RecentGameSummary[] => { const next = [summary, ...loadRecentGames ().filter (item => (item.savedAt !== summary.savedAt @@ -459,8 +456,7 @@ const resettableExtraQuestionState = (): { const recoveredCandidateMapFromStored = ( items: RecoveredCandidatePost[], - scores: [number, number][], -): Map => { + scores: [number, number][]): Map => { const storedScores = new Map (scores) return new Map (items.map (item => [item.postId, { @@ -470,8 +466,7 @@ const recoveredCandidateMapFromStored = ( const storedRecoveredCandidatesFromMap = ( - recoveredCandidatePosts: Map, -): RecoveredCandidatePost[] => + recoveredCandidatePosts: Map): RecoveredCandidatePost[] => [...recoveredCandidatePosts.entries ()] .map (([postId, recoveredCandidate]) => ({ postId, @@ -513,8 +508,7 @@ const distributionEntropy = (weights: number[]): number => const questionCategoryPenalty = ( question: GekanatorQuestion, answerCount: number, - repeatPenalty: number, -): number => { + repeatPenalty: number): number => { const earlyFactor = Math.max (0, (3 - answerCount) / 3) const titleLengthPenalty = (() => { if (titleLengthMinimumForCondition (question.condition) == null) @@ -553,8 +547,7 @@ const relatedPostIdsOf = (post: Post): number[] => { const userPriorWeightsFor = ( posts: Post[], - recentGames: RecentGameSummary[], -): Map => { + recentGames: RecentGameSummary[]): Map => { const postById = new Map (posts.map (post => [post.id, post])) const weights = new Map () const addWeight = (postId: number, weight: number) => { @@ -581,14 +574,12 @@ const userPriorWeightsFor = ( const answerWeightFor = ( questionId: string, - softenedQuestionIds: Set, -): number => softenedQuestionIds.has (questionId) ? softenedAnswerWeight : 1 + softenedQuestionIds: Set): number => softenedQuestionIds.has (questionId) ? softenedAnswerWeight : 1 const scoreWeightForAnswer = ( answer: GekanatorAnswerLog, - softenedQuestionIds: Set, -): number => + softenedQuestionIds: Set): number => answerWeightFor (answer.questionId, softenedQuestionIds) * ( answer.questionPurpose === 'learning_user_suggested' @@ -638,8 +629,7 @@ const titleTermPattern = const addPostIdToIndex = ( index: Map>, key: K, - postId: number, -) => { + postId: number) => { const current = index.get (key) if (current) { @@ -652,8 +642,7 @@ const addPostIdToIndex = ( const buildMaterialIndex = ( - posts: Post[], -): GekanatorQuestionMaterialIndex => { + posts: Post[]): GekanatorQuestionMaterialIndex => { const postById = new Map () const tagKeysByPostId = new Map () const postIdsByTagKey = new Map> () @@ -784,8 +773,7 @@ const originalDateQuestionTextFor = ( condition: Extract< GekanatorQuestionCondition, { type: 'original-year' | 'original-month' | 'original-month-day' } - >, -): string => { + >): string => { switch (condition.type) { case 'original-year': @@ -852,8 +840,7 @@ const isLearnableTagKey = (key: string): boolean => !(key.startsWith ('nico:')) const isUserSuggestedLearnedSemanticQuestion = ( - question: GekanatorQuestion, -): boolean => isLearnedSemanticQuestion (question) + question: GekanatorQuestion): boolean => isLearnedSemanticQuestion (question) type LearnedSemanticCandidateStats = { @@ -872,8 +859,7 @@ const learnedSemanticStatsForCandidateIds = ( question }: { candidateIds: number[] posts: Post[] - question: GekanatorQuestion }, -): LearnedSemanticCandidateStats => { + question: GekanatorQuestion }): LearnedSemanticCandidateStats => { const candidateIdSet = new Set (candidateIds) const positiveIds = new Set () const negativeIds = new Set () @@ -915,8 +901,7 @@ const learnedSemanticQuestionIsEffectiveForCandidateIds = ( question }: { candidateIds: number[] posts: Post[] - question: GekanatorQuestion }, -): boolean => { + question: GekanatorQuestion }): boolean => { if (!(isUserSuggestedLearnedSemanticQuestion (question))) return false @@ -936,8 +921,7 @@ const learnedSemanticQuestionIsEffectiveForCandidateIds = ( const directSemanticAnswerForPost = ( question: GekanatorQuestion, - post: Post, -): GekanatorAnswerValue | null => { + post: Post): GekanatorAnswerValue | null => { const direct = question.exampleAnswers?.[String (post.id) as `${ number }`] if (direct) return direct @@ -956,8 +940,7 @@ const learnedSemanticLearningValueForTopPosts = ( question: GekanatorQuestion learningTargetPosts: Post[] candidateIds: number[] - posts: Post[] }, -): { missingTopCount: number + posts: Post[] }): { missingTopCount: number knownCount: number hasLearningValue: boolean } => { const missingTopCount = @@ -1000,8 +983,7 @@ const learningTargetPostsForCandidates = ({ const questionPurposeCountsFor = ( - answers: GekanatorAnswerLog[], -): { + answers: GekanatorAnswerLog[]): { effectiveUserSuggestedCount: number learningUserSuggestedCount: number normalQuestionCount: number @@ -1041,8 +1023,7 @@ const questionPurposeCountsFor = ( const learnedSemanticNarrowPenaltyForStats = ( candidateCount: number, - stats: LearnedSemanticCandidateStats, -): number => { + stats: LearnedSemanticCandidateStats): number => { const minSide = candidateCount < 10 ? 1 : Math.max (3, candidateCount * .08) return stats.positiveCount < minSide || stats.negativeCount < minSide ? .15 : 0 } @@ -1050,8 +1031,7 @@ const learnedSemanticNarrowPenaltyForStats = ( const learnedSemanticScoreDeltaForExpectedAnswer = ( userAnswer: GekanatorAnswerValue, - expectedAnswer: GekanatorAnswerValue | null, -): number => { + expectedAnswer: GekanatorAnswerValue | null): number => { switch (userAnswer) { case 'yes': @@ -1089,8 +1069,7 @@ const learnedSemanticScoreDeltaForExpectedAnswer = ( const scoreDropDeltaForRecoveredPost = ( postId: number, totalScore: number, - recoveredCandidatePosts: Map, -): number => { + recoveredCandidatePosts: Map): number => { const recoveredCandidate = recoveredCandidatePosts.get (postId) if (recoveredCandidate == null) return totalScore @@ -1114,8 +1093,7 @@ const postPassesScoreDrop = ( recoveredCandidatePosts }: { postId: number scores: Map - recoveredCandidatePosts: Map }, -): boolean => { + recoveredCandidatePosts: Map }): boolean => { if (!(activeCandidateScoreDropEnabled (scores))) return true @@ -1129,8 +1107,7 @@ const postPassesScoreDrop = ( // `post_similarities` is the score-propagation graph, not the question kind. const questionUsesPostSimilarityPropagationGraphForScoring = ( - question: GekanatorQuestion, -): boolean => + question: GekanatorQuestion): boolean => (question.kind === 'post_similarity' && !(isUserSuggestedLearnedSemanticQuestion (question))) || (question.kind === 'tag' @@ -1139,8 +1116,7 @@ const questionUsesPostSimilarityPropagationGraphForScoring = ( const questionSupportsAnswerBasedHardFiltering = ( - question: GekanatorQuestion, -): boolean => !(questionUsesPostSimilarityPropagationGraphForScoring (question)) + question: GekanatorQuestion): boolean => !(questionUsesPostSimilarityPropagationGraphForScoring (question)) && !(isUserSuggestedLearnedSemanticQuestion (question)) @@ -1160,8 +1136,7 @@ const usesLearnedTagExamples = (question: GekanatorQuestion): boolean => const searchedQuestionsFor = ( questions: GekanatorQuestion[], - search: string, -): GekanatorQuestion[] => { + search: string): GekanatorQuestion[] => { const needle = search.trim () if (!(needle)) return [] @@ -1235,8 +1210,7 @@ type QuestionMatchResolver = { const buildGekanatorMatchIndex = ( posts: Post[], - questions: GekanatorQuestion[], -): GekanatorMatchIndex => new Map ( + questions: GekanatorQuestion[]): GekanatorMatchIndex => new Map ( questions.map (question => [ question.id, new Set ( @@ -1285,8 +1259,7 @@ const matchingPostIdsForQuestion = ({ const positiveMatchingPostIdsForQuestion = ( - resolver: QuestionMatchResolver, -): Set => { + resolver: QuestionMatchResolver): Set => { if (isUserSuggestedLearnedSemanticQuestion (resolver.question)) { const cached = resolver.dynamicMatchIndex?.get (resolver.question.id) @@ -1356,8 +1329,7 @@ const matchingWeightInCandidates = ( materialIndex: GekanatorQuestionMaterialIndex matchIndex: GekanatorMatchIndex question: GekanatorQuestion - dynamicMatchIndex?: GekanatorMatchIndex }, -): number => { + dynamicMatchIndex?: GekanatorMatchIndex }): number => { const matched = positiveMatchingPostIdsForQuestion ({ posts, materialIndex, @@ -1380,8 +1352,7 @@ const signatureForCandidateIds = ( materialIndex: GekanatorQuestionMaterialIndex matchIndex: GekanatorMatchIndex question: GekanatorQuestion - dynamicMatchIndex?: GekanatorMatchIndex }, -): string => { + dynamicMatchIndex?: GekanatorMatchIndex }): string => { if (isUserSuggestedLearnedSemanticQuestion (question)) { const postById = new Map (posts.map (post => [post.id, post])) @@ -1421,8 +1392,7 @@ const postIdsForHardAnswer = ( posts: Post[] materialIndex: GekanatorQuestionMaterialIndex matchIndex: GekanatorMatchIndex - dynamicMatchIndex?: GekanatorMatchIndex }, -): number[] => { + dynamicMatchIndex?: GekanatorMatchIndex }): number[] => { if (!(questionSupportsAnswerBasedHardFiltering (question))) return candidateIds @@ -1562,8 +1532,7 @@ const buildIndexedQuestion = ( text: string kind: GekanatorQuestionKind priorityWeight: number - materialIndex: GekanatorQuestionMaterialIndex }, -): GekanatorQuestion => ({ + materialIndex: GekanatorQuestionMaterialIndex }): GekanatorQuestion => ({ id: questionIdForCondition (condition), text, kind, @@ -1578,8 +1547,7 @@ const buildIndexedQuestion = ( const rankedEntriesForCounts = ( { counts, total, cap }: { counts: Map total: number - cap: number }, -): [T, number][] => + cap: number }): [T, number][] => ([...counts.entries ()] .filter (([, count]) => count > 0 && count < total) .sort ((a, b) => Math.abs (total / 2 - a[1]) - Math.abs (total / 2 - b[1])) @@ -1594,8 +1562,7 @@ const buildQuestionsForCandidateIds = ( materialIndex: GekanatorQuestionMaterialIndex acceptedQuestions: GekanatorQuestion[] mode?: QuestionBuildMode - confirmationPostId?: number | null }, -): GekanatorQuestion[] => { + confirmationPostId?: number | null }): GekanatorQuestion[] => { const total = candidateIds.length const confirmationPost = (() => { if (confirmationPostId == null) @@ -1652,8 +1619,7 @@ const buildQuestionsForCandidateIds = ( condition: Extract< GekanatorQuestionCondition, { type: 'original-year' | 'original-month' | 'original-month-day' } - >, - ): GekanatorQuestion => { + >): GekanatorQuestion => { const priorityWeight = (() => { if (condition.type === 'original-year') return 1.04 @@ -1673,8 +1639,7 @@ const buildQuestionsForCandidateIds = ( const specialMonthDays = rankedEntriesForCounts ({ counts: monthDayCounts, total, - cap: factCap - }).filter (([monthDay]) => specialOriginalMonthDayLabelFor (String (monthDay)) != null) + cap: factCap}).filter (([monthDay]) => specialOriginalMonthDayLabelFor (String (monthDay)) != null) if (mode === 'split') { @@ -2124,8 +2089,7 @@ type ExclusiveConditionGroup = const exclusiveConditionGroupFor = ( - condition: GekanatorQuestion['condition'], -): ExclusiveConditionGroup | null => { + condition: GekanatorQuestion['condition']): ExclusiveConditionGroup | null => { switch (condition.type) { case 'original-month': @@ -2144,8 +2108,7 @@ const exclusiveConditionGroupFor = ( const sameConditionValue = ( left: GekanatorQuestion['condition'], - right: GekanatorQuestion['condition'], -): boolean => { + right: GekanatorQuestion['condition']): boolean => { const leftTitleLength = titleLengthMinimumForCondition (left) const rightTitleLength = titleLengthMinimumForCondition (right) if (leftTitleLength != null || rightTitleLength != null) @@ -2187,8 +2150,7 @@ const sameConditionValue = ( const isMonthCrossMatch = ( candidate: GekanatorQuestion['condition'], - previous: GekanatorQuestion['condition'], -): boolean => { + previous: GekanatorQuestion['condition']): boolean => { const candidateMonth = monthForCondition (candidate) const previousMonth = monthForCondition (previous) if (candidateMonth == null || previousMonth == null) @@ -2204,8 +2166,7 @@ const isMonthCrossMatch = ( const isExclusiveContradiction = ( candidate: GekanatorQuestion['condition'], - previous: GekanatorQuestion['condition'], -): boolean => { + previous: GekanatorQuestion['condition']): boolean => { const candidateGroup = exclusiveConditionGroupFor (candidate) const previousGroup = exclusiveConditionGroupFor (previous) @@ -2242,16 +2203,14 @@ const contradictionPenaltyFor = ({ case 'no': if ( sameConditionValue (question.condition, previous) - || isMonthCrossMatch (question.condition, previous) - ) + || isMonthCrossMatch (question.condition, previous)) return sum + 40 return sum case 'probably_no': if ( sameConditionValue (question.condition, previous) - || isMonthCrossMatch (question.condition, previous) - ) + || isMonthCrossMatch (question.condition, previous)) return sum + 20 return sum @@ -2281,8 +2240,7 @@ const chooseQuestion = ( recentFirstQuestionPenaltyById: Map userPriorWeights: Map materialIndex: GekanatorQuestionMaterialIndex - matchIndex: GekanatorMatchIndex }, -): QuestionSelection | null => { + matchIndex: GekanatorMatchIndex }): QuestionSelection | null => { const dynamicMatchIndex = new Map> () const invertedSignature = (signature: string): string => @@ -2327,8 +2285,7 @@ const chooseQuestion = ( const rank = ( questionsToRank: GekanatorQuestion[], candidates: { post: Post; score: number }[], - weightedCandidates: { post: Post; score: number; weight: number }[], - ) => { + weightedCandidates: { post: Post; score: number; weight: number }[]) => { const redundant = redundantSignatures (candidates.map (item => item.post)) const candidateById = new Map (candidates.map (item => [item.post.id, item.post])) const candidateIds = candidates.map (item => item.post.id) @@ -2613,8 +2570,7 @@ const chooseQuestion = ( if ( effectiveRatio < targetEffectiveUserSuggestedQuestionRatio && totalUserSuggestedRatio < targetTotalUserSuggestedQuestionRatio - && effectiveUserSuggestedPool.length > 0 - ) + && effectiveUserSuggestedPool.length > 0) { selectedPool = effectiveUserSuggestedPool selectedPurpose = 'effective_user_suggested' @@ -2622,8 +2578,7 @@ const chooseQuestion = ( else if ( learningRatio < targetLearningUserSuggestedQuestionRatio && totalUserSuggestedRatio < targetTotalUserSuggestedQuestionRatio - && learningUserSuggestedPool.length > 0 - ) + && learningUserSuggestedPool.length > 0) { selectedPool = learningUserSuggestedPool selectedPurpose = 'learning_user_suggested' @@ -2683,8 +2638,7 @@ const chooseQuestion = ( const winningRunPriorityFor = ( - expected: GekanatorAnswerValue, -): number | null => { + expected: GekanatorAnswerValue): number | null => { if (expected === 'yes') return 0 if (expected === 'partial') @@ -2719,8 +2673,7 @@ const chooseWinningRunQuestion = ({ materialIndex, acceptedQuestions, mode: 'confirmation', - confirmationPostId: targetPost.id - }) + confirmationPostId: targetPost.id}) .filter (question => { if (askedIds.has (question.id)) return false @@ -2829,8 +2782,7 @@ const chooseFallbackQuestion = ({ materialIndex, acceptedQuestions: [], mode: 'confirmation', - confirmationPostId: post.id - }))) + confirmationPostId: post.id}))) .slice (0, 32) const dynamicMatchIndex = new Map> () const ranked = mergeQuestions ([ @@ -2905,8 +2857,7 @@ const chooseFallbackQuestion = ({ const shouldEnterGuessPhase = ( - reason: GuessReason | null, -): reason is 'hard_max_questions' | 'winning_run_finished' | 'question_count_checkpoint' => + reason: GuessReason | null): reason is 'hard_max_questions' | 'winning_run_finished' | 'question_count_checkpoint' => (reason === 'hard_max_questions' || reason === 'winning_run_finished' || reason === 'question_count_checkpoint') @@ -2914,15 +2865,13 @@ const shouldEnterGuessPhase = ( const isWinningRunActive = ( winningRunTargetId: number | null, - winningRunStartAnswerCount: number | null, -): boolean => + winningRunStartAnswerCount: number | null): boolean => winningRunTargetId != null && winningRunStartAnswerCount != null const winningRunQuestionCount = ( answers: GekanatorAnswerLog[], - winningRunStartAnswerCount: number | null, -): number => { + winningRunStartAnswerCount: number | null): number => { if (winningRunStartAnswerCount == null) return 0 @@ -2962,8 +2911,7 @@ const nextQuestionPlanFor = ( matchIndex: GekanatorMatchIndex lastGuessQuestionCount: number winningRunTargetId: number | null - winningRunStartAnswerCount: number | null }, -): { question: GekanatorQuestion | null + winningRunStartAnswerCount: number | null }): { question: GekanatorQuestion | null guess: Post | null guessReason: GuessReason | null questionMode: QuestionMode @@ -3013,8 +2961,7 @@ const nextQuestionPlanFor = ( if ( isWinningRunActive (winningRunTargetId, winningRunStartAnswerCount) && winningRunTargetId === nextWinningRunTargetId - && winningRunStartAnswerCount != null - ) + && winningRunStartAnswerCount != null) return winningRunStartAnswerCount return answers.length @@ -3188,8 +3135,7 @@ const mascotStateFor = ( resultWon: boolean | null, eligiblePostCount: number, bestConfidencePercent: number, - winningRunActive: boolean, -): MascotState => { + winningRunActive: boolean): MascotState => { const resultPhase = phase === 'end' || phase === 'review' @@ -3210,13 +3156,11 @@ const mascotStateFor = ( if ( winningRunActive || eligiblePostCount <= 2 - || bestConfidencePercent >= 70 - ) + || bestConfidencePercent >= 70) return 'thinking_near' if ( eligiblePostCount >= 15 - && bestConfidencePercent < 45 - ) + && bestConfidencePercent < 45) return 'thinking_far' return 'thinking_mid' case 'guess': @@ -3327,8 +3271,7 @@ const GekanatorBackdrop: FC<{ const settingsForMode = useCallback ( ( - mode: 'normal' | 'winning_run' | 'guess', - ): { columns: number; rows: number; opacity: number } => { + mode: 'normal' | 'winning_run' | 'guess'): { columns: number; rows: number; opacity: number } => { if (mode === 'winning_run' || mode === 'guess') return { columns: 8, rows: 8, opacity: motionMode === 'calm' ? .18 : .24 } @@ -3342,8 +3285,7 @@ const GekanatorBackdrop: FC<{ const scaleForMode = useCallback ( ( mode: 'normal' | 'winning_run' | 'guess', - displayedWinningCount: number, - ): number => { + displayedWinningCount: number): number => { if (mode === 'guess') return 8 @@ -3355,8 +3297,7 @@ const GekanatorBackdrop: FC<{ []) const postsForMode = useCallback (( - mode: 'normal' | 'winning_run' | 'guess', - ): Post[] => { + mode: 'normal' | 'winning_run' | 'guess'): Post[] => { if (mode === 'guess' && displayedGuess) return [displayedGuess] if (mode === 'winning_run' && winningRunTargetPost) @@ -3366,8 +3307,7 @@ const GekanatorBackdrop: FC<{ const thumbnailsForMode = useCallback (( mode: 'normal' | 'winning_run' | 'guess', - count: number, - ): string[] => { + count: number): string[] => { const modePosts = postsForMode (mode) if (modePosts.length === 0) return [] @@ -3734,8 +3674,7 @@ const GekanatorBackdrop: FC<{ const expectedAnswerFor = ( question: GekanatorQuestion | undefined, - correctPost: Post | null, -): GekanatorAnswerValue | null => + correctPost: Post | null): GekanatorAnswerValue | null => expectedAnswerForQuestion (question, correctPost) @@ -4169,8 +4108,7 @@ const GekanatorPage: FC<{ user: User | null }> = ({ user }) => { setSaved (true) setSavedGameId (data.id) setLearnedExampleCount (data.learnedExampleCount) - setResultWon (variables.guessedPostId === variables.correctPostId) - }}) + setResultWon (variables.guessedPostId === variables.correctPostId)}}) const questionSuggestionMutation = useMutation ({ mutationFn: saveGekanatorQuestionSuggestion, onSuccess: async data => { @@ -4180,15 +4118,13 @@ const GekanatorPage: FC<{ user: User | null }> = ({ user }) => { setQuestionSuggestionSearch ('') setQuestionSuggestionSelectedId (null) setQuestionSuggestion ('') - setQuestionSuggestionAnswer ('yes') - }}) + setQuestionSuggestionAnswer ('yes')}}) const extraQuestionAnswersMutation = useMutation ({ mutationFn: saveGekanatorExtraQuestionAnswers, onSuccess: async () => { await queryClient.refetchQueries ({ queryKey: gekanatorKeys.questions () }) setExtraQuestionState ('saved') - setPhase ('end') - }}) + setPhase ('end')}}) const resetExtraQuestionState = () => { const next = resettableExtraQuestionState () @@ -4330,8 +4266,7 @@ const GekanatorPage: FC<{ user: User | null }> = ({ user }) => { if ( !(allowPreQuestionRecovery) || recoveredEligiblePosts.length === 0 - || recoveredEligiblePosts.length === 1 - ) + || recoveredEligiblePosts.length === 1) return false const nextQuestion = chooseQuestion ({ @@ -4493,8 +4428,7 @@ const GekanatorPage: FC<{ user: User | null }> = ({ user }) => { if ( !(nextPlan.question) && !(shouldEnterGuessPhase (nextPlan.guessReason)) - && recovered.eligiblePosts.length !== 1 - ) + && recovered.eligiblePosts.length !== 1) { const recoveredForQuestion = recoverQuestionState ({ nextAnswers, @@ -4602,8 +4536,7 @@ const GekanatorPage: FC<{ user: User | null }> = ({ user }) => { !(canPersistGame) || reviewGuessedPostId == null || reviewCorrectPostId == null - || saveMutation.isPending - ) + || saveMutation.isPending) return if (savedGameId != null) @@ -4666,8 +4599,7 @@ const GekanatorPage: FC<{ user: User | null }> = ({ user }) => { !(canPersistGame) || savedGameId == null || extraQuestionAnswersMutation.isPending - || extraQuestions.some (question => !(extraQuestionAnswers[String (question.id)])) - ) + || extraQuestions.some (question => !(extraQuestionAnswers[String (question.id)]))) return extraQuestionAnswersMutation.mutate ({ @@ -4870,8 +4802,7 @@ const GekanatorPage: FC<{ user: User | null }> = ({ user }) => { const answerExtraQuestion = ( questionId: number, - value: GekanatorAnswerValue, - ) => { + value: GekanatorAnswerValue) => { setExtraQuestionAnswers ({ ...extraQuestionAnswers, [String (questionId)]: value }) @@ -4909,8 +4840,7 @@ const GekanatorPage: FC<{ user: User | null }> = ({ user }) => { || isLoading || acceptedQuestionsLoading || shouldEnterGuessPhase (questionPlan.guessReason) - || eligiblePosts.length === 1 - ) + || eligiblePosts.length === 1) return const recovered = recoverQuestionState ({ @@ -4926,8 +4856,7 @@ const GekanatorPage: FC<{ user: User | null }> = ({ user }) => { if ( recovered.recoveryStepCount === recoveryStepCount && recovered.recoveredCandidatePosts.size === recoveredCandidatePosts.size - && recovered.softenedQuestionIds.size === softenedQuestionIds.size - ) + && recovered.softenedQuestionIds.size === softenedQuestionIds.size) return setSoftenedQuestionIds (recovered.softenedQuestionIds) @@ -4954,15 +4883,13 @@ const GekanatorPage: FC<{ user: User | null }> = ({ user }) => { if ( phase !== 'question' || isLoading - || acceptedQuestionsLoading - ) + || acceptedQuestionsLoading) return if ( currentQuestion || !(questionPlan.guess) - || !(shouldEnterGuessPhase (questionPlan.guessReason)) - ) + || !(shouldEnterGuessPhase (questionPlan.guessReason))) return setWinningRunTargetId (questionPlan.winningRunTargetId) diff --git a/frontend/src/pages/deerjikists/DeerjikistDetailPage.tsx b/frontend/src/pages/deerjikists/DeerjikistDetailPage.tsx index 99dd7ad..e3daec4 100644 --- a/frontend/src/pages/deerjikists/DeerjikistDetailPage.tsx +++ b/frontend/src/pages/deerjikists/DeerjikistDetailPage.tsx @@ -145,8 +145,7 @@ const DeerjikistDetailPage: FC = () => { return rtn })}/>)} - - ))} + ))}
    -
    - )} +
    )} ) } diff --git a/frontend/src/pages/materials/MaterialDetailPage.test.tsx b/frontend/src/pages/materials/MaterialDetailPage.test.tsx index 2b697a9..8a5819a 100644 --- a/frontend/src/pages/materials/MaterialDetailPage.test.tsx +++ b/frontend/src/pages/materials/MaterialDetailPage.test.tsx @@ -7,8 +7,9 @@ import { buildMaterial, buildTag } from '@/test/factories' import { renderWithProviders } from '@/test/render' const api = vi.hoisted (() => ({ - apiGet: vi.fn (), - apiPut: vi.fn (), + apiGet: vi.fn (), + apiPut: vi.fn (), + isApiError: vi.fn (), })) const wikiApi = vi.hoisted (() => ({ @@ -26,7 +27,7 @@ vi.mock ('@/components/ui/use-toast', () => toastApi) const renderPage = () => renderWithProviders ( - }/> + }/> , { route: '/materials/8' }, ) @@ -34,7 +35,11 @@ const renderPage = () => describe ('MaterialDetailPage', () => { beforeEach (() => { vi.clearAllMocks () - api.apiGet.mockResolvedValue ([]) + api.apiGet.mockImplementation (async (path: string) => + path === '/tags/autocomplete' + ? [] + : buildMaterial ({ id: 8 })) + api.isApiError.mockReturnValue (false) wikiApi.fetchWikiPages.mockResolvedValue ([]) vi.stubGlobal ('fetch', vi.fn (async () => ({ blob: async () => new Blob (['image'], { type: 'image/png' }), @@ -73,6 +78,7 @@ describe ('MaterialDetailPage', () => { const textboxes = screen.getAllByRole ('textbox') fireEvent.change (textboxes[0], { target: { value: 'new' } }) fireEvent.change (textboxes[1], { target: { value: 'https://example.com/ref' } }) + fireEvent.change (textboxes[2], { target: { value: '素材/new.png' } }) fireEvent.click (screen.getByRole ('button', { name: '更新' })) await waitFor (() => { @@ -81,6 +87,25 @@ describe ('MaterialDetailPage', () => { const formData = api.apiPut.mock.calls[0]?.[1] as FormData expect (formData.get ('tag')).toBe ('new') expect (formData.get ('url')).toBe ('https://example.com/ref') + expect (formData.get ('export_paths[legacy_drive]')).toBe ('素材/new.png') expect (toastApi.toast).toHaveBeenCalledWith ({ title: '更新成功!' }) }) + + it ('shows not found when material API returns 404', async () => { + api.isApiError.mockReturnValue (true) + api.apiGet.mockRejectedValueOnce ({ response: { status: 404 } }) + + renderPage () + + expect (await screen.findByText ('素材が見つかりませんでした．')).toBeInTheDocument () + }) + + it ('shows an error for non-404 material API failures', async () => { + api.isApiError.mockReturnValue (false) + api.apiGet.mockRejectedValueOnce (new Error ('network error')) + + renderPage () + + expect (await screen.findByText ('素材の取得に失敗しました．')).toBeInTheDocument () + }) }) diff --git a/frontend/src/pages/materials/MaterialDetailPage.tsx b/frontend/src/pages/materials/MaterialDetailPage.tsx index 09f537e..2be6299 100644 --- a/frontend/src/pages/materials/MaterialDetailPage.tsx +++ b/frontend/src/pages/materials/MaterialDetailPage.tsx @@ -1,3 +1,4 @@ +import { useMutation, useQuery, useQueryClient } from '@tanstack/react-query' import { useEffect, useState } from 'react' import { Helmet } from 'react-helmet-async' import { useParams } from 'react-router-dom' @@ -13,124 +14,130 @@ import MainArea from '@/components/layout/MainArea' import { Button } from '@/components/ui/button' import { toast } from '@/components/ui/use-toast' import { SITE_TITLE } from '@/config' -import { apiGet, apiPut } from '@/lib/api' +import { + fetchMaterial, + updateMaterial, +} from '@/lib/materials' +import { materialsKeys } from '@/lib/queryKeys' import { inputClass } from '@/lib/utils' import { useValidationErrors } from '@/lib/useValidationErrors' import type { FC } from 'react' -import type { Material, Tag } from '@/types' - -type MaterialWithTag = Material & { tag: Tag } - -type MaterialFormField = 'tag' | 'file' | 'url' +type MaterialFormField = 'tag' | 'file' | 'url' | 'exportPaths' const MaterialDetailPage: FC = () => { const { id } = useParams () + const qc = useQueryClient () + const [exportPath, setExportPath] = useState ('') const [file, setFile] = useState (null) const [filePreview, setFilePreview] = useState ('') - const [loading, setLoading] = useState (false) - const [material, setMaterial] = useState (null) - const [sending, setSending] = useState (false) const [tag, setTag] = useState ('') const [url, setURL] = useState ('') const { baseErrors, fieldErrors, clearValidationErrors, applyValidationError } = useValidationErrors () - const handleSubmit = async () => { - clearValidationErrors () - - const formData = new FormData - if (tag.trim ()) - formData.append ('tag', tag) - if (file) - formData.append ('file', file) - if (url.trim ()) - formData.append ('url', url) - - try - { - setSending (true) - const data = await apiPut (`/materials/${ id }`, formData) - setMaterial (data) - toast ({ title: '更新成功!' }) - } - catch (e) - { - applyValidationError (e) - toast ({ title: '更新失敗……', description: '入力を見直してください．' }) - } - finally - { - setSending (false) - } - } + const { data: material, isError, isLoading } = useQuery ({ + queryKey: materialsKeys.show (id ?? ''), + queryFn: () => fetchMaterial (id ?? ''), + enabled: id != null}) + const materialTitle = material + ? material.tag?.name ?? `素材 #${ material.id }` + : '' useEffect (() => { - if (!(id)) + if (!(material)) return - void (async () => { - try + setTag (material.tag?.name ?? '') + setURL (material.url ?? '') + setExportPath (material.exportPaths.legacyDrive ?? '') + if (material.file && material.contentType) { - setLoading (true) - const data = await apiGet (`/materials/${ id }`) - setMaterial (data) - setTag (data.tag.name) - if (data.file && data.contentType) - { - setFilePreview (data.file) - setFile (new File ([await (await fetch (data.file)).blob ()], - data.file, - { type: data.contentType })) - } - setURL (data.url ?? '') + setFilePreview (material.file) + setFile (null) } - finally - { - setLoading (false) - } - }) () - }, [id]) + }, [material]) + + const invalidateMaterialQueries = async () => { + await qc.invalidateQueries ({ queryKey: materialsKeys.root }) + } + + const updateMutation = useMutation ({ + mutationFn: async () => { + const formData = new FormData + if (tag.trim ()) + formData.append ('tag', tag) + if (file) + formData.append ('file', file) + if (url.trim ()) + formData.append ('url', url) + formData.append ('export_paths[legacy_drive]', exportPath) + + return await updateMaterial (id ?? '', formData) + }, + onSuccess: async data => { + qc.setQueryData (materialsKeys.show (id ?? ''), data) + await invalidateMaterialQueries () + toast ({ title: '更新成功!' }) + }, + onError: error => { + applyValidationError (error) + toast ({ title: '更新失敗……', description: '入力を見直してください．' }) + }}) + + const handleSubmit = () => { + clearValidationErrors () + updateMutation.mutate () + } return ( {material && ( - {`${ material.tag.name } 素材照会 | ${ SITE_TITLE }`} + {`${ materialTitle } 素材照会 | ${ SITE_TITLE }`} )} - {loading ? 'Loading...' : (material && ( + {isLoading ? 'Loading...' : isError ? ( +

    + 素材の取得に失敗しました． +

    ) : material == null ? ( +

    + 素材が見つかりませんでした． +

    ) : ( <> - + {material.tag + ? ( + ) + : materialTitle} {(material.file && material.contentType) && ( (/image\/.*/.test (material.contentType) && ( - {material.tag.name)) + {material.tag?.name)) || (/video\/.*/.test (material.contentType) && (