コミットを比較
36 コミット
| 作成者 | SHA1 | 日付 | |
|---|---|---|---|
| 58828597d7 | |||
| b0c24f319a | |||
| f9463f383f | |||
| 6d037192c4 | |||
| d035da99ad | |||
| bc660676ef | |||
| 38650b5671 | |||
| 97b132e5ab | |||
| be68841bd3 | |||
| ac82adc6b3 | |||
| aa96ec95d1 | |||
| df17f20907 | |||
| 769966648b | |||
| 7bcb76516c | |||
| 8970edc59f | |||
| 9b1ea56e36 | |||
| 155edfe018 | |||
| 23f1ffb04b | |||
| ccfe65a6a4 | |||
| 06d6e512e4 | |||
| 5bd097bcfe | |||
| f636d2a177 | |||
| 10dc776313 | |||
| a9e16735f8 | |||
| 440a6c9961 | |||
| 4535a9d260 | |||
| 440d3d38be | |||
| cb33d9ca25 | |||
| 19bf24432a | |||
| ac41385962 | |||
| 08bf92ff79 | |||
| c51d7b98ba | |||
| b7b284c076 | |||
| 6c451d260f | |||
| c1902fbc99 | |||
| 2f2b6e2afa |
@@ -114,19 +114,95 @@ npm run preview
|
|||||||
- Ruby: `render` 系メソッド呼び出しでは、keyword 引数付きでも括弧を書かない。
|
- Ruby: `render` 系メソッド呼び出しでは、keyword 引数付きでも括弧を書かない。
|
||||||
- Ruby: never put a line break immediately before `)`.
|
- Ruby: never put a line break immediately before `)`.
|
||||||
- Ruby: do not use `%w` or `%i`.
|
- Ruby: do not use `%w` or `%i`.
|
||||||
|
- Ruby: TypeScript / TSX の associative syntax に近い感覚で読むこと。
|
||||||
|
Hash の `}` は block の `}` ではない。call の `)` も function 定義の
|
||||||
|
parameter-list `)` ではない。delimiter の役割を見誤らないこと。
|
||||||
- In Ruby, when an `if` condition is split across multiple lines and combines
|
- In Ruby, when an `if` condition is split across multiple lines and combines
|
||||||
clauses with `&&` or `||`, wrap the whole condition in parentheses.
|
clauses with `&&` or `||`, wrap the whole condition in parentheses.
|
||||||
- Ruby hashes are not blocks; keep `}` on the same line as the final pair.
|
- Ruby hash / block / call delimiter の詳細は、下の
|
||||||
- Ruby hashes keep the first pair on the same line as `{` unless line length
|
`Ruby delimiter and wrapping rules` を正本として扱ふこと。
|
||||||
requires a break.
|
|
||||||
- Short Ruby hashes may stay visually compact across two lines with the first
|
|
||||||
pair kept on the opening line and aligned continuation pairs below it.
|
|
||||||
- Ruby blocks use separate `{ ... }` rules from hashes, with 2-space body
|
|
||||||
indentation.
|
|
||||||
- For arrays, never put whitespace or a line break immediately before `]`.
|
- For arrays, never put whitespace or a line break immediately before `]`.
|
||||||
- Keep the first element on the same line as `[` by default.
|
- Keep the first element on the same line as `[` by default.
|
||||||
- If an array would exceed the line limit, break after `[` and indent
|
- If an array would exceed the line limit, break after `[` and indent
|
||||||
elements by 4 spaces.
|
elements by 4 spaces.
|
||||||
|
|
||||||
|
### Ruby delimiter and wrapping rules
|
||||||
|
|
||||||
|
- Ruby でも、closing delimiter は glyph ではなく syntax role で判定する。
|
||||||
|
`}` が block close なのか hash close なのか、`)` が method call なのか
|
||||||
|
grouping なのかを区別してから配置すること。
|
||||||
|
- Ruby の multi-line method call では、receiver / method と opening `(` を
|
||||||
|
同じ行に保ち、closing `)` を単独行へ落とさない。
|
||||||
|
- Ruby の multi-line hash literal / keyword-like argument hash では、opening
|
||||||
|
`{` を最初の pair と同じ行に置き、closing `}` を最後の pair と同じ行に
|
||||||
|
置く。Prettier 的な縦開き・縦閉じをしない。
|
||||||
|
- Ruby の guard 条件は、1 行で収まるなら modifier 形式を優先する。
|
||||||
|
99 文字を超えるなら block 形式へ切り替へるか、message 定数化などで縮める。
|
||||||
|
- Ruby の method chain や call argument を折り返す際、call-site の `)` を
|
||||||
|
block close のやうに独立させない。
|
||||||
|
|
||||||
|
Bad:
|
||||||
|
|
||||||
|
```rb
|
||||||
|
response = Example.fetch(
|
||||||
|
value,
|
||||||
|
option: option,
|
||||||
|
)
|
||||||
|
```
|
||||||
|
|
||||||
|
Good:
|
||||||
|
|
||||||
|
```rb
|
||||||
|
response = Example.fetch(
|
||||||
|
value,
|
||||||
|
option: option)
|
||||||
|
```
|
||||||
|
|
||||||
|
Bad:
|
||||||
|
|
||||||
|
```rb
|
||||||
|
payload = {
|
||||||
|
title: title,
|
||||||
|
url: url,
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
Good:
|
||||||
|
|
||||||
|
```rb
|
||||||
|
payload = {
|
||||||
|
title: title,
|
||||||
|
url: url }
|
||||||
|
```
|
||||||
|
|
||||||
|
Bad:
|
||||||
|
|
||||||
|
```rb
|
||||||
|
raise ArgumentError, 'URL が長すぎます.' if url.bytesize > MAX_URL_BYTES && flag.present?
|
||||||
|
```
|
||||||
|
|
||||||
|
Good:
|
||||||
|
|
||||||
|
```rb
|
||||||
|
if url.bytesize > MAX_URL_BYTES && flag.present?
|
||||||
|
raise ArgumentError, 'URL が長すぎます.'
|
||||||
|
end
|
||||||
|
```
|
||||||
|
|
||||||
|
Bad:
|
||||||
|
|
||||||
|
```rb
|
||||||
|
records.each {
|
||||||
|
do_work(_1) }
|
||||||
|
```
|
||||||
|
|
||||||
|
Good:
|
||||||
|
|
||||||
|
```rb
|
||||||
|
records.each {
|
||||||
|
do_work(_1)
|
||||||
|
}
|
||||||
|
```
|
||||||
- TypeScript and Python: use GNU-style spacing before parentheses where
|
- TypeScript and Python: use GNU-style spacing before parentheses where
|
||||||
syntactically valid.
|
syntactically valid.
|
||||||
- Never write Ruby, TypeScript, or TSX lines longer than 99 characters.
|
- Never write Ruby, TypeScript, or TSX lines longer than 99 characters.
|
||||||
|
|||||||
@@ -89,15 +89,13 @@ class MaterialsController < ApplicationController
|
|||||||
|
|
||||||
begin
|
begin
|
||||||
Material.transaction do
|
Material.transaction do
|
||||||
tag_name = TagName.find_undiscard_or_create_by!(name: tag_name_raw)
|
tag = resolve_material_tag!(tag_name_raw)
|
||||||
tag = tag_name.tag
|
|
||||||
tag = Tag.create!(tag_name:, category: :material) unless tag
|
|
||||||
|
|
||||||
material = Material.new(tag:, url:,
|
material = Material.new(tag:, url:,
|
||||||
created_by_user: current_user,
|
created_by_user: current_user,
|
||||||
updated_by_user: current_user)
|
updated_by_user: current_user)
|
||||||
material.file.attach(uploaded_blob) if uploaded_blob
|
material.file.attach(uploaded_blob) if uploaded_blob
|
||||||
material.save!
|
material.save!
|
||||||
|
TagVersioning.record_tag_snapshot!(tag, created_by_user: current_user)
|
||||||
upsert_export_paths!(material)
|
upsert_export_paths!(material)
|
||||||
MaterialVersionRecorder.record!(material:, event_type: :create,
|
MaterialVersionRecorder.record!(material:, event_type: :create,
|
||||||
created_by_user: current_user)
|
created_by_user: current_user)
|
||||||
@@ -139,10 +137,7 @@ class MaterialsController < ApplicationController
|
|||||||
begin
|
begin
|
||||||
Material.transaction do
|
Material.transaction do
|
||||||
MaterialVersionRecorder.ensure_snapshot!(material, created_by_user: current_user)
|
MaterialVersionRecorder.ensure_snapshot!(material, created_by_user: current_user)
|
||||||
tag_name = TagName.find_undiscard_or_create_by!(name: tag_name_raw)
|
tag = resolve_material_tag!(tag_name_raw)
|
||||||
tag = tag_name.tag
|
|
||||||
tag = Tag.create!(tag_name:, category: :material) unless tag
|
|
||||||
|
|
||||||
material.assign_attributes(tag:, url:, updated_by_user: current_user)
|
material.assign_attributes(tag:, url:, updated_by_user: current_user)
|
||||||
if uploaded_blob
|
if uploaded_blob
|
||||||
material.file.attach(uploaded_blob)
|
material.file.attach(uploaded_blob)
|
||||||
@@ -150,6 +145,7 @@ class MaterialsController < ApplicationController
|
|||||||
material.file.detach
|
material.file.detach
|
||||||
end
|
end
|
||||||
material.save!
|
material.save!
|
||||||
|
TagVersioning.record_tag_snapshot!(tag, created_by_user: current_user)
|
||||||
upsert_export_paths!(material)
|
upsert_export_paths!(material)
|
||||||
MaterialVersionRecorder.record!(material:, event_type: :update,
|
MaterialVersionRecorder.record!(material:, event_type: :update,
|
||||||
created_by_user: current_user)
|
created_by_user: current_user)
|
||||||
@@ -240,6 +236,12 @@ class MaterialsController < ApplicationController
|
|||||||
nil
|
nil
|
||||||
end
|
end
|
||||||
|
|
||||||
|
def resolve_material_tag! tag_name_raw
|
||||||
|
tag_name = TagName.find_undiscard_or_create_by!(name: tag_name_raw)
|
||||||
|
tag = tag_name.tag
|
||||||
|
tag || Tag.create!(tag_name:, category: :material)
|
||||||
|
end
|
||||||
|
|
||||||
def material_index_needs_tag_name? filters
|
def material_index_needs_tag_name? filters
|
||||||
filters[:q].present? || filters[:sort] == 'tag_name'
|
filters[:q].present? || filters[:sort] == 'tag_name'
|
||||||
end
|
end
|
||||||
|
|||||||
@@ -0,0 +1,44 @@
|
|||||||
|
class PostImportsController < ApplicationController
|
||||||
|
before_action :require_member!
|
||||||
|
|
||||||
|
def preview
|
||||||
|
rows = PostImportPreviewer.new.preview_rows(
|
||||||
|
rows: PostImportUrlListParser.parse(params[:source]))
|
||||||
|
render json: { rows: }
|
||||||
|
rescue ArgumentError => e
|
||||||
|
render_bad_request e.message
|
||||||
|
end
|
||||||
|
|
||||||
|
def validate
|
||||||
|
rows = normalised_import_rows allow_warning_fields: true
|
||||||
|
changed_row = Integer(params[:changed_row], exception: false)
|
||||||
|
result =
|
||||||
|
PostImportPreviewer.new.preview_rows(rows:,
|
||||||
|
fetch_metadata: changed_row,
|
||||||
|
metadata_cache: { })
|
||||||
|
render json: { rows: result }
|
||||||
|
rescue ArgumentError => e
|
||||||
|
render_bad_request e.message
|
||||||
|
end
|
||||||
|
|
||||||
|
def create
|
||||||
|
result = PostImportRunner.new(actor: current_user,
|
||||||
|
rows: normalised_import_rows).run
|
||||||
|
render json: result, status: result[:created].positive? ? :created : :ok
|
||||||
|
rescue ArgumentError => e
|
||||||
|
render_bad_request e.message
|
||||||
|
end
|
||||||
|
|
||||||
|
private
|
||||||
|
|
||||||
|
def require_member!
|
||||||
|
return head :unauthorized unless current_user
|
||||||
|
return if current_user.gte_member?
|
||||||
|
|
||||||
|
head :forbidden
|
||||||
|
end
|
||||||
|
|
||||||
|
def normalised_import_rows allow_warning_fields: false
|
||||||
|
PostImportRowNormaliser.normalise!(params[:rows], allow_warning_fields:)
|
||||||
|
end
|
||||||
|
end
|
||||||
@@ -142,37 +142,14 @@ class PostsController < ApplicationController
|
|||||||
return head :unauthorized unless current_user
|
return head :unauthorized unless current_user
|
||||||
return head :forbidden unless current_user.gte_member?
|
return head :forbidden unless current_user.gte_member?
|
||||||
|
|
||||||
# TODO: サイトに応じて thumbnail_base 設定
|
post = PostCreator.new(actor: current_user,
|
||||||
title = params[:title].presence
|
attributes: { title: params[:title], url: params[:url],
|
||||||
url = params[:url]
|
thumbnail: params[:thumbnail], tags: params[:tags],
|
||||||
thumbnail = params[:thumbnail]
|
original_created_from: params[:original_created_from],
|
||||||
tag_names = params[:tags].to_s.split
|
original_created_before: params[:original_created_before],
|
||||||
original_created_from = params[:original_created_from]
|
parent_post_ids: parse_parent_post_ids,
|
||||||
original_created_before = params[:original_created_before]
|
video_ms: params[:video_ms],
|
||||||
parent_post_ids = parse_parent_post_ids
|
duration: params[:duration] }).create!
|
||||||
resized_thumbnail = thumbnail.present? ? Post.resized_thumbnail_attachment(thumbnail) : nil
|
|
||||||
|
|
||||||
post = Post.new(title:, url:, thumbnail_base: nil, uploaded_user: current_user,
|
|
||||||
original_created_from:, original_created_before:)
|
|
||||||
post.thumbnail.attach(resized_thumbnail) if resized_thumbnail
|
|
||||||
|
|
||||||
ApplicationRecord.transaction do
|
|
||||||
post.save!
|
|
||||||
|
|
||||||
Tag.normalise_tags!(tag_names, deny_deprecated: true, with_sections: true) =>
|
|
||||||
{ tags:, sections: }
|
|
||||||
TagVersioning.record_tag_snapshots!(tags, created_by_user: current_user)
|
|
||||||
|
|
||||||
tags = Tag.expand_parent_tags(tags).reject(&:deprecated?)
|
|
||||||
post.video_ms = normalise_video_ms(tags)
|
|
||||||
validate_video_sections!(post.video_ms, sections)
|
|
||||||
post.save!
|
|
||||||
sync_post_tags!(post, tags, sections)
|
|
||||||
|
|
||||||
sync_parent_posts!(post, parent_post_ids)
|
|
||||||
|
|
||||||
PostVersionRecorder.record!(post:, event_type: :create, created_by_user: current_user)
|
|
||||||
end
|
|
||||||
|
|
||||||
post.reload
|
post.reload
|
||||||
render json: PostRepr.base(post), status: :created
|
render json: PostRepr.base(post), status: :created
|
||||||
@@ -182,7 +159,7 @@ class PostsController < ApplicationController
|
|||||||
render_unprocessable_entity '廃止済みタグは付与できません.', field: :tags
|
render_unprocessable_entity '廃止済みタグは付与できません.', field: :tags
|
||||||
rescue Tag::SectionLiteralParseError
|
rescue Tag::SectionLiteralParseError
|
||||||
render_validation_error fields: { tags: ['タグ区間の記法が不正です.'] }
|
render_validation_error fields: { tags: ['タグ区間の記法が不正です.'] }
|
||||||
rescue VideoMsParseError
|
rescue PostCreator::VideoMsParseError
|
||||||
render_validation_error fields: { video_ms: ['動画時間の記法が不正です.'] }
|
render_validation_error fields: { video_ms: ['動画時間の記法が不正です.'] }
|
||||||
rescue MiniMagick::Error
|
rescue MiniMagick::Error
|
||||||
render_validation_error fields: { thumbnail: ['サムネイル画像の変換に失敗しました.'] }
|
render_validation_error fields: { thumbnail: ['サムネイル画像の変換に失敗しました.'] }
|
||||||
|
|||||||
@@ -1,50 +1,57 @@
|
|||||||
class PreviewController < ApplicationController
|
class PreviewController < ApplicationController
|
||||||
|
before_action :require_member!
|
||||||
|
|
||||||
def title
|
def title
|
||||||
# TODO: # 既知サイトなら決まったフォーマットで title 取得するやぅに.
|
return render_bad_request('URL は必須です.') if params[:url].blank?
|
||||||
return head :unauthorized unless current_user
|
|
||||||
|
|
||||||
url = params[:url]
|
render json: { title: Preview::ThumbnailFetcher.title(params[:url]) }
|
||||||
return render_bad_request('URL は必須です.') unless url.present?
|
rescue Preview::UrlSafety::UnsafeUrl => e
|
||||||
|
|
||||||
unless url.start_with?(/http(s)?:\/\//)
|
|
||||||
url = 'http://' + url
|
|
||||||
end
|
|
||||||
|
|
||||||
html = URI.open(url, open_timeout: 5, read_timeout: 5).read
|
|
||||||
doc = Nokogiri::HTML.parse(html)
|
|
||||||
title = doc.at('title')&.text&.strip
|
|
||||||
|
|
||||||
render json: { title: title }
|
|
||||||
rescue => e
|
|
||||||
render_bad_request(e.message)
|
render_bad_request(e.message)
|
||||||
|
rescue Preview::HttpFetcher::FetchTimeout => e
|
||||||
|
render_preview_error(e.message, :gateway_timeout)
|
||||||
|
rescue Preview::HttpFetcher::ResponseTooLarge => e
|
||||||
|
render_preview_error(e.message, :payload_too_large)
|
||||||
|
rescue Preview::HttpFetcher::FetchFailed => e
|
||||||
|
render_preview_error(e.message, :bad_gateway)
|
||||||
end
|
end
|
||||||
|
|
||||||
def thumbnail
|
def thumbnail
|
||||||
# TODO: 既知ドメインであれば指定のアドレスからサムネールを取得するやぅにする.
|
return render_bad_request('URL は必須です.') if params[:url].blank?
|
||||||
|
|
||||||
|
image = MiniMagick::Image.read(Preview::ThumbnailFetcher.fetch(params[:url]))
|
||||||
|
image.auto_orient
|
||||||
|
image.resize '180x180>'
|
||||||
|
image.format 'png'
|
||||||
|
width, height = image.dimensions
|
||||||
|
raise Preview::ThumbnailFetcher::GenerationFailed, 'サムネール画像の変換に失敗しました.' if width > 180 || height > 180
|
||||||
|
|
||||||
|
send_data image.to_blob, type: 'image/png', disposition: 'inline'
|
||||||
|
rescue Preview::UrlSafety::UnsafeUrl => e
|
||||||
|
render_bad_request(e.message)
|
||||||
|
rescue Preview::HttpFetcher::FetchTimeout => e
|
||||||
|
render_preview_error(e.message, :gateway_timeout)
|
||||||
|
rescue Preview::HttpFetcher::ResponseTooLarge => e
|
||||||
|
render_preview_error(e.message, :payload_too_large)
|
||||||
|
rescue Preview::HttpFetcher::FetchFailed => e
|
||||||
|
render_preview_error(e.message, :bad_gateway)
|
||||||
|
rescue Preview::ThumbnailFetcher::GenerationFailed, MiniMagick::Error => e
|
||||||
|
render_unprocessable_entity(e.message)
|
||||||
|
end
|
||||||
|
|
||||||
|
private
|
||||||
|
|
||||||
|
def require_member!
|
||||||
return head :unauthorized unless current_user
|
return head :unauthorized unless current_user
|
||||||
|
return if current_user.gte_member?
|
||||||
|
|
||||||
url = params[:url]
|
head :forbidden
|
||||||
return render_bad_request('URL は必須です.') if url.blank?
|
end
|
||||||
|
|
||||||
unless url.start_with?(/http(s)?:\/\//)
|
def render_preview_error(message, status)
|
||||||
url = 'http://' + url
|
render json: { type: status.to_s,
|
||||||
end
|
message:,
|
||||||
|
errors: { },
|
||||||
path = Rails.root.join('tmp', "thumb_#{ SecureRandom.hex }.png")
|
base_errors: [message] },
|
||||||
system("node #{ Rails.root }/lib/screenshot.js #{ Shellwords.escape(url) } #{ path }")
|
status:
|
||||||
|
|
||||||
if File.exist?(path)
|
|
||||||
image = MiniMagick::Image.open(path)
|
|
||||||
image.resize '180x180'
|
|
||||||
File.delete(path) rescue nil
|
|
||||||
send_file image.path, type: 'image/png', disposition: 'inline'
|
|
||||||
else
|
|
||||||
render json: { type: 'internal_server_error',
|
|
||||||
message: 'サムネールを生成できませんでした.',
|
|
||||||
errors: { },
|
|
||||||
base_errors: ['サムネールを生成できませんでした.'] },
|
|
||||||
status: :internal_server_error
|
|
||||||
end
|
|
||||||
end
|
end
|
||||||
end
|
end
|
||||||
|
|||||||
@@ -57,9 +57,9 @@ class Post < ApplicationRecord
|
|||||||
|
|
||||||
attribute :version_no, :integer, default: 1
|
attribute :version_no, :integer, default: 1
|
||||||
|
|
||||||
before_validation :normalise_url
|
before_validation :normalise_url, if: :will_save_change_to_url?
|
||||||
|
|
||||||
validates :url, presence: true, uniqueness: true
|
validates :url, presence: true, uniqueness: true, length: { maximum: 768 }
|
||||||
validates :video_ms, numericality: { only_integer: true, greater_than: 0 }, allow_nil: true
|
validates :video_ms, numericality: { only_integer: true, greater_than: 0 }, allow_nil: true
|
||||||
|
|
||||||
validate :validate_original_created_range
|
validate :validate_original_created_range
|
||||||
@@ -173,15 +173,6 @@ class Post < ApplicationRecord
|
|||||||
def normalise_url
|
def normalise_url
|
||||||
return if url.blank?
|
return if url.blank?
|
||||||
|
|
||||||
self.url = url.strip
|
self.url = PostUrlNormaliser.normalise(url) || url.strip
|
||||||
|
|
||||||
u = URI.parse(url)
|
|
||||||
return unless u in URI::HTTP
|
|
||||||
|
|
||||||
u.host = u.host.downcase if u.host
|
|
||||||
u.path = u.path.sub(/\/\Z/, '') if u.path.present?
|
|
||||||
self.url = u.to_s
|
|
||||||
rescue URI::InvalidURIError
|
|
||||||
;
|
|
||||||
end
|
end
|
||||||
end
|
end
|
||||||
|
|||||||
@@ -0,0 +1,163 @@
|
|||||||
|
class PostUrlSanitisationRule < ApplicationRecord
|
||||||
|
include Discard::Model
|
||||||
|
|
||||||
|
class InvalidUrlError < StandardError
|
||||||
|
attr_reader :invalid_rows
|
||||||
|
|
||||||
|
def initialize(invalid_rows)
|
||||||
|
@invalid_rows = invalid_rows
|
||||||
|
ids = invalid_rows.map { _1.fetch(:post_id) }.join(', ')
|
||||||
|
super("post URL sanitisation produced invalid URLs for posts #{ ids }")
|
||||||
|
end
|
||||||
|
end
|
||||||
|
|
||||||
|
class UrlConflictError < StandardError
|
||||||
|
attr_reader :conflicts
|
||||||
|
|
||||||
|
def initialize(conflicts)
|
||||||
|
@conflicts = conflicts
|
||||||
|
urls = conflicts.map { _1.fetch(:url) }.uniq.join(', ')
|
||||||
|
super("post URL sanitisation conflicts detected for #{ urls }")
|
||||||
|
end
|
||||||
|
end
|
||||||
|
|
||||||
|
self.primary_key = :priority
|
||||||
|
|
||||||
|
default_scope -> { kept }
|
||||||
|
|
||||||
|
validates :source_pattern, presence: true, uniqueness: true
|
||||||
|
|
||||||
|
validate :source_pattern_must_be_regexp
|
||||||
|
|
||||||
|
class << self
|
||||||
|
def sanitise(url) = sanitise_with_rules(url, rules)
|
||||||
|
|
||||||
|
def apply!
|
||||||
|
rewrites = nil
|
||||||
|
|
||||||
|
Post.transaction do
|
||||||
|
compiled_rules = rules
|
||||||
|
|
||||||
|
rewrites = Post.order(:id)
|
||||||
|
.lock('FOR UPDATE')
|
||||||
|
.pluck(:id, :url)
|
||||||
|
.map do |post_id, original_url|
|
||||||
|
{ post_id:,
|
||||||
|
original_url:,
|
||||||
|
sanitised_url: sanitise_with_rules(original_url, compiled_rules) }
|
||||||
|
end
|
||||||
|
|
||||||
|
invalid_rows = rewrites.filter { invalid_sanitised_url?(_1.fetch(:sanitised_url)) }
|
||||||
|
.map { { post_id: _1.fetch(:post_id),
|
||||||
|
original_url: _1.fetch(:original_url),
|
||||||
|
sanitised_url: _1.fetch(:sanitised_url) } }
|
||||||
|
raise InvalidUrlError.new(invalid_rows) if invalid_rows.present?
|
||||||
|
|
||||||
|
conflicts = build_conflicts(rewrites)
|
||||||
|
raise UrlConflictError.new(conflicts) if conflicts.present?
|
||||||
|
|
||||||
|
changed = rewrites.filter { _1.fetch(:original_url) != _1.fetch(:sanitised_url) }
|
||||||
|
return if changed.empty?
|
||||||
|
|
||||||
|
token = SecureRandom.hex(6)
|
||||||
|
|
||||||
|
changed.each do |row|
|
||||||
|
Post.where(id: row.fetch(:post_id))
|
||||||
|
.update_all(url: temporary_url_for(row.fetch(:post_id), token))
|
||||||
|
end
|
||||||
|
|
||||||
|
changed.each do |row|
|
||||||
|
Post.where(id: row.fetch(:post_id))
|
||||||
|
.update_all(url: row.fetch(:sanitised_url))
|
||||||
|
end
|
||||||
|
end
|
||||||
|
rescue ActiveRecord::RecordNotUnique => error
|
||||||
|
conflicts = build_persisted_conflicts(rewrites)
|
||||||
|
raise error if conflicts.empty?
|
||||||
|
|
||||||
|
raise UrlConflictError.new(conflicts), cause: error
|
||||||
|
end
|
||||||
|
|
||||||
|
private
|
||||||
|
|
||||||
|
def rules = kept.order(:priority).map { |r| [Regexp.new(r.source_pattern), r.replacement] }
|
||||||
|
|
||||||
|
def sanitise_with_rules(url, compiled_rules)
|
||||||
|
compiled_rules.reduce(url.dup) do |value, (pattern, replacement)|
|
||||||
|
value.sub(pattern, replacement)
|
||||||
|
end
|
||||||
|
end
|
||||||
|
|
||||||
|
def temporary_url_for(post_id, token) =
|
||||||
|
"https://post-url-sanitising.invalid/#{ token }/#{ post_id }"
|
||||||
|
|
||||||
|
def invalid_sanitised_url?(url)
|
||||||
|
return true if url.blank?
|
||||||
|
return true if url.length > 768
|
||||||
|
|
||||||
|
parsed = URI.parse(url)
|
||||||
|
return true if !(parsed in URI::HTTP)
|
||||||
|
return true if parsed.host.blank?
|
||||||
|
|
||||||
|
false
|
||||||
|
rescue URI::InvalidURIError
|
||||||
|
true
|
||||||
|
end
|
||||||
|
|
||||||
|
def build_conflicts(rewrites)
|
||||||
|
rewrites
|
||||||
|
.group_by { _1.fetch(:sanitised_url).downcase }
|
||||||
|
.values
|
||||||
|
.filter { _1.size > 1 }
|
||||||
|
.flatten
|
||||||
|
.map { { url: _1.fetch(:sanitised_url),
|
||||||
|
post_id: _1.fetch(:post_id),
|
||||||
|
original_url: _1.fetch(:original_url) } }
|
||||||
|
end
|
||||||
|
|
||||||
|
def build_persisted_conflicts(rewrites)
|
||||||
|
return [] if rewrites.blank?
|
||||||
|
|
||||||
|
target_rows = rewrites.filter { _1.fetch(:original_url) != _1.fetch(:sanitised_url) }
|
||||||
|
target_keys = target_rows.map { _1.fetch(:sanitised_url).downcase }.uniq
|
||||||
|
return [] if target_keys.empty?
|
||||||
|
|
||||||
|
target_pairs = target_rows.to_h do |row|
|
||||||
|
[row.fetch(:post_id), row.fetch(:sanitised_url).downcase]
|
||||||
|
end
|
||||||
|
|
||||||
|
persisted_rows = Post.order(:id)
|
||||||
|
.where('LOWER(url) IN (?)', target_keys)
|
||||||
|
.pluck(:id, :url)
|
||||||
|
.reject { |post_id, original_url| target_pairs[post_id] == original_url.downcase }
|
||||||
|
.map { |post_id, original_url|
|
||||||
|
{ url: original_url,
|
||||||
|
post_id:,
|
||||||
|
original_url:,
|
||||||
|
conflict_key: original_url.downcase }
|
||||||
|
}
|
||||||
|
|
||||||
|
target_conflicts = target_rows.map { { url: _1.fetch(:sanitised_url),
|
||||||
|
post_id: _1.fetch(:post_id),
|
||||||
|
original_url: _1.fetch(:original_url),
|
||||||
|
conflict_key: _1.fetch(:sanitised_url).downcase } }
|
||||||
|
|
||||||
|
(persisted_rows + target_conflicts)
|
||||||
|
.group_by { _1.fetch(:conflict_key) }
|
||||||
|
.values
|
||||||
|
.filter { _1.size > 1 }
|
||||||
|
.flatten
|
||||||
|
.map { _1.except(:conflict_key) }
|
||||||
|
end
|
||||||
|
end
|
||||||
|
|
||||||
|
private
|
||||||
|
|
||||||
|
def source_pattern_must_be_regexp
|
||||||
|
return if source_pattern.blank?
|
||||||
|
|
||||||
|
Regexp.new(source_pattern)
|
||||||
|
rescue RegexpError
|
||||||
|
errors.add :source_pattern, '変な正規表現だね〜(笑)'
|
||||||
|
end
|
||||||
|
end
|
||||||
@@ -0,0 +1,122 @@
|
|||||||
|
class PostCreator
|
||||||
|
class VideoMsParseError < ArgumentError; end
|
||||||
|
|
||||||
|
def initialize actor:, attributes:
|
||||||
|
@actor = actor
|
||||||
|
@attributes = attributes.symbolize_keys
|
||||||
|
end
|
||||||
|
|
||||||
|
def create!
|
||||||
|
post = Post.new(title: @attributes[:title].presence,
|
||||||
|
url: @attributes[:url],
|
||||||
|
thumbnail_base: @attributes[:thumbnail_base].presence,
|
||||||
|
uploaded_user: @actor,
|
||||||
|
original_created_from: @attributes[:original_created_from].presence,
|
||||||
|
original_created_before: @attributes[:original_created_before].presence)
|
||||||
|
thumbnail = @attributes[:thumbnail]
|
||||||
|
post.thumbnail.attach(Post.resized_thumbnail_attachment(thumbnail)) if thumbnail.present?
|
||||||
|
|
||||||
|
ApplicationRecord.transaction do
|
||||||
|
post.save!
|
||||||
|
Tag.normalise_tags!(tag_names, deny_deprecated: true, with_sections: true) =>
|
||||||
|
{ tags:, sections: }
|
||||||
|
TagVersioning.record_tag_snapshots!(tags, created_by_user: @actor)
|
||||||
|
tags = Tag.expand_parent_tags(tags).reject(&:deprecated?)
|
||||||
|
post.video_ms = normalise_video_ms(tags)
|
||||||
|
validate_video_sections!(post.video_ms, sections)
|
||||||
|
post.save!
|
||||||
|
sync_post_tags!(post, tags, sections)
|
||||||
|
sync_parent_posts!(post, parent_post_ids)
|
||||||
|
PostVersionRecorder.record!(post:, event_type: :create, created_by_user: @actor)
|
||||||
|
end
|
||||||
|
post
|
||||||
|
end
|
||||||
|
|
||||||
|
private
|
||||||
|
|
||||||
|
def tag_names = @attributes[:tags].to_s.split
|
||||||
|
|
||||||
|
def parent_post_ids
|
||||||
|
Array(@attributes[:parent_post_ids]).flat_map { _1.to_s.split }.map { |token|
|
||||||
|
id = Integer(token, exception: false)
|
||||||
|
raise ArgumentError, "親投稿 Id. が不正です: #{ token }" if id.nil? || id <= 0
|
||||||
|
|
||||||
|
id
|
||||||
|
}.uniq
|
||||||
|
end
|
||||||
|
|
||||||
|
def normalise_video_ms tags
|
||||||
|
return nil unless tags.any? { _1.id == Tag.video.id }
|
||||||
|
|
||||||
|
video_ms = @attributes[:video_ms]
|
||||||
|
if video_ms.present?
|
||||||
|
value = Integer(video_ms, exception: false)
|
||||||
|
raise VideoMsParseError unless value&.positive?
|
||||||
|
|
||||||
|
return value
|
||||||
|
end
|
||||||
|
duration = @attributes[:duration]
|
||||||
|
return nil if duration.blank?
|
||||||
|
|
||||||
|
return duration.to_i if duration.is_a?(Numeric) && duration.to_i.positive?
|
||||||
|
|
||||||
|
value = Tag.time_to_ms!(duration.to_s, tag_name: '動画時間')
|
||||||
|
raise VideoMsParseError unless value.positive?
|
||||||
|
|
||||||
|
value
|
||||||
|
rescue Tag::SectionLiteralParseError
|
||||||
|
raise VideoMsParseError
|
||||||
|
end
|
||||||
|
|
||||||
|
def validate_video_sections! video_ms, sections
|
||||||
|
return unless video_ms
|
||||||
|
|
||||||
|
sections.each_value do |ranges|
|
||||||
|
ranges.each do |begin_ms, end_ms|
|
||||||
|
post = Post.new
|
||||||
|
if begin_ms >= video_ms
|
||||||
|
post.errors.add :video_ms, 'タグ区間の開始が動画時間以上です.'
|
||||||
|
raise ActiveRecord::RecordInvalid, post
|
||||||
|
end
|
||||||
|
if end_ms && end_ms > video_ms
|
||||||
|
post.errors.add :video_ms, 'タグ区間の終端が動画時間を超えてゐます.'
|
||||||
|
raise ActiveRecord::RecordInvalid, post
|
||||||
|
end
|
||||||
|
end
|
||||||
|
end
|
||||||
|
end
|
||||||
|
|
||||||
|
def sync_post_tags! post, desired_tags, sections
|
||||||
|
desired_ids = desired_tags.map(&:id).to_set
|
||||||
|
current_ids = post.tags.pluck(:id).to_set
|
||||||
|
Tag.where(id: desired_ids - current_ids).find_each do |tag|
|
||||||
|
PostTag.create_or_find_by!(post:, tag:, created_user: @actor)
|
||||||
|
end
|
||||||
|
PostTagSection.where(post_id: post.id).destroy_all
|
||||||
|
sections.each do |tag_id, ranges|
|
||||||
|
ranges.each do |begin_ms, end_ms|
|
||||||
|
PostTagSection.create!(post_id: post.id,
|
||||||
|
tag_id:,
|
||||||
|
begin_ms:,
|
||||||
|
end_ms:)
|
||||||
|
end
|
||||||
|
end
|
||||||
|
PostTag.where(post_id: post.id,
|
||||||
|
tag_id: (current_ids - desired_ids).to_a).kept.find_each do |post_tag|
|
||||||
|
post_tag.discard_by!(@actor)
|
||||||
|
end
|
||||||
|
end
|
||||||
|
|
||||||
|
def sync_parent_posts! post, ids
|
||||||
|
if ids.include?(post.id)
|
||||||
|
post.errors.add :parent_post_ids, '自分自身を親投稿にはできません.'
|
||||||
|
raise ActiveRecord::RecordInvalid, post
|
||||||
|
end
|
||||||
|
missing = ids - Post.where(id: ids).pluck(:id)
|
||||||
|
if missing.present?
|
||||||
|
post.errors.add :parent_post_ids, "存在しない親投稿 Id. があります: #{ missing.join(' ') }"
|
||||||
|
raise ActiveRecord::RecordInvalid, post
|
||||||
|
end
|
||||||
|
ids.each { |parent_post_id| PostImplication.create_or_find_by!(post:, parent_post_id:) }
|
||||||
|
end
|
||||||
|
end
|
||||||
@@ -0,0 +1,422 @@
|
|||||||
|
require 'timeout'
|
||||||
|
|
||||||
|
class PostImportPreviewer
|
||||||
|
FIELDS = [
|
||||||
|
'title',
|
||||||
|
'thumbnail_base',
|
||||||
|
'original_created_from',
|
||||||
|
'original_created_before',
|
||||||
|
'duration',
|
||||||
|
'tags',
|
||||||
|
'parent_post_ids'].freeze
|
||||||
|
FETCH_WARNING_FIELDS = ['url', 'title', 'thumbnail_base'].freeze
|
||||||
|
EXISTING_SKIP_WARNING = '既存投稿のためスキップします.'.freeze
|
||||||
|
TITLE_FETCH_WARNING = 'タイトルを取得できませんでした.'.freeze
|
||||||
|
THUMBNAIL_FETCH_WARNING = 'サムネールを取得できませんでした.'.freeze
|
||||||
|
METADATA_FETCH_WARNING = 'メタデータを取得できませんでした.'.freeze
|
||||||
|
|
||||||
|
def preview_rows rows:, fetch_metadata: true, metadata_cache: { }
|
||||||
|
prepared_rows = rows.map { prepare_row(_1) }
|
||||||
|
url_counts = prepared_rows.filter_map { _1[:normal_url] }.tally
|
||||||
|
existing_posts =
|
||||||
|
Post.where(url: prepared_rows.map { _1[:normal_url] }.compact.uniq).index_by(&:url)
|
||||||
|
existing_parent_ids = preload_parent_ids(prepared_rows)
|
||||||
|
preload_metadata!(
|
||||||
|
prepared_rows,
|
||||||
|
fetch_metadata,
|
||||||
|
metadata_cache,
|
||||||
|
existing_posts,
|
||||||
|
url_counts)
|
||||||
|
known_tags =
|
||||||
|
preload_known_tags(
|
||||||
|
prepared_rows,
|
||||||
|
fetch_metadata,
|
||||||
|
metadata_cache,
|
||||||
|
existing_posts,
|
||||||
|
url_counts)
|
||||||
|
prepared_rows.map { |row|
|
||||||
|
preview_row(row,
|
||||||
|
fetch_metadata:,
|
||||||
|
metadata_cache:,
|
||||||
|
existing_posts:,
|
||||||
|
url_counts:,
|
||||||
|
known_tags:,
|
||||||
|
existing_parent_ids:)
|
||||||
|
}
|
||||||
|
end
|
||||||
|
|
||||||
|
def normalised_url value
|
||||||
|
PostUrlNormaliser.normalise(value)
|
||||||
|
end
|
||||||
|
|
||||||
|
private
|
||||||
|
|
||||||
|
def prepare_row row
|
||||||
|
source = row.symbolize_keys
|
||||||
|
url = source[:url].to_s.strip
|
||||||
|
normal_url = normalised_url(url)
|
||||||
|
source.merge(url_text: url, normal_url:)
|
||||||
|
end
|
||||||
|
|
||||||
|
def preview_row row,
|
||||||
|
fetch_metadata:,
|
||||||
|
metadata_cache:,
|
||||||
|
existing_posts:,
|
||||||
|
url_counts:,
|
||||||
|
known_tags:,
|
||||||
|
existing_parent_ids:
|
||||||
|
attributes = initial_attributes(row)
|
||||||
|
provenance = initial_provenance(row)
|
||||||
|
tag_sources = initial_tag_sources(row, attributes, provenance)
|
||||||
|
field_warnings = initial_field_warnings(row)
|
||||||
|
base_warnings = initial_base_warnings(row)
|
||||||
|
url = row[:url_text]
|
||||||
|
provenance['url'] = 'manual'
|
||||||
|
normal_url = row[:normal_url]
|
||||||
|
url_for_metadata = normal_url || url
|
||||||
|
existing_post = normal_url.present? ? existing_posts[normal_url] : nil
|
||||||
|
|
||||||
|
validation_errors = {}
|
||||||
|
validation_errors[:url] = ['URL が不正です.'] if normal_url.blank?
|
||||||
|
if normal_url.present? && url_counts[normal_url].to_i > 1
|
||||||
|
validation_errors[:url] = ['URL が重複しています.']
|
||||||
|
end
|
||||||
|
|
||||||
|
if row[:metadata_url].present? && row[:metadata_url] != url_for_metadata
|
||||||
|
clear_automatic_values!(attributes, provenance, tag_sources)
|
||||||
|
clear_fetch_warnings!(field_warnings)
|
||||||
|
end
|
||||||
|
|
||||||
|
if validation_errors.blank? && normal_url.present? && existing_post
|
||||||
|
add_field_warning!(field_warnings, 'url', EXISTING_SKIP_WARNING)
|
||||||
|
attributes['tags'] = merged_tags(tag_sources, provenance['tags'])
|
||||||
|
warnings_present = field_warnings.values.any?(&:present?) || base_warnings.present?
|
||||||
|
return { source_row: row[:source_row],
|
||||||
|
url: normal_url,
|
||||||
|
attributes:,
|
||||||
|
provenance:,
|
||||||
|
tag_sources:,
|
||||||
|
metadata_url: url_for_metadata,
|
||||||
|
skip_reason: 'existing',
|
||||||
|
existing_post_id: existing_post.id,
|
||||||
|
field_warnings:,
|
||||||
|
base_warnings:,
|
||||||
|
validation_errors:,
|
||||||
|
status: warnings_present ? 'warning' : 'ready' }
|
||||||
|
end
|
||||||
|
|
||||||
|
should_fetch = validation_errors.blank?
|
||||||
|
should_fetch &&= should_fetch_metadata?(fetch_metadata, row[:source_row].to_i)
|
||||||
|
if should_fetch
|
||||||
|
clear_fetch_warnings!(field_warnings)
|
||||||
|
metadata = metadata_for(url_for_metadata, metadata_cache)
|
||||||
|
apply_metadata!(attributes, provenance, tag_sources, metadata[:data])
|
||||||
|
apply_fetch_warnings!(field_warnings, metadata[:warnings])
|
||||||
|
end
|
||||||
|
|
||||||
|
attributes['url'] = url
|
||||||
|
validate_basic_data(attributes, validation_errors)
|
||||||
|
validate_preview_tags(merged_tags(tag_sources, provenance['tags']),
|
||||||
|
validation_errors,
|
||||||
|
field_warnings,
|
||||||
|
known_tags)
|
||||||
|
validate_parents(attributes['parent_post_ids'], validation_errors, existing_parent_ids)
|
||||||
|
attributes.delete('url')
|
||||||
|
attributes['tags'] = merged_tags(tag_sources, provenance['tags'])
|
||||||
|
|
||||||
|
warnings_present = field_warnings.values.any?(&:present?) || base_warnings.present?
|
||||||
|
{ source_row: row[:source_row],
|
||||||
|
url: normal_url || url,
|
||||||
|
attributes:,
|
||||||
|
provenance:,
|
||||||
|
tag_sources:,
|
||||||
|
metadata_url: url_for_metadata,
|
||||||
|
skip_reason: nil,
|
||||||
|
existing_post_id: nil,
|
||||||
|
field_warnings:,
|
||||||
|
base_warnings:,
|
||||||
|
validation_errors:,
|
||||||
|
status: validation_errors.present? ? 'error' : (warnings_present ? 'warning' : 'ready') }
|
||||||
|
end
|
||||||
|
|
||||||
|
def should_fetch_metadata? fetch_metadata, source_row
|
||||||
|
case fetch_metadata
|
||||||
|
when true then true
|
||||||
|
when Integer then fetch_metadata == source_row
|
||||||
|
when false, nil then false
|
||||||
|
else raise ArgumentError, 'メタデータ取得対象が不正です.'
|
||||||
|
end
|
||||||
|
end
|
||||||
|
|
||||||
|
def initial_attributes row
|
||||||
|
attributes = row[:attributes]&.stringify_keys || { }
|
||||||
|
FIELDS.to_h { |field| [field, attributes[field].to_s] }
|
||||||
|
end
|
||||||
|
|
||||||
|
def initial_provenance row
|
||||||
|
provenance = row[:provenance]&.stringify_keys || { }
|
||||||
|
FIELDS.to_h { |field| [field, provenance[field].presence || 'automatic'] }
|
||||||
|
.merge('url' => 'manual')
|
||||||
|
end
|
||||||
|
|
||||||
|
def initial_tag_sources row, attributes, provenance
|
||||||
|
sources = row[:tag_sources]&.stringify_keys || { 'automatic' => '', 'manual' => '' }
|
||||||
|
sources['automatic'] = sources['automatic'].to_s
|
||||||
|
sources['manual'] =
|
||||||
|
provenance['tags'] == 'manual' ? attributes['tags'].to_s : sources['manual'].to_s
|
||||||
|
sources
|
||||||
|
end
|
||||||
|
|
||||||
|
def initial_field_warnings row
|
||||||
|
(row[:field_warnings] || { })
|
||||||
|
.stringify_keys
|
||||||
|
.transform_values { |value| Array(value).map(&:to_s) }
|
||||||
|
end
|
||||||
|
|
||||||
|
def initial_base_warnings row
|
||||||
|
Array(row[:base_warnings]).map(&:to_s)
|
||||||
|
end
|
||||||
|
|
||||||
|
def clear_automatic_values! attributes, provenance, tag_sources
|
||||||
|
['title', 'thumbnail_base', 'original_created_from',
|
||||||
|
'original_created_before', 'duration'].each do |field|
|
||||||
|
attributes[field] = '' if provenance[field] == 'automatic'
|
||||||
|
end
|
||||||
|
tag_sources['automatic'] = ''
|
||||||
|
attributes['tags'] = merged_tags(tag_sources, provenance['tags'])
|
||||||
|
end
|
||||||
|
|
||||||
|
def clear_fetch_warnings! field_warnings
|
||||||
|
FETCH_WARNING_FIELDS.each do |field|
|
||||||
|
field_warnings.delete(field)
|
||||||
|
end
|
||||||
|
end
|
||||||
|
|
||||||
|
def metadata_for url, cache
|
||||||
|
cache[url] ||= fetch_metadata(url)
|
||||||
|
end
|
||||||
|
|
||||||
|
def fetch_metadata url
|
||||||
|
return { data: { }, warnings: { 'url' => ['URL が空です.'] } } if url.blank?
|
||||||
|
|
||||||
|
data = PostMetadataFetcher.fetch(url).stringify_keys.compact
|
||||||
|
warnings = { }
|
||||||
|
add_field_warning!(warnings, 'title', TITLE_FETCH_WARNING) if data['title'].blank?
|
||||||
|
if data['thumbnail_base'].blank?
|
||||||
|
add_field_warning!(warnings, 'thumbnail_base', THUMBNAIL_FETCH_WARNING)
|
||||||
|
end
|
||||||
|
{ data:, warnings: }
|
||||||
|
rescue Preview::UrlSafety::UnsafeUrl,
|
||||||
|
Preview::HttpFetcher::FetchFailed,
|
||||||
|
Preview::HttpFetcher::ResponseTooLarge => e
|
||||||
|
Rails.logger.info(
|
||||||
|
"post_import_metadata_fetch_failure "\
|
||||||
|
"#{ { error: e.class.name, message: e.message }.to_json }")
|
||||||
|
{ data: { }, warnings: { 'url' => [METADATA_FETCH_WARNING] } }
|
||||||
|
end
|
||||||
|
|
||||||
|
def preload_metadata! prepared_rows, fetch_metadata, metadata_cache, existing_posts, url_counts
|
||||||
|
urls = prepared_rows.filter_map { |row|
|
||||||
|
next unless row[:normal_url].present?
|
||||||
|
next unless should_fetch_metadata?(fetch_metadata, row[:source_row].to_i)
|
||||||
|
next if url_counts[row[:normal_url]].to_i > 1
|
||||||
|
next if existing_posts.key?(row[:normal_url])
|
||||||
|
|
||||||
|
row[:normal_url]
|
||||||
|
}.uniq
|
||||||
|
urls = urls.reject { metadata_cache.key?(_1) }
|
||||||
|
return if urls.empty?
|
||||||
|
|
||||||
|
url_queue = Queue.new
|
||||||
|
result_queue = Queue.new
|
||||||
|
urls.each { url_queue << _1 }
|
||||||
|
workers = [urls.length, 4].min.times.map {
|
||||||
|
Thread.new {
|
||||||
|
Rails.application.executor.wrap do
|
||||||
|
loop do
|
||||||
|
url = url_queue.pop(true)
|
||||||
|
result_queue << [url, safe_fetch_metadata(url)]
|
||||||
|
rescue ThreadError
|
||||||
|
break
|
||||||
|
end
|
||||||
|
end
|
||||||
|
}
|
||||||
|
}
|
||||||
|
Timeout.timeout(15) { workers.each(&:join) }
|
||||||
|
rescue Timeout::Error
|
||||||
|
workers&.each(&:kill)
|
||||||
|
ensure
|
||||||
|
workers&.each(&:join)
|
||||||
|
while result_queue&.size.to_i.positive?
|
||||||
|
url, result = result_queue.pop
|
||||||
|
metadata_cache[url] = result
|
||||||
|
end
|
||||||
|
urls&.each do |url|
|
||||||
|
metadata_cache[url] ||= { data: { }, warnings: { 'url' => [METADATA_FETCH_WARNING] } }
|
||||||
|
end
|
||||||
|
end
|
||||||
|
|
||||||
|
def safe_fetch_metadata url
|
||||||
|
fetch_metadata(url)
|
||||||
|
rescue StandardError => e
|
||||||
|
Rails.logger.error(
|
||||||
|
"post_import_metadata_fetch_unexpected_failure "\
|
||||||
|
"#{ { error: e.class.name, message: e.message }.to_json }")
|
||||||
|
{ data: { }, warnings: { 'url' => [METADATA_FETCH_WARNING] } }
|
||||||
|
end
|
||||||
|
|
||||||
|
def preload_known_tags prepared_rows, fetch_metadata, metadata_cache, existing_posts, url_counts
|
||||||
|
names = prepared_rows.flat_map { |row|
|
||||||
|
attributes = initial_attributes(row)
|
||||||
|
provenance = initial_provenance(row)
|
||||||
|
tag_sources = initial_tag_sources(row, attributes, provenance)
|
||||||
|
if metadata_url_changed?(row)
|
||||||
|
clear_automatic_values!(attributes, provenance, tag_sources)
|
||||||
|
end
|
||||||
|
if should_apply_metadata_to_row?(row, fetch_metadata, existing_posts, url_counts)
|
||||||
|
metadata = metadata_for(row[:normal_url], metadata_cache)
|
||||||
|
apply_metadata!(attributes, provenance, tag_sources, metadata[:data])
|
||||||
|
end
|
||||||
|
preview_tag_names(merged_tags(tag_sources, provenance['tags']))
|
||||||
|
}.compact.uniq
|
||||||
|
return { } if names.empty?
|
||||||
|
|
||||||
|
Tag.joins(:tag_name)
|
||||||
|
.where(tag_names: { name: names })
|
||||||
|
.includes(:tag_name)
|
||||||
|
.to_a
|
||||||
|
.index_by(&:name)
|
||||||
|
end
|
||||||
|
|
||||||
|
def preload_parent_ids prepared_rows
|
||||||
|
ids = prepared_rows.flat_map { |row|
|
||||||
|
attributes = initial_attributes(row)
|
||||||
|
preview_parent_ids(attributes['parent_post_ids'])
|
||||||
|
}.uniq
|
||||||
|
return { } if ids.empty?
|
||||||
|
|
||||||
|
Post.where(id: ids).pluck(:id).to_h { [_1, true] }
|
||||||
|
end
|
||||||
|
|
||||||
|
def metadata_url_changed? row
|
||||||
|
row[:metadata_url].present? && row[:metadata_url] != (row[:normal_url] || row[:url_text])
|
||||||
|
end
|
||||||
|
|
||||||
|
def should_apply_metadata_to_row? row, fetch_metadata, existing_posts, url_counts
|
||||||
|
normal_url = row[:normal_url]
|
||||||
|
return false if normal_url.blank?
|
||||||
|
return false if url_counts[normal_url].to_i > 1
|
||||||
|
return false if existing_posts.key?(normal_url)
|
||||||
|
|
||||||
|
should_fetch_metadata?(fetch_metadata, row[:source_row].to_i)
|
||||||
|
end
|
||||||
|
|
||||||
|
def apply_metadata! attributes, provenance, tag_sources, metadata
|
||||||
|
metadata.each do |field, value|
|
||||||
|
if field == 'tags'
|
||||||
|
next if provenance['tags'] == 'manual'
|
||||||
|
|
||||||
|
tag_sources['automatic'] = value.to_s
|
||||||
|
attributes['tags'] = merged_tags(tag_sources)
|
||||||
|
next
|
||||||
|
end
|
||||||
|
next unless provenance[field] == 'automatic'
|
||||||
|
|
||||||
|
attributes[field] = value
|
||||||
|
provenance[field] = 'automatic'
|
||||||
|
end
|
||||||
|
end
|
||||||
|
|
||||||
|
def apply_fetch_warnings! field_warnings, warnings
|
||||||
|
warnings.each do |field, values|
|
||||||
|
values.each do |value|
|
||||||
|
add_field_warning!(field_warnings, field, value)
|
||||||
|
end
|
||||||
|
end
|
||||||
|
end
|
||||||
|
|
||||||
|
def add_field_warning! field_warnings, field, message
|
||||||
|
field_warnings[field] ||= []
|
||||||
|
field_warnings[field] << message unless field_warnings[field].include?(message)
|
||||||
|
end
|
||||||
|
|
||||||
|
def merged_tags sources, origin = nil
|
||||||
|
return sources['manual'].to_s if origin == 'manual'
|
||||||
|
|
||||||
|
sources['automatic'].to_s
|
||||||
|
end
|
||||||
|
|
||||||
|
def preview_tag_names raw
|
||||||
|
names = raw.to_s.split
|
||||||
|
return [] if names.empty?
|
||||||
|
if names.any? { _1.downcase.start_with?('nico:') }
|
||||||
|
return []
|
||||||
|
end
|
||||||
|
|
||||||
|
names.map { |name| TagName.canonicalise(name.sub(/\[.*\]\z/, '')).first }
|
||||||
|
rescue Tag::SectionLiteralParseError
|
||||||
|
[]
|
||||||
|
end
|
||||||
|
|
||||||
|
def validate_preview_tags raw, errors, field_warnings, known_tags
|
||||||
|
names = raw.to_s.split
|
||||||
|
return if names.empty?
|
||||||
|
if names.any? { _1.downcase.start_with?('nico:') }
|
||||||
|
errors[:tags] = ['ニコニコ・タグは直接指定できません.']
|
||||||
|
return
|
||||||
|
end
|
||||||
|
|
||||||
|
parsed = names.map { |name| TagName.canonicalise(name.sub(/\[.*\]\z/, '')).first }
|
||||||
|
existing = parsed.filter_map { known_tags[_1] }
|
||||||
|
deprecated = existing.select(&:deprecated?).map(&:name)
|
||||||
|
errors[:tags] = ["廃止済みタグがあります: #{ deprecated.join(' ') }"] if deprecated.present?
|
||||||
|
known = existing.reject(&:deprecated?).map(&:name)
|
||||||
|
new_tags = parsed.uniq - known
|
||||||
|
if new_tags.present?
|
||||||
|
add_field_warning!(field_warnings, 'tags', "新規タグを作成します: #{ new_tags.join(' ') }")
|
||||||
|
end
|
||||||
|
rescue Tag::SectionLiteralParseError
|
||||||
|
errors[:tags] = ['タグ区間の記法が不正です.']
|
||||||
|
end
|
||||||
|
|
||||||
|
def validate_basic_data attributes, errors
|
||||||
|
post = Post.new(title: attributes['title'].presence,
|
||||||
|
url: attributes['url'],
|
||||||
|
thumbnail_base: attributes['thumbnail_base'].presence,
|
||||||
|
original_created_from: attributes['original_created_from'].presence,
|
||||||
|
original_created_before: attributes['original_created_before'].presence,
|
||||||
|
video_ms: parse_duration(attributes['duration'], errors))
|
||||||
|
post.valid?
|
||||||
|
post.errors.each do |error|
|
||||||
|
next if error.attribute == :url && error.type == :taken
|
||||||
|
|
||||||
|
(errors[error.attribute] ||= []) << error.message
|
||||||
|
end
|
||||||
|
end
|
||||||
|
|
||||||
|
def parse_duration value, errors
|
||||||
|
return nil if value.blank?
|
||||||
|
|
||||||
|
value.is_a?(Numeric) ? value.to_i : Tag.time_to_ms!(value.to_s, tag_name: '動画時間')
|
||||||
|
rescue Tag::SectionLiteralParseError
|
||||||
|
errors[:video_ms] = ['動画時間の記法が不正です.']
|
||||||
|
nil
|
||||||
|
end
|
||||||
|
|
||||||
|
def preview_parent_ids raw
|
||||||
|
raw.to_s.split.map { Integer(_1, exception: false) }.compact
|
||||||
|
end
|
||||||
|
|
||||||
|
def validate_parents raw, errors, existing_parent_ids
|
||||||
|
ids = raw.to_s.split.map { Integer(_1, exception: false) }
|
||||||
|
return if ids.compact.empty? && raw.to_s.blank?
|
||||||
|
|
||||||
|
if ids.any? { _1.nil? || _1 <= 0 }
|
||||||
|
errors[:parent_post_ids] = ['親投稿 Id. が不正です.']
|
||||||
|
return
|
||||||
|
end
|
||||||
|
if ids.uniq.any? { !existing_parent_ids[_1] }
|
||||||
|
errors[:parent_post_ids] = ['存在しない親投稿 Id. があります.']
|
||||||
|
end
|
||||||
|
end
|
||||||
|
end
|
||||||
@@ -0,0 +1,171 @@
|
|||||||
|
class PostImportRowNormaliser
|
||||||
|
ORIGINS = ['automatic', 'manual'].freeze
|
||||||
|
STRING_FIELDS = [
|
||||||
|
'title',
|
||||||
|
'thumbnail_base',
|
||||||
|
'original_created_from',
|
||||||
|
'original_created_before',
|
||||||
|
'tags',
|
||||||
|
'parent_post_ids'].freeze
|
||||||
|
FLEXIBLE_FIELDS = ['duration', 'video_ms'].freeze
|
||||||
|
ATTRIBUTE_FIELDS = (STRING_FIELDS + FLEXIBLE_FIELDS).freeze
|
||||||
|
|
||||||
|
def self.normalise! rows, allow_warning_fields: false
|
||||||
|
raise ArgumentError, '取込行の形式が不正です.' unless rows.is_a?(Array)
|
||||||
|
raise ArgumentError, '取込件数が多すぎます.' if rows.length > PostImportUrlListParser::MAX_ROWS
|
||||||
|
|
||||||
|
normalised_rows = rows.map { normalise_row!(_1, allow_warning_fields:) }
|
||||||
|
source_rows = normalised_rows.map { _1['source_row'] }
|
||||||
|
raise ArgumentError, '元行番号が重複しています.' if source_rows.uniq.length != source_rows.length
|
||||||
|
if normalised_rows.sum { row_bytesize(_1) } > PostImportUrlListParser::MAX_BYTES
|
||||||
|
raise ArgumentError, '取込データが大きすぎます.'
|
||||||
|
end
|
||||||
|
|
||||||
|
normalised_rows
|
||||||
|
end
|
||||||
|
|
||||||
|
def self.normalise_row! row, allow_warning_fields:
|
||||||
|
unless row.is_a?(Hash) || row.is_a?(ActionController::Parameters)
|
||||||
|
raise ArgumentError, '取込行の形式が不正です.'
|
||||||
|
end
|
||||||
|
|
||||||
|
parameters =
|
||||||
|
row.is_a?(ActionController::Parameters) ? row : ActionController::Parameters.new(row)
|
||||||
|
permitted = parameters.permit(*permitted_keys(allow_warning_fields))
|
||||||
|
normalised = permitted.to_h.deep_transform_keys { _1.to_s.underscore }
|
||||||
|
|
||||||
|
normalised['source_row'] = normalise_source_row!(normalised['source_row'])
|
||||||
|
normalise_url!(normalised['url'])
|
||||||
|
normalise_metadata_url!(normalised['metadata_url'])
|
||||||
|
normalise_attributes!(normalised.fetch('attributes', { }))
|
||||||
|
normalise_provenance!(normalised.fetch('provenance', { }))
|
||||||
|
normalise_tag_sources!(normalised['tag_sources'])
|
||||||
|
normalise_warning_values!(normalised, allow_warning_fields:)
|
||||||
|
if row_bytesize(normalised) > PostImportUrlListParser::MAX_BYTES
|
||||||
|
raise ArgumentError, '取込行が大きすぎます.'
|
||||||
|
end
|
||||||
|
|
||||||
|
normalised
|
||||||
|
end
|
||||||
|
|
||||||
|
def self.permitted_keys allow_warning_fields
|
||||||
|
keys = [
|
||||||
|
:source_row,
|
||||||
|
:sourceRow,
|
||||||
|
:url,
|
||||||
|
:metadata_url,
|
||||||
|
:metadataUrl,
|
||||||
|
{ attributes: {} },
|
||||||
|
{ provenance: {} },
|
||||||
|
{ tag_sources: {} },
|
||||||
|
{ tagSources: {} }]
|
||||||
|
return keys unless allow_warning_fields
|
||||||
|
|
||||||
|
keys + [
|
||||||
|
{ field_warnings: {} },
|
||||||
|
{ fieldWarnings: {} },
|
||||||
|
{ base_warnings: [] },
|
||||||
|
{ baseWarnings: [] }]
|
||||||
|
end
|
||||||
|
private_class_method :permitted_keys
|
||||||
|
|
||||||
|
def self.normalise_source_row! value
|
||||||
|
source_row = Integer(value, exception: false)
|
||||||
|
raise ArgumentError, '元行番号が不正です.' if source_row.nil? || source_row <= 0
|
||||||
|
|
||||||
|
source_row
|
||||||
|
end
|
||||||
|
private_class_method :normalise_source_row!
|
||||||
|
|
||||||
|
def self.normalise_url! value
|
||||||
|
raise ArgumentError, 'URL の形式が不正です.' unless value.is_a?(String)
|
||||||
|
raise ArgumentError, 'URL が長すぎます.' if value.bytesize > PostImportUrlListParser::MAX_URL_BYTES
|
||||||
|
end
|
||||||
|
private_class_method :normalise_url!
|
||||||
|
|
||||||
|
def self.normalise_metadata_url! value
|
||||||
|
raise ArgumentError, 'metadata_url の形式が不正です.' unless value.nil? || value.is_a?(String)
|
||||||
|
if value.to_s.bytesize > PostImportUrlListParser::MAX_URL_BYTES
|
||||||
|
raise ArgumentError, 'metadata_url が長すぎます.'
|
||||||
|
end
|
||||||
|
end
|
||||||
|
private_class_method :normalise_metadata_url!
|
||||||
|
|
||||||
|
def self.normalise_attributes! attributes
|
||||||
|
raise ArgumentError, 'attributes の形式が不正です.' unless attributes.is_a?(Hash)
|
||||||
|
raise ArgumentError, '取込項目が不正です.' unless (attributes.keys - ATTRIBUTE_FIELDS).empty?
|
||||||
|
|
||||||
|
attributes.each do |key, value|
|
||||||
|
case key
|
||||||
|
when *STRING_FIELDS
|
||||||
|
raise ArgumentError, '取込項目の型が不正です.' unless value.nil? || value.is_a?(String)
|
||||||
|
when *FLEXIBLE_FIELDS
|
||||||
|
unless value.nil? || value.is_a?(String) || value.is_a?(Numeric)
|
||||||
|
raise ArgumentError, '取込項目の型が不正です.'
|
||||||
|
end
|
||||||
|
end
|
||||||
|
if value.to_s.bytesize > PostImportUrlListParser::MAX_URL_BYTES
|
||||||
|
raise ArgumentError, '取込項目が大きすぎます.'
|
||||||
|
end
|
||||||
|
end
|
||||||
|
end
|
||||||
|
private_class_method :normalise_attributes!
|
||||||
|
|
||||||
|
def self.normalise_provenance! provenance
|
||||||
|
raise ArgumentError, 'provenance の形式が不正です.' unless provenance.is_a?(Hash)
|
||||||
|
|
||||||
|
allowed = ATTRIBUTE_FIELDS + ['url']
|
||||||
|
raise ArgumentError, '値の由来が不正です.' unless (provenance.keys - allowed).empty?
|
||||||
|
raise ArgumentError, '値の由来が不正です.' unless provenance.values.all? { ORIGINS.include?(_1) }
|
||||||
|
end
|
||||||
|
private_class_method :normalise_provenance!
|
||||||
|
|
||||||
|
def self.normalise_tag_sources! tag_sources
|
||||||
|
return if tag_sources.nil?
|
||||||
|
|
||||||
|
raise ArgumentError, 'タグ由来の形式が不正です.' unless tag_sources.is_a?(Hash)
|
||||||
|
raise ArgumentError, 'タグ由来の形式が不正です.' unless (tag_sources.keys - ORIGINS).empty?
|
||||||
|
raise ArgumentError, 'タグ由来の形式が不正です.' unless tag_sources.values.all? { _1.is_a?(String) }
|
||||||
|
if tag_sources.values.any? { _1.bytesize > PostImportUrlListParser::MAX_URL_BYTES }
|
||||||
|
raise ArgumentError, 'タグ由来が大きすぎます.'
|
||||||
|
end
|
||||||
|
if tag_sources.values.sum(&:bytesize) > PostImportUrlListParser::MAX_URL_BYTES
|
||||||
|
raise ArgumentError, 'タグ由来が大きすぎます.'
|
||||||
|
end
|
||||||
|
end
|
||||||
|
private_class_method :normalise_tag_sources!
|
||||||
|
|
||||||
|
def self.normalise_warning_values! normalised, allow_warning_fields:
|
||||||
|
return unless allow_warning_fields
|
||||||
|
|
||||||
|
field_warnings = normalised['field_warnings']
|
||||||
|
unless field_warnings.nil? || field_warnings.is_a?(Hash)
|
||||||
|
raise ArgumentError, '警告の形式が不正です.'
|
||||||
|
end
|
||||||
|
field_warnings&.each do |key, values|
|
||||||
|
raise ArgumentError, '警告の形式が不正です.' unless ATTRIBUTE_FIELDS.include?(key) || key == 'url'
|
||||||
|
unless values.is_a?(Array) && values.all? { _1.is_a?(String) }
|
||||||
|
raise ArgumentError, '警告の形式が不正です.'
|
||||||
|
end
|
||||||
|
if values.any? { _1.bytesize > PostImportUrlListParser::MAX_URL_BYTES }
|
||||||
|
raise ArgumentError, '警告が大きすぎます.'
|
||||||
|
end
|
||||||
|
end
|
||||||
|
|
||||||
|
base_warnings = normalised['base_warnings']
|
||||||
|
return if base_warnings.nil?
|
||||||
|
|
||||||
|
unless base_warnings.is_a?(Array) && base_warnings.all? { _1.is_a?(String) }
|
||||||
|
raise ArgumentError, '警告の形式が不正です.'
|
||||||
|
end
|
||||||
|
if base_warnings.any? { _1.bytesize > PostImportUrlListParser::MAX_URL_BYTES }
|
||||||
|
raise ArgumentError, '警告が大きすぎます.'
|
||||||
|
end
|
||||||
|
end
|
||||||
|
private_class_method :normalise_warning_values!
|
||||||
|
|
||||||
|
def self.row_bytesize row
|
||||||
|
row.to_json.bytesize
|
||||||
|
end
|
||||||
|
private_class_method :row_bytesize
|
||||||
|
end
|
||||||
@@ -0,0 +1,77 @@
|
|||||||
|
class PostImportRunner
|
||||||
|
def initialize actor:, rows:
|
||||||
|
@actor = actor
|
||||||
|
@rows = rows
|
||||||
|
end
|
||||||
|
|
||||||
|
def run
|
||||||
|
normalised_rows = PostImportRowNormaliser.normalise!(@rows)
|
||||||
|
previews = PostImportPreviewer.new.preview_rows(rows: normalised_rows,
|
||||||
|
fetch_metadata: false)
|
||||||
|
preview_map = previews.index_by { _1[:source_row] }
|
||||||
|
|
||||||
|
results = normalised_rows.map do |row|
|
||||||
|
run_row(row, preview_map.fetch(row['source_row']))
|
||||||
|
end
|
||||||
|
|
||||||
|
{ created: results.count { _1[:status] == 'created' },
|
||||||
|
skipped: results.count { _1[:status] == 'skipped' },
|
||||||
|
failed: results.count { _1[:status] == 'failed' },
|
||||||
|
rows: results }
|
||||||
|
end
|
||||||
|
|
||||||
|
private
|
||||||
|
|
||||||
|
def run_row row, preview
|
||||||
|
attributes = row.fetch('attributes', { }).transform_keys { _1.to_s.underscore }
|
||||||
|
return { source_row: row['source_row'],
|
||||||
|
status: 'failed',
|
||||||
|
errors: preview[:validation_errors] } if preview[:validation_errors].present?
|
||||||
|
if preview[:skip_reason] == 'existing'
|
||||||
|
return { source_row: row['source_row'],
|
||||||
|
status: 'skipped',
|
||||||
|
existing_post_id: preview[:existing_post_id] }
|
||||||
|
end
|
||||||
|
|
||||||
|
attributes['tags'] = preview[:attributes]['tags']
|
||||||
|
attributes['url'] = row['url']
|
||||||
|
post = PostCreator.new(actor: @actor, attributes:).create!
|
||||||
|
{ source_row: row['source_row'], status: 'created', post: PostRepr.base(post) }
|
||||||
|
rescue ActiveRecord::RecordInvalid => e
|
||||||
|
existing_post = existing_post_for_race(row, e.record)
|
||||||
|
if existing_post
|
||||||
|
return { source_row: row['source_row'],
|
||||||
|
status: 'skipped',
|
||||||
|
existing_post_id: existing_post.id }
|
||||||
|
end
|
||||||
|
|
||||||
|
{ source_row: row['source_row'], status: 'failed', errors: e.record.errors.to_hash }
|
||||||
|
rescue Tag::NicoTagNormalisationError
|
||||||
|
{ source_row: row['source_row'],
|
||||||
|
status: 'failed',
|
||||||
|
errors: { tags: ['ニコニコ・タグは直接指定できません.'] } }
|
||||||
|
rescue Tag::DeprecatedTagNormalisationError
|
||||||
|
{ source_row: row['source_row'],
|
||||||
|
status: 'failed',
|
||||||
|
errors: { tags: ['廃止済みタグは付与できません.'] } }
|
||||||
|
rescue ArgumentError
|
||||||
|
{ source_row: row['source_row'],
|
||||||
|
status: 'failed',
|
||||||
|
errors: { base: ['入力値が不正です.'] } }
|
||||||
|
rescue StandardError => e
|
||||||
|
Rails.logger.error("post_import_runner_failure #{ { error: e.class.name,
|
||||||
|
message: e.message }.to_json }")
|
||||||
|
{ source_row: row['source_row'],
|
||||||
|
status: 'failed',
|
||||||
|
errors: { base: ['登録中にエラーが発生しました.'] } }
|
||||||
|
end
|
||||||
|
|
||||||
|
def existing_post_for_race row, record
|
||||||
|
return nil unless record.errors.of_kind?(:url, :taken)
|
||||||
|
|
||||||
|
normal_url = PostUrlNormaliser.normalise(row['url'])
|
||||||
|
return nil if normal_url.blank?
|
||||||
|
|
||||||
|
Post.find_by(url: normal_url)
|
||||||
|
end
|
||||||
|
end
|
||||||
@@ -0,0 +1,25 @@
|
|||||||
|
class PostImportUrlListParser
|
||||||
|
MAX_ROWS = 100
|
||||||
|
MAX_BYTES = 1.megabyte
|
||||||
|
MAX_URL_BYTES = 20.kilobytes
|
||||||
|
|
||||||
|
def self.parse source
|
||||||
|
raw = source.to_s
|
||||||
|
raise ArgumentError, '入力が大きすぎます.' if raw.bytesize > MAX_BYTES
|
||||||
|
|
||||||
|
rows = raw.split(/\r\n|\n|\r/).each_with_index.filter_map { |line, index|
|
||||||
|
url = line.strip
|
||||||
|
next if url.blank?
|
||||||
|
|
||||||
|
if url.bytesize > MAX_URL_BYTES
|
||||||
|
raise ArgumentError, "#{ index + 1 } 行目: URL が長すぎます."
|
||||||
|
end
|
||||||
|
|
||||||
|
{ source_row: index + 1, url: }
|
||||||
|
}
|
||||||
|
raise ArgumentError, 'URL を入力してください.' if rows.empty?
|
||||||
|
raise ArgumentError, "取込件数は #{ MAX_ROWS } 件までです." if rows.length > MAX_ROWS
|
||||||
|
|
||||||
|
rows
|
||||||
|
end
|
||||||
|
end
|
||||||
@@ -0,0 +1,122 @@
|
|||||||
|
class PostMetadataFetcher
|
||||||
|
TIMESTAMP_PATTERN =
|
||||||
|
Regexp.new(
|
||||||
|
'\A(\d{4})-(\d{2})-(\d{2})T(\d{2})' \
|
||||||
|
'(?::(\d{2})(?::(\d{2})(?:\.(\d+))?)?)?' \
|
||||||
|
'(Z|[+-]\d{2}:?\d{2})?\z')
|
||||||
|
|
||||||
|
def self.fetch raw_url
|
||||||
|
uri, = Preview::UrlSafety.validate(raw_url)
|
||||||
|
response = Preview::HttpFetcher.fetch(
|
||||||
|
uri.to_s,
|
||||||
|
max_bytes: Preview::ThumbnailFetcher::HTML_MAX_BYTES)
|
||||||
|
metadata = Preview::HtmlMetadataExtractor.extract(response)
|
||||||
|
document = Nokogiri::HTML.parse(response.body)
|
||||||
|
content = lambda { |name|
|
||||||
|
document
|
||||||
|
.at_css("meta[property='#{ name }'], meta[name='#{ name }']")
|
||||||
|
&.[]('content')
|
||||||
|
&.strip
|
||||||
|
&.presence
|
||||||
|
}
|
||||||
|
duration = content.call('og:video:duration') || content.call('video:duration')
|
||||||
|
published = content.call('article:published_time') || content.call('date')
|
||||||
|
created_range = original_created_range(published)
|
||||||
|
platform_tags = platform_tags(uri)
|
||||||
|
{ title: metadata[:title],
|
||||||
|
thumbnail_base:
|
||||||
|
Preview::KnownSiteExtractor.thumbnail_url(uri) || metadata[:image_url],
|
||||||
|
original_created_from: created_range&.first&.iso8601,
|
||||||
|
original_created_before: created_range&.last&.iso8601,
|
||||||
|
duration: duration&.to_f&.then { _1.positive? ? (_1 * 1_000).round : nil },
|
||||||
|
tags: platform_tags.join(' ') }
|
||||||
|
end
|
||||||
|
|
||||||
|
def self.platform_tags uri
|
||||||
|
return ['動画', 'YouTube'] if Preview::KnownSiteExtractor.youtube_video_id(uri)
|
||||||
|
return ['動画', 'ニコニコ'] if Preview::KnownSiteExtractor.niconico_video_id(uri)
|
||||||
|
|
||||||
|
[]
|
||||||
|
end
|
||||||
|
|
||||||
|
def self.original_created_range value
|
||||||
|
return nil if value.blank?
|
||||||
|
|
||||||
|
raw = value.to_s.strip
|
||||||
|
from, before =
|
||||||
|
case raw
|
||||||
|
when /\A(\d{4})\z/
|
||||||
|
year = Regexp.last_match(1).to_i
|
||||||
|
from = Time.zone.local(year, 1, 1)
|
||||||
|
[from, from + 1.year]
|
||||||
|
when /\A(\d{4})-(\d{2})\z/
|
||||||
|
year = Regexp.last_match(1).to_i
|
||||||
|
month = Regexp.last_match(2).to_i
|
||||||
|
from = Time.zone.local(year, month, 1)
|
||||||
|
[from, from + 1.month]
|
||||||
|
when /\A(\d{4})-(\d{2})-(\d{2})\z/
|
||||||
|
year = Regexp.last_match(1).to_i
|
||||||
|
month = Regexp.last_match(2).to_i
|
||||||
|
day = Regexp.last_match(3).to_i
|
||||||
|
from = Time.zone.local(year, month, day)
|
||||||
|
[from, from + 1.day]
|
||||||
|
else
|
||||||
|
parse_timestamp_range(raw)
|
||||||
|
end
|
||||||
|
return nil if from.nil? || before.nil?
|
||||||
|
|
||||||
|
[from, before]
|
||||||
|
rescue ArgumentError, TypeError
|
||||||
|
nil
|
||||||
|
end
|
||||||
|
|
||||||
|
def self.parse_timestamp_range raw
|
||||||
|
match = raw.match(TIMESTAMP_PATTERN)
|
||||||
|
return nil unless match
|
||||||
|
|
||||||
|
year = match[1].to_i
|
||||||
|
month = match[2].to_i
|
||||||
|
day = match[3].to_i
|
||||||
|
hour = match[4].to_i
|
||||||
|
minute = match[5]&.to_i || 0
|
||||||
|
second = match[6]&.to_i || 0
|
||||||
|
fraction = match[7]
|
||||||
|
offset = match[8]
|
||||||
|
whole_second = second + fractional_seconds(fraction)
|
||||||
|
from =
|
||||||
|
if offset.present?
|
||||||
|
Time.new(year, month, day, hour, minute, whole_second, parse_offset(offset)).in_time_zone
|
||||||
|
else
|
||||||
|
Time.zone.local(year, month, day, hour, minute, whole_second)
|
||||||
|
end
|
||||||
|
before =
|
||||||
|
if fraction.present?
|
||||||
|
from + (10**(-fraction.length))
|
||||||
|
elsif match[6].present?
|
||||||
|
from + 1.second
|
||||||
|
elsif match[5].present?
|
||||||
|
from + 1.minute
|
||||||
|
else
|
||||||
|
from + 1.hour
|
||||||
|
end
|
||||||
|
[from, before]
|
||||||
|
end
|
||||||
|
|
||||||
|
def self.parse_offset value
|
||||||
|
return '+00:00' if value == 'Z'
|
||||||
|
|
||||||
|
value.match?(/\A[+-]\d{2}:\d{2}\z/) ? value : "#{ value[0, 3] }:#{ value[3, 2] }"
|
||||||
|
end
|
||||||
|
|
||||||
|
def self.fractional_seconds value
|
||||||
|
return 0 if value.blank?
|
||||||
|
|
||||||
|
Rational(value.to_i, 10**value.length)
|
||||||
|
end
|
||||||
|
|
||||||
|
private_class_method :platform_tags,
|
||||||
|
:original_created_range,
|
||||||
|
:parse_timestamp_range,
|
||||||
|
:parse_offset,
|
||||||
|
:fractional_seconds
|
||||||
|
end
|
||||||
@@ -0,0 +1,13 @@
|
|||||||
|
class PostUrlNormaliser
|
||||||
|
def self.normalise raw_url
|
||||||
|
value = raw_url.to_s.strip
|
||||||
|
uri = URI.parse(value)
|
||||||
|
return nil unless uri.is_a?(URI::HTTP) && uri.host.present?
|
||||||
|
|
||||||
|
uri.host = uri.host.downcase
|
||||||
|
uri.path = uri.path.sub(/\/\z/, '') if uri.path.present?
|
||||||
|
PostUrlSanitisationRule.sanitise(uri.to_s)
|
||||||
|
rescue URI::InvalidURIError
|
||||||
|
nil
|
||||||
|
end
|
||||||
|
end
|
||||||
@@ -0,0 +1,21 @@
|
|||||||
|
module Preview
|
||||||
|
class HtmlMetadataExtractor
|
||||||
|
IMAGE_SELECTORS = [
|
||||||
|
'meta[property="og:image"]',
|
||||||
|
'meta[name="twitter:image"]',
|
||||||
|
'meta[name="thumbnail"]'
|
||||||
|
].freeze
|
||||||
|
|
||||||
|
def self.extract(response)
|
||||||
|
document = Nokogiri::HTML.parse(response.body)
|
||||||
|
image_url = IMAGE_SELECTORS.filter_map {
|
||||||
|
document.at_css(_1)&.[]('content')&.strip.presence
|
||||||
|
}.first
|
||||||
|
|
||||||
|
{ title: document.at_css('title')&.text&.strip,
|
||||||
|
image_url: image_url ? URI.join(response.url, image_url).to_s : nil }
|
||||||
|
rescue URI::InvalidURIError
|
||||||
|
{ title: document.at_css('title')&.text&.strip, image_url: nil }
|
||||||
|
end
|
||||||
|
end
|
||||||
|
end
|
||||||
@@ -0,0 +1,127 @@
|
|||||||
|
require 'net/http'
|
||||||
|
require 'json'
|
||||||
|
|
||||||
|
module Preview
|
||||||
|
class HttpFetcher
|
||||||
|
class FetchFailed < StandardError; end
|
||||||
|
class FetchTimeout < FetchFailed; end
|
||||||
|
class ResponseTooLarge < FetchFailed; end
|
||||||
|
|
||||||
|
MAX_REDIRECTS = 5
|
||||||
|
DEFAULT_MAX_BYTES = 5.megabytes
|
||||||
|
|
||||||
|
Response = Data.define(:body, :content_type, :url)
|
||||||
|
|
||||||
|
def self.fetch(raw_url,
|
||||||
|
max_bytes: DEFAULT_MAX_BYTES,
|
||||||
|
redirects: MAX_REDIRECTS)
|
||||||
|
uri, addresses = UrlSafety.validate(raw_url)
|
||||||
|
response = request(uri, addresses.first, max_bytes)
|
||||||
|
|
||||||
|
if response.is_a?(Net::HTTPRedirection)
|
||||||
|
location = response['location']
|
||||||
|
if redirects.zero?
|
||||||
|
log_failure(:redirect_limit,
|
||||||
|
url: uri.to_s,
|
||||||
|
redirects:,
|
||||||
|
location:,
|
||||||
|
content_type: response['content-type'],
|
||||||
|
content_length: response['content-length'])
|
||||||
|
raise FetchFailed, 'redirect が多すぎます.'
|
||||||
|
end
|
||||||
|
|
||||||
|
if location.blank?
|
||||||
|
log_failure(:blank_redirect_location,
|
||||||
|
url: uri.to_s,
|
||||||
|
redirects:,
|
||||||
|
content_type: response['content-type'],
|
||||||
|
content_length: response['content-length'])
|
||||||
|
raise FetchFailed, 'redirect 先が不正です.'
|
||||||
|
end
|
||||||
|
|
||||||
|
redirect_url =
|
||||||
|
begin
|
||||||
|
URI.join(uri, location).to_s
|
||||||
|
rescue URI::InvalidURIError => e
|
||||||
|
log_failure(:invalid_redirect_location,
|
||||||
|
url: uri.to_s,
|
||||||
|
redirects:,
|
||||||
|
location:,
|
||||||
|
error: e.class.name,
|
||||||
|
message: e.message)
|
||||||
|
raise FetchFailed, 'redirect 先が不正です.'
|
||||||
|
end
|
||||||
|
|
||||||
|
return fetch(redirect_url, max_bytes:, redirects: redirects - 1)
|
||||||
|
end
|
||||||
|
|
||||||
|
unless response.is_a?(Net::HTTPSuccess)
|
||||||
|
log_failure(:http_status,
|
||||||
|
url: uri.to_s,
|
||||||
|
code: response.code,
|
||||||
|
content_type: response['content-type'],
|
||||||
|
content_length: response['content-length'])
|
||||||
|
raise FetchFailed, "外部サーバーが HTTP #{ response.code } を返しました."
|
||||||
|
end
|
||||||
|
|
||||||
|
Response.new(response.body, response['content-type'].to_s, uri.to_s)
|
||||||
|
rescue Net::OpenTimeout, Net::ReadTimeout, Timeout::Error => e
|
||||||
|
log_failure(:timeout, url: uri&.to_s || raw_url, error: e.class.name, message: e.message)
|
||||||
|
raise FetchTimeout, e.message
|
||||||
|
rescue SocketError, SystemCallError, OpenSSL::SSL::SSLError, EOFError => e
|
||||||
|
log_failure(:network_error,
|
||||||
|
url: uri&.to_s || raw_url,
|
||||||
|
error: e.class.name,
|
||||||
|
message: e.message)
|
||||||
|
raise FetchFailed, e.message
|
||||||
|
end
|
||||||
|
|
||||||
|
def self.request(uri, ip_address, max_bytes)
|
||||||
|
http = Net::HTTP.new(uri.host, uri.port)
|
||||||
|
http.ipaddr = ip_address
|
||||||
|
http.use_ssl = uri.scheme == 'https'
|
||||||
|
http.open_timeout = 5
|
||||||
|
http.read_timeout = 8
|
||||||
|
http.write_timeout = 5
|
||||||
|
|
||||||
|
request = Net::HTTP::Get.new(uri)
|
||||||
|
request['User-Agent'] = 'BTRC-Hub thumbnail preview'
|
||||||
|
request['Accept'] = 'text/html,image/*;q=0.9,*/*;q=0.1'
|
||||||
|
|
||||||
|
http.request(request) do |response|
|
||||||
|
length = response['content-length'].to_i
|
||||||
|
if length > max_bytes
|
||||||
|
log_failure(:response_too_large,
|
||||||
|
url: uri.to_s,
|
||||||
|
content_type: response['content-type'],
|
||||||
|
content_length: response['content-length'],
|
||||||
|
max_bytes:)
|
||||||
|
raise ResponseTooLarge, '外部データが大きすぎます.'
|
||||||
|
end
|
||||||
|
|
||||||
|
body = +''
|
||||||
|
response.read_body do |chunk|
|
||||||
|
body << chunk
|
||||||
|
next unless body.bytesize > max_bytes
|
||||||
|
|
||||||
|
log_failure(:response_too_large,
|
||||||
|
url: uri.to_s,
|
||||||
|
content_type: response['content-type'],
|
||||||
|
content_length: response['content-length'],
|
||||||
|
bytes_read: body.bytesize,
|
||||||
|
max_bytes:)
|
||||||
|
raise ResponseTooLarge, '外部データが大きすぎます.'
|
||||||
|
end
|
||||||
|
response.instance_variable_set(:@body, body)
|
||||||
|
response.instance_variable_set(:@read, true)
|
||||||
|
return response
|
||||||
|
end
|
||||||
|
end
|
||||||
|
|
||||||
|
def self.log_failure(reason, **payload)
|
||||||
|
Rails.logger.warn("preview_http_fetcher_failure #{ { reason:, **payload }.to_json }")
|
||||||
|
end
|
||||||
|
|
||||||
|
private_class_method :request, :log_failure
|
||||||
|
end
|
||||||
|
end
|
||||||
@@ -0,0 +1,34 @@
|
|||||||
|
module Preview
|
||||||
|
class KnownSiteExtractor
|
||||||
|
def self.thumbnail_url(uri)
|
||||||
|
youtube_thumbnail(uri)
|
||||||
|
end
|
||||||
|
|
||||||
|
def self.youtube_video_id(uri)
|
||||||
|
case uri.host&.downcase
|
||||||
|
when 'youtu.be'
|
||||||
|
uri.path.split('/').reject(&:blank?).first
|
||||||
|
when 'www.youtube.com', 'youtube.com', 'm.youtube.com'
|
||||||
|
uri.path == '/watch' ? URI.decode_www_form(uri.query.to_s).to_h['v'] : nil
|
||||||
|
end&.then { _1 if _1.match?(/\A[A-Za-z0-9_-]{6,20}\z/) }
|
||||||
|
end
|
||||||
|
|
||||||
|
def self.niconico_video_id(uri)
|
||||||
|
case uri.host&.downcase
|
||||||
|
when 'www.nicovideo.jp', 'nicovideo.jp'
|
||||||
|
uri.path[%r{\A/watch/(sm\d+)\z}, 1]
|
||||||
|
when 'nico.ms'
|
||||||
|
uri.path[%r{\A/(sm\d+)\z}, 1]
|
||||||
|
end
|
||||||
|
end
|
||||||
|
|
||||||
|
def self.youtube_thumbnail(uri)
|
||||||
|
id = youtube_video_id(uri)
|
||||||
|
return unless id
|
||||||
|
|
||||||
|
"https://i.ytimg.com/vi/#{ id }/hqdefault.jpg"
|
||||||
|
end
|
||||||
|
|
||||||
|
private_class_method :youtube_thumbnail
|
||||||
|
end
|
||||||
|
end
|
||||||
@@ -0,0 +1,95 @@
|
|||||||
|
module Preview
|
||||||
|
class ThumbnailFetcher
|
||||||
|
class GenerationFailed < StandardError; end
|
||||||
|
ALLOWED_IMAGE_CONTENT_TYPES = [
|
||||||
|
'image/jpeg', 'image/png', 'image/gif', 'image/webp'
|
||||||
|
].freeze
|
||||||
|
HTML_MAX_BYTES = 1.megabyte
|
||||||
|
NICONICO_XML_MAX_BYTES = 256.kilobytes
|
||||||
|
|
||||||
|
def self.fetch(raw_url)
|
||||||
|
uri, = UrlSafety.validate(raw_url)
|
||||||
|
|
||||||
|
known_url = KnownSiteExtractor.thumbnail_url(uri)
|
||||||
|
image = fetch_image_or_nil(known_url) if known_url
|
||||||
|
return image if image
|
||||||
|
|
||||||
|
niconico_url = niconico_thumbnail_url(uri)
|
||||||
|
image = fetch_image_or_nil(niconico_url) if niconico_url
|
||||||
|
return image if image
|
||||||
|
|
||||||
|
page = HttpFetcher.fetch(uri.to_s, max_bytes: HTML_MAX_BYTES)
|
||||||
|
metadata = HtmlMetadataExtractor.extract(page)
|
||||||
|
raise GenerationFailed, 'サムネール画像が見つかりませんでした.' if metadata[:image_url].blank?
|
||||||
|
|
||||||
|
fetch_image!(metadata[:image_url])
|
||||||
|
end
|
||||||
|
|
||||||
|
def self.title(raw_url)
|
||||||
|
uri, = UrlSafety.validate(raw_url)
|
||||||
|
HtmlMetadataExtractor.extract(
|
||||||
|
HttpFetcher.fetch(uri.to_s, max_bytes: HTML_MAX_BYTES))[:title]
|
||||||
|
end
|
||||||
|
|
||||||
|
def self.fetch_image_or_nil(url)
|
||||||
|
return nil if url.blank?
|
||||||
|
|
||||||
|
response = HttpFetcher.fetch(url)
|
||||||
|
return nil unless allowed_image_content_type?(response.content_type)
|
||||||
|
|
||||||
|
response.body
|
||||||
|
rescue HttpFetcher::FetchTimeout
|
||||||
|
raise
|
||||||
|
rescue HttpFetcher::FetchFailed
|
||||||
|
nil
|
||||||
|
end
|
||||||
|
|
||||||
|
def self.fetch_image!(url)
|
||||||
|
response = HttpFetcher.fetch(url)
|
||||||
|
unless allowed_image_content_type?(response.content_type)
|
||||||
|
raise GenerationFailed, 'サムネール画像が見つかりませんでした.'
|
||||||
|
end
|
||||||
|
|
||||||
|
response.body
|
||||||
|
rescue HttpFetcher::FetchTimeout
|
||||||
|
raise
|
||||||
|
rescue HttpFetcher::ResponseTooLarge
|
||||||
|
raise
|
||||||
|
rescue HttpFetcher::FetchFailed
|
||||||
|
raise GenerationFailed, 'サムネール画像を取得できませんでした.'
|
||||||
|
end
|
||||||
|
|
||||||
|
def self.niconico_thumbnail_url(uri)
|
||||||
|
video_id = KnownSiteExtractor.niconico_video_id(uri)
|
||||||
|
return nil if video_id.blank?
|
||||||
|
|
||||||
|
response = HttpFetcher.fetch("https://ext.nicovideo.jp/api/getthumbinfo/#{ video_id }",
|
||||||
|
max_bytes: NICONICO_XML_MAX_BYTES)
|
||||||
|
xml = Nokogiri::XML(response.body)
|
||||||
|
return nil unless xml.at_xpath('/nicovideo_thumb_response/@status')&.value == 'ok'
|
||||||
|
|
||||||
|
xml.at_xpath('//thumbnail_url')&.text&.strip.presence
|
||||||
|
rescue HttpFetcher::FetchFailed, HttpFetcher::FetchTimeout => e
|
||||||
|
Rails.logger.info("preview_niconico_getthumbinfo_fallback #{ { url: uri.to_s,
|
||||||
|
video_id:,
|
||||||
|
error: e.class.name,
|
||||||
|
message: e.message }.to_json }")
|
||||||
|
nil
|
||||||
|
rescue Nokogiri::XML::SyntaxError => e
|
||||||
|
Rails.logger.info("preview_niconico_getthumbinfo_fallback #{ { url: uri.to_s,
|
||||||
|
video_id:,
|
||||||
|
error: e.class.name,
|
||||||
|
message: e.message }.to_json }")
|
||||||
|
nil
|
||||||
|
end
|
||||||
|
|
||||||
|
def self.allowed_image_content_type?(content_type)
|
||||||
|
mime_type = content_type.to_s.split(';', 2).first.downcase.strip
|
||||||
|
ALLOWED_IMAGE_CONTENT_TYPES.include?(mime_type)
|
||||||
|
end
|
||||||
|
|
||||||
|
private_class_method :fetch_image_or_nil, :fetch_image!,
|
||||||
|
:niconico_thumbnail_url,
|
||||||
|
:allowed_image_content_type?
|
||||||
|
end
|
||||||
|
end
|
||||||
@@ -0,0 +1,55 @@
|
|||||||
|
require 'resolv'
|
||||||
|
require 'ipaddr'
|
||||||
|
require 'uri'
|
||||||
|
|
||||||
|
module Preview
|
||||||
|
class UrlSafety
|
||||||
|
class UnsafeUrl < StandardError; end
|
||||||
|
|
||||||
|
FORBIDDEN_NETWORKS = [
|
||||||
|
'0.0.0.0/8', '10.0.0.0/8', '100.64.0.0/10', '127.0.0.0/8',
|
||||||
|
'169.254.0.0/16', '172.16.0.0/12', '192.0.0.0/24',
|
||||||
|
'192.0.2.0/24', '192.168.0.0/16', '198.18.0.0/15',
|
||||||
|
'198.51.100.0/24', '203.0.113.0/24', '224.0.0.0/4',
|
||||||
|
'240.0.0.0/4', '::/128', '::1/128', 'fc00::/7', 'fe80::/10',
|
||||||
|
'ff00::/8', '2001:db8::/32', '::ffff:0:0/96'
|
||||||
|
].map { IPAddr.new(_1) }.freeze
|
||||||
|
|
||||||
|
def self.validate(raw_url)
|
||||||
|
value = raw_url.to_s.strip
|
||||||
|
if value.match?(/\A[a-z][a-z0-9+\-.]*:/i)
|
||||||
|
unless value.match?(/\Ahttps?:\/\//i)
|
||||||
|
raise UnsafeUrl, 'http または https の URL を指定してください.'
|
||||||
|
end
|
||||||
|
else
|
||||||
|
value = "http://#{ value }"
|
||||||
|
end
|
||||||
|
uri = URI.parse(value)
|
||||||
|
|
||||||
|
unless ['http', 'https'].include?(uri.scheme&.downcase) && uri.host.present?
|
||||||
|
raise UnsafeUrl, 'http または https の URL を指定してください.'
|
||||||
|
end
|
||||||
|
raise UnsafeUrl, 'userinfo つき URL は使用できません.' if uri.userinfo.present?
|
||||||
|
|
||||||
|
addresses = Resolv.getaddresses(uri.host)
|
||||||
|
raise UnsafeUrl, 'URL のホストを解決できません.' if addresses.empty?
|
||||||
|
|
||||||
|
parsed_addresses = addresses.map { IPAddr.new(_1) }
|
||||||
|
if parsed_addresses.any? { |address| forbidden?(address) }
|
||||||
|
raise UnsafeUrl, '安全でない接続先は使用できません.'
|
||||||
|
end
|
||||||
|
|
||||||
|
[uri, parsed_addresses.map(&:to_s)]
|
||||||
|
rescue Resolv::ResolvError
|
||||||
|
raise UnsafeUrl, 'URL のホストを解決できません.'
|
||||||
|
rescue URI::InvalidURIError, IPAddr::InvalidAddressError
|
||||||
|
raise UnsafeUrl, 'URL が不正です.'
|
||||||
|
end
|
||||||
|
|
||||||
|
def self.forbidden?(address)
|
||||||
|
FORBIDDEN_NETWORKS.any? { _1.include?(address) }
|
||||||
|
end
|
||||||
|
|
||||||
|
private_class_method :forbidden?
|
||||||
|
end
|
||||||
|
end
|
||||||
@@ -33,6 +33,12 @@ Rails.application.routes.draw do
|
|||||||
get :thumbnail
|
get :thumbnail
|
||||||
end
|
end
|
||||||
|
|
||||||
|
scope 'posts/import', controller: :post_imports do
|
||||||
|
post :preview
|
||||||
|
post :validate
|
||||||
|
post '', action: :create
|
||||||
|
end
|
||||||
|
|
||||||
resources :wiki_pages, path: 'wiki', only: [:index, :show, :create, :update] do
|
resources :wiki_pages, path: 'wiki', only: [:index, :show, :create, :update] do
|
||||||
collection do
|
collection do
|
||||||
get :search
|
get :search
|
||||||
|
|||||||
@@ -0,0 +1,61 @@
|
|||||||
|
class CreatePostUrlSanitisationRules < ActiveRecord::Migration[8.0]
|
||||||
|
class PostUrlSanitisationRule < ActiveRecord::Base
|
||||||
|
self.table_name = 'post_url_sanitisation_rules'
|
||||||
|
end
|
||||||
|
|
||||||
|
def up
|
||||||
|
create_table :post_url_sanitisation_rules, id: :integer, primary_key: :priority do |t|
|
||||||
|
t.string :source_pattern, null: false
|
||||||
|
t.string :replacement, null: false
|
||||||
|
t.timestamps
|
||||||
|
t.datetime :discarded_at
|
||||||
|
|
||||||
|
t.index :source_pattern, unique: true
|
||||||
|
t.index :discarded_at
|
||||||
|
end
|
||||||
|
|
||||||
|
now = Time.current
|
||||||
|
|
||||||
|
PostUrlSanitisationRule.insert_all!([
|
||||||
|
{ priority: 10,
|
||||||
|
source_pattern: '\Ahttps?://youtu\.be/([^/?#]+)(?:[?#].*)?\z',
|
||||||
|
replacement: 'https://www.youtube.com/watch?v=\1',
|
||||||
|
created_at: now,
|
||||||
|
updated_at: now },
|
||||||
|
{ priority: 20,
|
||||||
|
source_pattern: '\Ahttps?://(?:www\.|m\.)?youtube\.com/live/([^/?#]+)(?:[?#].*)?\z',
|
||||||
|
replacement: 'https://www.youtube.com/watch?v=\1',
|
||||||
|
created_at: now,
|
||||||
|
updated_at: now },
|
||||||
|
{ priority: 30,
|
||||||
|
source_pattern: '\Ahttps?://(?:www\.|m\.)?youtube\.com/shorts/([^/?#]+)(?:[?#].*)?\z',
|
||||||
|
replacement: 'https://www.youtube.com/watch?v=\1',
|
||||||
|
created_at: now,
|
||||||
|
updated_at: now },
|
||||||
|
{ priority: 40,
|
||||||
|
source_pattern: '\Ahttps?://(?:www\.|m\.)?youtube\.com/embed/([^/?#]+)(?:[?#].*)?\z',
|
||||||
|
replacement: 'https://www.youtube.com/watch?v=\1',
|
||||||
|
created_at: now,
|
||||||
|
updated_at: now },
|
||||||
|
{ priority: 50,
|
||||||
|
source_pattern:
|
||||||
|
'\Ahttps?://(?:www\.|m\.)?youtube\.com/watch\?(?:[^#&]+&)*v=([^&#]+)(?:[&#].*)?\z',
|
||||||
|
replacement: 'https://www.youtube.com/watch?v=\1',
|
||||||
|
created_at: now,
|
||||||
|
updated_at: now },
|
||||||
|
{ priority: 60,
|
||||||
|
source_pattern: '\Ahttps?://nico\.ms/([^/?#]+)(?:[?#].*)?\z',
|
||||||
|
replacement: 'https://www.nicovideo.jp/watch/\1',
|
||||||
|
created_at: now,
|
||||||
|
updated_at: now },
|
||||||
|
{ priority: 70,
|
||||||
|
source_pattern: '\Ahttps?://(?:www\.)?nicovideo\.jp/watch/([^?#/]+)(?:[?#].*)?\z',
|
||||||
|
replacement: 'https://www.nicovideo.jp/watch/\1',
|
||||||
|
created_at: now,
|
||||||
|
updated_at: now }])
|
||||||
|
end
|
||||||
|
|
||||||
|
def down
|
||||||
|
drop_table :post_url_sanitisation_rules
|
||||||
|
end
|
||||||
|
end
|
||||||
生成ファイル
+11
-1
@@ -10,7 +10,7 @@
|
|||||||
#
|
#
|
||||||
# It's strongly recommended that you check this file into your version control system.
|
# It's strongly recommended that you check this file into your version control system.
|
||||||
|
|
||||||
ActiveRecord::Schema[8.0].define(version: 2026_07_05_000000) do
|
ActiveRecord::Schema[8.0].define(version: 2026_07_13_000000) do
|
||||||
create_table "active_storage_attachments", charset: "utf8mb4", collation: "utf8mb4_0900_ai_ci", force: :cascade do |t|
|
create_table "active_storage_attachments", charset: "utf8mb4", collation: "utf8mb4_0900_ai_ci", force: :cascade do |t|
|
||||||
t.string "name", null: false
|
t.string "name", null: false
|
||||||
t.string "record_type", null: false
|
t.string "record_type", null: false
|
||||||
@@ -331,6 +331,16 @@ ActiveRecord::Schema[8.0].define(version: 2026_07_05_000000) do
|
|||||||
t.index ["tag_id"], name: "index_post_tags_on_tag_id"
|
t.index ["tag_id"], name: "index_post_tags_on_tag_id"
|
||||||
end
|
end
|
||||||
|
|
||||||
|
create_table "post_url_sanitisation_rules", primary_key: "priority", id: :integer, charset: "utf8mb4", collation: "utf8mb4_0900_ai_ci", force: :cascade do |t|
|
||||||
|
t.string "source_pattern", null: false
|
||||||
|
t.string "replacement", null: false
|
||||||
|
t.datetime "created_at", null: false
|
||||||
|
t.datetime "updated_at", null: false
|
||||||
|
t.datetime "discarded_at"
|
||||||
|
t.index ["discarded_at"], name: "index_post_url_sanitisation_rules_on_discarded_at"
|
||||||
|
t.index ["source_pattern"], name: "index_post_url_sanitisation_rules_on_source_pattern", unique: true
|
||||||
|
end
|
||||||
|
|
||||||
create_table "post_versions", charset: "utf8mb4", collation: "utf8mb4_0900_ai_ci", force: :cascade do |t|
|
create_table "post_versions", charset: "utf8mb4", collation: "utf8mb4_0900_ai_ci", force: :cascade do |t|
|
||||||
t.bigint "post_id", null: false
|
t.bigint "post_id", null: false
|
||||||
t.integer "version_no", null: false
|
t.integer "version_no", null: false
|
||||||
|
|||||||
@@ -8,6 +8,43 @@
|
|||||||
# MovieGenre.find_or_create_by!(name: genre_name)
|
# MovieGenre.find_or_create_by!(name: genre_name)
|
||||||
# end
|
# end
|
||||||
|
|
||||||
|
post_url_sanitisation_rules = [
|
||||||
|
{ priority: 10,
|
||||||
|
source_pattern: '\Ahttps?://youtu\.be/([^/?#]+)(?:[?#].*)?\z',
|
||||||
|
replacement: 'https://www.youtube.com/watch?v=\1' },
|
||||||
|
{ priority: 20,
|
||||||
|
source_pattern: '\Ahttps?://(?:www\.|m\.)?youtube\.com/live/([^/?#]+)(?:[?#].*)?\z',
|
||||||
|
replacement: 'https://www.youtube.com/watch?v=\1' },
|
||||||
|
{ priority: 30,
|
||||||
|
source_pattern: '\Ahttps?://(?:www\.|m\.)?youtube\.com/shorts/([^/?#]+)(?:[?#].*)?\z',
|
||||||
|
replacement: 'https://www.youtube.com/watch?v=\1' },
|
||||||
|
{ priority: 40,
|
||||||
|
source_pattern: '\Ahttps?://(?:www\.|m\.)?youtube\.com/embed/([^/?#]+)(?:[?#].*)?\z',
|
||||||
|
replacement: 'https://www.youtube.com/watch?v=\1' },
|
||||||
|
{ priority: 50,
|
||||||
|
source_pattern:
|
||||||
|
'\Ahttps?://(?:www\.|m\.)?youtube\.com/watch\?(?:[^#&]+&)*v=([^&#]+)(?:[&#].*)?\z',
|
||||||
|
replacement: 'https://www.youtube.com/watch?v=\1' },
|
||||||
|
{ priority: 60,
|
||||||
|
source_pattern: '\Ahttps?://nico\.ms/([^/?#]+)(?:[?#].*)?\z',
|
||||||
|
replacement: 'https://www.nicovideo.jp/watch/\1' },
|
||||||
|
{ priority: 70,
|
||||||
|
source_pattern: '\Ahttps?://(?:www\.)?nicovideo\.jp/watch/([^?#/]+)(?:[?#].*)?\z',
|
||||||
|
replacement: 'https://www.nicovideo.jp/watch/\1' }
|
||||||
|
]
|
||||||
|
|
||||||
|
post_url_sanitisation_rule_scope = PostUrlSanitisationRule.unscoped
|
||||||
|
|
||||||
|
post_url_sanitisation_rules.each do |attributes|
|
||||||
|
priority = attributes.fetch(:priority)
|
||||||
|
source_pattern = attributes.fetch(:source_pattern)
|
||||||
|
|
||||||
|
next if post_url_sanitisation_rule_scope.exists?(priority:)
|
||||||
|
next if post_url_sanitisation_rule_scope.exists?(source_pattern:)
|
||||||
|
|
||||||
|
post_url_sanitisation_rule_scope.create!(attributes)
|
||||||
|
end
|
||||||
|
|
||||||
material_sync_source_uri = ENV['MATERIAL_SYNC_SOURCE_URI']
|
material_sync_source_uri = ENV['MATERIAL_SYNC_SOURCE_URI']
|
||||||
material_sync_source_file_id = ENV['MATERIAL_SYNC_SOURCE_FILE_ID']
|
material_sync_source_file_id = ENV['MATERIAL_SYNC_SOURCE_FILE_ID']
|
||||||
|
|
||||||
|
|||||||
-1102
ファイル差分が大きすぎるため省略します
差分を読込み
@@ -1,15 +0,0 @@
|
|||||||
{
|
|
||||||
"name": "lib",
|
|
||||||
"version": "1.0.0",
|
|
||||||
"main": "screenshot.js",
|
|
||||||
"scripts": {
|
|
||||||
"test": "echo \"Error: no test specified\" && exit 1"
|
|
||||||
},
|
|
||||||
"keywords": [],
|
|
||||||
"author": "",
|
|
||||||
"license": "ISC",
|
|
||||||
"description": "",
|
|
||||||
"dependencies": {
|
|
||||||
"puppeteer": "^24.10.0"
|
|
||||||
}
|
|
||||||
}
|
|
||||||
@@ -1,18 +0,0 @@
|
|||||||
const puppeteer = require ('puppeteer')
|
|
||||||
const fs = require ('fs')
|
|
||||||
|
|
||||||
|
|
||||||
void (async () => {
|
|
||||||
const url = process.argv[2]
|
|
||||||
const output = process.argv[3]
|
|
||||||
|
|
||||||
const browser = await puppeteer.launch ({
|
|
||||||
args: ['--no-sandbox', '--disable-setuid-sandbox'] })
|
|
||||||
|
|
||||||
const page = await browser.newPage ()
|
|
||||||
await page.setViewport ({ width: 960, height: 960 })
|
|
||||||
await page.goto (url, { waitUntil: 'networkidle2', timeout: 15000 })
|
|
||||||
|
|
||||||
await page.screenshot ({ path: output })
|
|
||||||
await browser.close ()
|
|
||||||
}) ()
|
|
||||||
@@ -0,0 +1,65 @@
|
|||||||
|
require 'rails_helper'
|
||||||
|
|
||||||
|
RSpec.describe 'database seeds' do
|
||||||
|
before do
|
||||||
|
PostUrlSanitisationRule.unscoped.delete_all
|
||||||
|
end
|
||||||
|
|
||||||
|
it 'registers the initial post URL sanitisation rules' do
|
||||||
|
load_seeds
|
||||||
|
|
||||||
|
urls = {
|
||||||
|
'https://youtu.be/abc123?si=share' => youtube_url('abc123'),
|
||||||
|
'https://www.youtube.com/live/abc123?t=10' => youtube_url('abc123'),
|
||||||
|
'https://youtube.com/shorts/abc123?feature=share' => youtube_url('abc123'),
|
||||||
|
'https://m.youtube.com/embed/abc123' => youtube_url('abc123'),
|
||||||
|
'https://youtube.com/watch?feature=share&v=abc123&t=10' => youtube_url('abc123'),
|
||||||
|
'https://nico.ms/sm123?from=share#fragment' => nico_url('sm123'),
|
||||||
|
'https://www.nicovideo.jp/watch/sm123?ref=share#fragment' => nico_url('sm123')
|
||||||
|
}
|
||||||
|
|
||||||
|
expect(PostUrlSanitisationRule.count).to eq(7)
|
||||||
|
urls.each do |url, canonical_url|
|
||||||
|
expect(PostUrlSanitisationRule.sanitise(url)).to eq(canonical_url)
|
||||||
|
end
|
||||||
|
end
|
||||||
|
|
||||||
|
it 'does not overwrite or restore an existing rule' do
|
||||||
|
rule = PostUrlSanitisationRule.create!(
|
||||||
|
priority: 10,
|
||||||
|
source_pattern: '\Ahttps://example\.com/custom\z',
|
||||||
|
replacement: 'https://example.com/replacement'
|
||||||
|
)
|
||||||
|
rule.discard!
|
||||||
|
original_attributes = rule.reload.attributes
|
||||||
|
|
||||||
|
2.times { load_seeds }
|
||||||
|
|
||||||
|
persisted_rule = PostUrlSanitisationRule.unscoped.find(10)
|
||||||
|
expect(persisted_rule.attributes).to eq(original_attributes)
|
||||||
|
expect(PostUrlSanitisationRule.unscoped.count).to eq(7)
|
||||||
|
end
|
||||||
|
|
||||||
|
it 'does not duplicate a rule moved to another priority' do
|
||||||
|
source_pattern = '\Ahttps?://youtu\.be/([^/?#]+)(?:[?#].*)?\z'
|
||||||
|
PostUrlSanitisationRule.create!(
|
||||||
|
priority: 80,
|
||||||
|
source_pattern:,
|
||||||
|
replacement: 'https://example.com/custom/\1'
|
||||||
|
)
|
||||||
|
|
||||||
|
load_seeds
|
||||||
|
|
||||||
|
rules = PostUrlSanitisationRule.unscoped
|
||||||
|
expect(rules.where(source_pattern:).count).to eq(1)
|
||||||
|
expect(rules.find(80).replacement).to eq('https://example.com/custom/\1')
|
||||||
|
end
|
||||||
|
|
||||||
|
def load_seeds
|
||||||
|
load Rails.root.join('db/seeds.rb')
|
||||||
|
end
|
||||||
|
|
||||||
|
def youtube_url(video_id) = "https://www.youtube.com/watch?v=#{ video_id }"
|
||||||
|
|
||||||
|
def nico_url(video_id) = "https://www.nicovideo.jp/watch/#{ video_id }"
|
||||||
|
end
|
||||||
@@ -0,0 +1,68 @@
|
|||||||
|
require 'rails_helper'
|
||||||
|
|
||||||
|
RSpec.describe Post, type: :model do
|
||||||
|
before do
|
||||||
|
PostUrlSanitisationRule.unscoped.delete_all
|
||||||
|
end
|
||||||
|
|
||||||
|
describe 'URL normalisation' do
|
||||||
|
it 'normalises the HTTP URL before applying sanitisation rules' do
|
||||||
|
PostUrlSanitisationRule.create!(
|
||||||
|
priority: 10,
|
||||||
|
source_pattern: '\\Ahttps://example\\.com/videos/([^/]+)\\z',
|
||||||
|
replacement: 'https://example.com/watch/\\1'
|
||||||
|
)
|
||||||
|
|
||||||
|
post = described_class.create!(
|
||||||
|
title: 'normalised URL',
|
||||||
|
url: ' https://EXAMPLE.com/videos/123/ '
|
||||||
|
)
|
||||||
|
|
||||||
|
expect(post.url).to eq('https://example.com/watch/123')
|
||||||
|
end
|
||||||
|
|
||||||
|
it 'does not normalise an unchanged URL when another attribute changes' do
|
||||||
|
post = create(:post)
|
||||||
|
post.update_column(:url, 'https://EXAMPLE.com/unchanged/')
|
||||||
|
|
||||||
|
post.update!(title: 'updated title')
|
||||||
|
|
||||||
|
expect(post.reload.url).to eq('https://EXAMPLE.com/unchanged/')
|
||||||
|
end
|
||||||
|
|
||||||
|
it 'validates the sanitised URL length' do
|
||||||
|
path = 'a' * 375
|
||||||
|
|
||||||
|
PostUrlSanitisationRule.create!(
|
||||||
|
priority: 10,
|
||||||
|
source_pattern: '\\Ahttps://example\\.com/(a+)\\z',
|
||||||
|
replacement: 'https://example.com/\\1\\1'
|
||||||
|
)
|
||||||
|
|
||||||
|
post = described_class.new(
|
||||||
|
title: 'long URL',
|
||||||
|
url: "https://example.com/#{ path }"
|
||||||
|
)
|
||||||
|
|
||||||
|
expect(post).to be_invalid
|
||||||
|
expect(post.errors.details.fetch(:url)).to include(
|
||||||
|
error: :too_long,
|
||||||
|
count: 768
|
||||||
|
)
|
||||||
|
end
|
||||||
|
|
||||||
|
it 'validates uniqueness after sanitisation' do
|
||||||
|
PostUrlSanitisationRule.create!(
|
||||||
|
priority: 10,
|
||||||
|
source_pattern: '\\Ahttps://example\\.com/alias\\z',
|
||||||
|
replacement: 'https://example.com/canonical'
|
||||||
|
)
|
||||||
|
create(:post, url: 'https://example.com/canonical')
|
||||||
|
|
||||||
|
post = described_class.new(title: 'duplicate URL', url: 'https://example.com/alias')
|
||||||
|
|
||||||
|
expect(post).to be_invalid
|
||||||
|
expect(post.errors.details.fetch(:url)).to include(error: :taken, value: post.url)
|
||||||
|
end
|
||||||
|
end
|
||||||
|
end
|
||||||
@@ -0,0 +1,265 @@
|
|||||||
|
require 'rails_helper'
|
||||||
|
|
||||||
|
RSpec.describe PostUrlSanitisationRule, type: :model do
|
||||||
|
before do
|
||||||
|
described_class.unscoped.delete_all
|
||||||
|
end
|
||||||
|
|
||||||
|
describe 'validations' do
|
||||||
|
it 'requires a source pattern' do
|
||||||
|
rule = described_class.new(priority: 10, source_pattern: nil, replacement: '')
|
||||||
|
|
||||||
|
expect(rule).to be_invalid
|
||||||
|
expect(rule.errors.details.fetch(:source_pattern)).to eq([{ error: :blank }])
|
||||||
|
end
|
||||||
|
|
||||||
|
it 'requires a unique source pattern' do
|
||||||
|
described_class.create!(priority: 10, source_pattern: 'source', replacement: 'first')
|
||||||
|
rule = described_class.new(
|
||||||
|
priority: 20,
|
||||||
|
source_pattern: 'source',
|
||||||
|
replacement: 'second'
|
||||||
|
)
|
||||||
|
|
||||||
|
expect(rule).to be_invalid
|
||||||
|
expect(rule.errors.details.fetch(:source_pattern)).to include(
|
||||||
|
error: :taken,
|
||||||
|
value: 'source'
|
||||||
|
)
|
||||||
|
end
|
||||||
|
|
||||||
|
it 'rejects an invalid regexp' do
|
||||||
|
rule = described_class.new(priority: 10, source_pattern: '[', replacement: '')
|
||||||
|
|
||||||
|
expect(rule).to be_invalid
|
||||||
|
expect(rule.errors[:source_pattern]).to include('変な正規表現だね〜(笑)')
|
||||||
|
end
|
||||||
|
end
|
||||||
|
|
||||||
|
describe '.sanitise' do
|
||||||
|
it 'applies each active rule once in priority order' do
|
||||||
|
described_class.create!(priority: 30, source_pattern: 'c', replacement: 'd')
|
||||||
|
described_class.create!(priority: 10, source_pattern: 'a', replacement: 'aa')
|
||||||
|
described_class.create!(priority: 20, source_pattern: 'aa', replacement: 'c')
|
||||||
|
discarded = described_class.create!(
|
||||||
|
priority: 5,
|
||||||
|
source_pattern: '.',
|
||||||
|
replacement: 'discarded'
|
||||||
|
)
|
||||||
|
discarded.discard!
|
||||||
|
|
||||||
|
expect(described_class.sanitise('a')).to eq('d')
|
||||||
|
end
|
||||||
|
|
||||||
|
it 'canonicalises the initial YouTube and Nico URL forms' do
|
||||||
|
create_initial_rules
|
||||||
|
|
||||||
|
urls = {
|
||||||
|
'https://youtu.be/abc123?si=share' => youtube_url('abc123'),
|
||||||
|
'https://www.youtube.com/live/abc123?t=10' => youtube_url('abc123'),
|
||||||
|
'https://youtube.com/shorts/abc123?feature=share' => youtube_url('abc123'),
|
||||||
|
'https://m.youtube.com/embed/abc123' => youtube_url('abc123'),
|
||||||
|
'https://youtube.com/watch?feature=share&v=abc123&t=10' => youtube_url('abc123'),
|
||||||
|
'https://nico.ms/sm123?from=share#fragment' => nico_url('sm123'),
|
||||||
|
'https://www.nicovideo.jp/watch/sm123?ref=share#fragment' => nico_url('sm123')
|
||||||
|
}
|
||||||
|
|
||||||
|
urls.each do |url, canonical_url|
|
||||||
|
expect(described_class.sanitise(url)).to eq(canonical_url)
|
||||||
|
end
|
||||||
|
end
|
||||||
|
end
|
||||||
|
|
||||||
|
describe '.apply!' do
|
||||||
|
it 'locks posts in ID order and updates through temporary URLs' do
|
||||||
|
first = create(:post, url: 'https://example.com/source/1')
|
||||||
|
second = create(:post, url: 'https://example.com/source/2')
|
||||||
|
create_source_rule
|
||||||
|
sql = capture_sql { described_class.apply! }
|
||||||
|
|
||||||
|
lock_sql = sql.find { _1.match?(/SELECT .*posts.*FOR UPDATE/i) }
|
||||||
|
expect(lock_sql).to match(/ORDER BY .*posts.*id.* ASC/i)
|
||||||
|
expect(sql.grep(/post-url-sanitising\.invalid/).size).to eq(2)
|
||||||
|
expect(first.reload.url).to eq('https://example.com/canonical/1')
|
||||||
|
expect(second.reload.url).to eq('https://example.com/canonical/2')
|
||||||
|
end
|
||||||
|
|
||||||
|
it 'loads and compiles rules only once' do
|
||||||
|
create(:post, url: 'https://example.com/source/1')
|
||||||
|
create(:post, url: 'https://example.com/source/2')
|
||||||
|
create_source_rule
|
||||||
|
|
||||||
|
expect(described_class).to receive(:rules).once.and_call_original
|
||||||
|
|
||||||
|
described_class.apply!
|
||||||
|
end
|
||||||
|
|
||||||
|
it 'does not change post versions, version_no, or updated_at' do
|
||||||
|
post = create(:post, url: 'https://example.com/source/1')
|
||||||
|
post.update_columns(version_no: 7, updated_at: 1.day.ago)
|
||||||
|
original_updated_at = post.reload.updated_at
|
||||||
|
create_source_rule
|
||||||
|
|
||||||
|
expect { described_class.apply! }.not_to change(PostVersion, :count)
|
||||||
|
|
||||||
|
post.reload
|
||||||
|
expect(post.url).to eq('https://example.com/canonical/1')
|
||||||
|
expect(post.version_no).to eq(7)
|
||||||
|
expect(post.updated_at).to eq(original_updated_at)
|
||||||
|
end
|
||||||
|
|
||||||
|
invalid_urls = {
|
||||||
|
'a blank URL' => '',
|
||||||
|
'an unparseable URL' => 'https://[',
|
||||||
|
'a non-HTTP URL' => 'ftp://example.com/file',
|
||||||
|
'an HTTP URL without a host' => 'https:/path'
|
||||||
|
}
|
||||||
|
|
||||||
|
invalid_urls.each do |description, sanitised_url|
|
||||||
|
it "rolls back every update when sanitisation produces #{ description }" do
|
||||||
|
valid_post = create(:post, url: 'https://example.com/source/1')
|
||||||
|
invalid_post = create(:post, url: 'https://example.com/invalid')
|
||||||
|
create_source_rule
|
||||||
|
described_class.create!(
|
||||||
|
priority: 20,
|
||||||
|
source_pattern: '\\Ahttps://example\\.com/invalid\\z',
|
||||||
|
replacement: sanitised_url
|
||||||
|
)
|
||||||
|
|
||||||
|
expect { described_class.apply! }
|
||||||
|
.to raise_error(described_class::InvalidUrlError) { |error|
|
||||||
|
expect(error.invalid_rows).to eq([
|
||||||
|
{ post_id: invalid_post.id,
|
||||||
|
original_url: 'https://example.com/invalid',
|
||||||
|
sanitised_url: }
|
||||||
|
])
|
||||||
|
}
|
||||||
|
expect(valid_post.reload.url).to eq('https://example.com/source/1')
|
||||||
|
expect(invalid_post.reload.url).to eq('https://example.com/invalid')
|
||||||
|
end
|
||||||
|
end
|
||||||
|
|
||||||
|
it 'rolls back every update when sanitisation produces a URL longer than 768 characters' do
|
||||||
|
path = 'a' * 375
|
||||||
|
original_url = "https://example.com/#{ path }"
|
||||||
|
sanitised_url = "https://example.com/#{ path }#{ path }"
|
||||||
|
valid_post = create(:post, url: 'https://example.com/source/1')
|
||||||
|
invalid_post = create(:post, url: original_url)
|
||||||
|
create_source_rule
|
||||||
|
described_class.create!(
|
||||||
|
priority: 20,
|
||||||
|
source_pattern: '\\Ahttps://example\\.com/(a+)\\z',
|
||||||
|
replacement: 'https://example.com/\\1\\1'
|
||||||
|
)
|
||||||
|
|
||||||
|
expect { described_class.apply! }
|
||||||
|
.to raise_error(described_class::InvalidUrlError) { |error|
|
||||||
|
expect(error.invalid_rows).to eq([
|
||||||
|
{ post_id: invalid_post.id,
|
||||||
|
original_url:,
|
||||||
|
sanitised_url: }
|
||||||
|
])
|
||||||
|
}
|
||||||
|
expect(valid_post.reload.url).to eq('https://example.com/source/1')
|
||||||
|
expect(invalid_post.reload.url).to eq(original_url)
|
||||||
|
end
|
||||||
|
|
||||||
|
it 'rejects sanitised URL collisions case-insensitively without changing posts' do
|
||||||
|
source = create(:post, url: 'https://example.com/source/Foo')
|
||||||
|
canonical = create(:post, url: 'https://example.com/canonical/foo')
|
||||||
|
create_source_rule
|
||||||
|
|
||||||
|
expect { described_class.apply! }
|
||||||
|
.to raise_error(described_class::UrlConflictError) { |error|
|
||||||
|
expect(error.conflicts).to contain_exactly(
|
||||||
|
{ url: 'https://example.com/canonical/Foo',
|
||||||
|
post_id: source.id,
|
||||||
|
original_url: 'https://example.com/source/Foo' },
|
||||||
|
{ url: 'https://example.com/canonical/foo',
|
||||||
|
post_id: canonical.id,
|
||||||
|
original_url: 'https://example.com/canonical/foo' }
|
||||||
|
)
|
||||||
|
}
|
||||||
|
expect(source.reload.url).to eq('https://example.com/source/Foo')
|
||||||
|
expect(canonical.reload.url).to eq('https://example.com/canonical/foo')
|
||||||
|
expect(Post.count).to eq(2)
|
||||||
|
end
|
||||||
|
|
||||||
|
it 'converts a persisted URL constraint race into UrlConflictError' do
|
||||||
|
post = create(:post, url: 'https://example.com/source/1')
|
||||||
|
create_source_rule
|
||||||
|
conflict = { url: 'https://example.com/canonical/1',
|
||||||
|
post_id: post.id,
|
||||||
|
original_url: post.url }
|
||||||
|
database_error = ActiveRecord::RecordNotUnique.new('duplicate URL')
|
||||||
|
allow_any_instance_of(ActiveRecord::Relation)
|
||||||
|
.to receive(:update_all).and_raise(database_error)
|
||||||
|
allow(described_class).to receive(:build_persisted_conflicts).and_return([conflict])
|
||||||
|
|
||||||
|
expect { described_class.apply! }
|
||||||
|
.to raise_error(described_class::UrlConflictError) { |error|
|
||||||
|
expect(error.conflicts).to eq([conflict])
|
||||||
|
expect(error.cause).to equal(database_error)
|
||||||
|
}
|
||||||
|
expect(post.reload.url).to eq('https://example.com/source/1')
|
||||||
|
end
|
||||||
|
|
||||||
|
it 're-raises an unidentified RecordNotUnique error' do
|
||||||
|
post = create(:post, url: 'https://example.com/source/1')
|
||||||
|
create_source_rule
|
||||||
|
database_error = ActiveRecord::RecordNotUnique.new('another unique constraint')
|
||||||
|
allow_any_instance_of(ActiveRecord::Relation)
|
||||||
|
.to receive(:update_all).and_raise(database_error)
|
||||||
|
allow(described_class).to receive(:build_persisted_conflicts).and_return([])
|
||||||
|
|
||||||
|
expect { described_class.apply! }.to raise_error(database_error)
|
||||||
|
expect(post.reload.url).to eq('https://example.com/source/1')
|
||||||
|
end
|
||||||
|
end
|
||||||
|
|
||||||
|
def capture_sql
|
||||||
|
statements = []
|
||||||
|
subscriber = lambda do |_name, _start, _finish, _id, payload|
|
||||||
|
binds = payload.fetch(:binds, []).map { _1.value_for_database.to_s }
|
||||||
|
statements << ([payload.fetch(:sql)] + binds).join(' ')
|
||||||
|
end
|
||||||
|
ActiveSupport::Notifications.subscribed(subscriber, 'sql.active_record') { yield }
|
||||||
|
statements
|
||||||
|
end
|
||||||
|
|
||||||
|
def create_source_rule
|
||||||
|
described_class.create!(
|
||||||
|
priority: 10,
|
||||||
|
source_pattern: '\\Ahttps://example\\.com/source/(.+)\\z',
|
||||||
|
replacement: 'https://example.com/canonical/\\1'
|
||||||
|
)
|
||||||
|
end
|
||||||
|
|
||||||
|
def create_initial_rules
|
||||||
|
rules = [
|
||||||
|
['\\Ahttps?://youtu\\.be/([^/?#]+)(?:[?#].*)?\\z', youtube_url('\\1')],
|
||||||
|
['\\Ahttps?://(?:www\\.|m\\.)?youtube\\.com/live/([^/?#]+)(?:[?#].*)?\\z',
|
||||||
|
youtube_url('\\1')],
|
||||||
|
['\\Ahttps?://(?:www\\.|m\\.)?youtube\\.com/shorts/([^/?#]+)(?:[?#].*)?\\z',
|
||||||
|
youtube_url('\\1')],
|
||||||
|
['\\Ahttps?://(?:www\\.|m\\.)?youtube\\.com/embed/([^/?#]+)(?:[?#].*)?\\z',
|
||||||
|
youtube_url('\\1')],
|
||||||
|
['\\Ahttps?://(?:www\\.|m\\.)?youtube\\.com/watch\\?(?:[^#&]+&)*' \
|
||||||
|
'v=([^&#]+)(?:[&#].*)?\\z', youtube_url('\\1')],
|
||||||
|
['\\Ahttps?://nico\\.ms/([^/?#]+)(?:[?#].*)?\\z', nico_url('\\1')],
|
||||||
|
['\\Ahttps?://(?:www\\.)?nicovideo\\.jp/watch/([^?#/]+)(?:[?#].*)?\\z',
|
||||||
|
nico_url('\\1')]
|
||||||
|
]
|
||||||
|
rules.each_with_index do |(source_pattern, replacement), index|
|
||||||
|
described_class.create!(
|
||||||
|
priority: (index + 1) * 10,
|
||||||
|
source_pattern:,
|
||||||
|
replacement:
|
||||||
|
)
|
||||||
|
end
|
||||||
|
end
|
||||||
|
|
||||||
|
def youtube_url(video_id) = "https://www.youtube.com/watch?v=#{ video_id }"
|
||||||
|
|
||||||
|
def nico_url(video_id) = "https://www.nicovideo.jp/watch/#{ video_id }"
|
||||||
|
end
|
||||||
@@ -287,6 +287,25 @@ RSpec.describe 'Materials API', type: :request do
|
|||||||
expect(json.dig('export_paths', 'legacy_drive')).to eq('伊地知ニジカ/created.png')
|
expect(json.dig('export_paths', 'legacy_drive')).to eq('伊地知ニジカ/created.png')
|
||||||
end
|
end
|
||||||
|
|
||||||
|
it 'creates a create tag_version for a newly created material tag' do
|
||||||
|
expect do
|
||||||
|
post '/materials', params: {
|
||||||
|
tag: 'material_create_versioned_tag',
|
||||||
|
file: dummy_upload(filename: 'created.png')
|
||||||
|
}
|
||||||
|
end.to change(TagVersion, :count).by(1)
|
||||||
|
|
||||||
|
expect(response).to have_http_status(:created)
|
||||||
|
|
||||||
|
tag = Tag.joins(:tag_name).find_by!(tag_names: { name: 'material_create_versioned_tag' })
|
||||||
|
version = tag.tag_versions.order(:version_no).last
|
||||||
|
|
||||||
|
expect(version.event_type).to eq('create')
|
||||||
|
expect(version.name).to eq('material_create_versioned_tag')
|
||||||
|
expect(version.category).to eq('material')
|
||||||
|
expect(version.created_by_user).to eq(member_user)
|
||||||
|
end
|
||||||
|
|
||||||
it 'snapshots attached file metadata and sha256' do
|
it 'snapshots attached file metadata and sha256' do
|
||||||
post '/materials', params: {
|
post '/materials', params: {
|
||||||
tag: 'material_create_file_version',
|
tag: 'material_create_file_version',
|
||||||
@@ -466,6 +485,73 @@ RSpec.describe 'Materials API', type: :request do
|
|||||||
expect(json.dig('tag', 'name')).to eq('material_update_new')
|
expect(json.dig('tag', 'name')).to eq('material_update_new')
|
||||||
end
|
end
|
||||||
|
|
||||||
|
it 'creates a create tag_version when update creates a new tag' do
|
||||||
|
expect do
|
||||||
|
put "/materials/#{ material.id }", params: {
|
||||||
|
tag: 'material_update_versioned_tag',
|
||||||
|
file: dummy_upload(filename: 'updated.png')
|
||||||
|
}
|
||||||
|
end.to change(Tag, :count).by(1)
|
||||||
|
.and change(TagName, :count).by(1)
|
||||||
|
.and change(TagVersion, :count).by(1)
|
||||||
|
|
||||||
|
expect(response).to have_http_status(:ok)
|
||||||
|
|
||||||
|
tag = Tag.joins(:tag_name).find_by!(tag_names: { name: 'material_update_versioned_tag' })
|
||||||
|
version = tag.tag_versions.order(:version_no).last
|
||||||
|
|
||||||
|
expect(version.event_type).to eq('create')
|
||||||
|
expect(version.name).to eq('material_update_versioned_tag')
|
||||||
|
expect(version.category).to eq('material')
|
||||||
|
expect(version.created_by_user).to eq(member_user)
|
||||||
|
end
|
||||||
|
|
||||||
|
it 'backfills a create tag_version for an existing material tag without history' do
|
||||||
|
existing_tag =
|
||||||
|
Tag.create!(tag_name: TagName.create!(name: 'material_update_existing_no_history'),
|
||||||
|
category: :material)
|
||||||
|
|
||||||
|
expect(existing_tag.tag_versions).to be_empty
|
||||||
|
|
||||||
|
expect do
|
||||||
|
put "/materials/#{ material.id }", params: {
|
||||||
|
tag: 'material_update_existing_no_history',
|
||||||
|
file: dummy_upload(filename: 'updated.png')
|
||||||
|
}
|
||||||
|
end.to change(TagVersion, :count).by(1)
|
||||||
|
|
||||||
|
expect(response).to have_http_status(:ok)
|
||||||
|
|
||||||
|
version = existing_tag.reload.tag_versions.order(:version_no).last
|
||||||
|
expect(version.event_type).to eq('create')
|
||||||
|
expect(version.name).to eq('material_update_existing_no_history')
|
||||||
|
expect(version.category).to eq('material')
|
||||||
|
expect(version.created_by_user).to eq(member_user)
|
||||||
|
end
|
||||||
|
|
||||||
|
it 'backfills a create tag_version for an existing character tag without history' do
|
||||||
|
existing_tag =
|
||||||
|
Tag.create!(tag_name: TagName.create!(name: 'material_update_character_no_history'),
|
||||||
|
category: :character)
|
||||||
|
|
||||||
|
expect(existing_tag.tag_versions).to be_empty
|
||||||
|
|
||||||
|
expect do
|
||||||
|
put "/materials/#{ material.id }", params: {
|
||||||
|
tag: 'material_update_character_no_history',
|
||||||
|
file: dummy_upload(filename: 'updated.png')
|
||||||
|
}
|
||||||
|
end.to change(TagVersion, :count).by(1)
|
||||||
|
|
||||||
|
expect(response).to have_http_status(:ok)
|
||||||
|
|
||||||
|
version = existing_tag.reload.tag_versions.order(:version_no).last
|
||||||
|
expect(version.event_type).to eq('create')
|
||||||
|
expect(version.name).to eq('material_update_character_no_history')
|
||||||
|
expect(version.category).to eq('character')
|
||||||
|
expect(version.created_by_user).to eq(member_user)
|
||||||
|
end
|
||||||
|
|
||||||
it 'detaches the existing file without purging blob when url replaces file' do
|
it 'detaches the existing file without purging blob when url replaces file' do
|
||||||
old_blob_id = material.file.blob.id
|
old_blob_id = material.file.blob.id
|
||||||
|
|
||||||
@@ -494,6 +580,7 @@ RSpec.describe 'Materials API', type: :request do
|
|||||||
it 'does not increase version for the same snapshot update' do
|
it 'does not increase version for the same snapshot update' do
|
||||||
MaterialVersionRecorder.record!(material:, event_type: :create,
|
MaterialVersionRecorder.record!(material:, event_type: :create,
|
||||||
created_by_user: member_user)
|
created_by_user: member_user)
|
||||||
|
TagVersioning.ensure_snapshot!(tag, created_by_user: member_user)
|
||||||
|
|
||||||
expect do
|
expect do
|
||||||
put "/materials/#{ material.id }", params: {
|
put "/materials/#{ material.id }", params: {
|
||||||
@@ -501,6 +588,8 @@ RSpec.describe 'Materials API', type: :request do
|
|||||||
}
|
}
|
||||||
end.not_to change(MaterialVersion, :count)
|
end.not_to change(MaterialVersion, :count)
|
||||||
|
|
||||||
|
expect(tag.reload.tag_versions.count).to eq(1)
|
||||||
|
|
||||||
expect(response).to have_http_status(:ok)
|
expect(response).to have_http_status(:ok)
|
||||||
expect(material.reload.version_no).to eq(1)
|
expect(material.reload.version_no).to eq(1)
|
||||||
end
|
end
|
||||||
|
|||||||
@@ -1,28 +1,46 @@
|
|||||||
require "rails_helper"
|
require 'rails_helper'
|
||||||
|
|
||||||
|
|
||||||
RSpec.describe "Preview", type: :request do
|
RSpec.describe 'Preview', type: :request do
|
||||||
describe "GET /preview/title" do
|
describe 'GET /preview/title' do
|
||||||
it "401 unless logged in" do
|
it '401 unless logged in' do
|
||||||
sign_out
|
sign_out
|
||||||
get "/preview/title", params: { url: "example.com" }
|
get '/preview/title', params: { url: 'example.com' }
|
||||||
expect(response).to have_http_status(:unauthorized)
|
expect(response).to have_http_status(:unauthorized)
|
||||||
end
|
end
|
||||||
|
|
||||||
it "400 when url blank" do
|
it '403 when logged in as guest' do
|
||||||
sign_in_as(create(:user))
|
sign_in_as(create(:user, :guest))
|
||||||
get "/preview/title", params: { url: "" }
|
get '/preview/title', params: { url: 'example.com' }
|
||||||
|
expect(response).to have_http_status(:forbidden)
|
||||||
|
end
|
||||||
|
|
||||||
|
it '400 when url blank' do
|
||||||
|
sign_in_as(create(:user, :member))
|
||||||
|
get '/preview/title', params: { url: '' }
|
||||||
expect(response).to have_http_status(:bad_request)
|
expect(response).to have_http_status(:bad_request)
|
||||||
end
|
end
|
||||||
|
|
||||||
it "returns parsed title (stubbing URI.open)" do
|
it 'returns parsed title' do
|
||||||
sign_in_as(create(:user))
|
sign_in_as(create(:user, :member))
|
||||||
fake_html = "<html><head><title> Hello </title></head></html>"
|
allow(Preview::ThumbnailFetcher)
|
||||||
allow(URI).to receive(:open).and_return(StringIO.new(fake_html))
|
.to receive(:title)
|
||||||
|
.with('example.com')
|
||||||
|
.and_return('Hello')
|
||||||
|
|
||||||
get "/preview/title", params: { url: "example.com" }
|
get '/preview/title', params: { url: 'example.com' }
|
||||||
expect(response).to have_http_status(:ok)
|
expect(response).to have_http_status(:ok)
|
||||||
expect(json["title"]).to eq("Hello")
|
expect(json['title']).to eq('Hello')
|
||||||
|
end
|
||||||
|
|
||||||
|
it '413 when fetched response is too large' do
|
||||||
|
sign_in_as(create(:user, :member))
|
||||||
|
allow(Preview::ThumbnailFetcher)
|
||||||
|
.to receive(:title)
|
||||||
|
.and_raise(Preview::HttpFetcher::ResponseTooLarge, '外部データが大きすぎます.')
|
||||||
|
|
||||||
|
get '/preview/title', params: { url: 'example.com' }
|
||||||
|
expect(response).to have_http_status(:payload_too_large)
|
||||||
end
|
end
|
||||||
end
|
end
|
||||||
end
|
end
|
||||||
|
|||||||
@@ -0,0 +1,23 @@
|
|||||||
|
require 'rails_helper'
|
||||||
|
|
||||||
|
RSpec.describe Preview::HttpFetcher do
|
||||||
|
describe '.fetch' do
|
||||||
|
it 'raises FetchFailed when redirect location is invalid' do
|
||||||
|
redirect = Net::HTTPFound.new('1.1', '302', 'Found')
|
||||||
|
redirect['location'] = 'http://[invalid'
|
||||||
|
|
||||||
|
allow(Preview::UrlSafety).to receive(:validate)
|
||||||
|
.with('https://example.com/page')
|
||||||
|
.and_return([URI.parse('https://example.com/page'), ['203.0.113.10']])
|
||||||
|
allow(described_class).to receive(:request)
|
||||||
|
.and_return(redirect)
|
||||||
|
allow(Rails.logger).to receive(:warn)
|
||||||
|
|
||||||
|
expect {
|
||||||
|
described_class.fetch('https://example.com/page')
|
||||||
|
}.to raise_error(Preview::HttpFetcher::FetchFailed)
|
||||||
|
expect(Rails.logger).to have_received(:warn)
|
||||||
|
.with(/invalid_redirect_location/)
|
||||||
|
end
|
||||||
|
end
|
||||||
|
end
|
||||||
@@ -0,0 +1,51 @@
|
|||||||
|
require 'rails_helper'
|
||||||
|
|
||||||
|
RSpec.describe Preview::ThumbnailFetcher do
|
||||||
|
describe '.fetch' do
|
||||||
|
it 'rejects svg thumbnails' do
|
||||||
|
page = Preview::HttpFetcher::Response.new(
|
||||||
|
'<meta property="og:image" content="https://example.com/thumb.svg">',
|
||||||
|
'text/html',
|
||||||
|
'https://example.com/page')
|
||||||
|
svg = Preview::HttpFetcher::Response.new(
|
||||||
|
'<svg></svg>',
|
||||||
|
'image/svg+xml',
|
||||||
|
'https://example.com/thumb.svg')
|
||||||
|
|
||||||
|
allow(Preview::UrlSafety).to receive(:validate)
|
||||||
|
.and_return([URI.parse('https://example.com/page'), ['203.0.113.10']])
|
||||||
|
allow(Preview::HttpFetcher).to receive(:fetch)
|
||||||
|
.with('https://example.com/page', max_bytes: described_class::HTML_MAX_BYTES)
|
||||||
|
.and_return(page)
|
||||||
|
allow(Preview::HttpFetcher).to receive(:fetch)
|
||||||
|
.with('https://example.com/thumb.svg')
|
||||||
|
.and_return(svg)
|
||||||
|
|
||||||
|
expect {
|
||||||
|
described_class.fetch('https://example.com/page')
|
||||||
|
}.to raise_error(Preview::ThumbnailFetcher::GenerationFailed)
|
||||||
|
end
|
||||||
|
|
||||||
|
it 'accepts allowed image content type with parameters' do
|
||||||
|
page = Preview::HttpFetcher::Response.new(
|
||||||
|
'<meta property="og:image" content="https://example.com/thumb.jpg">',
|
||||||
|
'text/html',
|
||||||
|
'https://example.com/page')
|
||||||
|
image = Preview::HttpFetcher::Response.new(
|
||||||
|
'jpeg-bytes',
|
||||||
|
'image/jpeg; charset=binary',
|
||||||
|
'https://example.com/thumb.jpg')
|
||||||
|
|
||||||
|
allow(Preview::UrlSafety).to receive(:validate)
|
||||||
|
.and_return([URI.parse('https://example.com/page'), ['203.0.113.10']])
|
||||||
|
allow(Preview::HttpFetcher).to receive(:fetch)
|
||||||
|
.with('https://example.com/page', max_bytes: described_class::HTML_MAX_BYTES)
|
||||||
|
.and_return(page)
|
||||||
|
allow(Preview::HttpFetcher).to receive(:fetch)
|
||||||
|
.with('https://example.com/thumb.jpg')
|
||||||
|
.and_return(image)
|
||||||
|
|
||||||
|
expect(described_class.fetch('https://example.com/page')).to eq('jpeg-bytes')
|
||||||
|
end
|
||||||
|
end
|
||||||
|
end
|
||||||
@@ -0,0 +1,15 @@
|
|||||||
|
require 'rails_helper'
|
||||||
|
|
||||||
|
RSpec.describe Preview::UrlSafety do
|
||||||
|
describe '.validate' do
|
||||||
|
it 'raises UnsafeUrl when DNS resolution fails' do
|
||||||
|
allow(Resolv).to receive(:getaddresses)
|
||||||
|
.with('missing.example')
|
||||||
|
.and_raise(Resolv::ResolvError)
|
||||||
|
|
||||||
|
expect {
|
||||||
|
described_class.validate('https://missing.example')
|
||||||
|
}.to raise_error(Preview::UrlSafety::UnsafeUrl)
|
||||||
|
end
|
||||||
|
end
|
||||||
|
end
|
||||||
@@ -62,7 +62,7 @@ RSpec.describe 'nico:export' do
|
|||||||
|
|
||||||
it 'deduplicates video ids' do
|
it 'deduplicates video ids' do
|
||||||
create_post('https://www.nicovideo.jp/watch/sm12345')
|
create_post('https://www.nicovideo.jp/watch/sm12345')
|
||||||
create_post('https://www.nicovideo.jp/watch/sm12345?from=1')
|
create_post('https://sp.nicovideo.jp/watch/sm12345')
|
||||||
|
|
||||||
expect(Open3).to receive(:capture3) do |_env, *args, **_kwargs|
|
expect(Open3).to receive(:capture3) do |_env, *args, **_kwargs|
|
||||||
expect(args.drop(3)).to eq(['sm12345'])
|
expect(args.drop(3)).to eq(['sm12345'])
|
||||||
|
|||||||
@@ -41,6 +41,9 @@ import NotFound from '@/pages/NotFound'
|
|||||||
import TOSPage from '@/pages/TOSPage.mdx'
|
import TOSPage from '@/pages/TOSPage.mdx'
|
||||||
import PostDetailPage from '@/pages/posts/PostDetailPage'
|
import PostDetailPage from '@/pages/posts/PostDetailPage'
|
||||||
import PostHistoryPage from '@/pages/posts/PostHistoryPage'
|
import PostHistoryPage from '@/pages/posts/PostHistoryPage'
|
||||||
|
import PostImportResultPage from '@/pages/posts/PostImportResultPage'
|
||||||
|
import PostImportReviewPage from '@/pages/posts/PostImportReviewPage'
|
||||||
|
import PostImportSourcePage from '@/pages/posts/PostImportSourcePage'
|
||||||
import PostListPage from '@/pages/posts/PostListPage'
|
import PostListPage from '@/pages/posts/PostListPage'
|
||||||
import PostNewPage from '@/pages/posts/PostNewPage'
|
import PostNewPage from '@/pages/posts/PostNewPage'
|
||||||
import PostSearchPage from '@/pages/posts/PostSearchPage'
|
import PostSearchPage from '@/pages/posts/PostSearchPage'
|
||||||
@@ -75,6 +78,9 @@ const RouteTransitionWrapper = ({ animationMode, user, setUser }: {
|
|||||||
<Route path="/" element={<Navigate to="/posts" replace/>}/>
|
<Route path="/" element={<Navigate to="/posts" replace/>}/>
|
||||||
<Route path="/posts" element={<PostListPage/>}/>
|
<Route path="/posts" element={<PostListPage/>}/>
|
||||||
<Route path="/posts/new" element={<PostNewPage user={user}/>}/>
|
<Route path="/posts/new" element={<PostNewPage user={user}/>}/>
|
||||||
|
<Route path="/posts/import" element={<PostImportSourcePage user={user}/>}/>
|
||||||
|
<Route path="/posts/import/:sessionId/review" element={<PostImportReviewPage user={user}/>}/>
|
||||||
|
<Route path="/posts/import/:sessionId/result" element={<PostImportResultPage user={user}/>}/>
|
||||||
<Route path="/posts/search" element={<PostSearchPage/>}/>
|
<Route path="/posts/search" element={<PostSearchPage/>}/>
|
||||||
<Route path="/posts/:id" element={<PostDetailRoute user={user}/>}/>
|
<Route path="/posts/:id" element={<PostDetailRoute user={user}/>}/>
|
||||||
<Route path="/posts/changes" element={<PostHistoryPage/>}/>
|
<Route path="/posts/changes" element={<PostHistoryPage/>}/>
|
||||||
@@ -113,6 +119,9 @@ const RouteTransitionWrapper = ({ animationMode, user, setUser }: {
|
|||||||
<Route path="/" element={<Navigate to="/posts" replace/>}/>
|
<Route path="/" element={<Navigate to="/posts" replace/>}/>
|
||||||
<Route path="/posts" element={<PostListPage/>}/>
|
<Route path="/posts" element={<PostListPage/>}/>
|
||||||
<Route path="/posts/new" element={<PostNewPage user={user}/>}/>
|
<Route path="/posts/new" element={<PostNewPage user={user}/>}/>
|
||||||
|
<Route path="/posts/import" element={<PostImportSourcePage user={user}/>}/>
|
||||||
|
<Route path="/posts/import/:sessionId/review" element={<PostImportReviewPage user={user}/>}/>
|
||||||
|
<Route path="/posts/import/:sessionId/result" element={<PostImportResultPage user={user}/>}/>
|
||||||
<Route path="/posts/search" element={<PostSearchPage/>}/>
|
<Route path="/posts/search" element={<PostSearchPage/>}/>
|
||||||
<Route path="/posts/:id" element={<PostDetailRoute user={user}/>}/>
|
<Route path="/posts/:id" element={<PostDetailRoute user={user}/>}/>
|
||||||
<Route path="/posts/changes" element={<PostHistoryPage/>}/>
|
<Route path="/posts/changes" element={<PostHistoryPage/>}/>
|
||||||
|
|||||||
@@ -0,0 +1,75 @@
|
|||||||
|
import { beforeEach, describe, expect, it, vi } from 'vitest'
|
||||||
|
|
||||||
|
import DraggableDroppableTagRow from '@/components/DraggableDroppableTagRow'
|
||||||
|
import { buildTag } from '@/test/factories'
|
||||||
|
import { renderWithProviders } from '@/test/render'
|
||||||
|
|
||||||
|
const dndKit = vi.hoisted (() => ({
|
||||||
|
useDraggable: vi.fn (),
|
||||||
|
useDroppable: vi.fn (),
|
||||||
|
}))
|
||||||
|
|
||||||
|
vi.mock ('@dnd-kit/core', () => dndKit)
|
||||||
|
|
||||||
|
const tag = buildTag ({ id: 7, name: 'ドラッグ元', postCount: 3 })
|
||||||
|
|
||||||
|
const renderRow = (activeDndId?: string) => {
|
||||||
|
renderWithProviders (
|
||||||
|
<DraggableDroppableTagRow
|
||||||
|
activeDndId={activeDndId}
|
||||||
|
tag={tag}
|
||||||
|
nestLevel={2}
|
||||||
|
pathKey="cat-general-7"
|
||||||
|
suppressClickRef={{ current: false }}/>,
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
const tagBody = (): HTMLElement => {
|
||||||
|
const link = document.querySelector<HTMLElement> ('a[title="ドラッグ元"]')
|
||||||
|
|
||||||
|
if (link == null)
|
||||||
|
throw new Error ('tag link not found')
|
||||||
|
|
||||||
|
const body = link.closest ('div')
|
||||||
|
|
||||||
|
if (body == null)
|
||||||
|
throw new Error ('tag body not found')
|
||||||
|
|
||||||
|
return body
|
||||||
|
}
|
||||||
|
|
||||||
|
describe ('DraggableDroppableTagRow', () => {
|
||||||
|
beforeEach (() => {
|
||||||
|
vi.clearAllMocks ()
|
||||||
|
dndKit.useDraggable.mockReturnValue ({
|
||||||
|
attributes: { 'aria-describedby': 'drag-source' },
|
||||||
|
listeners: { onPointerDown: vi.fn () },
|
||||||
|
setNodeRef: vi.fn (),
|
||||||
|
transform: null })
|
||||||
|
dndKit.useDroppable.mockReturnValue ({
|
||||||
|
isOver: false,
|
||||||
|
setNodeRef: vi.fn () })
|
||||||
|
})
|
||||||
|
|
||||||
|
it ('passes dndId through draggable data for active-row tracking', () => {
|
||||||
|
renderRow ()
|
||||||
|
|
||||||
|
expect (dndKit.useDraggable).toHaveBeenCalledWith (
|
||||||
|
expect.objectContaining ({
|
||||||
|
id: 'tag-node:cat-general-7',
|
||||||
|
data: expect.objectContaining ({
|
||||||
|
dndId: 'tag-node:cat-general-7',
|
||||||
|
nestLevel: 2,
|
||||||
|
tagId: 7 }) }))
|
||||||
|
})
|
||||||
|
|
||||||
|
it ('hides only the active drag source from the explicit active dnd id', () => {
|
||||||
|
renderRow ('tag-node:cat-general-7')
|
||||||
|
expect (tagBody ()).toHaveStyle ({ visibility: 'hidden' })
|
||||||
|
})
|
||||||
|
|
||||||
|
it ('keeps inactive tag rows visible while another tag is dragged', () => {
|
||||||
|
renderRow ('tag-node:other')
|
||||||
|
expect (tagBody ()).toHaveStyle ({ visibility: 'visible' })
|
||||||
|
})
|
||||||
|
})
|
||||||
@@ -13,6 +13,7 @@ import type { CSSProperties, FC, MutableRefObject } from 'react'
|
|||||||
import type { Tag } from '@/types'
|
import type { Tag } from '@/types'
|
||||||
|
|
||||||
type Props = {
|
type Props = {
|
||||||
|
activeDndId?: string
|
||||||
tag: Tag
|
tag: Tag
|
||||||
nestLevel: number
|
nestLevel: number
|
||||||
pathKey: string
|
pathKey: string
|
||||||
@@ -21,7 +22,15 @@ type Props = {
|
|||||||
sp?: boolean }
|
sp?: boolean }
|
||||||
|
|
||||||
|
|
||||||
const DraggableDroppableTagRow: FC<Props> = ({ tag, nestLevel, pathKey, parentTagId, suppressClickRef, sp }) => {
|
const DraggableDroppableTagRow: FC<Props> = ({
|
||||||
|
activeDndId,
|
||||||
|
tag,
|
||||||
|
nestLevel,
|
||||||
|
pathKey,
|
||||||
|
parentTagId,
|
||||||
|
suppressClickRef,
|
||||||
|
sp,
|
||||||
|
}) => {
|
||||||
const behaviourSettings = useClientBehaviourSettings ()
|
const behaviourSettings = useClientBehaviourSettings ()
|
||||||
const animationMode = behaviourSettings.animation ?? 'normal'
|
const animationMode = behaviourSettings.animation ?? 'normal'
|
||||||
const layoutTransition = clientAnimationTransition (
|
const layoutTransition = clientAnimationTransition (
|
||||||
@@ -53,18 +62,23 @@ const DraggableDroppableTagRow: FC<Props> = ({ tag, nestLevel, pathKey, parentTa
|
|||||||
const { attributes,
|
const { attributes,
|
||||||
listeners,
|
listeners,
|
||||||
setNodeRef: setDragRef,
|
setNodeRef: setDragRef,
|
||||||
transform,
|
transform } = useDraggable ({ id: dndId,
|
||||||
isDragging: dragging } = useDraggable ({ id: dndId,
|
data: { kind: 'tag',
|
||||||
data: { kind: 'tag',
|
dndId,
|
||||||
tagId: tag.id,
|
tagId: tag.id,
|
||||||
parentTagId } })
|
parentTagId,
|
||||||
|
nestLevel } })
|
||||||
|
|
||||||
const { setNodeRef: setDropRef, isOver: over } = useDroppable ({
|
const { setNodeRef: setDropRef, isOver: over } = useDroppable ({
|
||||||
id: dndId,
|
id: dndId,
|
||||||
data: { kind: 'tag', tagId: tag.id } })
|
data: { kind: 'tag', tagId: tag.id } })
|
||||||
|
|
||||||
|
const activeDragging = activeDndId === dndId
|
||||||
const style: CSSProperties = { transform: CSS.Translate.toString (transform),
|
const style: CSSProperties = { transform: CSS.Translate.toString (transform),
|
||||||
visibility: dragging ? 'hidden' : 'visible' }
|
visibility: activeDragging ? 'hidden' : 'visible' }
|
||||||
|
const innerClassName = cn (
|
||||||
|
'inline-flex min-w-0 max-w-full items-baseline overflow-hidden',
|
||||||
|
sp && 'touch-pan-y')
|
||||||
|
|
||||||
return (
|
return (
|
||||||
<div
|
<div
|
||||||
@@ -77,12 +91,16 @@ const DraggableDroppableTagRow: FC<Props> = ({ tag, nestLevel, pathKey, parentTa
|
|||||||
return
|
return
|
||||||
const dx = e.clientX - p.x
|
const dx = e.clientX - p.x
|
||||||
const dy = e.clientY - p.y
|
const dy = e.clientY - p.y
|
||||||
|
|
||||||
if (dx * dx + dy * dy >= 9)
|
if (dx * dx + dy * dy >= 9)
|
||||||
armEatNextClick ()
|
armEatNextClick ()
|
||||||
}}
|
}}
|
||||||
onPointerUpCapture={() => {
|
onPointerUpCapture={() => {
|
||||||
downPosRef.current = null
|
downPosRef.current = null
|
||||||
}}
|
}}
|
||||||
|
onPointerCancelCapture={() => {
|
||||||
|
downPosRef.current = null
|
||||||
|
}}
|
||||||
onClickCapture={e => {
|
onClickCapture={e => {
|
||||||
if (suppressClickRef.current)
|
if (suppressClickRef.current)
|
||||||
{
|
{
|
||||||
@@ -90,24 +108,35 @@ const DraggableDroppableTagRow: FC<Props> = ({ tag, nestLevel, pathKey, parentTa
|
|||||||
e.stopPropagation ()
|
e.stopPropagation ()
|
||||||
}
|
}
|
||||||
}}
|
}}
|
||||||
ref={node => {
|
ref={setDropRef}
|
||||||
setDragRef (node)
|
|
||||||
setDropRef (node)
|
|
||||||
}}
|
|
||||||
style={style}
|
|
||||||
className={cn (
|
className={cn (
|
||||||
'min-w-0 max-w-full overflow-hidden rounded select-none',
|
'min-w-0 max-w-full overflow-hidden rounded select-none',
|
||||||
|
sp && 'touch-pan-y',
|
||||||
over && 'ring-2 ring-offset-2')}
|
over && 'ring-2 ring-offset-2')}
|
||||||
{...attributes}
|
>
|
||||||
{...listeners}>
|
{activeDragging
|
||||||
<motion.div
|
? (
|
||||||
className="flex min-w-0 max-w-full items-baseline overflow-hidden"
|
<div
|
||||||
transition={{ layout: layoutTransition }}
|
ref={setDragRef}
|
||||||
layoutId={animationMode === 'off'
|
style={style}
|
||||||
? undefined
|
className={innerClassName}
|
||||||
: `tag-${ sp ? 'sp-' : '' }${ tag.id }`}>
|
{...attributes}
|
||||||
<TagLink tag={tag} nestLevel={nestLevel}/>
|
{...listeners}>
|
||||||
</motion.div>
|
<TagLink tag={tag} nestLevel={nestLevel}/>
|
||||||
|
</div>)
|
||||||
|
: (
|
||||||
|
<motion.div
|
||||||
|
ref={setDragRef}
|
||||||
|
style={style}
|
||||||
|
className={innerClassName}
|
||||||
|
{...attributes}
|
||||||
|
{...listeners}
|
||||||
|
transition={{ layout: layoutTransition }}
|
||||||
|
layoutId={animationMode === 'off'
|
||||||
|
? undefined
|
||||||
|
: `tag-${ sp ? 'sp-' : '' }${ tag.id }`}>
|
||||||
|
<TagLink tag={tag} nestLevel={nestLevel}/>
|
||||||
|
</motion.div>)}
|
||||||
</div>)
|
</div>)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@@ -2,10 +2,11 @@ import DateTimeField from '@/components/common/DateTimeField'
|
|||||||
import FormField from '@/components/common/FormField'
|
import FormField from '@/components/common/FormField'
|
||||||
import { Button } from '@/components/ui/button'
|
import { Button } from '@/components/ui/button'
|
||||||
|
|
||||||
import type { FC } from 'react'
|
import type { FC, ReactNode } from 'react'
|
||||||
|
|
||||||
type Props = {
|
type Props = {
|
||||||
disabled?: boolean
|
disabled?: boolean
|
||||||
|
labelAddon?: ReactNode
|
||||||
originalCreatedFrom: string | null
|
originalCreatedFrom: string | null
|
||||||
setOriginalCreatedFrom: (x: string | null) => void
|
setOriginalCreatedFrom: (x: string | null) => void
|
||||||
originalCreatedBefore: string | null
|
originalCreatedBefore: string | null
|
||||||
@@ -15,18 +16,25 @@ type Props = {
|
|||||||
|
|
||||||
const PostOriginalCreatedTimeField: FC<Props> = (
|
const PostOriginalCreatedTimeField: FC<Props> = (
|
||||||
{ disabled,
|
{ disabled,
|
||||||
|
labelAddon,
|
||||||
originalCreatedFrom,
|
originalCreatedFrom,
|
||||||
setOriginalCreatedFrom,
|
setOriginalCreatedFrom,
|
||||||
originalCreatedBefore,
|
originalCreatedBefore,
|
||||||
setOriginalCreatedBefore,
|
setOriginalCreatedBefore,
|
||||||
errors }: Props) => (
|
errors }: Props) => (
|
||||||
<FormField label="オリジナルの作成日時" messages={errors}>
|
<FormField
|
||||||
|
label={
|
||||||
|
<span className="flex items-center gap-2">
|
||||||
|
<span>オリジナルの作成日時</span>
|
||||||
|
{labelAddon}
|
||||||
|
</span>}
|
||||||
|
messages={errors}>
|
||||||
{({ describedBy, invalid }) => (
|
{({ describedBy, invalid }) => (
|
||||||
<>
|
<>
|
||||||
<div className="my-1 flex">
|
<div className="my-1 flex flex-col gap-2 sm:flex-row sm:items-start">
|
||||||
<div className="w-80">
|
<div className="min-w-0 flex-1">
|
||||||
<DateTimeField
|
<DateTimeField
|
||||||
className="mr-2"
|
className="w-full"
|
||||||
disabled={disabled ?? false}
|
disabled={disabled ?? false}
|
||||||
aria-describedby={describedBy}
|
aria-describedby={describedBy}
|
||||||
aria-invalid={invalid}
|
aria-invalid={invalid}
|
||||||
@@ -49,6 +57,7 @@ const PostOriginalCreatedTimeField: FC<Props> = (
|
|||||||
</div>
|
</div>
|
||||||
<div>
|
<div>
|
||||||
<Button
|
<Button
|
||||||
|
type="button"
|
||||||
className="bg-gray-600 text-white rounded"
|
className="bg-gray-600 text-white rounded"
|
||||||
disabled={disabled}
|
disabled={disabled}
|
||||||
onClick={() => {
|
onClick={() => {
|
||||||
@@ -59,10 +68,10 @@ const PostOriginalCreatedTimeField: FC<Props> = (
|
|||||||
</div>
|
</div>
|
||||||
</div>
|
</div>
|
||||||
|
|
||||||
<div className="my-1 flex">
|
<div className="my-1 flex flex-col gap-2 sm:flex-row sm:items-start">
|
||||||
<div className="w-80">
|
<div className="min-w-0 flex-1">
|
||||||
<DateTimeField
|
<DateTimeField
|
||||||
className="mr-2"
|
className="w-full"
|
||||||
disabled={disabled}
|
disabled={disabled}
|
||||||
aria-describedby={describedBy}
|
aria-describedby={describedBy}
|
||||||
aria-invalid={invalid}
|
aria-invalid={invalid}
|
||||||
@@ -73,6 +82,7 @@ const PostOriginalCreatedTimeField: FC<Props> = (
|
|||||||
</div>
|
</div>
|
||||||
<div>
|
<div>
|
||||||
<Button
|
<Button
|
||||||
|
type="button"
|
||||||
className="bg-gray-600 text-white rounded"
|
className="bg-gray-600 text-white rounded"
|
||||||
disabled={disabled}
|
disabled={disabled}
|
||||||
onClick={() => {
|
onClick={() => {
|
||||||
|
|||||||
@@ -1,11 +1,13 @@
|
|||||||
import { DndContext,
|
import { DndContext,
|
||||||
DragOverlay,
|
DragOverlay,
|
||||||
|
MeasuringStrategy,
|
||||||
MouseSensor,
|
MouseSensor,
|
||||||
TouchSensor,
|
TouchSensor,
|
||||||
|
pointerWithin,
|
||||||
useDroppable,
|
useDroppable,
|
||||||
useSensor,
|
useSensor,
|
||||||
useSensors } from '@dnd-kit/core'
|
useSensors } from '@dnd-kit/core'
|
||||||
import { restrictToWindowEdges } from '@dnd-kit/modifiers'
|
import { restrictToWindowEdges, snapCenterToCursor } from '@dnd-kit/modifiers'
|
||||||
import { useQueryClient } from '@tanstack/react-query'
|
import { useQueryClient } from '@tanstack/react-query'
|
||||||
import { motion } from 'framer-motion'
|
import { motion } from 'framer-motion'
|
||||||
import { useEffect, useMemo, useRef, useState } from 'react'
|
import { useEffect, useMemo, useRef, useState } from 'react'
|
||||||
@@ -25,15 +27,23 @@ import { postsKeys, tagsKeys } from '@/lib/queryKeys'
|
|||||||
import { useClientBehaviourSettings } from '@/lib/useClientBehaviourSettings'
|
import { useClientBehaviourSettings } from '@/lib/useClientBehaviourSettings'
|
||||||
import { dateString, originalCreatedAtString } from '@/lib/utils'
|
import { dateString, originalCreatedAtString } from '@/lib/utils'
|
||||||
|
|
||||||
import type { DragEndEvent } from '@dnd-kit/core'
|
import type { CollisionDetection, DragEndEvent } from '@dnd-kit/core'
|
||||||
import type { FC, MutableRefObject, ReactNode } from 'react'
|
import type { FC, MutableRefObject, ReactNode } from 'react'
|
||||||
|
|
||||||
import type { Category, Post, TagWithSections } from '@/types'
|
import type { Category, Post, TagWithSections } from '@/types'
|
||||||
|
|
||||||
type TagByCategory = { [key in Category]: TagWithSections[] }
|
type TagByCategory = { [key in Category]: TagWithSections[] }
|
||||||
|
|
||||||
|
const tagCollisionDetection: CollisionDetection = args => {
|
||||||
|
return pointerWithin (args)
|
||||||
|
}
|
||||||
|
|
||||||
|
const alwaysMeasureDroppables = {
|
||||||
|
droppable: { strategy: MeasuringStrategy.Always } } as const
|
||||||
|
|
||||||
|
|
||||||
const renderTagTree = (
|
const renderTagTree = (
|
||||||
|
activeDndId: string | undefined,
|
||||||
tag: TagWithSections,
|
tag: TagWithSections,
|
||||||
nestLevel: number,
|
nestLevel: number,
|
||||||
path: string,
|
path: string,
|
||||||
@@ -45,6 +55,7 @@ const renderTagTree = (
|
|||||||
const self = (
|
const self = (
|
||||||
<li key={key} className="mb-1">
|
<li key={key} className="mb-1">
|
||||||
<DraggableDroppableTagRow
|
<DraggableDroppableTagRow
|
||||||
|
activeDndId={activeDndId}
|
||||||
tag={tag}
|
tag={tag}
|
||||||
nestLevel={nestLevel}
|
nestLevel={nestLevel}
|
||||||
pathKey={key}
|
pathKey={key}
|
||||||
@@ -58,7 +69,14 @@ const renderTagTree = (
|
|||||||
...((tag.children
|
...((tag.children
|
||||||
?.sort ((a, b) => a.name < b.name ? -1 : 1)
|
?.sort ((a, b) => a.name < b.name ? -1 : 1)
|
||||||
.flatMap (child =>
|
.flatMap (child =>
|
||||||
renderTagTree (child, nestLevel + 1, key, suppressClickRef, tag.id, sp)))
|
renderTagTree (
|
||||||
|
activeDndId,
|
||||||
|
child,
|
||||||
|
nestLevel + 1,
|
||||||
|
key,
|
||||||
|
suppressClickRef,
|
||||||
|
tag.id,
|
||||||
|
sp)))
|
||||||
?? [])]
|
?? [])]
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -177,12 +195,35 @@ const DropSlot = ({ cat }: { cat: Category }) => {
|
|||||||
</li>)
|
</li>)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
const EmptyCategoryDropSection = (
|
||||||
|
{ cat, children }: { cat: Category
|
||||||
|
children: ReactNode }) => {
|
||||||
|
const { setNodeRef, isOver: over } = useDroppable ({
|
||||||
|
id: `slot:${ cat }`,
|
||||||
|
data: { kind: 'slot', cat } })
|
||||||
|
|
||||||
|
return (
|
||||||
|
<div ref={setNodeRef} className="my-3">
|
||||||
|
{children}
|
||||||
|
<ul>
|
||||||
|
<li className="h-1">
|
||||||
|
{over && <div className="h-0.5 w-full rounded bg-sky-400"/>}
|
||||||
|
</li>
|
||||||
|
</ul>
|
||||||
|
</div>)
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
type Props = {
|
type Props = {
|
||||||
className?: string
|
className?: string
|
||||||
post: Post
|
post: Post
|
||||||
sp?: boolean }
|
sp?: boolean }
|
||||||
|
|
||||||
|
type ActiveTagDrag = {
|
||||||
|
dndId: string
|
||||||
|
tagId: number
|
||||||
|
nestLevel: number }
|
||||||
|
|
||||||
|
|
||||||
const TagDetailSidebar: FC<Props> = ({ className, post, sp }) => {
|
const TagDetailSidebar: FC<Props> = ({ className, post, sp }) => {
|
||||||
sp = Boolean (sp)
|
sp = Boolean (sp)
|
||||||
@@ -212,7 +253,7 @@ const TagDetailSidebar: FC<Props> = ({ className, post, sp }) => {
|
|||||||
return tagsTmp
|
return tagsTmp
|
||||||
}, [post])
|
}, [post])
|
||||||
|
|
||||||
const [activeTagId, setActiveTagId] = useState<number | null> (null)
|
const [activeTagDrag, setActiveTagDrag] = useState<ActiveTagDrag | null> (null)
|
||||||
const [dragging, setDragging] = useState (false)
|
const [dragging, setDragging] = useState (false)
|
||||||
const [saving, setSaving] = useState (false)
|
const [saving, setSaving] = useState (false)
|
||||||
const [tags, setTags] = useState (baseTags)
|
const [tags, setTags] = useState (baseTags)
|
||||||
@@ -220,6 +261,7 @@ const TagDetailSidebar: FC<Props> = ({ className, post, sp }) => {
|
|||||||
() => buildFlatTagByCategory (tags),
|
() => buildFlatTagByCategory (tags),
|
||||||
[tags],
|
[tags],
|
||||||
)
|
)
|
||||||
|
const activeDndId = activeTagDrag?.dndId
|
||||||
|
|
||||||
const suppressClickRef = useRef (false)
|
const suppressClickRef = useRef (false)
|
||||||
|
|
||||||
@@ -328,9 +370,19 @@ const TagDetailSidebar: FC<Props> = ({ className, post, sp }) => {
|
|||||||
<TagSearch/>
|
<TagSearch/>
|
||||||
<DndContext
|
<DndContext
|
||||||
sensors={sensors}
|
sensors={sensors}
|
||||||
|
collisionDetection={tagCollisionDetection}
|
||||||
|
measuring={alwaysMeasureDroppables}
|
||||||
onDragStart={e => {
|
onDragStart={e => {
|
||||||
if (e.active.data.current?.kind === 'tag')
|
if (e.active.data.current?.kind === 'tag')
|
||||||
setActiveTagId (e.active.data.current?.tagId ?? null)
|
{
|
||||||
|
const tagId = e.active.data.current?.tagId
|
||||||
|
const nestLevel = e.active.data.current?.nestLevel
|
||||||
|
const dndId = e.active.data.current?.dndId
|
||||||
|
setActiveTagDrag (
|
||||||
|
tagId == null || nestLevel == null || dndId == null
|
||||||
|
? null
|
||||||
|
: { dndId, tagId, nestLevel })
|
||||||
|
}
|
||||||
setDragging (true)
|
setDragging (true)
|
||||||
suppressClickRef.current = true
|
suppressClickRef.current = true
|
||||||
document.body.style.userSelect = 'none'
|
document.body.style.userSelect = 'none'
|
||||||
@@ -341,13 +393,13 @@ const TagDetailSidebar: FC<Props> = ({ className, post, sp }) => {
|
|||||||
suppressClickRef.current = false}, { capture: true, once: true })
|
suppressClickRef.current = false}, { capture: true, once: true })
|
||||||
}}
|
}}
|
||||||
onDragCancel={() => {
|
onDragCancel={() => {
|
||||||
setActiveTagId (null)
|
setActiveTagDrag (null)
|
||||||
setDragging (false)
|
setDragging (false)
|
||||||
document.body.style.userSelect = ''
|
document.body.style.userSelect = ''
|
||||||
suppressClickRef.current = false
|
suppressClickRef.current = false
|
||||||
}}
|
}}
|
||||||
onDragEnd={async e => {
|
onDragEnd={async e => {
|
||||||
setActiveTagId (null)
|
setActiveTagDrag (null)
|
||||||
setDragging (false)
|
setDragging (false)
|
||||||
await onDragEnd (e)
|
await onDragEnd (e)
|
||||||
document.body.style.userSelect = ''
|
document.body.style.userSelect = ''
|
||||||
@@ -362,22 +414,31 @@ const TagDetailSidebar: FC<Props> = ({ className, post, sp }) => {
|
|||||||
if (!(categoryTags.length > 0 || dragging))
|
if (!(categoryTags.length > 0 || dragging))
|
||||||
return null
|
return null
|
||||||
|
|
||||||
|
const sectionTitle = (
|
||||||
|
<SubsectionTitle>
|
||||||
|
<motion.div
|
||||||
|
layoutId={animationMode === 'off'
|
||||||
|
? undefined
|
||||||
|
: `tag-${ sp ? 'sp-' : '' }${ cat }`}
|
||||||
|
transition={{ layout: layoutTransition }}>
|
||||||
|
{CATEGORY_NAMES[cat]}
|
||||||
|
</motion.div>
|
||||||
|
</SubsectionTitle>)
|
||||||
|
|
||||||
|
if (!(categoryTags.length > 0))
|
||||||
|
return (
|
||||||
|
<EmptyCategoryDropSection cat={cat} key={cat}>
|
||||||
|
{sectionTitle}
|
||||||
|
</EmptyCategoryDropSection>)
|
||||||
|
|
||||||
return (
|
return (
|
||||||
<div className="my-3" key={cat}>
|
<div className="my-3" key={cat}>
|
||||||
<SubsectionTitle>
|
{sectionTitle}
|
||||||
<motion.div
|
|
||||||
layoutId={animationMode === 'off'
|
|
||||||
? undefined
|
|
||||||
: `tag-${ sp ? 'sp-' : '' }${ cat }`}
|
|
||||||
transition={{ layout: layoutTransition }}>
|
|
||||||
{CATEGORY_NAMES[cat]}
|
|
||||||
</motion.div>
|
|
||||||
</SubsectionTitle>
|
|
||||||
|
|
||||||
<ul>
|
<ul>
|
||||||
{(tagRelationDisplay === 'grouped'
|
{(tagRelationDisplay === 'grouped'
|
||||||
? (tags[cat] ?? []).flatMap (tag =>
|
? (tags[cat] ?? []).flatMap (tag =>
|
||||||
renderTagTree (
|
renderTagTree (
|
||||||
|
activeDndId,
|
||||||
tag,
|
tag,
|
||||||
0,
|
0,
|
||||||
`cat-${ cat }`,
|
`cat-${ cat }`,
|
||||||
@@ -389,6 +450,7 @@ const TagDetailSidebar: FC<Props> = ({ className, post, sp }) => {
|
|||||||
: (flatTagsByCategory[cat] ?? []).map (tag => (
|
: (flatTagsByCategory[cat] ?? []).map (tag => (
|
||||||
<li key={`flat-${ cat }-${ tag.id }`} className="mb-1">
|
<li key={`flat-${ cat }-${ tag.id }`} className="mb-1">
|
||||||
<DraggableDroppableTagRow
|
<DraggableDroppableTagRow
|
||||||
|
activeDndId={activeDndId}
|
||||||
tag={tag}
|
tag={tag}
|
||||||
nestLevel={0}
|
nestLevel={0}
|
||||||
pathKey={`flat-${ cat }-${ tag.id }`}
|
pathKey={`flat-${ cat }-${ tag.id }`}
|
||||||
@@ -441,11 +503,14 @@ const TagDetailSidebar: FC<Props> = ({ className, post, sp }) => {
|
|||||||
</ul>
|
</ul>
|
||||||
</motion.div>)}
|
</motion.div>)}
|
||||||
|
|
||||||
<DragOverlay adjustScale={false}>
|
<DragOverlay
|
||||||
|
adjustScale={false}
|
||||||
|
modifiers={[snapCenterToCursor]}
|
||||||
|
dropAnimation={animationMode === 'off' ? null : undefined}>
|
||||||
<div className="pointer-events-none">
|
<div className="pointer-events-none">
|
||||||
{activeTagId != null && (() => {
|
{activeTagDrag != null && (() => {
|
||||||
const tag = findTag (tags, activeTagId)
|
const tag = findTag (tags, activeTagDrag.tagId)
|
||||||
return tag && <TagLink tag={tag}/>
|
return tag && <TagLink tag={tag} nestLevel={activeTagDrag.nestLevel}/>
|
||||||
}) ()}
|
}) ()}
|
||||||
</div>
|
</div>
|
||||||
</DragOverlay>
|
</DragOverlay>
|
||||||
|
|||||||
@@ -26,7 +26,7 @@ export const menuOutline = (
|
|||||||
tag?: Tag | null
|
tag?: Tag | null
|
||||||
material?: Material | null
|
material?: Material | null
|
||||||
wikiId: number | null
|
wikiId: number | null
|
||||||
user: User | null,
|
user: User | null
|
||||||
pathName: string },
|
pathName: string },
|
||||||
): Menu => {
|
): Menu => {
|
||||||
const postCount = tag?.postCount ?? material?.tag?.postCount ?? 0
|
const postCount = tag?.postCount ?? material?.tag?.postCount ?? 0
|
||||||
@@ -43,6 +43,7 @@ export const menuOutline = (
|
|||||||
{ name: '一覧', to: '/posts' },
|
{ name: '一覧', to: '/posts' },
|
||||||
{ name: '検索', to: '/posts/search' },
|
{ name: '検索', to: '/posts/search' },
|
||||||
{ name: '追加', to: '/posts/new' },
|
{ name: '追加', to: '/posts/new' },
|
||||||
|
{ name: '取込', to: '/posts/import' },
|
||||||
{ name: '全体履歴', to: '/posts/changes' },
|
{ name: '全体履歴', to: '/posts/changes' },
|
||||||
{ name: 'ヘルプ', to: '/wiki/ヘルプ:広場' }] },
|
{ name: 'ヘルプ', to: '/wiki/ヘルプ:広場' }] },
|
||||||
{ name: 'タグ', to: '/tags', subMenu: [
|
{ name: 'タグ', to: '/tags', subMenu: [
|
||||||
|
|||||||
@@ -0,0 +1,18 @@
|
|||||||
|
import type { FC } from 'react'
|
||||||
|
|
||||||
|
type Props = { id?: string
|
||||||
|
messages?: string[] }
|
||||||
|
|
||||||
|
|
||||||
|
export const FieldWarning: FC<Props> = ({ id, messages }: Props) => {
|
||||||
|
if (messages == null || messages.length === 0)
|
||||||
|
return null
|
||||||
|
|
||||||
|
return (
|
||||||
|
<ul id={id} className="mt-1 space-y-1 text-amber-700 dark:text-amber-200">
|
||||||
|
{messages.map ((message, i) => <li key={i}>{message}</li>)}
|
||||||
|
</ul>)
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
export default FieldWarning
|
||||||
@@ -0,0 +1,287 @@
|
|||||||
|
import PostOriginalCreatedTimeField from '@/components/PostOriginalCreatedTimeField'
|
||||||
|
import FieldError from '@/components/common/FieldError'
|
||||||
|
import FieldWarning from '@/components/common/FieldWarning'
|
||||||
|
import FormField from '@/components/common/FormField'
|
||||||
|
import PostImportStatusBadge from '@/components/posts/import/PostImportStatusBadge'
|
||||||
|
import ThumbnailPreview from '@/components/posts/import/ThumbnailPreview'
|
||||||
|
import { effectivePostImportStatus } from '@/components/posts/import/postImportRowStatus'
|
||||||
|
import { Button } from '@/components/ui/button'
|
||||||
|
import { Dialog,
|
||||||
|
DialogContent,
|
||||||
|
DialogDescription,
|
||||||
|
DialogFooter,
|
||||||
|
DialogHeader,
|
||||||
|
DialogTitle } from '@/components/ui/dialog'
|
||||||
|
import { inputClass } from '@/lib/utils'
|
||||||
|
|
||||||
|
import type { FC } from 'react'
|
||||||
|
import { useEffect, useState } from 'react'
|
||||||
|
|
||||||
|
import type { PostImportOrigin,
|
||||||
|
PostImportRow } from '@/lib/postImportSession'
|
||||||
|
|
||||||
|
type Draft = {
|
||||||
|
url: string
|
||||||
|
title: string
|
||||||
|
thumbnailBase: string
|
||||||
|
originalCreatedFrom: string
|
||||||
|
originalCreatedBefore: string
|
||||||
|
duration: string
|
||||||
|
tags: string
|
||||||
|
parentPostIds: string }
|
||||||
|
|
||||||
|
type Props = {
|
||||||
|
open: boolean
|
||||||
|
row: PostImportRow | null
|
||||||
|
saving: boolean
|
||||||
|
onOpenChange: (open: boolean) => void
|
||||||
|
onSave: (draft: Draft) => Promise<boolean> }
|
||||||
|
|
||||||
|
const originOf = (
|
||||||
|
row: PostImportRow,
|
||||||
|
field: string,
|
||||||
|
): PostImportOrigin =>
|
||||||
|
row.provenance[field] ?? 'automatic'
|
||||||
|
|
||||||
|
const changedOrigin = (
|
||||||
|
changed: boolean,
|
||||||
|
row: PostImportRow,
|
||||||
|
field: string,
|
||||||
|
): PostImportOrigin =>
|
||||||
|
changed ? 'manual' : originOf (row, field)
|
||||||
|
|
||||||
|
const originalCreatedOrigin = (
|
||||||
|
row: PostImportRow,
|
||||||
|
originalDraft: Draft,
|
||||||
|
draft: Draft,
|
||||||
|
): PostImportOrigin => {
|
||||||
|
const changed =
|
||||||
|
originalDraft.originalCreatedFrom !== draft.originalCreatedFrom
|
||||||
|
|| originalDraft.originalCreatedBefore !== draft.originalCreatedBefore
|
||||||
|
if (changed)
|
||||||
|
return 'manual'
|
||||||
|
|
||||||
|
const manualOrigin =
|
||||||
|
originOf (row, 'originalCreatedFrom') === 'manual'
|
||||||
|
|| originOf (row, 'originalCreatedBefore') === 'manual'
|
||||||
|
return manualOrigin ? 'manual' : 'automatic'
|
||||||
|
}
|
||||||
|
|
||||||
|
const buildDraft = (row: PostImportRow): Draft => ({
|
||||||
|
url: row.url,
|
||||||
|
title: String (row.attributes.title ?? ''),
|
||||||
|
thumbnailBase: String (row.attributes.thumbnailBase ?? ''),
|
||||||
|
originalCreatedFrom: String (row.attributes.originalCreatedFrom ?? ''),
|
||||||
|
originalCreatedBefore: String (row.attributes.originalCreatedBefore ?? ''),
|
||||||
|
duration: String (row.attributes.duration ?? ''),
|
||||||
|
tags: String (row.attributes.tags ?? ''),
|
||||||
|
parentPostIds: String (row.attributes.parentPostIds ?? '') })
|
||||||
|
|
||||||
|
const groupedMessages = (
|
||||||
|
...values: Array<string[] | undefined>
|
||||||
|
): string[] =>
|
||||||
|
values.flatMap (value => value ?? [])
|
||||||
|
|
||||||
|
|
||||||
|
const PostImportRowDialog: FC<Props> = (
|
||||||
|
{ open, row, saving, onOpenChange, onSave },
|
||||||
|
) => {
|
||||||
|
const [draft, setDraft] = useState<Draft | null> (null)
|
||||||
|
|
||||||
|
useEffect (() => {
|
||||||
|
if (open && row)
|
||||||
|
setDraft (buildDraft (row))
|
||||||
|
}, [open, row])
|
||||||
|
|
||||||
|
if (row == null || draft == null)
|
||||||
|
return null
|
||||||
|
|
||||||
|
const originalDraft = buildDraft (row)
|
||||||
|
const fieldOrigin = (field: keyof Draft): PostImportOrigin =>
|
||||||
|
changedOrigin (draft[field] !== originalDraft[field], row, field)
|
||||||
|
|
||||||
|
const update = <Key extends keyof Draft,> (
|
||||||
|
key: Key,
|
||||||
|
value: Draft[Key],
|
||||||
|
) => {
|
||||||
|
setDraft (current =>
|
||||||
|
current ? { ...current, [key]: value } : current)
|
||||||
|
}
|
||||||
|
|
||||||
|
const save = async () => {
|
||||||
|
if (await onSave (draft))
|
||||||
|
onOpenChange (false)
|
||||||
|
}
|
||||||
|
|
||||||
|
return (
|
||||||
|
<Dialog open={open} onOpenChange={onOpenChange}>
|
||||||
|
<DialogContent className="flex max-h-[calc(100dvh-1rem)] max-w-3xl flex-col
|
||||||
|
overflow-hidden p-0">
|
||||||
|
<DialogHeader className="shrink-0 px-6 pb-4 pt-7">
|
||||||
|
<DialogTitle>投稿を編輯</DialogTitle>
|
||||||
|
<DialogDescription>
|
||||||
|
投稿 {row.sourceRow} の内容を確認し、必要な項目を編輯してください.
|
||||||
|
</DialogDescription>
|
||||||
|
</DialogHeader>
|
||||||
|
|
||||||
|
<div className="min-h-0 flex-1 overflow-y-auto overscroll-contain px-6 pb-6">
|
||||||
|
<div className="grid gap-6 md:grid-cols-[7rem_minmax(0,1fr)]">
|
||||||
|
<div className="space-y-3">
|
||||||
|
<ThumbnailPreview
|
||||||
|
url={draft.thumbnailBase}
|
||||||
|
className="h-28 w-28"/>
|
||||||
|
<div className="flex flex-wrap gap-2">
|
||||||
|
<PostImportStatusBadge value={effectivePostImportStatus (row)}/>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<div className="space-y-4">
|
||||||
|
<DialogTextField
|
||||||
|
label="URL"
|
||||||
|
value={draft.url}
|
||||||
|
origin={fieldOrigin ('url')}
|
||||||
|
warnings={row.fieldWarnings.url}
|
||||||
|
errors={groupedMessages (row.validationErrors.url, row.importErrors?.url)}
|
||||||
|
onChange={value => update ('url', value)}/>
|
||||||
|
<DialogTextField
|
||||||
|
label="タイトル"
|
||||||
|
value={draft.title}
|
||||||
|
origin={fieldOrigin ('title')}
|
||||||
|
warnings={row.fieldWarnings.title}
|
||||||
|
errors={groupedMessages (row.validationErrors.title, row.importErrors?.title)}
|
||||||
|
onChange={value => update ('title', value)}/>
|
||||||
|
<DialogTextField
|
||||||
|
label="サムネール基底 URL"
|
||||||
|
value={draft.thumbnailBase}
|
||||||
|
origin={fieldOrigin ('thumbnailBase')}
|
||||||
|
warnings={row.fieldWarnings.thumbnailBase}
|
||||||
|
errors={groupedMessages (
|
||||||
|
row.validationErrors.thumbnailBase,
|
||||||
|
row.importErrors?.thumbnailBase,
|
||||||
|
)}
|
||||||
|
onChange={value => update ('thumbnailBase', value)}/>
|
||||||
|
<PostOriginalCreatedTimeField
|
||||||
|
labelAddon={
|
||||||
|
<PostImportStatusBadge
|
||||||
|
value={originalCreatedOrigin (row, originalDraft, draft)}/>
|
||||||
|
}
|
||||||
|
originalCreatedFrom={draft.originalCreatedFrom || null}
|
||||||
|
setOriginalCreatedFrom={value => update ('originalCreatedFrom', value ?? '')}
|
||||||
|
originalCreatedBefore={draft.originalCreatedBefore || null}
|
||||||
|
setOriginalCreatedBefore={value => update ('originalCreatedBefore', value ?? '')}
|
||||||
|
errors={groupedMessages (
|
||||||
|
row.validationErrors.originalCreatedAt,
|
||||||
|
row.validationErrors.originalCreatedFrom,
|
||||||
|
row.validationErrors.originalCreatedBefore,
|
||||||
|
row.importErrors?.originalCreatedAt,
|
||||||
|
row.importErrors?.originalCreatedFrom,
|
||||||
|
row.importErrors?.originalCreatedBefore,
|
||||||
|
)}/>
|
||||||
|
<DialogTextField
|
||||||
|
label="動画時間"
|
||||||
|
value={draft.duration}
|
||||||
|
origin={fieldOrigin ('duration')}
|
||||||
|
errors={groupedMessages (
|
||||||
|
row.validationErrors.duration,
|
||||||
|
row.validationErrors.videoMs,
|
||||||
|
row.importErrors?.duration,
|
||||||
|
row.importErrors?.videoMs,
|
||||||
|
)}
|
||||||
|
onChange={value => update ('duration', value)}/>
|
||||||
|
<DialogAreaField
|
||||||
|
label="タグ"
|
||||||
|
value={draft.tags}
|
||||||
|
origin={fieldOrigin ('tags')}
|
||||||
|
warnings={row.fieldWarnings.tags}
|
||||||
|
errors={groupedMessages (row.validationErrors.tags, row.importErrors?.tags)}
|
||||||
|
onChange={value => update ('tags', value)}/>
|
||||||
|
<DialogTextField
|
||||||
|
label="親投稿"
|
||||||
|
value={draft.parentPostIds}
|
||||||
|
origin={fieldOrigin ('parentPostIds')}
|
||||||
|
errors={groupedMessages (
|
||||||
|
row.validationErrors.parentPostIds,
|
||||||
|
row.importErrors?.parentPostIds,
|
||||||
|
)}
|
||||||
|
onChange={value => update ('parentPostIds', value)}/>
|
||||||
|
<FieldWarning messages={row.baseWarnings}/>
|
||||||
|
<FieldError messages={row.validationErrors.base}/>
|
||||||
|
<FieldError messages={row.importErrors?.base}/>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<DialogFooter className="shrink-0 px-6 pb-6 pt-4">
|
||||||
|
<Button
|
||||||
|
type="button"
|
||||||
|
variant="outline"
|
||||||
|
onClick={() => onOpenChange (false)}>
|
||||||
|
取消し
|
||||||
|
</Button>
|
||||||
|
<Button
|
||||||
|
type="button"
|
||||||
|
onClick={save}
|
||||||
|
disabled={saving}>
|
||||||
|
編輯内容を保存
|
||||||
|
</Button>
|
||||||
|
</DialogFooter>
|
||||||
|
</DialogContent>
|
||||||
|
</Dialog>)
|
||||||
|
}
|
||||||
|
|
||||||
|
const DialogTextField = (
|
||||||
|
{ label, value, origin, warnings, errors, onChange }: {
|
||||||
|
label: string
|
||||||
|
value: string
|
||||||
|
origin: PostImportOrigin
|
||||||
|
warnings?: string[]
|
||||||
|
errors?: string[]
|
||||||
|
onChange: (value: string) => void },
|
||||||
|
) => (
|
||||||
|
<FormField label={<DialogLabel label={label} origin={origin}/>} messages={errors}>
|
||||||
|
{({ describedBy, invalid }) => (
|
||||||
|
<>
|
||||||
|
<input
|
||||||
|
value={value}
|
||||||
|
onChange={ev => onChange (ev.target.value)}
|
||||||
|
aria-describedby={describedBy}
|
||||||
|
aria-invalid={invalid}
|
||||||
|
className={inputClass (invalid)}/>
|
||||||
|
<FieldWarning messages={warnings}/>
|
||||||
|
</>)}
|
||||||
|
</FormField>)
|
||||||
|
|
||||||
|
const DialogAreaField = (
|
||||||
|
{ label, value, origin, warnings, errors, onChange }: {
|
||||||
|
label: string
|
||||||
|
value: string
|
||||||
|
origin: PostImportOrigin
|
||||||
|
warnings?: string[]
|
||||||
|
errors?: string[]
|
||||||
|
onChange: (value: string) => void },
|
||||||
|
) => (
|
||||||
|
<FormField label={<DialogLabel label={label} origin={origin}/>} messages={errors}>
|
||||||
|
{({ describedBy, invalid }) => (
|
||||||
|
<>
|
||||||
|
<textarea
|
||||||
|
value={value}
|
||||||
|
rows={4}
|
||||||
|
onChange={ev => onChange (ev.target.value)}
|
||||||
|
aria-describedby={describedBy}
|
||||||
|
aria-invalid={invalid}
|
||||||
|
className={inputClass (invalid)}/>
|
||||||
|
<FieldWarning messages={warnings}/>
|
||||||
|
</>)}
|
||||||
|
</FormField>)
|
||||||
|
|
||||||
|
const DialogLabel = (
|
||||||
|
{ label, origin }: {
|
||||||
|
label: string
|
||||||
|
origin: PostImportOrigin },
|
||||||
|
) => (
|
||||||
|
<span className="flex items-center gap-2">
|
||||||
|
<span>{label}</span>
|
||||||
|
<PostImportStatusBadge value={origin}/>
|
||||||
|
</span>)
|
||||||
|
|
||||||
|
export default PostImportRowDialog
|
||||||
@@ -0,0 +1,160 @@
|
|||||||
|
import { Button } from '@/components/ui/button'
|
||||||
|
import { cn } from '@/lib/utils'
|
||||||
|
|
||||||
|
import PostImportStatusBadge from '@/components/posts/import/PostImportStatusBadge'
|
||||||
|
import ThumbnailPreview from '@/components/posts/import/ThumbnailPreview'
|
||||||
|
import { effectivePostImportStatus } from '@/components/posts/import/postImportRowStatus'
|
||||||
|
|
||||||
|
import type { FC } from 'react'
|
||||||
|
|
||||||
|
import type { PostImportRow } from '@/lib/postImportSession'
|
||||||
|
|
||||||
|
type Props = {
|
||||||
|
row: PostImportRow
|
||||||
|
onEdit: () => void }
|
||||||
|
|
||||||
|
const toneClass = (row: PostImportRow): string[] => {
|
||||||
|
const value = effectivePostImportStatus (row)
|
||||||
|
switch (value)
|
||||||
|
{
|
||||||
|
case 'created':
|
||||||
|
return [
|
||||||
|
'border-sky-200 bg-sky-50',
|
||||||
|
'dark:border-sky-900 dark:bg-sky-950/30']
|
||||||
|
case 'skipped':
|
||||||
|
return [
|
||||||
|
'border-stone-200 bg-stone-50',
|
||||||
|
'dark:border-stone-800 dark:bg-stone-900/60']
|
||||||
|
case 'failed':
|
||||||
|
case 'error':
|
||||||
|
return [
|
||||||
|
'border-rose-200 bg-rose-50',
|
||||||
|
'dark:border-rose-900 dark:bg-rose-950/30']
|
||||||
|
case 'warning':
|
||||||
|
return [
|
||||||
|
'border-amber-200 bg-amber-50',
|
||||||
|
'dark:border-amber-900 dark:bg-amber-950/30']
|
||||||
|
default:
|
||||||
|
return [
|
||||||
|
'border-border bg-white',
|
||||||
|
'dark:border-neutral-700 dark:bg-neutral-900']
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
const summaryError = (row: PostImportRow): string | null => {
|
||||||
|
const validation = Object.values (row.validationErrors ?? { }).flat ()[0]
|
||||||
|
if (validation)
|
||||||
|
return validation
|
||||||
|
|
||||||
|
return Object.values (row.importErrors ?? { }).flat ()[0] ?? null
|
||||||
|
}
|
||||||
|
|
||||||
|
const summaryWarning = (row: PostImportRow): string | null =>
|
||||||
|
Object.values (row.fieldWarnings ?? { }).flat ()[0]
|
||||||
|
?? row.baseWarnings?.[0]
|
||||||
|
?? null
|
||||||
|
|
||||||
|
const summaryDate = (row: PostImportRow): string =>
|
||||||
|
[row.attributes.originalCreatedFrom, row.attributes.originalCreatedBefore]
|
||||||
|
.filter (_1 => typeof _1 === 'string' && _1 !== '')
|
||||||
|
.join (' ~ ')
|
||||||
|
|
||||||
|
|
||||||
|
const PostImportRowSummary: FC<Props> = ({ row, onEdit }) => {
|
||||||
|
const error = summaryError (row)
|
||||||
|
const warning = error == null ? summaryWarning (row) : null
|
||||||
|
const effectiveStatus = effectivePostImportStatus (row)
|
||||||
|
|
||||||
|
return (
|
||||||
|
<>
|
||||||
|
<div
|
||||||
|
className={cn (
|
||||||
|
'hidden items-center gap-4 rounded-lg border p-4 md:grid',
|
||||||
|
'md:grid-cols-[4rem_5rem_minmax(0,1fr)_auto_auto]',
|
||||||
|
toneClass (row),
|
||||||
|
'transition-shadow hover:shadow-sm',
|
||||||
|
)}>
|
||||||
|
<div className="space-y-1">
|
||||||
|
<div className="text-sm font-medium">#{row.sourceRow}</div>
|
||||||
|
</div>
|
||||||
|
<ThumbnailPreview
|
||||||
|
url={String (row.attributes.thumbnailBase ?? '')}
|
||||||
|
className="h-16 w-16"/>
|
||||||
|
<div className="min-w-0 space-y-1">
|
||||||
|
<div className="line-clamp-2 text-sm font-medium">
|
||||||
|
{String (row.attributes.title ?? '') || '(タイトル未取得)'}
|
||||||
|
</div>
|
||||||
|
<div className="truncate text-xs text-neutral-600 dark:text-neutral-300">
|
||||||
|
{row.url}
|
||||||
|
</div>
|
||||||
|
<div className="truncate text-xs text-neutral-500 dark:text-neutral-400">
|
||||||
|
{String (row.attributes.tags ?? '') || 'タグなし'}
|
||||||
|
</div>
|
||||||
|
<div className="text-xs text-neutral-500 dark:text-neutral-400">
|
||||||
|
{summaryDate (row) || '日時未取得'}
|
||||||
|
{row.attributes.duration ? ` / ${ row.attributes.duration }` : ''}
|
||||||
|
</div>
|
||||||
|
{warning && (
|
||||||
|
<div className="text-xs text-amber-700 dark:text-amber-200">
|
||||||
|
{warning}
|
||||||
|
</div>)}
|
||||||
|
{error && (
|
||||||
|
<div className="text-xs text-red-700 dark:text-red-200">
|
||||||
|
{error}
|
||||||
|
</div>)}
|
||||||
|
</div>
|
||||||
|
<div className="space-y-1">
|
||||||
|
<PostImportStatusBadge value={effectiveStatus}/>
|
||||||
|
</div>
|
||||||
|
<div className="flex justify-end">
|
||||||
|
<Button
|
||||||
|
type="button"
|
||||||
|
variant="outline"
|
||||||
|
onClick={onEdit}
|
||||||
|
disabled={row.importStatus === 'created'}>
|
||||||
|
編輯
|
||||||
|
</Button>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<div
|
||||||
|
className={cn (
|
||||||
|
'space-y-3 rounded-lg border p-4 md:hidden',
|
||||||
|
toneClass (row),
|
||||||
|
)}>
|
||||||
|
<div className="flex items-start gap-3">
|
||||||
|
<ThumbnailPreview
|
||||||
|
url={String (row.attributes.thumbnailBase ?? '')}
|
||||||
|
className="h-20 w-20 shrink-0"/>
|
||||||
|
<div className="min-w-0 flex-1 space-y-2">
|
||||||
|
<div className="line-clamp-2 text-sm font-medium">
|
||||||
|
{String (row.attributes.title ?? '') || '(タイトル未取得)'}
|
||||||
|
</div>
|
||||||
|
<div className="truncate text-xs text-neutral-600 dark:text-neutral-300">
|
||||||
|
{row.url}
|
||||||
|
</div>
|
||||||
|
<div className="flex flex-wrap gap-2">
|
||||||
|
<PostImportStatusBadge value={effectiveStatus}/>
|
||||||
|
</div>
|
||||||
|
{error && (
|
||||||
|
<div className="text-xs text-red-700 dark:text-red-200">
|
||||||
|
{error}
|
||||||
|
</div>)}
|
||||||
|
{warning && (
|
||||||
|
<div className="text-xs text-amber-700 dark:text-amber-200">
|
||||||
|
{warning}
|
||||||
|
</div>)}
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
<Button
|
||||||
|
type="button"
|
||||||
|
variant="outline"
|
||||||
|
onClick={onEdit}
|
||||||
|
disabled={row.importStatus === 'created'}>
|
||||||
|
編輯
|
||||||
|
</Button>
|
||||||
|
</div>
|
||||||
|
</>)
|
||||||
|
}
|
||||||
|
|
||||||
|
export default PostImportRowSummary
|
||||||
@@ -0,0 +1,60 @@
|
|||||||
|
import { cn } from '@/lib/utils'
|
||||||
|
|
||||||
|
import type { FC } from 'react'
|
||||||
|
|
||||||
|
import type { PostImportBadgeValue } from '@/components/posts/import/postImportRowStatus'
|
||||||
|
|
||||||
|
type Props = {
|
||||||
|
value: PostImportBadgeValue }
|
||||||
|
|
||||||
|
const LABELS: Record<PostImportBadgeValue, string> = {
|
||||||
|
ready: '登録可能',
|
||||||
|
warning: '警告',
|
||||||
|
error: '要修正',
|
||||||
|
pending: '未登録',
|
||||||
|
created: '登録済み',
|
||||||
|
skipped: 'スキップ',
|
||||||
|
failed: '失敗',
|
||||||
|
automatic: '自動取得',
|
||||||
|
manual: '手修正' }
|
||||||
|
|
||||||
|
const STYLES: Record<PostImportBadgeValue, string[]> = {
|
||||||
|
ready: [
|
||||||
|
'border-emerald-300 bg-emerald-50 text-emerald-700',
|
||||||
|
'dark:border-emerald-900 dark:bg-emerald-950 dark:text-emerald-200'],
|
||||||
|
warning: [
|
||||||
|
'border-amber-300 bg-amber-50 text-amber-700',
|
||||||
|
'dark:border-amber-900 dark:bg-amber-950 dark:text-amber-200'],
|
||||||
|
error: [
|
||||||
|
'border-red-300 bg-red-50 text-red-700',
|
||||||
|
'dark:border-red-900 dark:bg-red-950 dark:text-red-200'],
|
||||||
|
pending: [
|
||||||
|
'border-slate-300 bg-slate-50 text-slate-700',
|
||||||
|
'dark:border-slate-700 dark:bg-slate-900 dark:text-slate-200'],
|
||||||
|
created: [
|
||||||
|
'border-sky-300 bg-sky-50 text-sky-700',
|
||||||
|
'dark:border-sky-900 dark:bg-sky-950 dark:text-sky-200'],
|
||||||
|
skipped: [
|
||||||
|
'border-stone-300 bg-stone-50 text-stone-700',
|
||||||
|
'dark:border-stone-700 dark:bg-stone-900 dark:text-stone-200'],
|
||||||
|
failed: [
|
||||||
|
'border-rose-300 bg-rose-50 text-rose-700',
|
||||||
|
'dark:border-rose-900 dark:bg-rose-950 dark:text-rose-200'],
|
||||||
|
automatic: [
|
||||||
|
'border-slate-300 bg-slate-50 text-slate-700',
|
||||||
|
'dark:border-slate-700 dark:bg-slate-900 dark:text-slate-200'],
|
||||||
|
manual: [
|
||||||
|
'border-purple-300 bg-purple-50 text-purple-700',
|
||||||
|
'dark:border-purple-900 dark:bg-purple-950 dark:text-purple-200'] }
|
||||||
|
|
||||||
|
|
||||||
|
const PostImportStatusBadge: FC<Props> = ({ value }) => (
|
||||||
|
<span
|
||||||
|
className={cn (
|
||||||
|
'inline-flex items-center rounded-full border px-2.5 py-1 text-xs font-medium',
|
||||||
|
STYLES[value],
|
||||||
|
)}>
|
||||||
|
{LABELS[value]}
|
||||||
|
</span>)
|
||||||
|
|
||||||
|
export default PostImportStatusBadge
|
||||||
@@ -0,0 +1,49 @@
|
|||||||
|
import { useEffect, useState } from 'react'
|
||||||
|
|
||||||
|
import type { FC } from 'react'
|
||||||
|
|
||||||
|
|
||||||
|
type Props = {
|
||||||
|
url: string
|
||||||
|
alt?: string
|
||||||
|
className?: string }
|
||||||
|
|
||||||
|
|
||||||
|
const ThumbnailPreview: FC<Props> = ({ url, alt = 'サムネール', className = 'h-16 w-16' }) => {
|
||||||
|
const [failed, setFailed] = useState (false)
|
||||||
|
|
||||||
|
useEffect (() => {
|
||||||
|
setFailed (false)
|
||||||
|
}, [url])
|
||||||
|
|
||||||
|
if (!(url))
|
||||||
|
{
|
||||||
|
return (
|
||||||
|
<div
|
||||||
|
className={`${ className } flex items-center justify-center rounded border
|
||||||
|
border-border bg-muted text-xs text-muted-foreground`}>
|
||||||
|
なし
|
||||||
|
</div>)
|
||||||
|
}
|
||||||
|
|
||||||
|
if (failed)
|
||||||
|
{
|
||||||
|
return (
|
||||||
|
<div
|
||||||
|
className={`${ className } flex items-center justify-center rounded border
|
||||||
|
border-amber-300 bg-amber-50 p-2 text-center text-xs
|
||||||
|
text-amber-700 dark:border-amber-900 dark:bg-amber-950
|
||||||
|
dark:text-amber-200`}>
|
||||||
|
サムネールを表示できません
|
||||||
|
</div>)
|
||||||
|
}
|
||||||
|
|
||||||
|
return (
|
||||||
|
<img
|
||||||
|
src={url}
|
||||||
|
alt={alt}
|
||||||
|
className={`${ className } rounded border border-border object-cover`}
|
||||||
|
onError={() => setFailed (true)}/>)
|
||||||
|
}
|
||||||
|
|
||||||
|
export default ThumbnailPreview
|
||||||
@@ -0,0 +1,33 @@
|
|||||||
|
import type { PostImportOrigin,
|
||||||
|
PostImportRow } from '@/lib/postImportSession'
|
||||||
|
|
||||||
|
export type PostImportEffectiveStatus =
|
||||||
|
'created'
|
||||||
|
| 'skipped'
|
||||||
|
| 'failed'
|
||||||
|
| 'error'
|
||||||
|
| 'warning'
|
||||||
|
| 'ready'
|
||||||
|
|
||||||
|
export type PostImportBadgeValue =
|
||||||
|
PostImportEffectiveStatus
|
||||||
|
| 'pending'
|
||||||
|
| PostImportOrigin
|
||||||
|
|
||||||
|
|
||||||
|
export const effectivePostImportStatus = (
|
||||||
|
row: PostImportRow,
|
||||||
|
): PostImportEffectiveStatus => {
|
||||||
|
if (Object.keys (row.validationErrors ?? { }).length > 0)
|
||||||
|
return 'error'
|
||||||
|
|
||||||
|
switch (row.importStatus)
|
||||||
|
{
|
||||||
|
case 'created':
|
||||||
|
case 'skipped':
|
||||||
|
case 'failed':
|
||||||
|
return row.importStatus
|
||||||
|
default:
|
||||||
|
return row.status
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -239,6 +239,11 @@ body
|
|||||||
background: var(--top-nav-submenu-bg);
|
background: var(--top-nav-submenu-bg);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
.top-nav-mobile-menu .top-nav-submenu
|
||||||
|
{
|
||||||
|
background: var(--top-nav-mobile-active-bg);
|
||||||
|
}
|
||||||
|
|
||||||
.top-nav-mobile-menu
|
.top-nav-mobile-menu
|
||||||
{
|
{
|
||||||
background: var(--top-nav-mobile-menu-bg);
|
background: var(--top-nav-mobile-menu-bg);
|
||||||
|
|||||||
@@ -0,0 +1,22 @@
|
|||||||
|
import { readFileSync } from 'node:fs'
|
||||||
|
import { resolve } from 'node:path'
|
||||||
|
|
||||||
|
import { describe, expect, it } from 'vitest'
|
||||||
|
|
||||||
|
|
||||||
|
const indexCss = readFileSync (
|
||||||
|
resolve (process.cwd (), 'src/index.css'),
|
||||||
|
'utf8')
|
||||||
|
const compactIndexCss = indexCss.replace (/\s+/g, ' ')
|
||||||
|
const mobileActiveRule = '.top-nav-mobile-active {'
|
||||||
|
+ ' background: var(--top-nav-mobile-active-bg); }'
|
||||||
|
const mobileSubmenuRule = '.top-nav-mobile-menu .top-nav-submenu {'
|
||||||
|
+ ' background: var(--top-nav-mobile-active-bg); }'
|
||||||
|
|
||||||
|
|
||||||
|
describe ('TopNav mobile menu colours', () => {
|
||||||
|
it ('uses one background variable for active rows and expanded submenus', () => {
|
||||||
|
expect (compactIndexCss).toContain (mobileActiveRule)
|
||||||
|
expect (compactIndexCss).toContain (mobileSubmenuRule)
|
||||||
|
})
|
||||||
|
})
|
||||||
@@ -0,0 +1,216 @@
|
|||||||
|
import type { PostImportResultRow,
|
||||||
|
PostImportRow } from '@/lib/postImportTypes'
|
||||||
|
|
||||||
|
const hasSkipReason = (row: PostImportRow): boolean =>
|
||||||
|
row.skipReason === 'existing'
|
||||||
|
|
||||||
|
const hasValidationErrors = (row: PostImportRow): boolean =>
|
||||||
|
Object.keys (row.validationErrors ?? { }).length > 0
|
||||||
|
|
||||||
|
const rowChanged = (
|
||||||
|
previous: PostImportRow,
|
||||||
|
next: PostImportRow,
|
||||||
|
): boolean =>
|
||||||
|
previous.url !== next.url
|
||||||
|
|| JSON.stringify (previous.attributes) !== JSON.stringify (next.attributes)
|
||||||
|
|| JSON.stringify (previous.provenance) !== JSON.stringify (next.provenance)
|
||||||
|
|| JSON.stringify (previous.tagSources ?? { }) !== JSON.stringify (next.tagSources ?? { })
|
||||||
|
|
||||||
|
|
||||||
|
export const processableImportRows = (rows: PostImportRow[]): PostImportRow[] =>
|
||||||
|
rows.filter (row => {
|
||||||
|
if (row.importStatus === 'created')
|
||||||
|
return false
|
||||||
|
if (row.importStatus === 'skipped')
|
||||||
|
return false
|
||||||
|
if (row.importStatus === 'failed')
|
||||||
|
return false
|
||||||
|
if (hasValidationErrors (row))
|
||||||
|
return false
|
||||||
|
return row.importStatus == null || row.importStatus === 'pending'
|
||||||
|
})
|
||||||
|
|
||||||
|
export const creatableImportRows = (rows: PostImportRow[]): PostImportRow[] =>
|
||||||
|
processableImportRows (rows).filter (row => !(hasSkipReason (row)))
|
||||||
|
|
||||||
|
|
||||||
|
export const reviewSummaryCounts = (rows: PostImportRow[]) => ({
|
||||||
|
total: rows.length,
|
||||||
|
submittable: rows.filter (row =>
|
||||||
|
creatableImportRows ([row]).length > 0
|
||||||
|
&& !(hasValidationErrors (row))).length,
|
||||||
|
invalid: rows.filter (row => hasValidationErrors (row)).length,
|
||||||
|
skipPlanned: rows.filter (row =>
|
||||||
|
hasSkipReason (row)
|
||||||
|
&& !(hasValidationErrors (row))
|
||||||
|
&& row.importStatus !== 'created').length,
|
||||||
|
created: rows.filter (row => row.importStatus === 'created').length,
|
||||||
|
failed: rows.filter (row => row.importStatus === 'failed').length })
|
||||||
|
|
||||||
|
|
||||||
|
export const resultSummaryCounts = (rows: PostImportRow[]) =>
|
||||||
|
rows.reduce (
|
||||||
|
(counts, row) => {
|
||||||
|
if (hasValidationErrors (row))
|
||||||
|
{
|
||||||
|
++counts.invalid
|
||||||
|
return counts
|
||||||
|
}
|
||||||
|
if (row.importStatus === 'created')
|
||||||
|
{
|
||||||
|
++counts.created
|
||||||
|
return counts
|
||||||
|
}
|
||||||
|
if (row.importStatus === 'skipped')
|
||||||
|
{
|
||||||
|
++counts.skipped
|
||||||
|
return counts
|
||||||
|
}
|
||||||
|
if (row.importStatus === 'failed')
|
||||||
|
++counts.failed
|
||||||
|
return counts
|
||||||
|
},
|
||||||
|
{ created: 0, skipped: 0, failed: 0, invalid: 0 })
|
||||||
|
|
||||||
|
|
||||||
|
export const mergeValidatedImportRows = (
|
||||||
|
current: PostImportRow[],
|
||||||
|
validated: PostImportRow[],
|
||||||
|
): PostImportRow[] => {
|
||||||
|
const validatedMap = new Map (validated.map (row => [row.sourceRow, row]))
|
||||||
|
return current.map (previous => {
|
||||||
|
if (previous.importStatus === 'created')
|
||||||
|
return previous
|
||||||
|
|
||||||
|
const row = validatedMap.get (previous.sourceRow)
|
||||||
|
if (row == null)
|
||||||
|
return previous
|
||||||
|
|
||||||
|
const fieldWarnings =
|
||||||
|
Object.keys (row.fieldWarnings).length > 0 || row.metadataUrl !== previous.metadataUrl
|
||||||
|
? { ...row.fieldWarnings }
|
||||||
|
: { ...previous.fieldWarnings }
|
||||||
|
for (const [field, origin] of Object.entries (row.provenance))
|
||||||
|
{
|
||||||
|
if (origin === 'manual')
|
||||||
|
delete fieldWarnings[field]
|
||||||
|
}
|
||||||
|
|
||||||
|
return {
|
||||||
|
...previous,
|
||||||
|
url: row.url,
|
||||||
|
attributes: row.attributes,
|
||||||
|
provenance: row.provenance,
|
||||||
|
tagSources: row.tagSources,
|
||||||
|
skipReason: row.skipReason,
|
||||||
|
existingPostId: row.existingPostId,
|
||||||
|
fieldWarnings,
|
||||||
|
baseWarnings:
|
||||||
|
row.baseWarnings.length > 0 || row.metadataUrl !== previous.metadataUrl
|
||||||
|
? row.baseWarnings
|
||||||
|
: previous.baseWarnings,
|
||||||
|
validationErrors: row.validationErrors,
|
||||||
|
status: row.status,
|
||||||
|
metadataUrl: row.metadataUrl }
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
export const mergePreviewImportRows = (
|
||||||
|
current: PostImportRow[],
|
||||||
|
preview: PostImportRow[],
|
||||||
|
): PostImportRow[] => {
|
||||||
|
const currentMap = new Map (current.map (row => [row.sourceRow, row]))
|
||||||
|
return preview.map (row => {
|
||||||
|
const previous = currentMap.get (row.sourceRow)
|
||||||
|
if (previous == null)
|
||||||
|
return row
|
||||||
|
if (previous.importStatus === 'created')
|
||||||
|
return previous
|
||||||
|
|
||||||
|
const mergedAttributes = { ...row.attributes }
|
||||||
|
const mergedProvenance = { ...row.provenance }
|
||||||
|
|
||||||
|
for (const [field, origin] of Object.entries (previous.provenance))
|
||||||
|
{
|
||||||
|
if (origin !== 'manual' || field === 'url')
|
||||||
|
continue
|
||||||
|
if (field in previous.attributes)
|
||||||
|
mergedAttributes[field] = previous.attributes[field]
|
||||||
|
mergedProvenance[field] = 'manual'
|
||||||
|
}
|
||||||
|
|
||||||
|
const mergedTagSources =
|
||||||
|
previous.provenance.tags === 'manual'
|
||||||
|
? {
|
||||||
|
automatic: row.tagSources?.automatic ?? '',
|
||||||
|
manual: previous.tagSources?.manual ?? String (previous.attributes.tags ?? '') }
|
||||||
|
: row.tagSources
|
||||||
|
|
||||||
|
const nextRow = {
|
||||||
|
...row,
|
||||||
|
attributes: mergedAttributes,
|
||||||
|
provenance: mergedProvenance,
|
||||||
|
tagSources: mergedTagSources,
|
||||||
|
skipReason: row.skipReason,
|
||||||
|
existingPostId: row.existingPostId,
|
||||||
|
createdPostId: previous.createdPostId,
|
||||||
|
importStatus: previous.importStatus,
|
||||||
|
importErrors: previous.importErrors }
|
||||||
|
|
||||||
|
if (rowChanged (previous, nextRow))
|
||||||
|
{
|
||||||
|
nextRow.importStatus = 'pending'
|
||||||
|
nextRow.importErrors = undefined
|
||||||
|
}
|
||||||
|
return nextRow
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
export const mergeImportResults = (
|
||||||
|
rows: PostImportRow[],
|
||||||
|
results: PostImportResultRow[],
|
||||||
|
): PostImportRow[] => {
|
||||||
|
const resultMap = new Map (results.map (row => [row.sourceRow, row]))
|
||||||
|
return rows.map (row => {
|
||||||
|
const result = resultMap.get (row.sourceRow)
|
||||||
|
if (result == null)
|
||||||
|
return row
|
||||||
|
|
||||||
|
switch (result.status)
|
||||||
|
{
|
||||||
|
case 'created':
|
||||||
|
return {
|
||||||
|
...row,
|
||||||
|
importStatus: 'created',
|
||||||
|
createdPostId: result.post?.id,
|
||||||
|
existingPostId: undefined,
|
||||||
|
importErrors: result.errors }
|
||||||
|
case 'skipped':
|
||||||
|
return {
|
||||||
|
...row,
|
||||||
|
importStatus: 'skipped',
|
||||||
|
createdPostId: undefined,
|
||||||
|
existingPostId: result.existingPostId,
|
||||||
|
importErrors: result.errors }
|
||||||
|
case 'failed':
|
||||||
|
return {
|
||||||
|
...row,
|
||||||
|
importStatus: 'failed',
|
||||||
|
createdPostId: undefined,
|
||||||
|
existingPostId: undefined,
|
||||||
|
importErrors: result.errors }
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
export const retryImportRow = (
|
||||||
|
rows: PostImportRow[],
|
||||||
|
sourceRow: number,
|
||||||
|
): PostImportRow[] =>
|
||||||
|
rows.map (row =>
|
||||||
|
row.sourceRow === sourceRow && row.importStatus === 'failed'
|
||||||
|
? { ...row, importStatus: 'pending', importErrors: undefined }
|
||||||
|
: row)
|
||||||
@@ -0,0 +1,4 @@
|
|||||||
|
export * from '@/lib/postImportTypes'
|
||||||
|
export * from '@/lib/postImportStorage'
|
||||||
|
export * from '@/lib/postImportSourceValidation'
|
||||||
|
export * from '@/lib/postImportRows'
|
||||||
@@ -0,0 +1,110 @@
|
|||||||
|
import type { PostImportSourceIssue } from '@/lib/postImportTypes'
|
||||||
|
|
||||||
|
const MAX_ROWS = 100
|
||||||
|
const MAX_URL_BYTES = 20 * 1024
|
||||||
|
|
||||||
|
|
||||||
|
const truncateUrl = (value: string): string =>
|
||||||
|
value.length > 120 ? `${ value.slice (0, 117) }…` : value
|
||||||
|
|
||||||
|
|
||||||
|
const bytesize = (value: string): number =>
|
||||||
|
new TextEncoder ().encode (value).length
|
||||||
|
|
||||||
|
|
||||||
|
const normaliseImportUrl = (value: string): string | null => {
|
||||||
|
const trimmed = value.trim ()
|
||||||
|
if (!(trimmed))
|
||||||
|
return null
|
||||||
|
|
||||||
|
try
|
||||||
|
{
|
||||||
|
const url = new URL (trimmed)
|
||||||
|
if (!(url.protocol === 'http:' || url.protocol === 'https:'))
|
||||||
|
return null
|
||||||
|
if (!(url.host))
|
||||||
|
return null
|
||||||
|
|
||||||
|
url.hostname = url.hostname.toLowerCase ()
|
||||||
|
if (url.pathname.endsWith ('/'))
|
||||||
|
url.pathname = url.pathname.replace (/\/+$/, '')
|
||||||
|
return url.toString ()
|
||||||
|
}
|
||||||
|
catch
|
||||||
|
{
|
||||||
|
return null
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
export const countImportSourceLines = (source: string): number =>
|
||||||
|
source
|
||||||
|
.split (/\r\n|\n|\r/)
|
||||||
|
.map (_1 => _1.trim ())
|
||||||
|
.filter (_1 => _1 !== '')
|
||||||
|
.length
|
||||||
|
|
||||||
|
|
||||||
|
export const validateImportSource = (
|
||||||
|
source: string,
|
||||||
|
): PostImportSourceIssue[] => {
|
||||||
|
const lines = source.split (/\r\n|\n|\r/)
|
||||||
|
const issues: PostImportSourceIssue[] = []
|
||||||
|
const seen = new Map<string, number> ()
|
||||||
|
let count = 0
|
||||||
|
|
||||||
|
lines.forEach ((rawLine, index) => {
|
||||||
|
const value = rawLine.trim ()
|
||||||
|
if (!(value))
|
||||||
|
return
|
||||||
|
|
||||||
|
++count
|
||||||
|
const sourceRow = index + 1
|
||||||
|
const displayUrl = truncateUrl (value)
|
||||||
|
const normalised = normaliseImportUrl (value)
|
||||||
|
if (count > MAX_ROWS)
|
||||||
|
{
|
||||||
|
issues.push ({
|
||||||
|
sourceRow,
|
||||||
|
message: `取込件数は ${ MAX_ROWS } 件までです.`,
|
||||||
|
url: displayUrl })
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if (!(value.startsWith ('http://') || value.startsWith ('https://')))
|
||||||
|
{
|
||||||
|
issues.push ({
|
||||||
|
sourceRow,
|
||||||
|
message: 'HTTP または HTTPS の URL ではありません.',
|
||||||
|
url: displayUrl })
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if (bytesize (value) > MAX_URL_BYTES)
|
||||||
|
{
|
||||||
|
issues.push ({
|
||||||
|
sourceRow,
|
||||||
|
message: 'URL が長すぎます.',
|
||||||
|
url: displayUrl })
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if (normalised == null)
|
||||||
|
{
|
||||||
|
issues.push ({
|
||||||
|
sourceRow,
|
||||||
|
message: 'URL の形式が不正です.',
|
||||||
|
url: displayUrl })
|
||||||
|
return
|
||||||
|
}
|
||||||
|
const duplicateRow = seen.get (normalised)
|
||||||
|
if (duplicateRow != null)
|
||||||
|
{
|
||||||
|
issues.push ({
|
||||||
|
sourceRow,
|
||||||
|
message: `${ duplicateRow } 行目と同じ URL です.`,
|
||||||
|
url: displayUrl })
|
||||||
|
return
|
||||||
|
}
|
||||||
|
seen.set (normalised, sourceRow)
|
||||||
|
})
|
||||||
|
|
||||||
|
return issues
|
||||||
|
}
|
||||||
@@ -0,0 +1,357 @@
|
|||||||
|
import type { PostImportOrigin,
|
||||||
|
PostImportRow,
|
||||||
|
PostImportSession,
|
||||||
|
PostImportStatus,
|
||||||
|
PostImportSkipReason,
|
||||||
|
StorageErrorHandler } from '@/lib/postImportTypes'
|
||||||
|
|
||||||
|
const SESSION_VERSION = 2
|
||||||
|
const SESSION_PREFIX = 'post-import-session:'
|
||||||
|
const SOURCE_DRAFT_KEY = 'post-import-source-draft'
|
||||||
|
const SESSION_MAX_AGE_MS = 24 * 60 * 60 * 1000
|
||||||
|
const ATTRIBUTE_KEYS = [
|
||||||
|
'title',
|
||||||
|
'thumbnailBase',
|
||||||
|
'originalCreatedFrom',
|
||||||
|
'originalCreatedBefore',
|
||||||
|
'duration',
|
||||||
|
'videoMs',
|
||||||
|
'tags',
|
||||||
|
'parentPostIds'] as const
|
||||||
|
const PROVENANCE_KEYS = [...ATTRIBUTE_KEYS, 'url'] as const
|
||||||
|
const TAG_SOURCE_KEYS = ['automatic', 'manual'] as const
|
||||||
|
|
||||||
|
|
||||||
|
const isPlainObject = (value: unknown): value is Record<string, unknown> =>
|
||||||
|
typeof value === 'object' && value != null && !(Array.isArray (value))
|
||||||
|
|
||||||
|
|
||||||
|
const sessionKey = (sessionId: string): string => `${ SESSION_PREFIX }${ sessionId }`
|
||||||
|
|
||||||
|
|
||||||
|
const readStorage = (
|
||||||
|
key: string,
|
||||||
|
onError?: StorageErrorHandler,
|
||||||
|
): string | null => {
|
||||||
|
if (typeof window === 'undefined')
|
||||||
|
return null
|
||||||
|
|
||||||
|
try
|
||||||
|
{
|
||||||
|
return sessionStorage.getItem (key)
|
||||||
|
}
|
||||||
|
catch
|
||||||
|
{
|
||||||
|
onError?.('保存済みデータを読み込めませんでした.')
|
||||||
|
return null
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
const writeStorage = (
|
||||||
|
key: string,
|
||||||
|
value: string,
|
||||||
|
onError?: StorageErrorHandler,
|
||||||
|
): boolean => {
|
||||||
|
if (typeof window === 'undefined')
|
||||||
|
return false
|
||||||
|
|
||||||
|
try
|
||||||
|
{
|
||||||
|
sessionStorage.setItem (key, value)
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
catch
|
||||||
|
{
|
||||||
|
onError?.('ブラウザへ保存できませんでした.')
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
const removeStorage = (
|
||||||
|
key: string,
|
||||||
|
onError?: StorageErrorHandler,
|
||||||
|
) => {
|
||||||
|
if (typeof window === 'undefined')
|
||||||
|
return
|
||||||
|
|
||||||
|
try
|
||||||
|
{
|
||||||
|
sessionStorage.removeItem (key)
|
||||||
|
}
|
||||||
|
catch
|
||||||
|
{
|
||||||
|
onError?.('保存済みデータを削除できませんでした.')
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
const ensureStringListRecord = (value: unknown): Record<string, string[]> | null => {
|
||||||
|
if (!(isPlainObject (value)))
|
||||||
|
return null
|
||||||
|
|
||||||
|
const result: Record<string, string[]> = { }
|
||||||
|
for (const [key, entry] of Object.entries (value))
|
||||||
|
{
|
||||||
|
if (!(Array.isArray (entry)) || !(entry.every (_1 => typeof _1 === 'string')))
|
||||||
|
return null
|
||||||
|
result[key] = entry
|
||||||
|
}
|
||||||
|
return result
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
const isValidStatus = (
|
||||||
|
value: unknown,
|
||||||
|
): value is PostImportRow['status'] =>
|
||||||
|
value === 'ready' || value === 'warning' || value === 'error'
|
||||||
|
|
||||||
|
|
||||||
|
const isValidImportStatus = (
|
||||||
|
value: unknown,
|
||||||
|
): value is PostImportStatus =>
|
||||||
|
value === 'pending'
|
||||||
|
|| value === 'created'
|
||||||
|
|| value === 'skipped'
|
||||||
|
|| value === 'failed'
|
||||||
|
|
||||||
|
|
||||||
|
const isValidOrigin = (
|
||||||
|
value: unknown,
|
||||||
|
): value is PostImportOrigin =>
|
||||||
|
value === 'automatic' || value === 'manual'
|
||||||
|
|
||||||
|
|
||||||
|
const isValidSkipReason = (
|
||||||
|
value: unknown,
|
||||||
|
): value is PostImportSkipReason =>
|
||||||
|
value === 'existing'
|
||||||
|
|
||||||
|
const isPositiveInteger = (value: unknown): value is number =>
|
||||||
|
Number.isInteger (value) && Number (value) > 0
|
||||||
|
|
||||||
|
const hasOnlyKeys = (
|
||||||
|
value: Record<string, unknown>,
|
||||||
|
allowedKeys: readonly string[],
|
||||||
|
): boolean =>
|
||||||
|
Object.keys (value).every (key => allowedKeys.includes (key))
|
||||||
|
|
||||||
|
|
||||||
|
const sanitiseRow = (value: unknown): PostImportRow | null => {
|
||||||
|
if (!(isPlainObject (value)))
|
||||||
|
return null
|
||||||
|
if (!(Number.isInteger (value.sourceRow)) || Number (value.sourceRow) <= 0)
|
||||||
|
return null
|
||||||
|
if (typeof value.url !== 'string')
|
||||||
|
return null
|
||||||
|
if (!(isPlainObject (value.attributes)))
|
||||||
|
return null
|
||||||
|
if (!(isPlainObject (value.provenance)))
|
||||||
|
return null
|
||||||
|
if (!(isValidStatus (value.status)))
|
||||||
|
return null
|
||||||
|
if (value.importStatus != null && !(isValidImportStatus (value.importStatus)))
|
||||||
|
return null
|
||||||
|
if (value.skipReason != null && !(isValidSkipReason (value.skipReason)))
|
||||||
|
return null
|
||||||
|
if (value.skipReason === 'existing' && !(isPositiveInteger (value.existingPostId)))
|
||||||
|
return null
|
||||||
|
if (value.skipReason !== 'existing' && value.existingPostId != null)
|
||||||
|
return null
|
||||||
|
if (value.importStatus === 'created' && !(isPositiveInteger (value.createdPostId)))
|
||||||
|
return null
|
||||||
|
if (value.importStatus !== 'created' && value.createdPostId != null)
|
||||||
|
return null
|
||||||
|
|
||||||
|
const validationErrors = ensureStringListRecord (value.validationErrors)
|
||||||
|
const fieldWarnings = ensureStringListRecord (value.fieldWarnings)
|
||||||
|
if (validationErrors == null || fieldWarnings == null)
|
||||||
|
return null
|
||||||
|
|
||||||
|
const importErrors =
|
||||||
|
value.importErrors == null
|
||||||
|
? undefined
|
||||||
|
: ensureStringListRecord (value.importErrors)
|
||||||
|
if (value.importErrors != null && importErrors == null)
|
||||||
|
return null
|
||||||
|
if (!(Array.isArray (value.baseWarnings))
|
||||||
|
|| !(value.baseWarnings.every (_1 => typeof _1 === 'string')))
|
||||||
|
return null
|
||||||
|
|
||||||
|
const provenanceEntries = Object.entries (value.provenance)
|
||||||
|
if (!(hasOnlyKeys (value.attributes, ATTRIBUTE_KEYS)))
|
||||||
|
return null
|
||||||
|
if (!(Object.values (value.attributes).every (entry =>
|
||||||
|
typeof entry === 'string' || typeof entry === 'number')))
|
||||||
|
return null
|
||||||
|
if (!(hasOnlyKeys (value.provenance, PROVENANCE_KEYS)))
|
||||||
|
return null
|
||||||
|
if (!(provenanceEntries.every (([, origin]) => isValidOrigin (origin))))
|
||||||
|
return null
|
||||||
|
|
||||||
|
if (value.tagSources != null)
|
||||||
|
{
|
||||||
|
if (!(isPlainObject (value.tagSources)))
|
||||||
|
return null
|
||||||
|
if (!(hasOnlyKeys (value.tagSources, TAG_SOURCE_KEYS)))
|
||||||
|
return null
|
||||||
|
if (!(Object.values (value.tagSources).every (_1 => typeof _1 === 'string')))
|
||||||
|
return null
|
||||||
|
}
|
||||||
|
|
||||||
|
return {
|
||||||
|
sourceRow: Number (value.sourceRow),
|
||||||
|
url: value.url,
|
||||||
|
attributes: value.attributes as Record<string, string | number>,
|
||||||
|
fieldWarnings,
|
||||||
|
baseWarnings: value.baseWarnings,
|
||||||
|
validationErrors,
|
||||||
|
importErrors,
|
||||||
|
provenance: value.provenance as Record<string, PostImportOrigin>,
|
||||||
|
tagSources: value.tagSources as Record<PostImportOrigin, string> | undefined,
|
||||||
|
status: value.status,
|
||||||
|
skipReason: value.skipReason,
|
||||||
|
existingPostId:
|
||||||
|
isPositiveInteger (value.existingPostId) ? Number (value.existingPostId) : undefined,
|
||||||
|
metadataUrl: typeof value.metadataUrl === 'string' ? value.metadataUrl : undefined,
|
||||||
|
createdPostId:
|
||||||
|
isPositiveInteger (value.createdPostId) ? Number (value.createdPostId) : undefined,
|
||||||
|
importStatus: value.importStatus }
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
const isExpiredSession = (savedAt: string): boolean => {
|
||||||
|
const value = Date.parse (savedAt)
|
||||||
|
return Number.isNaN (value) || Date.now () - value > SESSION_MAX_AGE_MS
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
export const createPostImportSessionId = (): string =>
|
||||||
|
typeof crypto !== 'undefined' && 'randomUUID' in crypto
|
||||||
|
? crypto.randomUUID ()
|
||||||
|
: `${ Date.now () }-${ Math.random ().toString (36).slice (2) }`
|
||||||
|
|
||||||
|
|
||||||
|
export const cleanupExpiredPostImportSessions = (
|
||||||
|
onError?: StorageErrorHandler,
|
||||||
|
) => {
|
||||||
|
if (typeof window === 'undefined')
|
||||||
|
return
|
||||||
|
|
||||||
|
try
|
||||||
|
{
|
||||||
|
for (let i = 0; i < sessionStorage.length; ++i)
|
||||||
|
{
|
||||||
|
const key = sessionStorage.key (i)
|
||||||
|
if (key == null || !(key.startsWith (SESSION_PREFIX)))
|
||||||
|
continue
|
||||||
|
|
||||||
|
const raw = sessionStorage.getItem (key)
|
||||||
|
if (raw == null)
|
||||||
|
continue
|
||||||
|
try
|
||||||
|
{
|
||||||
|
const value = JSON.parse (raw) as { savedAt?: string }
|
||||||
|
if (typeof value.savedAt !== 'string' || isExpiredSession (value.savedAt))
|
||||||
|
{
|
||||||
|
sessionStorage.removeItem (key)
|
||||||
|
--i
|
||||||
|
}
|
||||||
|
}
|
||||||
|
catch
|
||||||
|
{
|
||||||
|
sessionStorage.removeItem (key)
|
||||||
|
--i
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
catch
|
||||||
|
{
|
||||||
|
onError?.('保存済みデータを整理できませんでした.')
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
export const loadPostImportSourceDraft = (
|
||||||
|
onError?: StorageErrorHandler,
|
||||||
|
): { source: string } => {
|
||||||
|
const raw = readStorage (SOURCE_DRAFT_KEY, onError)
|
||||||
|
if (raw == null)
|
||||||
|
return { source: '' }
|
||||||
|
|
||||||
|
try
|
||||||
|
{
|
||||||
|
const value = JSON.parse (raw) as { source?: string }
|
||||||
|
return { source: typeof value.source === 'string' ? value.source : '' }
|
||||||
|
}
|
||||||
|
catch
|
||||||
|
{
|
||||||
|
return { source: '' }
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
export const savePostImportSourceDraft = (
|
||||||
|
source: string,
|
||||||
|
onError?: StorageErrorHandler,
|
||||||
|
): boolean =>
|
||||||
|
writeStorage (SOURCE_DRAFT_KEY, JSON.stringify ({ source }), onError)
|
||||||
|
|
||||||
|
|
||||||
|
export const clearPostImportSourceDraft = (
|
||||||
|
onError?: StorageErrorHandler,
|
||||||
|
) => {
|
||||||
|
removeStorage (SOURCE_DRAFT_KEY, onError)
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
export const savePostImportSession = (
|
||||||
|
sessionId: string,
|
||||||
|
session: Omit<PostImportSession, 'version' | 'savedAt'>,
|
||||||
|
onError?: StorageErrorHandler,
|
||||||
|
): boolean =>
|
||||||
|
writeStorage (
|
||||||
|
sessionKey (sessionId),
|
||||||
|
JSON.stringify ({
|
||||||
|
...session,
|
||||||
|
version: SESSION_VERSION,
|
||||||
|
savedAt: new Date ().toISOString () }),
|
||||||
|
onError)
|
||||||
|
|
||||||
|
|
||||||
|
export const loadPostImportSession = (
|
||||||
|
sessionId: string,
|
||||||
|
onError?: StorageErrorHandler,
|
||||||
|
): PostImportSession | null => {
|
||||||
|
const raw = readStorage (sessionKey (sessionId), onError)
|
||||||
|
if (raw == null)
|
||||||
|
return null
|
||||||
|
|
||||||
|
try
|
||||||
|
{
|
||||||
|
const value = JSON.parse (raw) as Partial<PostImportSession>
|
||||||
|
if (value.version !== SESSION_VERSION || !(Array.isArray (value.rows)))
|
||||||
|
return null
|
||||||
|
if (typeof value.savedAt !== 'string' || isExpiredSession (value.savedAt))
|
||||||
|
{
|
||||||
|
removeStorage (sessionKey (sessionId), onError)
|
||||||
|
return null
|
||||||
|
}
|
||||||
|
|
||||||
|
const rows = value.rows.map (sanitiseRow)
|
||||||
|
if (rows.some (_1 => _1 == null))
|
||||||
|
return null
|
||||||
|
|
||||||
|
return {
|
||||||
|
version: SESSION_VERSION,
|
||||||
|
savedAt: value.savedAt,
|
||||||
|
source: typeof value.source === 'string' ? value.source : '',
|
||||||
|
rows: rows as PostImportRow[],
|
||||||
|
repairMode: value.repairMode === 'failed' ? 'failed' : 'all' }
|
||||||
|
}
|
||||||
|
catch
|
||||||
|
{
|
||||||
|
return null
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,48 @@
|
|||||||
|
export type PostImportOrigin = 'automatic' | 'manual'
|
||||||
|
export type PostImportRepairMode = 'all' | 'failed'
|
||||||
|
export type PostImportStatus =
|
||||||
|
'pending'
|
||||||
|
| 'created'
|
||||||
|
| 'skipped'
|
||||||
|
| 'failed'
|
||||||
|
export type PostImportSkipReason = 'existing'
|
||||||
|
export type PostImportResultStatus = 'created' | 'skipped' | 'failed'
|
||||||
|
export type PostImportAttributeValue = string | number
|
||||||
|
|
||||||
|
export type PostImportRow = {
|
||||||
|
sourceRow: number
|
||||||
|
url: string
|
||||||
|
attributes: Record<string, PostImportAttributeValue>
|
||||||
|
fieldWarnings: Record<string, string[]>
|
||||||
|
baseWarnings: string[]
|
||||||
|
validationErrors: Record<string, string[]>
|
||||||
|
importErrors?: Record<string, string[]>
|
||||||
|
provenance: Record<string, PostImportOrigin>
|
||||||
|
tagSources?: Record<PostImportOrigin, string>
|
||||||
|
status: 'ready' | 'warning' | 'error'
|
||||||
|
skipReason?: PostImportSkipReason
|
||||||
|
existingPostId?: number
|
||||||
|
metadataUrl?: string
|
||||||
|
createdPostId?: number
|
||||||
|
importStatus?: PostImportStatus }
|
||||||
|
|
||||||
|
export type PostImportResultRow = {
|
||||||
|
sourceRow: number
|
||||||
|
status: PostImportResultStatus
|
||||||
|
post?: { id: number }
|
||||||
|
existingPostId?: number
|
||||||
|
errors?: Record<string, string[]> }
|
||||||
|
|
||||||
|
export type PostImportSession = {
|
||||||
|
version: number
|
||||||
|
savedAt: string
|
||||||
|
source: string
|
||||||
|
rows: PostImportRow[]
|
||||||
|
repairMode: PostImportRepairMode }
|
||||||
|
|
||||||
|
export type PostImportSourceIssue = {
|
||||||
|
sourceRow: number
|
||||||
|
message: string
|
||||||
|
url: string }
|
||||||
|
|
||||||
|
export type StorageErrorHandler = (message: string) => void
|
||||||
@@ -89,8 +89,12 @@ describe ('settings', () => {
|
|||||||
})
|
})
|
||||||
|
|
||||||
it ('derives TopNav colours without mixing mobile active backgrounds', () => {
|
it ('derives TopNav colours without mixing mobile active backgrounds', () => {
|
||||||
expect (buildThemeTokens ('light', { }).topNavMobileActiveBackground).toBe (
|
const tokens = buildThemeTokens ('light', { })
|
||||||
|
|
||||||
|
expect (tokens.topNavMobileActiveBackground).toBe (
|
||||||
LIGHT_THEME_TOKENS.topNavRootBackgroundDesktop)
|
LIGHT_THEME_TOKENS.topNavRootBackgroundDesktop)
|
||||||
|
expect (tokens.topNavMobileActiveBackground).not.toBe (
|
||||||
|
tokens.topNavRootBackgroundMobile)
|
||||||
expect (buildThemeTokens ('dark', { }).topNavMobileActiveBackground).toBe (
|
expect (buildThemeTokens ('dark', { }).topNavMobileActiveBackground).toBe (
|
||||||
DARK_THEME_TOKENS.topNavActiveBackground)
|
DARK_THEME_TOKENS.topNavActiveBackground)
|
||||||
})
|
})
|
||||||
|
|||||||
@@ -0,0 +1,304 @@
|
|||||||
|
import { useEffect, useMemo, useState } from 'react'
|
||||||
|
import { Helmet } from 'react-helmet-async'
|
||||||
|
import { useNavigate,
|
||||||
|
useParams } from 'react-router-dom'
|
||||||
|
|
||||||
|
import FieldError from '@/components/common/FieldError'
|
||||||
|
import PageTitle from '@/components/common/PageTitle'
|
||||||
|
import PrefetchLink from '@/components/PrefetchLink'
|
||||||
|
import MainArea from '@/components/layout/MainArea'
|
||||||
|
import PostImportStatusBadge from '@/components/posts/import/PostImportStatusBadge'
|
||||||
|
import { effectivePostImportStatus } from '@/components/posts/import/postImportRowStatus'
|
||||||
|
import { Button } from '@/components/ui/button'
|
||||||
|
import { toast } from '@/components/ui/use-toast'
|
||||||
|
import { SITE_TITLE } from '@/config'
|
||||||
|
import { apiPost } from '@/lib/api'
|
||||||
|
import { canEditContent } from '@/lib/users'
|
||||||
|
import { clearPostImportSourceDraft,
|
||||||
|
loadPostImportSession,
|
||||||
|
mergeImportResults,
|
||||||
|
mergeValidatedImportRows,
|
||||||
|
resultSummaryCounts,
|
||||||
|
retryImportRow,
|
||||||
|
savePostImportSession,
|
||||||
|
type PostImportResultRow,
|
||||||
|
type PostImportRow,
|
||||||
|
type PostImportSession } from '@/lib/postImportSession'
|
||||||
|
import Forbidden from '@/pages/Forbidden'
|
||||||
|
|
||||||
|
import type { FC } from 'react'
|
||||||
|
|
||||||
|
import type { User } from '@/types'
|
||||||
|
|
||||||
|
type Props = { user: User | null }
|
||||||
|
|
||||||
|
const resultToneClass = (
|
||||||
|
status: ReturnType<typeof effectivePostImportStatus>,
|
||||||
|
): string[] => {
|
||||||
|
switch (status)
|
||||||
|
{
|
||||||
|
case 'created':
|
||||||
|
return [
|
||||||
|
'border-sky-200 bg-sky-50',
|
||||||
|
'dark:border-sky-900 dark:bg-sky-950/30']
|
||||||
|
case 'skipped':
|
||||||
|
return [
|
||||||
|
'border-stone-200 bg-stone-50',
|
||||||
|
'dark:border-stone-800 dark:bg-stone-900/60']
|
||||||
|
case 'failed':
|
||||||
|
case 'error':
|
||||||
|
return [
|
||||||
|
'border-rose-200 bg-rose-50',
|
||||||
|
'dark:border-rose-900 dark:bg-rose-950/30']
|
||||||
|
case 'warning':
|
||||||
|
return [
|
||||||
|
'border-amber-200 bg-amber-50',
|
||||||
|
'dark:border-amber-900 dark:bg-amber-950/30']
|
||||||
|
default:
|
||||||
|
return [
|
||||||
|
'border-border bg-white',
|
||||||
|
'dark:border-neutral-700 dark:bg-neutral-900']
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
const rowMessages = (row: PostImportRow): string[] => [
|
||||||
|
...Object.values (row.validationErrors ?? { }).flat (),
|
||||||
|
...Object.values (row.importErrors ?? { }).flat ()]
|
||||||
|
|
||||||
|
|
||||||
|
const PostImportResultPage: FC<Props> = ({ user }) => {
|
||||||
|
const editable = canEditContent (user)
|
||||||
|
const navigate = useNavigate ()
|
||||||
|
const { sessionId } = useParams ()
|
||||||
|
|
||||||
|
const [session, setSession] = useState<PostImportSession | null> (null)
|
||||||
|
const [missing, setMissing] = useState (false)
|
||||||
|
const [loadingRow, setLoadingRow] = useState<number | null> (null)
|
||||||
|
|
||||||
|
useEffect (() => {
|
||||||
|
if (sessionId == null)
|
||||||
|
return
|
||||||
|
|
||||||
|
const loaded = loadPostImportSession (sessionId, message =>
|
||||||
|
toast ({ title: '取込状態を復元できませんでした', description: message }))
|
||||||
|
setSession (loaded)
|
||||||
|
setMissing (loaded == null)
|
||||||
|
}, [sessionId])
|
||||||
|
|
||||||
|
useEffect (() => {
|
||||||
|
if (sessionId == null || session == null)
|
||||||
|
return
|
||||||
|
|
||||||
|
savePostImportSession (sessionId, session, message =>
|
||||||
|
toast ({ title: '取込状態を保存できませんでした', description: message }))
|
||||||
|
}, [session, sessionId])
|
||||||
|
|
||||||
|
const counts = useMemo (
|
||||||
|
() => resultSummaryCounts (session?.rows ?? []),
|
||||||
|
[session])
|
||||||
|
|
||||||
|
const retry = async (sourceRow: number) => {
|
||||||
|
if (session == null)
|
||||||
|
return
|
||||||
|
|
||||||
|
setLoadingRow (sourceRow)
|
||||||
|
try
|
||||||
|
{
|
||||||
|
const pendingRows = retryImportRow (session.rows, sourceRow)
|
||||||
|
setSession ({ ...session, rows: pendingRows })
|
||||||
|
const validated = await apiPost<{ rows: PostImportRow[] }> ('/posts/import/validate', {
|
||||||
|
rows: pendingRows
|
||||||
|
.filter (row => row.importStatus !== 'created')
|
||||||
|
.map (row => ({
|
||||||
|
sourceRow: row.sourceRow,
|
||||||
|
url: row.url,
|
||||||
|
attributes: row.attributes,
|
||||||
|
provenance: row.provenance,
|
||||||
|
tagSources: row.tagSources,
|
||||||
|
metadataUrl: row.metadataUrl })),
|
||||||
|
changed_row: -1 })
|
||||||
|
const validatedRows = mergeValidatedImportRows (pendingRows, validated.rows)
|
||||||
|
setSession (current =>
|
||||||
|
current
|
||||||
|
? { ...current, rows: validatedRows }
|
||||||
|
: current)
|
||||||
|
const target = validatedRows.find (_1 => _1.sourceRow === sourceRow)
|
||||||
|
if (target == null)
|
||||||
|
return
|
||||||
|
if (Object.keys (target.validationErrors ?? { }).length > 0)
|
||||||
|
return
|
||||||
|
|
||||||
|
const result = await apiPost<{
|
||||||
|
created: number
|
||||||
|
skipped: number
|
||||||
|
failed: number
|
||||||
|
rows: PostImportResultRow[] }> ('/posts/import', {
|
||||||
|
rows: [{
|
||||||
|
sourceRow: target.sourceRow,
|
||||||
|
url: target.url,
|
||||||
|
attributes: target.attributes,
|
||||||
|
provenance: target.provenance,
|
||||||
|
tagSources: target.tagSources,
|
||||||
|
metadataUrl: target.metadataUrl }] })
|
||||||
|
setSession (current =>
|
||||||
|
current
|
||||||
|
? { ...current, rows: mergeImportResults (current.rows, result.rows) }
|
||||||
|
: current)
|
||||||
|
}
|
||||||
|
catch
|
||||||
|
{
|
||||||
|
setSession (session)
|
||||||
|
toast ({ title: '再試行に失敗しました' })
|
||||||
|
}
|
||||||
|
finally
|
||||||
|
{
|
||||||
|
setLoadingRow (null)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
const openRepair = (sourceRow: number) => {
|
||||||
|
if (session == null || sessionId == null)
|
||||||
|
return
|
||||||
|
|
||||||
|
const nextSession = { ...session, repairMode: 'failed' as const }
|
||||||
|
setSession (nextSession)
|
||||||
|
const saved = savePostImportSession (sessionId, nextSession, message =>
|
||||||
|
toast ({ title: '取込状態を保存できませんでした', description: message }))
|
||||||
|
if (!(saved))
|
||||||
|
return
|
||||||
|
navigate (`/posts/import/${ sessionId }/review?edit=${ sourceRow }`)
|
||||||
|
}
|
||||||
|
|
||||||
|
if (!(editable))
|
||||||
|
return <Forbidden/>
|
||||||
|
|
||||||
|
if (missing || sessionId == null || session == null)
|
||||||
|
{
|
||||||
|
return (
|
||||||
|
<MainArea>
|
||||||
|
<div className="mx-auto max-w-4xl space-y-4 p-4">
|
||||||
|
<PageTitle>投稿インポート結果</PageTitle>
|
||||||
|
<FieldError messages={['取込状態が見つかりません.']}/>
|
||||||
|
<Button type="button" onClick={() => navigate ('/posts/import')}>
|
||||||
|
URL リスト入力へ戻る
|
||||||
|
</Button>
|
||||||
|
</div>
|
||||||
|
</MainArea>)
|
||||||
|
}
|
||||||
|
|
||||||
|
return (
|
||||||
|
<MainArea>
|
||||||
|
<Helmet>
|
||||||
|
<title>{`投稿インポート結果 | ${ SITE_TITLE }`}</title>
|
||||||
|
</Helmet>
|
||||||
|
|
||||||
|
<div className="mx-auto max-w-5xl space-y-4 p-4">
|
||||||
|
<PageTitle>登録結果</PageTitle>
|
||||||
|
|
||||||
|
<div className="rounded-lg border bg-white p-4 dark:border-neutral-700
|
||||||
|
dark:bg-neutral-900">
|
||||||
|
<div className="flex flex-wrap gap-2">
|
||||||
|
<SummaryChip label="登録成功" value={counts.created} badge="created"/>
|
||||||
|
<SummaryChip label="スキップ" value={counts.skipped} badge="skipped"/>
|
||||||
|
<SummaryChip label="失敗" value={counts.failed} badge="failed"/>
|
||||||
|
<SummaryChip label="要修正" value={counts.invalid} badge="error"/>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<div className="space-y-3">
|
||||||
|
{session.rows.map (row => {
|
||||||
|
const status = effectivePostImportStatus (row)
|
||||||
|
return (
|
||||||
|
<div
|
||||||
|
key={row.sourceRow}
|
||||||
|
className={[
|
||||||
|
'rounded-lg border p-4',
|
||||||
|
...resultToneClass (status)].join (' ')}>
|
||||||
|
<div className="flex flex-col gap-3 md:flex-row md:items-start
|
||||||
|
md:justify-between">
|
||||||
|
<div className="space-y-2">
|
||||||
|
<div className="flex flex-wrap items-center gap-2">
|
||||||
|
<span className="text-sm font-medium">行 {row.sourceRow}</span>
|
||||||
|
<PostImportStatusBadge value={status}/>
|
||||||
|
</div>
|
||||||
|
<div className="text-sm text-neutral-700 dark:text-neutral-200">
|
||||||
|
{String (row.attributes.title ?? '') || row.url}
|
||||||
|
</div>
|
||||||
|
<div className="text-xs text-neutral-500 dark:text-neutral-400">
|
||||||
|
{row.url}
|
||||||
|
</div>
|
||||||
|
<FieldError messages={rowMessages (row)}/>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<div className="flex flex-col gap-2 sm:flex-row">
|
||||||
|
{(row.createdPostId != null || row.existingPostId != null) && (
|
||||||
|
<Button type="button" variant="outline" asChild>
|
||||||
|
<PrefetchLink
|
||||||
|
to={`/posts/${ row.createdPostId ?? row.existingPostId }`}>
|
||||||
|
投稿を開く
|
||||||
|
</PrefetchLink>
|
||||||
|
</Button>)}
|
||||||
|
{(status === 'error' || row.importStatus === 'failed') && (
|
||||||
|
<Button
|
||||||
|
type="button"
|
||||||
|
variant="outline"
|
||||||
|
onClick={() => openRepair (row.sourceRow)}>
|
||||||
|
編輯
|
||||||
|
</Button>)}
|
||||||
|
{row.importStatus === 'failed' && status !== 'error' && (
|
||||||
|
<>
|
||||||
|
<Button
|
||||||
|
type="button"
|
||||||
|
onClick={() => retry (row.sourceRow)}
|
||||||
|
disabled={loadingRow === row.sourceRow}>
|
||||||
|
再試行
|
||||||
|
</Button>
|
||||||
|
</>)}
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
</div>)})}
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<div className="flex flex-col gap-2 sm:flex-row">
|
||||||
|
<Button
|
||||||
|
type="button"
|
||||||
|
variant="outline"
|
||||||
|
onClick={() => {
|
||||||
|
const nextSession = { ...session, repairMode: 'all' as const }
|
||||||
|
setSession (nextSession)
|
||||||
|
const saved = savePostImportSession (sessionId, nextSession, message =>
|
||||||
|
toast ({ title: '取込状態を保存できませんでした', description: message }))
|
||||||
|
if (!(saved))
|
||||||
|
return
|
||||||
|
navigate (`/posts/import/${ sessionId }/review`)
|
||||||
|
}}>
|
||||||
|
確認画面へ戻る
|
||||||
|
</Button>
|
||||||
|
<Button
|
||||||
|
type="button"
|
||||||
|
variant="outline"
|
||||||
|
onClick={() => {
|
||||||
|
clearPostImportSourceDraft (message =>
|
||||||
|
toast ({ title: '入力内容を削除できませんでした', description: message }))
|
||||||
|
navigate ('/posts/import')
|
||||||
|
}}>
|
||||||
|
新しい URL リストを入力
|
||||||
|
</Button>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
</MainArea>)
|
||||||
|
}
|
||||||
|
|
||||||
|
const SummaryChip = (
|
||||||
|
{ label, value, badge }: {
|
||||||
|
label: string
|
||||||
|
value: number
|
||||||
|
badge: 'created' | 'skipped' | 'failed' | 'error' },
|
||||||
|
) => (
|
||||||
|
<div className="flex items-center gap-2 rounded-full border border-border
|
||||||
|
bg-background px-3 py-1 text-sm">
|
||||||
|
<PostImportStatusBadge value={badge}/>
|
||||||
|
<span>{label} {value}</span>
|
||||||
|
</div>)
|
||||||
|
|
||||||
|
export default PostImportResultPage
|
||||||
@@ -0,0 +1,351 @@
|
|||||||
|
import { useEffect, useMemo, useState } from 'react'
|
||||||
|
import { Helmet } from 'react-helmet-async'
|
||||||
|
import { useNavigate,
|
||||||
|
useParams,
|
||||||
|
useSearchParams } from 'react-router-dom'
|
||||||
|
|
||||||
|
import PageTitle from '@/components/common/PageTitle'
|
||||||
|
import MainArea from '@/components/layout/MainArea'
|
||||||
|
import PostImportRowDialog from '@/components/posts/import/PostImportRowDialog'
|
||||||
|
import PostImportRowSummary from '@/components/posts/import/PostImportRowSummary'
|
||||||
|
import PostImportStatusBadge from '@/components/posts/import/PostImportStatusBadge'
|
||||||
|
import { Button } from '@/components/ui/button'
|
||||||
|
import { toast } from '@/components/ui/use-toast'
|
||||||
|
import { SITE_TITLE } from '@/config'
|
||||||
|
import { apiPost } from '@/lib/api'
|
||||||
|
import { canEditContent } from '@/lib/users'
|
||||||
|
import { loadPostImportSession,
|
||||||
|
creatableImportRows,
|
||||||
|
mergeImportResults,
|
||||||
|
mergeValidatedImportRows,
|
||||||
|
processableImportRows,
|
||||||
|
reviewSummaryCounts,
|
||||||
|
savePostImportSession,
|
||||||
|
type PostImportResultRow,
|
||||||
|
type PostImportRow,
|
||||||
|
type PostImportSession } from '@/lib/postImportSession'
|
||||||
|
import Forbidden from '@/pages/Forbidden'
|
||||||
|
|
||||||
|
import type { FC } from 'react'
|
||||||
|
|
||||||
|
import type { User } from '@/types'
|
||||||
|
|
||||||
|
type Props = { user: User | null }
|
||||||
|
|
||||||
|
type Draft = {
|
||||||
|
url: string
|
||||||
|
title: string
|
||||||
|
thumbnailBase: string
|
||||||
|
originalCreatedFrom: string
|
||||||
|
originalCreatedBefore: string
|
||||||
|
duration: string
|
||||||
|
tags: string
|
||||||
|
parentPostIds: string }
|
||||||
|
|
||||||
|
|
||||||
|
const PostImportReviewPage: FC<Props> = ({ user }) => {
|
||||||
|
const editable = canEditContent (user)
|
||||||
|
const navigate = useNavigate ()
|
||||||
|
const { sessionId } = useParams ()
|
||||||
|
const [searchParams, setSearchParams] = useSearchParams ()
|
||||||
|
|
||||||
|
const [session, setSession] = useState<PostImportSession | null> (null)
|
||||||
|
const [loading, setLoading] = useState (false)
|
||||||
|
const [missing, setMissing] = useState (false)
|
||||||
|
const [savingRow, setSavingRow] = useState<number | null> (null)
|
||||||
|
|
||||||
|
useEffect (() => {
|
||||||
|
if (sessionId == null)
|
||||||
|
return
|
||||||
|
|
||||||
|
const loaded = loadPostImportSession (sessionId, message =>
|
||||||
|
toast ({ title: '取込状態を復元できませんでした', description: message }))
|
||||||
|
setSession (loaded)
|
||||||
|
setMissing (loaded == null)
|
||||||
|
}, [sessionId])
|
||||||
|
|
||||||
|
useEffect (() => {
|
||||||
|
if (sessionId == null || session == null)
|
||||||
|
return
|
||||||
|
|
||||||
|
savePostImportSession (sessionId, session, message =>
|
||||||
|
toast ({ title: '取込状態を保存できませんでした', description: message }))
|
||||||
|
}, [session, sessionId])
|
||||||
|
|
||||||
|
const rows = session?.rows ?? []
|
||||||
|
const editingSourceRow = Number (searchParams.get ('edit') ?? '')
|
||||||
|
const editingRow =
|
||||||
|
Number.isFinite (editingSourceRow)
|
||||||
|
? rows.find (_1 => _1.sourceRow === editingSourceRow) ?? null
|
||||||
|
: null
|
||||||
|
const counts = useMemo (() => reviewSummaryCounts (rows), [rows])
|
||||||
|
const processable = useMemo (
|
||||||
|
() => processableImportRows (rows),
|
||||||
|
[rows])
|
||||||
|
const creatable = useMemo (
|
||||||
|
() => creatableImportRows (rows),
|
||||||
|
[rows])
|
||||||
|
const reviewRows =
|
||||||
|
session?.repairMode === 'failed'
|
||||||
|
? [...rows].sort ((a, b) => {
|
||||||
|
const aFailed = a.importStatus === 'failed' ? 0 : 1
|
||||||
|
const bFailed = b.importStatus === 'failed' ? 0 : 1
|
||||||
|
return aFailed - bFailed || a.sourceRow - b.sourceRow
|
||||||
|
})
|
||||||
|
: rows
|
||||||
|
|
||||||
|
useEffect (() => {
|
||||||
|
if (editingRow == null || session?.repairMode !== 'failed')
|
||||||
|
return
|
||||||
|
|
||||||
|
const element = document.getElementById (`post-import-row-${ editingRow.sourceRow }`)
|
||||||
|
element?.scrollIntoView ({ block: 'center', behavior: 'smooth' })
|
||||||
|
}, [editingRow, session?.repairMode])
|
||||||
|
|
||||||
|
const updateSessionRows = (nextRows: PostImportRow[]) =>
|
||||||
|
setSession (current =>
|
||||||
|
current ? { ...current, rows: nextRows } : current)
|
||||||
|
|
||||||
|
const saveDraft = async (draft: Draft): Promise<boolean> => {
|
||||||
|
if (session == null)
|
||||||
|
return false
|
||||||
|
if (editingRow == null)
|
||||||
|
return false
|
||||||
|
|
||||||
|
const urlChanged = draft.url !== editingRow.url
|
||||||
|
const nextProvenance = { ...editingRow.provenance }
|
||||||
|
const nextAttributes = { ...editingRow.attributes }
|
||||||
|
const nextTagSources = {
|
||||||
|
automatic: editingRow.tagSources?.automatic ?? '',
|
||||||
|
manual: editingRow.tagSources?.manual ?? '' }
|
||||||
|
const draftFields = [
|
||||||
|
['title', draft.title],
|
||||||
|
['thumbnailBase', draft.thumbnailBase],
|
||||||
|
['originalCreatedFrom', draft.originalCreatedFrom],
|
||||||
|
['originalCreatedBefore', draft.originalCreatedBefore],
|
||||||
|
['duration', draft.duration],
|
||||||
|
['parentPostIds', draft.parentPostIds]] as const
|
||||||
|
draftFields.forEach (([field, value]) => {
|
||||||
|
nextAttributes[field] = value
|
||||||
|
nextProvenance[field] =
|
||||||
|
value !== String (editingRow.attributes[field] ?? '')
|
||||||
|
? 'manual'
|
||||||
|
: (editingRow.provenance[field] ?? 'automatic')
|
||||||
|
})
|
||||||
|
nextAttributes.tags = draft.tags
|
||||||
|
if (draft.tags !== String (editingRow.attributes.tags ?? ''))
|
||||||
|
{
|
||||||
|
nextProvenance.tags = 'manual'
|
||||||
|
nextTagSources.manual = draft.tags
|
||||||
|
}
|
||||||
|
else
|
||||||
|
{
|
||||||
|
nextProvenance.tags = editingRow.provenance.tags ?? 'automatic'
|
||||||
|
nextTagSources.manual = editingRow.tagSources?.manual ?? ''
|
||||||
|
}
|
||||||
|
|
||||||
|
setSavingRow (editingRow.sourceRow)
|
||||||
|
const nextRows = session.rows.map (row =>
|
||||||
|
row.sourceRow === editingRow.sourceRow
|
||||||
|
? {
|
||||||
|
...row,
|
||||||
|
url: draft.url,
|
||||||
|
attributes: nextAttributes,
|
||||||
|
provenance: {
|
||||||
|
...nextProvenance,
|
||||||
|
url: urlChanged ? 'manual' : (editingRow.provenance.url ?? 'manual') },
|
||||||
|
tagSources: nextTagSources,
|
||||||
|
importStatus: row.importStatus === 'created' ? 'created' : 'pending',
|
||||||
|
importErrors: undefined }
|
||||||
|
: row)
|
||||||
|
try
|
||||||
|
{
|
||||||
|
const validated = await apiPost<{ rows: PostImportRow[] }> ('/posts/import/validate', {
|
||||||
|
rows: nextRows
|
||||||
|
.filter (row => row.importStatus !== 'created')
|
||||||
|
.map (row => ({
|
||||||
|
sourceRow: row.sourceRow,
|
||||||
|
url: row.url,
|
||||||
|
attributes: row.attributes,
|
||||||
|
provenance: row.provenance,
|
||||||
|
tagSources: row.tagSources,
|
||||||
|
metadataUrl: row.metadataUrl })),
|
||||||
|
changed_row: urlChanged ? editingRow.sourceRow : -1 })
|
||||||
|
updateSessionRows (mergeValidatedImportRows (nextRows, validated.rows))
|
||||||
|
setSearchParams ({ })
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
catch
|
||||||
|
{
|
||||||
|
toast ({ title: '行の再検証に失敗しました' })
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
finally
|
||||||
|
{
|
||||||
|
setSavingRow (null)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
const submit = async () => {
|
||||||
|
if (sessionId == null || session == null || processable.length === 0)
|
||||||
|
return
|
||||||
|
|
||||||
|
setLoading (true)
|
||||||
|
try
|
||||||
|
{
|
||||||
|
const result = await apiPost<{
|
||||||
|
created: number
|
||||||
|
skipped: number
|
||||||
|
failed: number
|
||||||
|
rows: PostImportResultRow[] }> ('/posts/import', {
|
||||||
|
rows: processable.map (row => ({
|
||||||
|
sourceRow: row.sourceRow,
|
||||||
|
url: row.url,
|
||||||
|
attributes: row.attributes,
|
||||||
|
provenance: row.provenance,
|
||||||
|
tagSources: row.tagSources,
|
||||||
|
metadataUrl: row.metadataUrl })) })
|
||||||
|
const nextRows = mergeImportResults (session.rows, result.rows)
|
||||||
|
const nextSession = { ...session, rows: nextRows, repairMode: 'all' as const }
|
||||||
|
setSession (nextSession)
|
||||||
|
const saved = savePostImportSession (sessionId, nextSession, message =>
|
||||||
|
toast ({ title: '取込状態を保存できませんでした', description: message }))
|
||||||
|
if (!(saved))
|
||||||
|
return
|
||||||
|
navigate (`/posts/import/${ sessionId }/result`)
|
||||||
|
}
|
||||||
|
catch
|
||||||
|
{
|
||||||
|
toast ({ title: '登録に失敗しました' })
|
||||||
|
}
|
||||||
|
finally
|
||||||
|
{
|
||||||
|
setLoading (false)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
if (!(editable))
|
||||||
|
return <Forbidden/>
|
||||||
|
|
||||||
|
if (missing || sessionId == null || session == null)
|
||||||
|
{
|
||||||
|
return (
|
||||||
|
<MainArea>
|
||||||
|
<div className="mx-auto max-w-4xl space-y-4 p-4">
|
||||||
|
<PageTitle>投稿インポート</PageTitle>
|
||||||
|
<div className="text-red-700 dark:text-red-300">取込状態が見つかりません.</div>
|
||||||
|
<Button type="button" onClick={() => navigate ('/posts/import')}>
|
||||||
|
URL リスト入力へ戻る
|
||||||
|
</Button>
|
||||||
|
</div>
|
||||||
|
</MainArea>)
|
||||||
|
}
|
||||||
|
|
||||||
|
return (
|
||||||
|
<>
|
||||||
|
<Helmet>
|
||||||
|
<title>{`投稿インポート確認 | ${ SITE_TITLE }`}</title>
|
||||||
|
</Helmet>
|
||||||
|
|
||||||
|
<MainArea className="min-h-0">
|
||||||
|
<div className="mx-auto max-w-6xl space-y-4 p-4">
|
||||||
|
<PageTitle>投稿情報の確認・編輯</PageTitle>
|
||||||
|
|
||||||
|
<div className="rounded-lg border bg-white p-4 dark:border-neutral-700
|
||||||
|
dark:bg-neutral-900">
|
||||||
|
<div className="flex flex-wrap items-center gap-2">
|
||||||
|
<SummaryChip label="全件" value={counts.total}/>
|
||||||
|
<SummaryChip label="登録対象" value={counts.submittable}/>
|
||||||
|
<SummaryChip label="要修正" value={counts.invalid}/>
|
||||||
|
<SummaryChip label="スキップ予定" value={counts.skipPlanned}/>
|
||||||
|
<SummaryChip label="登録済み" value={counts.created}/>
|
||||||
|
<SummaryChip label="失敗" value={counts.failed}/>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<div className="space-y-3">
|
||||||
|
{reviewRows.map (row => (
|
||||||
|
<div key={row.sourceRow} id={`post-import-row-${ row.sourceRow }`}>
|
||||||
|
<PostImportRowSummary
|
||||||
|
row={row}
|
||||||
|
onEdit={() => {
|
||||||
|
setSearchParams ({ edit: String (row.sourceRow) })
|
||||||
|
}}/>
|
||||||
|
</div>))}
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
</MainArea>
|
||||||
|
|
||||||
|
<PostImportFooter
|
||||||
|
loading={loading}
|
||||||
|
invalidCount={counts.invalid}
|
||||||
|
processableCount={processable.length}
|
||||||
|
creatableCount={creatable.length}
|
||||||
|
skipPlannedCount={counts.skipPlanned}
|
||||||
|
onBack={() => navigate ('/posts/import')}
|
||||||
|
onSubmit={submit}/>
|
||||||
|
|
||||||
|
<PostImportRowDialog
|
||||||
|
open={editingRow != null}
|
||||||
|
row={editingRow}
|
||||||
|
saving={savingRow != null}
|
||||||
|
onOpenChange={open => {
|
||||||
|
if (!(open))
|
||||||
|
setSearchParams ({ })
|
||||||
|
}}
|
||||||
|
onSave={saveDraft}/>
|
||||||
|
</>)
|
||||||
|
}
|
||||||
|
|
||||||
|
const PostImportFooter = (
|
||||||
|
{ loading,
|
||||||
|
invalidCount,
|
||||||
|
processableCount,
|
||||||
|
creatableCount,
|
||||||
|
skipPlannedCount,
|
||||||
|
onBack,
|
||||||
|
onSubmit }: {
|
||||||
|
loading: boolean
|
||||||
|
invalidCount: number
|
||||||
|
processableCount: number
|
||||||
|
creatableCount: number
|
||||||
|
skipPlannedCount: number
|
||||||
|
onBack: () => void
|
||||||
|
onSubmit: () => void },
|
||||||
|
) => (
|
||||||
|
<div
|
||||||
|
className="shrink-0 border-t bg-white/95 p-4 backdrop-blur
|
||||||
|
dark:border-neutral-700 dark:bg-neutral-950/95">
|
||||||
|
<div className="mx-auto flex max-w-6xl flex-col gap-3 md:flex-row
|
||||||
|
md:items-center md:justify-between">
|
||||||
|
<div className="flex flex-wrap items-center gap-2 text-sm">
|
||||||
|
<PostImportStatusBadge value="pending"/>
|
||||||
|
<span>処理対象 {processableCount} 件</span>
|
||||||
|
<PostImportStatusBadge value="ready"/>
|
||||||
|
<span>登録対象 {creatableCount} 件</span>
|
||||||
|
<PostImportStatusBadge value="skipped"/>
|
||||||
|
<span>スキップ予定 {skipPlannedCount} 件</span>
|
||||||
|
<PostImportStatusBadge value="error"/>
|
||||||
|
<span>要修正 {invalidCount} 件</span>
|
||||||
|
</div>
|
||||||
|
<div className="flex flex-col gap-2 sm:flex-row">
|
||||||
|
<Button type="button" variant="outline" onClick={onBack}>
|
||||||
|
URL リスト入力へ戻る
|
||||||
|
</Button>
|
||||||
|
<Button
|
||||||
|
type="button"
|
||||||
|
onClick={onSubmit}
|
||||||
|
disabled={loading || processableCount === 0}>
|
||||||
|
取込を実行
|
||||||
|
</Button>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
</div>)
|
||||||
|
|
||||||
|
const SummaryChip = ({ label, value }: { label: string
|
||||||
|
value: number }) => (
|
||||||
|
<div className="rounded-full border border-border bg-background px-3 py-1 text-sm">
|
||||||
|
{label} {value}
|
||||||
|
</div>)
|
||||||
|
|
||||||
|
export default PostImportReviewPage
|
||||||
@@ -0,0 +1,212 @@
|
|||||||
|
import { useEffect, useRef, useState } from 'react'
|
||||||
|
import { Helmet } from 'react-helmet-async'
|
||||||
|
import { useNavigate } from 'react-router-dom'
|
||||||
|
|
||||||
|
import FieldError from '@/components/common/FieldError'
|
||||||
|
import FormField from '@/components/common/FormField'
|
||||||
|
import PageTitle from '@/components/common/PageTitle'
|
||||||
|
import MainArea from '@/components/layout/MainArea'
|
||||||
|
import { Button } from '@/components/ui/button'
|
||||||
|
import { toast } from '@/components/ui/use-toast'
|
||||||
|
import { SITE_TITLE } from '@/config'
|
||||||
|
import { apiPost, isApiError } from '@/lib/api'
|
||||||
|
import { canEditContent } from '@/lib/users'
|
||||||
|
import { inputClass } from '@/lib/utils'
|
||||||
|
import { countImportSourceLines,
|
||||||
|
cleanupExpiredPostImportSessions,
|
||||||
|
createPostImportSessionId,
|
||||||
|
loadPostImportSourceDraft,
|
||||||
|
savePostImportSession,
|
||||||
|
savePostImportSourceDraft,
|
||||||
|
validateImportSource,
|
||||||
|
type PostImportRow } from '@/lib/postImportSession'
|
||||||
|
import Forbidden from '@/pages/Forbidden'
|
||||||
|
|
||||||
|
import type { FC } from 'react'
|
||||||
|
|
||||||
|
import type { User } from '@/types'
|
||||||
|
|
||||||
|
type Props = { user: User | null }
|
||||||
|
|
||||||
|
const MAX_ROWS = 100
|
||||||
|
const SOURCE_ERROR_ID = 'post-import-source-error'
|
||||||
|
const SOURCE_ISSUES_ID = 'post-import-source-issues'
|
||||||
|
|
||||||
|
const urlIssuesFromRows = (rows: PostImportRow[]) =>
|
||||||
|
rows.flatMap (row =>
|
||||||
|
(row.validationErrors.url ?? []).map (message => ({
|
||||||
|
sourceRow: row.sourceRow,
|
||||||
|
message,
|
||||||
|
url: row.url })))
|
||||||
|
|
||||||
|
|
||||||
|
const PostImportSourcePage: FC<Props> = ({ user }) => {
|
||||||
|
const editable = canEditContent (user)
|
||||||
|
const navigate = useNavigate ()
|
||||||
|
const [source, setSource] = useState ('')
|
||||||
|
const [loading, setLoading] = useState (false)
|
||||||
|
const [sourceIssues, setSourceIssues] = useState<ReturnType<typeof validateImportSource>> ([])
|
||||||
|
const [sourceError, setSourceError] = useState<string | null> (null)
|
||||||
|
const saveTimer = useRef<number | null> (null)
|
||||||
|
const editedRef = useRef (false)
|
||||||
|
|
||||||
|
const lineCount = countImportSourceLines (source)
|
||||||
|
const messages = sourceError ? [sourceError] : []
|
||||||
|
const sourceDescribedBy = [
|
||||||
|
sourceError ? SOURCE_ERROR_ID : null,
|
||||||
|
sourceIssues.length > 0 ? SOURCE_ISSUES_ID : null]
|
||||||
|
.filter (_1 => _1 != null)
|
||||||
|
.join (' ')
|
||||||
|
|
||||||
|
useEffect (() => {
|
||||||
|
cleanupExpiredPostImportSessions (message =>
|
||||||
|
toast ({ title: '保存済みデータを整理できませんでした', description: message }))
|
||||||
|
|
||||||
|
const draft = loadPostImportSourceDraft (message =>
|
||||||
|
toast ({ title: '保存済み入力を復元できませんでした', description: message }))
|
||||||
|
if (!(editedRef.current))
|
||||||
|
{
|
||||||
|
setSource (current => current === '' ? draft.source : current)
|
||||||
|
}
|
||||||
|
}, [])
|
||||||
|
|
||||||
|
useEffect (() => {
|
||||||
|
if (saveTimer.current != null)
|
||||||
|
window.clearTimeout (saveTimer.current)
|
||||||
|
saveTimer.current = window.setTimeout (() => {
|
||||||
|
savePostImportSourceDraft (source, message =>
|
||||||
|
toast ({ title: '入力内容を保存できませんでした', description: message }))
|
||||||
|
}, 300)
|
||||||
|
return () => {
|
||||||
|
if (saveTimer.current != null)
|
||||||
|
window.clearTimeout (saveTimer.current)
|
||||||
|
}
|
||||||
|
}, [source])
|
||||||
|
|
||||||
|
const preview = async () => {
|
||||||
|
if (lineCount === 0)
|
||||||
|
{
|
||||||
|
setSourceIssues ([])
|
||||||
|
setSourceError ('URL を入力してください.')
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
const issues = validateImportSource (source)
|
||||||
|
if (issues.length > 0)
|
||||||
|
{
|
||||||
|
setSourceIssues (issues)
|
||||||
|
setSourceError (null)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
setLoading (true)
|
||||||
|
setSourceIssues ([])
|
||||||
|
setSourceError (null)
|
||||||
|
try
|
||||||
|
{
|
||||||
|
const data = await apiPost<{ rows: PostImportRow[] }> ('/posts/import/preview', {
|
||||||
|
source })
|
||||||
|
const urlIssues = urlIssuesFromRows (data.rows)
|
||||||
|
if (urlIssues.length > 0)
|
||||||
|
{
|
||||||
|
setSourceIssues (urlIssues)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
const sessionId = createPostImportSessionId ()
|
||||||
|
const saved = savePostImportSession (sessionId, {
|
||||||
|
source,
|
||||||
|
rows: data.rows,
|
||||||
|
repairMode: 'all' }, message =>
|
||||||
|
toast ({ title: '取込状態を保存できませんでした', description: message }))
|
||||||
|
if (!(saved))
|
||||||
|
return
|
||||||
|
navigate (`/posts/import/${ sessionId }/review`)
|
||||||
|
}
|
||||||
|
catch (requestError)
|
||||||
|
{
|
||||||
|
const message =
|
||||||
|
isApiError<{ message?: string, baseErrors?: string[] }> (requestError)
|
||||||
|
? requestError.response?.data?.message
|
||||||
|
?? requestError.response?.data?.baseErrors?.[0]
|
||||||
|
: undefined
|
||||||
|
setSourceError (message ?? '入力を確認してください.')
|
||||||
|
toast ({
|
||||||
|
title: '投稿情報の取得に失敗しました',
|
||||||
|
description: message ?? '入力を確認してください.' })
|
||||||
|
}
|
||||||
|
finally
|
||||||
|
{
|
||||||
|
setLoading (false)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
if (!(editable))
|
||||||
|
return <Forbidden/>
|
||||||
|
|
||||||
|
return (
|
||||||
|
<MainArea>
|
||||||
|
<Helmet>
|
||||||
|
<title>{`投稿インポート | ${ SITE_TITLE }`}</title>
|
||||||
|
</Helmet>
|
||||||
|
|
||||||
|
<div className="mx-auto max-w-4xl space-y-4 p-4">
|
||||||
|
<PageTitle>投稿インポート</PageTitle>
|
||||||
|
<div className="rounded-lg border bg-white p-4 dark:border-neutral-700
|
||||||
|
dark:bg-neutral-900 md:p-6">
|
||||||
|
<div className="space-y-4">
|
||||||
|
<FormField label="URL リスト">
|
||||||
|
{() => (
|
||||||
|
<textarea
|
||||||
|
value={source}
|
||||||
|
rows={14}
|
||||||
|
aria-invalid={messages.length > 0 || sourceIssues.length > 0}
|
||||||
|
aria-describedby={sourceDescribedBy || undefined}
|
||||||
|
className={inputClass (
|
||||||
|
messages.length > 0 || sourceIssues.length > 0,
|
||||||
|
'font-mono text-sm',
|
||||||
|
)}
|
||||||
|
onBlur={() => {
|
||||||
|
savePostImportSourceDraft (source, message =>
|
||||||
|
toast ({
|
||||||
|
title: '入力内容を保存できませんでした',
|
||||||
|
description: message }))
|
||||||
|
}}
|
||||||
|
onChange={ev => {
|
||||||
|
editedRef.current = true
|
||||||
|
setSource (ev.target.value)
|
||||||
|
setSourceError (null)
|
||||||
|
setSourceIssues ([])
|
||||||
|
}}/>)}
|
||||||
|
</FormField>
|
||||||
|
<div id={messages.length > 0 ? SOURCE_ERROR_ID : undefined}>
|
||||||
|
<FieldError messages={messages}/>
|
||||||
|
</div>
|
||||||
|
<ul
|
||||||
|
id={SOURCE_ISSUES_ID}
|
||||||
|
className="space-y-2 text-sm text-red-700 dark:text-red-300">
|
||||||
|
{sourceIssues.map (issue => (
|
||||||
|
<li key={`${ issue.sourceRow }-${ issue.message }-${ issue.url }`}>
|
||||||
|
<div>{issue.sourceRow} 行目: {issue.message}</div>
|
||||||
|
<div className="break-all font-mono text-xs">{issue.url}</div>
|
||||||
|
</li>))}
|
||||||
|
</ul>
|
||||||
|
|
||||||
|
<div className="flex items-center justify-between gap-3">
|
||||||
|
<div className="text-sm text-neutral-600 dark:text-neutral-300">
|
||||||
|
入力行数 {lineCount} / {MAX_ROWS}
|
||||||
|
</div>
|
||||||
|
<Button
|
||||||
|
type="button"
|
||||||
|
className="shrink-0"
|
||||||
|
onClick={preview}
|
||||||
|
disabled={loading}>
|
||||||
|
次へ
|
||||||
|
</Button>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
</MainArea>)
|
||||||
|
}
|
||||||
|
|
||||||
|
export default PostImportSourcePage
|
||||||
@@ -25,7 +25,13 @@ import type { User } from '@/types'
|
|||||||
type Props = { user: User | null }
|
type Props = { user: User | null }
|
||||||
|
|
||||||
type PostFormField =
|
type PostFormField =
|
||||||
'url' | 'title' | 'tags' | 'parentPostIds' | 'videoMs' | 'originalCreatedAt' | 'thumbnail'
|
'url'
|
||||||
|
| 'title'
|
||||||
|
| 'tags'
|
||||||
|
| 'parentPostIds'
|
||||||
|
| 'videoMs'
|
||||||
|
| 'originalCreatedAt'
|
||||||
|
| 'thumbnail'
|
||||||
|
|
||||||
|
|
||||||
const PostNewPage: FC<Props> = ({ user }) => {
|
const PostNewPage: FC<Props> = ({ user }) => {
|
||||||
@@ -50,8 +56,10 @@ const PostNewPage: FC<Props> = ({ user }) => {
|
|||||||
|
|
||||||
const thumbnailPreviewRef = useRef ('')
|
const thumbnailPreviewRef = useRef ('')
|
||||||
const videoFlg =
|
const videoFlg =
|
||||||
useMemo (() => tags.split (/\s+/).some (tag => tag.replace (/\[.*\]$/, '') === '動画'),
|
useMemo (() =>
|
||||||
[tags])
|
tags.split (/\s+/).some (
|
||||||
|
tag => tag.replace (/\[.*\]$/, '') === '動画'),
|
||||||
|
[tags])
|
||||||
|
|
||||||
const handleSubmit = async () => {
|
const handleSubmit = async () => {
|
||||||
clearValidationErrors ()
|
clearValidationErrors ()
|
||||||
@@ -79,7 +87,7 @@ const PostNewPage: FC<Props> = ({ user }) => {
|
|||||||
catch (e)
|
catch (e)
|
||||||
{
|
{
|
||||||
applyValidationError (e)
|
applyValidationError (e)
|
||||||
toast ({ title: '投稿失敗', description: '入力を確認してください。' })
|
toast ({ title: '投稿失敗', description: '入力を確認してください.' })
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -105,7 +113,8 @@ const PostNewPage: FC<Props> = ({ user }) => {
|
|||||||
try
|
try
|
||||||
{
|
{
|
||||||
const data = await apiGet<Blob> ('/preview/thumbnail',
|
const data = await apiGet<Blob> ('/preview/thumbnail',
|
||||||
{ params: { url }, responseType: 'blob' })
|
{ params: { url },
|
||||||
|
responseType: 'blob' })
|
||||||
const imageURL = URL.createObjectURL (data)
|
const imageURL = URL.createObjectURL (data)
|
||||||
setThumbnailPreview (imageURL)
|
setThumbnailPreview (imageURL)
|
||||||
setThumbnailFile (new File ([data],
|
setThumbnailFile (new File ([data],
|
||||||
|
|||||||
新しい課題から参照
ユーザをブロックする