コミットを比較
4 コミット
bc660676ef
...
b0c24f319a
| 作成者 | SHA1 | 日付 | |
|---|---|---|---|
| b0c24f319a | |||
| f9463f383f | |||
| 6d037192c4 | |||
| d035da99ad |
@@ -119,13 +119,8 @@ npm run preview
|
|||||||
parameter-list `)` ではない。delimiter の役割を見誤らないこと。
|
parameter-list `)` ではない。delimiter の役割を見誤らないこと。
|
||||||
- In Ruby, when an `if` condition is split across multiple lines and combines
|
- In Ruby, when an `if` condition is split across multiple lines and combines
|
||||||
clauses with `&&` or `||`, wrap the whole condition in parentheses.
|
clauses with `&&` or `||`, wrap the whole condition in parentheses.
|
||||||
- Ruby hashes are not blocks; keep `}` on the same line as the final pair.
|
- Ruby hash / block / call delimiter の詳細は、下の
|
||||||
- Ruby hashes keep the first pair on the same line as `{` unless line length
|
`Ruby delimiter and wrapping rules` を正本として扱ふこと。
|
||||||
requires a break.
|
|
||||||
- Short Ruby hashes may stay visually compact across two lines with the first
|
|
||||||
pair kept on the opening line and aligned continuation pairs below it.
|
|
||||||
- Ruby blocks use separate `{ ... }` rules from hashes, with 2-space body
|
|
||||||
indentation.
|
|
||||||
- For arrays, never put whitespace or a line break immediately before `]`.
|
- For arrays, never put whitespace or a line break immediately before `]`.
|
||||||
- Keep the first element on the same line as `[` by default.
|
- Keep the first element on the same line as `[` by default.
|
||||||
- If an array would exceed the line limit, break after `[` and indent
|
- If an array would exceed the line limit, break after `[` and indent
|
||||||
@@ -141,140 +136,72 @@ npm run preview
|
|||||||
- Ruby の multi-line hash literal / keyword-like argument hash では、opening
|
- Ruby の multi-line hash literal / keyword-like argument hash では、opening
|
||||||
`{` を最初の pair と同じ行に置き、closing `}` を最後の pair と同じ行に
|
`{` を最初の pair と同じ行に置き、closing `}` を最後の pair と同じ行に
|
||||||
置く。Prettier 的な縦開き・縦閉じをしない。
|
置く。Prettier 的な縦開き・縦閉じをしない。
|
||||||
- Ruby の `if` / `unless` / `case` 条件で、複数行に分けるだけで安易に
|
- Ruby の guard 条件は、1 行で収まるなら modifier 形式を優先する。
|
||||||
`if ... end` へ展開しない。局所の既存コードが modifier 形式ならそれに
|
99 文字を超えるなら block 形式へ切り替へるか、message 定数化などで縮める。
|
||||||
そろえる。
|
- Ruby の method chain や call argument を折り返す際、call-site の `)` を
|
||||||
- Ruby の guard 条件は、1 行で収まるなら modifier 形式を優先する。2 行以上に
|
block close のやうに独立させない。
|
||||||
なるなら通常の block 形式へ切り替へてよい。
|
|
||||||
- Ruby の method chain や call argument を折り返す際、call-site の `)`
|
|
||||||
直前で行を空けたり、closing delimiter を block のやうに独立させない。
|
|
||||||
|
|
||||||
Bad:
|
Bad:
|
||||||
|
|
||||||
```rb
|
```rb
|
||||||
source =
|
response = Example.fetch(
|
||||||
if params[:format] == 'google_sheets'
|
value,
|
||||||
PostImportGoogleSheetsFetcher.fetch!(params[:source],
|
option: option,
|
||||||
rate_key: current_user.id)
|
|
||||||
else
|
|
||||||
params[:source]
|
|
||||||
end
|
|
||||||
parsed = PostImportSourceParser.new(
|
|
||||||
source:,
|
|
||||||
format:,
|
|
||||||
has_header: params[:has_header],
|
|
||||||
json_path: params[:json_path],
|
|
||||||
).parse
|
|
||||||
```
|
|
||||||
|
|
||||||
Good:
|
|
||||||
|
|
||||||
```rb
|
|
||||||
source =
|
|
||||||
if params[:format] == 'google_sheets'
|
|
||||||
PostImportGoogleSheetsFetcher.fetch!(params[:source],
|
|
||||||
rate_key: current_user.id)
|
|
||||||
else
|
|
||||||
params[:source]
|
|
||||||
end
|
|
||||||
parsed = PostImportSourceParser.new(
|
|
||||||
source:,
|
|
||||||
format:,
|
|
||||||
has_header: params[:has_header],
|
|
||||||
json_path: params[:json_path]).parse
|
|
||||||
```
|
|
||||||
|
|
||||||
Bad:
|
|
||||||
|
|
||||||
```rb
|
|
||||||
result[field] = {
|
|
||||||
'kind' => kind,
|
|
||||||
'value' => value['value'].to_s,
|
|
||||||
'columns' => Array(columns).map { Integer(_1) },
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
Good:
|
|
||||||
|
|
||||||
```rb
|
|
||||||
result[field] = {
|
|
||||||
'kind' => kind,
|
|
||||||
'value' => value['value'].to_s,
|
|
||||||
'columns' => Array(columns).map { Integer(_1) } }
|
|
||||||
```
|
|
||||||
|
|
||||||
Bad:
|
|
||||||
|
|
||||||
```rb
|
|
||||||
unless rows.all? { |row|
|
|
||||||
values = Array(row['values'])
|
|
||||||
values.length <= columns.length &&
|
|
||||||
values.all? { |value|
|
|
||||||
value.is_a?(String) &&
|
|
||||||
value.bytesize <= PostImportSourceParser::MAX_CELL_BYTES
|
|
||||||
}
|
|
||||||
}
|
|
||||||
raise ArgumentError, '解析結果のセルが不正です.'
|
|
||||||
end
|
|
||||||
```
|
|
||||||
|
|
||||||
Good:
|
|
||||||
|
|
||||||
```rb
|
|
||||||
raise ArgumentError, '解析結果のセルが不正です.' unless rows.all? { |row|
|
|
||||||
values = Array(row['values'])
|
|
||||||
values.length <= columns.length && values.all? { |value|
|
|
||||||
value.is_a?(String) && value.bytesize <= PostImportSourceParser::MAX_CELL_BYTES
|
|
||||||
}
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
Bad:
|
|
||||||
|
|
||||||
```rb
|
|
||||||
if row['url'].to_s.bytesize > PostImportSourceParser::MAX_CELL_BYTES
|
|
||||||
raise ArgumentError, 'URL が長すぎます.'
|
|
||||||
end
|
|
||||||
```
|
|
||||||
|
|
||||||
Good:
|
|
||||||
|
|
||||||
```rb
|
|
||||||
raise ArgumentError, 'URL が長すぎます.' if row['url'].to_s.bytesize > PostImportSourceParser::MAX_CELL_BYTES
|
|
||||||
```
|
|
||||||
|
|
||||||
Bad:
|
|
||||||
|
|
||||||
```rb
|
|
||||||
parameters = row.is_a?(ActionController::Parameters) ? row :
|
|
||||||
ActionController::Parameters.new(row)
|
|
||||||
```
|
|
||||||
|
|
||||||
Good:
|
|
||||||
|
|
||||||
```rb
|
|
||||||
parameters =
|
|
||||||
row.is_a?(ActionController::Parameters) ? row : ActionController::Parameters.new(row)
|
|
||||||
```
|
|
||||||
|
|
||||||
Bad:
|
|
||||||
|
|
||||||
```rb
|
|
||||||
PostTagSection.create!(
|
|
||||||
post_id: post.id,
|
|
||||||
tag_id:,
|
|
||||||
begin_ms:,
|
|
||||||
end_ms:,
|
|
||||||
)
|
)
|
||||||
```
|
```
|
||||||
|
|
||||||
Good:
|
Good:
|
||||||
|
|
||||||
```rb
|
```rb
|
||||||
PostTagSection.create!(post_id: post.id,
|
response = Example.fetch(
|
||||||
tag_id:,
|
value,
|
||||||
begin_ms:,
|
option: option)
|
||||||
end_ms:)
|
```
|
||||||
|
|
||||||
|
Bad:
|
||||||
|
|
||||||
|
```rb
|
||||||
|
payload = {
|
||||||
|
title: title,
|
||||||
|
url: url,
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
Good:
|
||||||
|
|
||||||
|
```rb
|
||||||
|
payload = {
|
||||||
|
title: title,
|
||||||
|
url: url }
|
||||||
|
```
|
||||||
|
|
||||||
|
Bad:
|
||||||
|
|
||||||
|
```rb
|
||||||
|
raise ArgumentError, 'URL が長すぎます.' if url.bytesize > MAX_URL_BYTES && flag.present?
|
||||||
|
```
|
||||||
|
|
||||||
|
Good:
|
||||||
|
|
||||||
|
```rb
|
||||||
|
if url.bytesize > MAX_URL_BYTES && flag.present?
|
||||||
|
raise ArgumentError, 'URL が長すぎます.'
|
||||||
|
end
|
||||||
|
```
|
||||||
|
|
||||||
|
Bad:
|
||||||
|
|
||||||
|
```rb
|
||||||
|
records.each {
|
||||||
|
do_work(_1) }
|
||||||
|
```
|
||||||
|
|
||||||
|
Good:
|
||||||
|
|
||||||
|
```rb
|
||||||
|
records.each {
|
||||||
|
do_work(_1)
|
||||||
|
}
|
||||||
```
|
```
|
||||||
- TypeScript and Python: use GNU-style spacing before parentheses where
|
- TypeScript and Python: use GNU-style spacing before parentheses where
|
||||||
syntactically valid.
|
syntactically valid.
|
||||||
|
|||||||
@@ -6,11 +6,11 @@ class PostImportsController < ApplicationController
|
|||||||
rows: PostImportUrlListParser.parse(params[:source]))
|
rows: PostImportUrlListParser.parse(params[:source]))
|
||||||
render json: { rows: }
|
render json: { rows: }
|
||||||
rescue ArgumentError => e
|
rescue ArgumentError => e
|
||||||
render_bad_request(e.message)
|
render_bad_request e.message
|
||||||
end
|
end
|
||||||
|
|
||||||
def validate
|
def validate
|
||||||
rows = normalised_import_rows(allow_warning_fields: true)
|
rows = normalised_import_rows allow_warning_fields: true
|
||||||
changed_row = Integer(params[:changed_row], exception: false)
|
changed_row = Integer(params[:changed_row], exception: false)
|
||||||
result =
|
result =
|
||||||
PostImportPreviewer.new.preview_rows(rows:,
|
PostImportPreviewer.new.preview_rows(rows:,
|
||||||
@@ -18,7 +18,7 @@ class PostImportsController < ApplicationController
|
|||||||
metadata_cache: { })
|
metadata_cache: { })
|
||||||
render json: { rows: result }
|
render json: { rows: result }
|
||||||
rescue ArgumentError => e
|
rescue ArgumentError => e
|
||||||
render_bad_request(e.message)
|
render_bad_request e.message
|
||||||
end
|
end
|
||||||
|
|
||||||
def create
|
def create
|
||||||
@@ -26,7 +26,7 @@ class PostImportsController < ApplicationController
|
|||||||
rows: normalised_import_rows).run
|
rows: normalised_import_rows).run
|
||||||
render json: result, status: result[:created].positive? ? :created : :ok
|
render json: result, status: result[:created].positive? ? :created : :ok
|
||||||
rescue ArgumentError => e
|
rescue ArgumentError => e
|
||||||
render_bad_request(e.message)
|
render_bad_request e.message
|
||||||
end
|
end
|
||||||
|
|
||||||
private
|
private
|
||||||
|
|||||||
@@ -148,7 +148,8 @@ class PostsController < ApplicationController
|
|||||||
original_created_from: params[:original_created_from],
|
original_created_from: params[:original_created_from],
|
||||||
original_created_before: params[:original_created_before],
|
original_created_before: params[:original_created_before],
|
||||||
parent_post_ids: parse_parent_post_ids,
|
parent_post_ids: parse_parent_post_ids,
|
||||||
video_ms: params[:video_ms], duration: params[:duration] }).create!
|
video_ms: params[:video_ms],
|
||||||
|
duration: params[:duration] }).create!
|
||||||
|
|
||||||
post.reload
|
post.reload
|
||||||
render json: PostRepr.base(post), status: :created
|
render json: PostRepr.base(post), status: :created
|
||||||
|
|||||||
@@ -174,14 +174,5 @@ class Post < ApplicationRecord
|
|||||||
return if url.blank?
|
return if url.blank?
|
||||||
|
|
||||||
self.url = PostUrlNormaliser.normalise(url) || url.strip
|
self.url = PostUrlNormaliser.normalise(url) || url.strip
|
||||||
|
|
||||||
u = URI.parse(url)
|
|
||||||
return unless u in URI::HTTP
|
|
||||||
|
|
||||||
u.host = u.host.downcase if u.host
|
|
||||||
u.path = u.path.sub(/\/\Z/, '') if u.path.present?
|
|
||||||
self.url = PostUrlSanitisationRule.sanitise(u.to_s)
|
|
||||||
rescue URI::InvalidURIError
|
|
||||||
;
|
|
||||||
end
|
end
|
||||||
end
|
end
|
||||||
|
|||||||
@@ -73,11 +73,15 @@ class PostCreator
|
|||||||
|
|
||||||
sections.each_value do |ranges|
|
sections.each_value do |ranges|
|
||||||
ranges.each do |begin_ms, end_ms|
|
ranges.each do |begin_ms, end_ms|
|
||||||
next if begin_ms < video_ms && (!end_ms || end_ms <= video_ms)
|
|
||||||
|
|
||||||
post = Post.new
|
post = Post.new
|
||||||
post.errors.add :video_ms, 'タグ区間が動画時間の範囲外です.'
|
if begin_ms >= video_ms
|
||||||
raise ActiveRecord::RecordInvalid, post
|
post.errors.add :video_ms, 'タグ区間の開始が動画時間以上です.'
|
||||||
|
raise ActiveRecord::RecordInvalid, post
|
||||||
|
end
|
||||||
|
if end_ms && end_ms > video_ms
|
||||||
|
post.errors.add :video_ms, 'タグ区間の終端が動画時間を超えてゐます.'
|
||||||
|
raise ActiveRecord::RecordInvalid, post
|
||||||
|
end
|
||||||
end
|
end
|
||||||
end
|
end
|
||||||
end
|
end
|
||||||
|
|||||||
@@ -1,3 +1,5 @@
|
|||||||
|
require 'timeout'
|
||||||
|
|
||||||
class PostImportPreviewer
|
class PostImportPreviewer
|
||||||
FIELDS = [
|
FIELDS = [
|
||||||
'title',
|
'title',
|
||||||
@@ -6,8 +8,7 @@ class PostImportPreviewer
|
|||||||
'original_created_before',
|
'original_created_before',
|
||||||
'duration',
|
'duration',
|
||||||
'tags',
|
'tags',
|
||||||
'parent_post_ids',
|
'parent_post_ids'].freeze
|
||||||
].freeze
|
|
||||||
FETCH_WARNING_FIELDS = ['url', 'title', 'thumbnail_base'].freeze
|
FETCH_WARNING_FIELDS = ['url', 'title', 'thumbnail_base'].freeze
|
||||||
EXISTING_SKIP_WARNING = '既存投稿のためスキップします.'.freeze
|
EXISTING_SKIP_WARNING = '既存投稿のためスキップします.'.freeze
|
||||||
TITLE_FETCH_WARNING = 'タイトルを取得できませんでした.'.freeze
|
TITLE_FETCH_WARNING = 'タイトルを取得できませんでした.'.freeze
|
||||||
@@ -15,9 +16,26 @@ class PostImportPreviewer
|
|||||||
METADATA_FETCH_WARNING = 'メタデータを取得できませんでした.'.freeze
|
METADATA_FETCH_WARNING = 'メタデータを取得できませんでした.'.freeze
|
||||||
|
|
||||||
def preview_rows rows:, fetch_metadata: true, metadata_cache: { }
|
def preview_rows rows:, fetch_metadata: true, metadata_cache: { }
|
||||||
urls = {}
|
prepared_rows = rows.map { prepare_row(_1) }
|
||||||
rows.map { |row|
|
url_counts = prepared_rows.filter_map { _1[:normal_url] }.tally
|
||||||
preview_row(row, urls:, fetch_metadata:, metadata_cache:)
|
existing_posts =
|
||||||
|
Post.where(url: prepared_rows.map { _1[:normal_url] }.compact.uniq).index_by(&:url)
|
||||||
|
known_tags = preload_known_tags(prepared_rows)
|
||||||
|
existing_parent_ids = preload_parent_ids(prepared_rows)
|
||||||
|
preload_metadata!(
|
||||||
|
prepared_rows,
|
||||||
|
fetch_metadata,
|
||||||
|
metadata_cache,
|
||||||
|
existing_posts,
|
||||||
|
url_counts)
|
||||||
|
prepared_rows.map { |row|
|
||||||
|
preview_row(row,
|
||||||
|
fetch_metadata:,
|
||||||
|
metadata_cache:,
|
||||||
|
existing_posts:,
|
||||||
|
url_counts:,
|
||||||
|
known_tags:,
|
||||||
|
existing_parent_ids:)
|
||||||
}
|
}
|
||||||
end
|
end
|
||||||
|
|
||||||
@@ -27,48 +45,69 @@ class PostImportPreviewer
|
|||||||
|
|
||||||
private
|
private
|
||||||
|
|
||||||
def preview_row row, urls:, fetch_metadata:, metadata_cache:
|
def prepare_row row
|
||||||
row = row.symbolize_keys
|
source = row.symbolize_keys
|
||||||
|
url = source[:url].to_s.strip
|
||||||
|
normal_url = normalised_url(url)
|
||||||
|
source.merge(url_text: url, normal_url:)
|
||||||
|
end
|
||||||
|
|
||||||
|
def preview_row row,
|
||||||
|
fetch_metadata:,
|
||||||
|
metadata_cache:,
|
||||||
|
existing_posts:,
|
||||||
|
url_counts:,
|
||||||
|
known_tags:,
|
||||||
|
existing_parent_ids:
|
||||||
attributes = initial_attributes(row)
|
attributes = initial_attributes(row)
|
||||||
provenance = initial_provenance(row)
|
provenance = initial_provenance(row)
|
||||||
tag_sources = initial_tag_sources(row, attributes, provenance)
|
tag_sources = initial_tag_sources(row, attributes, provenance)
|
||||||
field_warnings = initial_field_warnings(row)
|
field_warnings = initial_field_warnings(row)
|
||||||
base_warnings = initial_base_warnings(row)
|
base_warnings = initial_base_warnings(row)
|
||||||
url = row[:url].to_s.strip
|
url = row[:url_text]
|
||||||
provenance['url'] = 'manual'
|
provenance['url'] = 'manual'
|
||||||
normal_url = normalised_url(url)
|
normal_url = row[:normal_url]
|
||||||
url_for_metadata = normal_url || url
|
url_for_metadata = normal_url || url
|
||||||
existing_post = normal_url.present? && Post.exists?(url: normal_url)
|
existing_post = normal_url.present? ? existing_posts[normal_url] : nil
|
||||||
|
|
||||||
validation_errors = {}
|
validation_errors = {}
|
||||||
validation_errors[:url] = ['URL が不正です.'] if normal_url.blank?
|
validation_errors[:url] = ['URL が不正です.'] if normal_url.blank?
|
||||||
validation_errors[:url] = ['URL が重複しています.'] if normal_url.present? && urls.key?(normal_url)
|
if normal_url.present? && url_counts[normal_url].to_i > 1
|
||||||
urls[normal_url] = true if normal_url.present?
|
validation_errors[:url] = ['URL が重複しています.']
|
||||||
|
end
|
||||||
|
|
||||||
if row[:metadata_url].present? && row[:metadata_url] != url_for_metadata
|
if row[:metadata_url].present? && row[:metadata_url] != url_for_metadata
|
||||||
clear_automatic_values!(attributes, provenance, tag_sources)
|
clear_automatic_values!(attributes, provenance, tag_sources)
|
||||||
clear_fetch_warnings!(field_warnings)
|
clear_fetch_warnings!(field_warnings)
|
||||||
end
|
end
|
||||||
|
|
||||||
if normal_url.present? && existing_post
|
if validation_errors.blank? && normal_url.present? && existing_post
|
||||||
add_field_warning!(field_warnings, 'url', EXISTING_SKIP_WARNING)
|
add_field_warning!(field_warnings, 'url', EXISTING_SKIP_WARNING)
|
||||||
attributes['tags'] = merged_tags(tag_sources, provenance['tags'])
|
attributes['tags'] = merged_tags(tag_sources, provenance['tags'])
|
||||||
warnings_present = field_warnings.values.any?(&:present?) || base_warnings.present?
|
warnings_present = field_warnings.values.any?(&:present?) || base_warnings.present?
|
||||||
return {
|
return { source_row: row[:source_row],
|
||||||
source_row: row[:source_row],
|
url: normal_url,
|
||||||
url: normal_url,
|
attributes:,
|
||||||
attributes:,
|
provenance:,
|
||||||
provenance:,
|
tag_sources:,
|
||||||
tag_sources:,
|
metadata_url: url_for_metadata,
|
||||||
metadata_url: url_for_metadata,
|
skip_reason: 'existing',
|
||||||
field_warnings:,
|
existing_post_id: existing_post.id,
|
||||||
base_warnings:,
|
field_warnings:,
|
||||||
validation_errors:,
|
base_warnings:,
|
||||||
status: validation_errors.present? ? 'error' : (warnings_present ? 'warning' : 'ready'),
|
validation_errors:,
|
||||||
}
|
status:
|
||||||
|
if validation_errors.present?
|
||||||
|
'error'
|
||||||
|
elsif warnings_present
|
||||||
|
'warning'
|
||||||
|
else
|
||||||
|
'ready'
|
||||||
|
end }
|
||||||
end
|
end
|
||||||
|
|
||||||
should_fetch = should_fetch_metadata?(fetch_metadata, row[:source_row].to_i)
|
should_fetch = validation_errors.blank?
|
||||||
|
should_fetch &&= should_fetch_metadata?(fetch_metadata, row[:source_row].to_i)
|
||||||
if should_fetch
|
if should_fetch
|
||||||
clear_fetch_warnings!(field_warnings)
|
clear_fetch_warnings!(field_warnings)
|
||||||
metadata = metadata_for(url_for_metadata, metadata_cache)
|
metadata = metadata_for(url_for_metadata, metadata_cache)
|
||||||
@@ -80,24 +119,25 @@ class PostImportPreviewer
|
|||||||
validate_basic_data(attributes, validation_errors)
|
validate_basic_data(attributes, validation_errors)
|
||||||
validate_preview_tags(merged_tags(tag_sources, provenance['tags']),
|
validate_preview_tags(merged_tags(tag_sources, provenance['tags']),
|
||||||
validation_errors,
|
validation_errors,
|
||||||
field_warnings)
|
field_warnings,
|
||||||
validate_parents(attributes['parent_post_ids'], validation_errors)
|
known_tags)
|
||||||
|
validate_parents(attributes['parent_post_ids'], validation_errors, existing_parent_ids)
|
||||||
attributes.delete('url')
|
attributes.delete('url')
|
||||||
attributes['tags'] = merged_tags(tag_sources, provenance['tags'])
|
attributes['tags'] = merged_tags(tag_sources, provenance['tags'])
|
||||||
|
|
||||||
warnings_present = field_warnings.values.any?(&:present?) || base_warnings.present?
|
warnings_present = field_warnings.values.any?(&:present?) || base_warnings.present?
|
||||||
{
|
{ source_row: row[:source_row],
|
||||||
source_row: row[:source_row],
|
|
||||||
url: normal_url || url,
|
url: normal_url || url,
|
||||||
attributes:,
|
attributes:,
|
||||||
provenance:,
|
provenance:,
|
||||||
tag_sources:,
|
tag_sources:,
|
||||||
metadata_url: url_for_metadata,
|
metadata_url: url_for_metadata,
|
||||||
|
skip_reason: nil,
|
||||||
|
existing_post_id: nil,
|
||||||
field_warnings:,
|
field_warnings:,
|
||||||
base_warnings:,
|
base_warnings:,
|
||||||
validation_errors:,
|
validation_errors:,
|
||||||
status: validation_errors.present? ? 'error' : (warnings_present ? 'warning' : 'ready'),
|
status: validation_errors.present? ? 'error' : (warnings_present ? 'warning' : 'ready') }
|
||||||
}
|
|
||||||
end
|
end
|
||||||
|
|
||||||
def should_fetch_metadata? fetch_metadata, source_row
|
def should_fetch_metadata? fetch_metadata, source_row
|
||||||
@@ -129,7 +169,9 @@ class PostImportPreviewer
|
|||||||
end
|
end
|
||||||
|
|
||||||
def initial_field_warnings row
|
def initial_field_warnings row
|
||||||
(row[:field_warnings] || { }).stringify_keys.transform_values { |value| Array(value).map(&:to_s) }
|
(row[:field_warnings] || { })
|
||||||
|
.stringify_keys
|
||||||
|
.transform_values { |value| Array(value).map(&:to_s) }
|
||||||
end
|
end
|
||||||
|
|
||||||
def initial_base_warnings row
|
def initial_base_warnings row
|
||||||
@@ -161,18 +203,77 @@ class PostImportPreviewer
|
|||||||
data = PostMetadataFetcher.fetch(url).stringify_keys.compact
|
data = PostMetadataFetcher.fetch(url).stringify_keys.compact
|
||||||
warnings = { }
|
warnings = { }
|
||||||
add_field_warning!(warnings, 'title', TITLE_FETCH_WARNING) if data['title'].blank?
|
add_field_warning!(warnings, 'title', TITLE_FETCH_WARNING) if data['title'].blank?
|
||||||
add_field_warning!(warnings, 'thumbnail_base', THUMBNAIL_FETCH_WARNING) if data['thumbnail_base'].blank?
|
if data['thumbnail_base'].blank?
|
||||||
|
add_field_warning!(warnings, 'thumbnail_base', THUMBNAIL_FETCH_WARNING)
|
||||||
|
end
|
||||||
{ data:, warnings: }
|
{ data:, warnings: }
|
||||||
rescue Preview::UrlSafety::UnsafeUrl,
|
rescue Preview::UrlSafety::UnsafeUrl,
|
||||||
Preview::HttpFetcher::FetchFailed,
|
Preview::HttpFetcher::FetchFailed,
|
||||||
Preview::HttpFetcher::ResponseTooLarge => e
|
Preview::HttpFetcher::ResponseTooLarge => e
|
||||||
Rails.logger.info(
|
Rails.logger.info(
|
||||||
"post_import_metadata_fetch_failure "\
|
"post_import_metadata_fetch_failure "\
|
||||||
"#{ { error: e.class.name, message: e.message }.to_json }",
|
"#{ { error: e.class.name, message: e.message }.to_json }")
|
||||||
)
|
|
||||||
{ data: { }, warnings: { 'url' => [METADATA_FETCH_WARNING] } }
|
{ data: { }, warnings: { 'url' => [METADATA_FETCH_WARNING] } }
|
||||||
end
|
end
|
||||||
|
|
||||||
|
def preload_metadata! prepared_rows, fetch_metadata, metadata_cache, existing_posts, url_counts
|
||||||
|
urls = prepared_rows.filter_map { |row|
|
||||||
|
next unless row[:normal_url].present?
|
||||||
|
next unless should_fetch_metadata?(fetch_metadata, row[:source_row].to_i)
|
||||||
|
next if url_counts[row[:normal_url]].to_i > 1
|
||||||
|
next if existing_posts.key?(row[:normal_url])
|
||||||
|
|
||||||
|
row[:normal_url]
|
||||||
|
}.uniq
|
||||||
|
urls = urls.reject { metadata_cache.key?(_1) }
|
||||||
|
return if urls.empty?
|
||||||
|
|
||||||
|
queue = Queue.new
|
||||||
|
urls.each { queue << _1 }
|
||||||
|
workers = [urls.length, 4].min.times.map {
|
||||||
|
Thread.new {
|
||||||
|
loop do
|
||||||
|
url = queue.pop(true)
|
||||||
|
metadata_cache[url] = fetch_metadata(url)
|
||||||
|
rescue ThreadError
|
||||||
|
break
|
||||||
|
end
|
||||||
|
}
|
||||||
|
}
|
||||||
|
Timeout.timeout(15) { workers.each(&:join) }
|
||||||
|
rescue Timeout::Error
|
||||||
|
workers&.each(&:kill)
|
||||||
|
urls.each do |url|
|
||||||
|
metadata_cache[url] ||= { data: { }, warnings: { 'url' => [METADATA_FETCH_WARNING] } }
|
||||||
|
end
|
||||||
|
end
|
||||||
|
|
||||||
|
def preload_known_tags prepared_rows
|
||||||
|
names = prepared_rows.flat_map { |row|
|
||||||
|
attributes = initial_attributes(row)
|
||||||
|
provenance = initial_provenance(row)
|
||||||
|
tag_sources = initial_tag_sources(row, attributes, provenance)
|
||||||
|
preview_tag_names(merged_tags(tag_sources, provenance['tags']))
|
||||||
|
}.compact.uniq
|
||||||
|
return { } if names.empty?
|
||||||
|
|
||||||
|
Tag.joins(:tag_name)
|
||||||
|
.where(tag_names: { name: names })
|
||||||
|
.includes(:tag_name)
|
||||||
|
.to_a
|
||||||
|
.index_by(&:name)
|
||||||
|
end
|
||||||
|
|
||||||
|
def preload_parent_ids prepared_rows
|
||||||
|
ids = prepared_rows.flat_map { |row|
|
||||||
|
attributes = initial_attributes(row)
|
||||||
|
preview_parent_ids(attributes['parent_post_ids'])
|
||||||
|
}.uniq
|
||||||
|
return { } if ids.empty?
|
||||||
|
|
||||||
|
Post.where(id: ids).pluck(:id).to_h { [_1, true] }
|
||||||
|
end
|
||||||
|
|
||||||
def apply_metadata! attributes, provenance, tag_sources, metadata
|
def apply_metadata! attributes, provenance, tag_sources, metadata
|
||||||
metadata.each do |field, value|
|
metadata.each do |field, value|
|
||||||
if field == 'tags'
|
if field == 'tags'
|
||||||
@@ -208,7 +309,19 @@ class PostImportPreviewer
|
|||||||
sources['automatic'].to_s
|
sources['automatic'].to_s
|
||||||
end
|
end
|
||||||
|
|
||||||
def validate_preview_tags raw, errors, field_warnings
|
def preview_tag_names raw
|
||||||
|
names = raw.to_s.split
|
||||||
|
return [] if names.empty?
|
||||||
|
if names.any? { _1.downcase.start_with?('nico:') }
|
||||||
|
return []
|
||||||
|
end
|
||||||
|
|
||||||
|
names.map { |name| TagName.canonicalise(name.sub(/\[.*\]\z/, '')).first }
|
||||||
|
rescue Tag::SectionLiteralParseError
|
||||||
|
[]
|
||||||
|
end
|
||||||
|
|
||||||
|
def validate_preview_tags raw, errors, field_warnings, known_tags
|
||||||
names = raw.to_s.split
|
names = raw.to_s.split
|
||||||
return if names.empty?
|
return if names.empty?
|
||||||
if names.any? { _1.downcase.start_with?('nico:') }
|
if names.any? { _1.downcase.start_with?('nico:') }
|
||||||
@@ -217,7 +330,7 @@ class PostImportPreviewer
|
|||||||
end
|
end
|
||||||
|
|
||||||
parsed = names.map { |name| TagName.canonicalise(name.sub(/\[.*\]\z/, '')).first }
|
parsed = names.map { |name| TagName.canonicalise(name.sub(/\[.*\]\z/, '')).first }
|
||||||
existing = Tag.joins(:tag_name).where(tag_names: { name: parsed }).includes(:tag_name).to_a
|
existing = parsed.filter_map { known_tags[_1] }
|
||||||
deprecated = existing.select(&:deprecated?).map(&:name)
|
deprecated = existing.select(&:deprecated?).map(&:name)
|
||||||
errors[:tags] = ["廃止済みタグがあります: #{ deprecated.join(' ') }"] if deprecated.present?
|
errors[:tags] = ["廃止済みタグがあります: #{ deprecated.join(' ') }"] if deprecated.present?
|
||||||
known = existing.reject(&:deprecated?).map(&:name)
|
known = existing.reject(&:deprecated?).map(&:name)
|
||||||
@@ -253,12 +366,19 @@ class PostImportPreviewer
|
|||||||
nil
|
nil
|
||||||
end
|
end
|
||||||
|
|
||||||
def validate_parents raw, errors
|
def preview_parent_ids raw
|
||||||
|
raw.to_s.split.map { Integer(_1, exception: false) }.compact
|
||||||
|
end
|
||||||
|
|
||||||
|
def validate_parents raw, errors, existing_parent_ids
|
||||||
ids = raw.to_s.split.map { Integer(_1, exception: false) }
|
ids = raw.to_s.split.map { Integer(_1, exception: false) }
|
||||||
return if ids.compact.empty? && raw.to_s.blank?
|
return if ids.compact.empty? && raw.to_s.blank?
|
||||||
|
|
||||||
errors[:parent_post_ids] = ['親投稿 Id. が不正です.'] and return if ids.any? { _1.nil? || _1 <= 0 }
|
if ids.any? { _1.nil? || _1 <= 0 }
|
||||||
if Post.where(id: ids).count != ids.uniq.length
|
errors[:parent_post_ids] = ['親投稿 Id. が不正です.']
|
||||||
|
return
|
||||||
|
end
|
||||||
|
if ids.uniq.any? { !existing_parent_ids[_1] }
|
||||||
errors[:parent_post_ids] = ['存在しない親投稿 Id. があります.']
|
errors[:parent_post_ids] = ['存在しない親投稿 Id. があります.']
|
||||||
end
|
end
|
||||||
end
|
end
|
||||||
|
|||||||
@@ -6,8 +6,7 @@ class PostImportRowNormaliser
|
|||||||
'original_created_from',
|
'original_created_from',
|
||||||
'original_created_before',
|
'original_created_before',
|
||||||
'tags',
|
'tags',
|
||||||
'parent_post_ids',
|
'parent_post_ids'].freeze
|
||||||
].freeze
|
|
||||||
FLEXIBLE_FIELDS = ['duration', 'video_ms'].freeze
|
FLEXIBLE_FIELDS = ['duration', 'video_ms'].freeze
|
||||||
ATTRIBUTE_FIELDS = (STRING_FIELDS + FLEXIBLE_FIELDS).freeze
|
ATTRIBUTE_FIELDS = (STRING_FIELDS + FLEXIBLE_FIELDS).freeze
|
||||||
|
|
||||||
@@ -59,16 +58,14 @@ class PostImportRowNormaliser
|
|||||||
{ attributes: {} },
|
{ attributes: {} },
|
||||||
{ provenance: {} },
|
{ provenance: {} },
|
||||||
{ tag_sources: {} },
|
{ tag_sources: {} },
|
||||||
{ tagSources: {} },
|
{ tagSources: {} }]
|
||||||
]
|
|
||||||
return keys unless allow_warning_fields
|
return keys unless allow_warning_fields
|
||||||
|
|
||||||
keys + [
|
keys + [
|
||||||
{ field_warnings: {} },
|
{ field_warnings: {} },
|
||||||
{ fieldWarnings: {} },
|
{ fieldWarnings: {} },
|
||||||
{ base_warnings: [] },
|
{ base_warnings: [] },
|
||||||
{ baseWarnings: [] },
|
{ baseWarnings: [] }]
|
||||||
]
|
|
||||||
end
|
end
|
||||||
private_class_method :permitted_keys
|
private_class_method :permitted_keys
|
||||||
|
|
||||||
@@ -161,7 +158,9 @@ class PostImportRowNormaliser
|
|||||||
unless base_warnings.is_a?(Array) && base_warnings.all? { _1.is_a?(String) }
|
unless base_warnings.is_a?(Array) && base_warnings.all? { _1.is_a?(String) }
|
||||||
raise ArgumentError, '警告の形式が不正です.'
|
raise ArgumentError, '警告の形式が不正です.'
|
||||||
end
|
end
|
||||||
raise ArgumentError, '警告が大きすぎます.' if base_warnings.any? { _1.bytesize > PostImportUrlListParser::MAX_URL_BYTES }
|
if base_warnings.any? { _1.bytesize > PostImportUrlListParser::MAX_URL_BYTES }
|
||||||
|
raise ArgumentError, '警告が大きすぎます.'
|
||||||
|
end
|
||||||
end
|
end
|
||||||
private_class_method :normalise_warning_values!
|
private_class_method :normalise_warning_values!
|
||||||
|
|
||||||
|
|||||||
@@ -6,40 +6,28 @@ class PostImportRunner
|
|||||||
|
|
||||||
def run
|
def run
|
||||||
normalised_rows = PostImportRowNormaliser.normalise!(@rows)
|
normalised_rows = PostImportRowNormaliser.normalise!(@rows)
|
||||||
|
previews = PostImportPreviewer.new.preview_rows(rows: normalised_rows,
|
||||||
|
fetch_metadata: false)
|
||||||
|
preview_map = previews.index_by { _1[:source_row] }
|
||||||
|
|
||||||
results = normalised_rows.map { |row| run_row(row) }
|
results = normalised_rows.map do |row|
|
||||||
{
|
run_row(row, preview_map.fetch(row['source_row']))
|
||||||
created: results.count { _1[:status] == 'created' },
|
end
|
||||||
|
|
||||||
|
{ created: results.count { _1[:status] == 'created' },
|
||||||
skipped: results.count { _1[:status] == 'skipped' },
|
skipped: results.count { _1[:status] == 'skipped' },
|
||||||
failed: results.count { _1[:status] == 'failed' },
|
failed: results.count { _1[:status] == 'failed' },
|
||||||
rows: results,
|
rows: results }
|
||||||
}
|
|
||||||
end
|
end
|
||||||
|
|
||||||
private
|
private
|
||||||
|
|
||||||
def run_row row
|
def run_row row, preview
|
||||||
attributes = row.fetch('attributes', { }).transform_keys { _1.to_s.underscore }
|
attributes = row.fetch('attributes', { }).transform_keys { _1.to_s.underscore }
|
||||||
preview = PostImportPreviewer.new.preview_rows(
|
return { source_row: row['source_row'],
|
||||||
rows: [{
|
status: 'failed',
|
||||||
source_row: row['source_row'],
|
errors: preview[:validation_errors] } if preview[:validation_errors].present?
|
||||||
url: row['url'],
|
return row.slice('source_row').merge(status: 'skipped') if preview[:skip_reason] == 'existing'
|
||||||
attributes:,
|
|
||||||
provenance: row['provenance'],
|
|
||||||
tag_sources: row['tag_sources'],
|
|
||||||
}],
|
|
||||||
fetch_metadata: false).first
|
|
||||||
if Array(preview.dig(:field_warnings, 'url')).include?(PostImportPreviewer::EXISTING_SKIP_WARNING)
|
|
||||||
return row.slice('source_row').merge(status: 'skipped')
|
|
||||||
end
|
|
||||||
|
|
||||||
if preview[:validation_errors].present?
|
|
||||||
return {
|
|
||||||
source_row: row['source_row'],
|
|
||||||
status: 'failed',
|
|
||||||
errors: preview[:validation_errors],
|
|
||||||
}
|
|
||||||
end
|
|
||||||
|
|
||||||
attributes['tags'] = preview[:attributes]['tags']
|
attributes['tags'] = preview[:attributes]['tags']
|
||||||
attributes['url'] = row['url']
|
attributes['url'] = row['url']
|
||||||
@@ -48,15 +36,22 @@ class PostImportRunner
|
|||||||
rescue ActiveRecord::RecordInvalid => e
|
rescue ActiveRecord::RecordInvalid => e
|
||||||
{ source_row: row['source_row'], status: 'failed', errors: e.record.errors.to_hash }
|
{ source_row: row['source_row'], status: 'failed', errors: e.record.errors.to_hash }
|
||||||
rescue Tag::NicoTagNormalisationError
|
rescue Tag::NicoTagNormalisationError
|
||||||
{ source_row: row['source_row'], status: 'failed', errors: { tags: ['ニコニコ・タグは直接指定できません.'] } }
|
{ source_row: row['source_row'],
|
||||||
|
status: 'failed',
|
||||||
|
errors: { tags: ['ニコニコ・タグは直接指定できません.'] } }
|
||||||
rescue Tag::DeprecatedTagNormalisationError
|
rescue Tag::DeprecatedTagNormalisationError
|
||||||
{ source_row: row['source_row'], status: 'failed', errors: { tags: ['廃止済みタグは付与できません.'] } }
|
{ source_row: row['source_row'],
|
||||||
rescue ArgumentError, PostCreator::VideoMsParseError
|
status: 'failed',
|
||||||
{ source_row: row['source_row'], status: 'failed', errors: { base: ['入力値が不正です.'] } }
|
errors: { tags: ['廃止済みタグは付与できません.'] } }
|
||||||
|
rescue ArgumentError
|
||||||
|
{ source_row: row['source_row'],
|
||||||
|
status: 'failed',
|
||||||
|
errors: { base: ['入力値が不正です.'] } }
|
||||||
rescue StandardError => e
|
rescue StandardError => e
|
||||||
Rails.logger.error(
|
Rails.logger.error("post_import_runner_failure #{ { error: e.class.name,
|
||||||
"post_import_runner_failure #{ { error: e.class.name, message: e.message }.to_json }",
|
message: e.message }.to_json }")
|
||||||
)
|
{ source_row: row['source_row'],
|
||||||
{ source_row: row['source_row'], status: 'failed', errors: { base: ['登録中にエラーが発生しました.'] } }
|
status: 'failed',
|
||||||
|
errors: { base: ['登録中にエラーが発生しました.'] } }
|
||||||
end
|
end
|
||||||
end
|
end
|
||||||
|
|||||||
@@ -11,7 +11,9 @@ class PostImportUrlListParser
|
|||||||
url = line.strip
|
url = line.strip
|
||||||
next if url.blank?
|
next if url.blank?
|
||||||
|
|
||||||
raise ArgumentError, "#{ index + 1 } 行目: URL が長すぎます." if url.bytesize > MAX_URL_BYTES
|
if url.bytesize > MAX_URL_BYTES
|
||||||
|
raise ArgumentError, "#{ index + 1 } 行目: URL が長すぎます."
|
||||||
|
end
|
||||||
|
|
||||||
{ source_row: index + 1, url: }
|
{ source_row: index + 1, url: }
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -1,10 +1,15 @@
|
|||||||
class PostMetadataFetcher
|
class PostMetadataFetcher
|
||||||
|
TIMESTAMP_PATTERN =
|
||||||
|
Regexp.new(
|
||||||
|
'\A(\d{4})-(\d{2})-(\d{2})T(\d{2})' \
|
||||||
|
'(?::(\d{2})(?::(\d{2})(?:\.(\d+))?)?)?' \
|
||||||
|
'(Z|[+-]\d{2}:?\d{2})?\z')
|
||||||
|
|
||||||
def self.fetch raw_url
|
def self.fetch raw_url
|
||||||
uri, = Preview::UrlSafety.validate(raw_url)
|
uri, = Preview::UrlSafety.validate(raw_url)
|
||||||
response = Preview::HttpFetcher.fetch(
|
response = Preview::HttpFetcher.fetch(
|
||||||
uri.to_s,
|
uri.to_s,
|
||||||
max_bytes: Preview::ThumbnailFetcher::HTML_MAX_BYTES,
|
max_bytes: Preview::ThumbnailFetcher::HTML_MAX_BYTES)
|
||||||
)
|
|
||||||
metadata = Preview::HtmlMetadataExtractor.extract(response)
|
metadata = Preview::HtmlMetadataExtractor.extract(response)
|
||||||
document = Nokogiri::HTML.parse(response.body)
|
document = Nokogiri::HTML.parse(response.body)
|
||||||
content = lambda { |name|
|
content = lambda { |name|
|
||||||
@@ -19,7 +24,8 @@ class PostMetadataFetcher
|
|||||||
created_range = original_created_range(published)
|
created_range = original_created_range(published)
|
||||||
platform_tags = platform_tags(uri)
|
platform_tags = platform_tags(uri)
|
||||||
{ title: metadata[:title],
|
{ title: metadata[:title],
|
||||||
thumbnail_base: Preview::KnownSiteExtractor.thumbnail_url(uri) || metadata[:image_url],
|
thumbnail_base:
|
||||||
|
Preview::KnownSiteExtractor.thumbnail_url(uri) || metadata[:image_url],
|
||||||
original_created_from: created_range&.first&.iso8601,
|
original_created_from: created_range&.first&.iso8601,
|
||||||
original_created_before: created_range&.last&.iso8601,
|
original_created_before: created_range&.last&.iso8601,
|
||||||
duration: duration&.to_f&.then { _1.positive? ? (_1 * 1_000).round : nil },
|
duration: duration&.to_f&.then { _1.positive? ? (_1 * 1_000).round : nil },
|
||||||
@@ -37,24 +43,80 @@ class PostMetadataFetcher
|
|||||||
return nil if value.blank?
|
return nil if value.blank?
|
||||||
|
|
||||||
raw = value.to_s.strip
|
raw = value.to_s.strip
|
||||||
from = Time.zone.parse(raw)
|
from, before =
|
||||||
zone = '(?:Z|[+-]\d{2}:?\d{2})?'
|
case raw
|
||||||
before =
|
when /\A(\d{4})\z/
|
||||||
case raw
|
year = Regexp.last_match(1).to_i
|
||||||
when /\A\d{4}\z/ then from + 1.year
|
from = Time.zone.local(year, 1, 1)
|
||||||
when /\A\d{4}-\d{2}\z/ then from + 1.month
|
[from, from + 1.year]
|
||||||
when /\A\d{4}-\d{2}-\d{2}\z/ then from + 1.day
|
when /\A(\d{4})-(\d{2})\z/
|
||||||
when /\A\d{4}-\d{2}-\d{2}T\d{2}#{ zone }\z/ then from + 1.hour
|
year = Regexp.last_match(1).to_i
|
||||||
when /\A\d{4}-\d{2}-\d{2}T\d{2}:\d{2}#{ zone }\z/ then from + 1.minute
|
month = Regexp.last_match(2).to_i
|
||||||
when /\A\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}#{ zone }\z/ then from + 1.second
|
from = Time.zone.local(year, month, 1)
|
||||||
when /\A\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}\.(\d+)#{ zone }\z/
|
[from, from + 1.month]
|
||||||
from + (10**(-Regexp.last_match(1).length))
|
when /\A(\d{4})-(\d{2})-(\d{2})\z/
|
||||||
else
|
year = Regexp.last_match(1).to_i
|
||||||
return nil
|
month = Regexp.last_match(2).to_i
|
||||||
end
|
day = Regexp.last_match(3).to_i
|
||||||
|
from = Time.zone.local(year, month, day)
|
||||||
|
[from, from + 1.day]
|
||||||
|
else
|
||||||
|
parse_timestamp_range(raw)
|
||||||
|
end
|
||||||
|
return nil if from.nil? || before.nil?
|
||||||
|
|
||||||
[from, before]
|
[from, before]
|
||||||
rescue ArgumentError, TypeError
|
rescue ArgumentError, TypeError
|
||||||
nil
|
nil
|
||||||
end
|
end
|
||||||
private_class_method :platform_tags, :original_created_range
|
|
||||||
|
def self.parse_timestamp_range raw
|
||||||
|
match = raw.match(TIMESTAMP_PATTERN)
|
||||||
|
return nil unless match
|
||||||
|
|
||||||
|
year = match[1].to_i
|
||||||
|
month = match[2].to_i
|
||||||
|
day = match[3].to_i
|
||||||
|
hour = match[4].to_i
|
||||||
|
minute = match[5]&.to_i || 0
|
||||||
|
second = match[6]&.to_i || 0
|
||||||
|
fraction = match[7]
|
||||||
|
offset = match[8]
|
||||||
|
whole_second = second + fractional_seconds(fraction)
|
||||||
|
from =
|
||||||
|
if offset.present?
|
||||||
|
Time.new(year, month, day, hour, minute, whole_second, parse_offset(offset)).in_time_zone
|
||||||
|
else
|
||||||
|
Time.zone.local(year, month, day, hour, minute, whole_second)
|
||||||
|
end
|
||||||
|
before =
|
||||||
|
if fraction.present?
|
||||||
|
from + (10**(-fraction.length))
|
||||||
|
elsif match[6].present?
|
||||||
|
from + 1.second
|
||||||
|
elsif match[5].present?
|
||||||
|
from + 1.minute
|
||||||
|
else
|
||||||
|
from + 1.hour
|
||||||
|
end
|
||||||
|
[from, before]
|
||||||
|
end
|
||||||
|
|
||||||
|
def self.parse_offset value
|
||||||
|
return '+00:00' if value == 'Z'
|
||||||
|
|
||||||
|
value.match?(/\A[+-]\d{2}:\d{2}\z/) ? value : "#{ value[0, 3] }:#{ value[3, 2] }"
|
||||||
|
end
|
||||||
|
|
||||||
|
def self.fractional_seconds value
|
||||||
|
return 0 if value.blank?
|
||||||
|
|
||||||
|
Rational(value.to_i, 10**value.length)
|
||||||
|
end
|
||||||
|
|
||||||
|
private_class_method :platform_tags,
|
||||||
|
:original_created_range,
|
||||||
|
:parse_timestamp_range,
|
||||||
|
:parse_offset,
|
||||||
|
:fractional_seconds
|
||||||
end
|
end
|
||||||
|
|||||||
@@ -6,7 +6,7 @@ class PostUrlNormaliser
|
|||||||
|
|
||||||
uri.host = uri.host.downcase
|
uri.host = uri.host.downcase
|
||||||
uri.path = uri.path.sub(/\/\z/, '') if uri.path.present?
|
uri.path = uri.path.sub(/\/\z/, '') if uri.path.present?
|
||||||
uri.to_s
|
PostUrlSanitisationRule.sanitise(uri.to_s)
|
||||||
rescue URI::InvalidURIError
|
rescue URI::InvalidURIError
|
||||||
nil
|
nil
|
||||||
end
|
end
|
||||||
|
|||||||
@@ -12,12 +12,10 @@ module Preview
|
|||||||
|
|
||||||
Response = Data.define(:body, :content_type, :url)
|
Response = Data.define(:body, :content_type, :url)
|
||||||
|
|
||||||
def self.fetch(raw_url, max_bytes: DEFAULT_MAX_BYTES, redirects: MAX_REDIRECTS,
|
def self.fetch(raw_url,
|
||||||
allowed_hosts: nil)
|
max_bytes: DEFAULT_MAX_BYTES,
|
||||||
|
redirects: MAX_REDIRECTS)
|
||||||
uri, addresses = UrlSafety.validate(raw_url)
|
uri, addresses = UrlSafety.validate(raw_url)
|
||||||
if allowed_hosts && !allowed_hosts.include?(uri.host.downcase)
|
|
||||||
raise FetchFailed, '許可されてゐない redirect 先です.'
|
|
||||||
end
|
|
||||||
response = request(uri, addresses.first, max_bytes)
|
response = request(uri, addresses.first, max_bytes)
|
||||||
|
|
||||||
if response.is_a?(Net::HTTPRedirection)
|
if response.is_a?(Net::HTTPRedirection)
|
||||||
@@ -54,7 +52,7 @@ module Preview
|
|||||||
raise FetchFailed, 'redirect 先が不正です.'
|
raise FetchFailed, 'redirect 先が不正です.'
|
||||||
end
|
end
|
||||||
|
|
||||||
return fetch(redirect_url, max_bytes:, redirects: redirects - 1, allowed_hosts:)
|
return fetch(redirect_url, max_bytes:, redirects: redirects - 1)
|
||||||
end
|
end
|
||||||
|
|
||||||
unless response.is_a?(Net::HTTPSuccess)
|
unless response.is_a?(Net::HTTPSuccess)
|
||||||
|
|||||||
@@ -43,11 +43,29 @@ const originOf = (
|
|||||||
): PostImportOrigin =>
|
): PostImportOrigin =>
|
||||||
row.provenance[field] ?? 'automatic'
|
row.provenance[field] ?? 'automatic'
|
||||||
|
|
||||||
const originalCreatedOrigin = (row: PostImportRow): PostImportOrigin =>
|
const changedOrigin = (
|
||||||
originOf (row, 'originalCreatedFrom') === 'manual'
|
changed: boolean,
|
||||||
|| originOf (row, 'originalCreatedBefore') === 'manual'
|
row: PostImportRow,
|
||||||
? 'manual'
|
field: string,
|
||||||
: 'automatic'
|
): PostImportOrigin =>
|
||||||
|
changed ? 'manual' : originOf (row, field)
|
||||||
|
|
||||||
|
const originalCreatedOrigin = (
|
||||||
|
row: PostImportRow,
|
||||||
|
originalDraft: Draft,
|
||||||
|
draft: Draft,
|
||||||
|
): PostImportOrigin => {
|
||||||
|
const changed =
|
||||||
|
originalDraft.originalCreatedFrom !== draft.originalCreatedFrom
|
||||||
|
|| originalDraft.originalCreatedBefore !== draft.originalCreatedBefore
|
||||||
|
if (changed)
|
||||||
|
return 'manual'
|
||||||
|
|
||||||
|
const manualOrigin =
|
||||||
|
originOf (row, 'originalCreatedFrom') === 'manual'
|
||||||
|
|| originOf (row, 'originalCreatedBefore') === 'manual'
|
||||||
|
return manualOrigin ? 'manual' : 'automatic'
|
||||||
|
}
|
||||||
|
|
||||||
const buildDraft = (row: PostImportRow): Draft => ({
|
const buildDraft = (row: PostImportRow): Draft => ({
|
||||||
url: row.url,
|
url: row.url,
|
||||||
@@ -75,9 +93,13 @@ const PostImportRowDialog: FC<Props> = (
|
|||||||
setDraft (buildDraft (row))
|
setDraft (buildDraft (row))
|
||||||
}, [open, row])
|
}, [open, row])
|
||||||
|
|
||||||
if (!(row) || !(draft))
|
if (row == null || draft == null)
|
||||||
return null
|
return null
|
||||||
|
|
||||||
|
const originalDraft = buildDraft (row)
|
||||||
|
const fieldOrigin = (field: keyof Draft): PostImportOrigin =>
|
||||||
|
changedOrigin (draft[field] !== originalDraft[field], row, field)
|
||||||
|
|
||||||
const update = <Key extends keyof Draft,> (
|
const update = <Key extends keyof Draft,> (
|
||||||
key: Key,
|
key: Key,
|
||||||
value: Draft[Key],
|
value: Draft[Key],
|
||||||
@@ -117,21 +139,21 @@ const PostImportRowDialog: FC<Props> = (
|
|||||||
<DialogTextField
|
<DialogTextField
|
||||||
label="URL"
|
label="URL"
|
||||||
value={draft.url}
|
value={draft.url}
|
||||||
origin={originOf (row, 'url')}
|
origin={fieldOrigin ('url')}
|
||||||
warnings={row.fieldWarnings.url}
|
warnings={row.fieldWarnings.url}
|
||||||
errors={groupedMessages (row.validationErrors.url, row.importErrors?.url)}
|
errors={groupedMessages (row.validationErrors.url, row.importErrors?.url)}
|
||||||
onChange={value => update ('url', value)}/>
|
onChange={value => update ('url', value)}/>
|
||||||
<DialogTextField
|
<DialogTextField
|
||||||
label="タイトル"
|
label="タイトル"
|
||||||
value={draft.title}
|
value={draft.title}
|
||||||
origin={originOf (row, 'title')}
|
origin={fieldOrigin ('title')}
|
||||||
warnings={row.fieldWarnings.title}
|
warnings={row.fieldWarnings.title}
|
||||||
errors={groupedMessages (row.validationErrors.title, row.importErrors?.title)}
|
errors={groupedMessages (row.validationErrors.title, row.importErrors?.title)}
|
||||||
onChange={value => update ('title', value)}/>
|
onChange={value => update ('title', value)}/>
|
||||||
<DialogTextField
|
<DialogTextField
|
||||||
label="サムネール基底 URL"
|
label="サムネール基底 URL"
|
||||||
value={draft.thumbnailBase}
|
value={draft.thumbnailBase}
|
||||||
origin={originOf (row, 'thumbnailBase')}
|
origin={fieldOrigin ('thumbnailBase')}
|
||||||
warnings={row.fieldWarnings.thumbnailBase}
|
warnings={row.fieldWarnings.thumbnailBase}
|
||||||
errors={groupedMessages (
|
errors={groupedMessages (
|
||||||
row.validationErrors.thumbnailBase,
|
row.validationErrors.thumbnailBase,
|
||||||
@@ -139,7 +161,10 @@ const PostImportRowDialog: FC<Props> = (
|
|||||||
)}
|
)}
|
||||||
onChange={value => update ('thumbnailBase', value)}/>
|
onChange={value => update ('thumbnailBase', value)}/>
|
||||||
<PostOriginalCreatedTimeField
|
<PostOriginalCreatedTimeField
|
||||||
labelAddon={<PostImportStatusBadge value={originalCreatedOrigin (row)}/>}
|
labelAddon={
|
||||||
|
<PostImportStatusBadge
|
||||||
|
value={originalCreatedOrigin (row, originalDraft, draft)}/>
|
||||||
|
}
|
||||||
originalCreatedFrom={draft.originalCreatedFrom || null}
|
originalCreatedFrom={draft.originalCreatedFrom || null}
|
||||||
setOriginalCreatedFrom={value => update ('originalCreatedFrom', value ?? '')}
|
setOriginalCreatedFrom={value => update ('originalCreatedFrom', value ?? '')}
|
||||||
originalCreatedBefore={draft.originalCreatedBefore || null}
|
originalCreatedBefore={draft.originalCreatedBefore || null}
|
||||||
@@ -155,7 +180,7 @@ const PostImportRowDialog: FC<Props> = (
|
|||||||
<DialogTextField
|
<DialogTextField
|
||||||
label="動画時間"
|
label="動画時間"
|
||||||
value={draft.duration}
|
value={draft.duration}
|
||||||
origin={originOf (row, 'duration')}
|
origin={fieldOrigin ('duration')}
|
||||||
errors={groupedMessages (
|
errors={groupedMessages (
|
||||||
row.validationErrors.duration,
|
row.validationErrors.duration,
|
||||||
row.validationErrors.videoMs,
|
row.validationErrors.videoMs,
|
||||||
@@ -166,14 +191,14 @@ const PostImportRowDialog: FC<Props> = (
|
|||||||
<DialogAreaField
|
<DialogAreaField
|
||||||
label="タグ"
|
label="タグ"
|
||||||
value={draft.tags}
|
value={draft.tags}
|
||||||
origin={originOf (row, 'tags')}
|
origin={fieldOrigin ('tags')}
|
||||||
warnings={row.fieldWarnings.tags}
|
warnings={row.fieldWarnings.tags}
|
||||||
errors={groupedMessages (row.validationErrors.tags, row.importErrors?.tags)}
|
errors={groupedMessages (row.validationErrors.tags, row.importErrors?.tags)}
|
||||||
onChange={value => update ('tags', value)}/>
|
onChange={value => update ('tags', value)}/>
|
||||||
<DialogTextField
|
<DialogTextField
|
||||||
label="親投稿"
|
label="親投稿"
|
||||||
value={draft.parentPostIds}
|
value={draft.parentPostIds}
|
||||||
origin={originOf (row, 'parentPostIds')}
|
origin={fieldOrigin ('parentPostIds')}
|
||||||
errors={groupedMessages (
|
errors={groupedMessages (
|
||||||
row.validationErrors.parentPostIds,
|
row.validationErrors.parentPostIds,
|
||||||
row.importErrors?.parentPostIds,
|
row.importErrors?.parentPostIds,
|
||||||
|
|||||||
@@ -1,7 +1,6 @@
|
|||||||
import { Button } from '@/components/ui/button'
|
import { Button } from '@/components/ui/button'
|
||||||
import { cn } from '@/lib/utils'
|
import { cn } from '@/lib/utils'
|
||||||
|
|
||||||
import PrefetchLink from '@/components/PrefetchLink'
|
|
||||||
import PostImportStatusBadge from '@/components/posts/import/PostImportStatusBadge'
|
import PostImportStatusBadge from '@/components/posts/import/PostImportStatusBadge'
|
||||||
import ThumbnailPreview from '@/components/posts/import/ThumbnailPreview'
|
import ThumbnailPreview from '@/components/posts/import/ThumbnailPreview'
|
||||||
import { effectivePostImportStatus } from '@/components/posts/import/postImportRowStatus'
|
import { effectivePostImportStatus } from '@/components/posts/import/postImportRowStatus'
|
||||||
@@ -71,10 +70,9 @@ const PostImportRowSummary: FC<Props> = ({ row, onEdit }) => {
|
|||||||
<div
|
<div
|
||||||
className={cn (
|
className={cn (
|
||||||
'hidden items-center gap-4 rounded-lg border p-4 md:grid',
|
'hidden items-center gap-4 rounded-lg border p-4 md:grid',
|
||||||
'md:grid-cols-[4rem_5rem_minmax(0,1fr)_auto_auto_auto]',
|
'md:grid-cols-[4rem_5rem_minmax(0,1fr)_auto_auto]',
|
||||||
toneClass (row),
|
toneClass (row),
|
||||||
'hover:bg-slate-50',
|
'transition-shadow hover:shadow-sm',
|
||||||
'dark:hover:bg-neutral-800',
|
|
||||||
)}>
|
)}>
|
||||||
<div className="space-y-1">
|
<div className="space-y-1">
|
||||||
<div className="text-sm font-medium">#{row.sourceRow}</div>
|
<div className="text-sm font-medium">#{row.sourceRow}</div>
|
||||||
@@ -96,13 +94,6 @@ const PostImportRowSummary: FC<Props> = ({ row, onEdit }) => {
|
|||||||
{summaryDate (row) || '日時未取得'}
|
{summaryDate (row) || '日時未取得'}
|
||||||
{row.attributes.duration ? ` / ${ row.attributes.duration }` : ''}
|
{row.attributes.duration ? ` / ${ row.attributes.duration }` : ''}
|
||||||
</div>
|
</div>
|
||||||
{row.createdPostId && (
|
|
||||||
<PrefetchLink
|
|
||||||
to={`/posts/${ row.createdPostId }`}
|
|
||||||
className="inline-block text-xs text-sky-700 underline
|
|
||||||
dark:text-sky-300">
|
|
||||||
投稿 #{row.createdPostId}
|
|
||||||
</PrefetchLink>)}
|
|
||||||
{warning && (
|
{warning && (
|
||||||
<div className="text-xs text-amber-700 dark:text-amber-200">
|
<div className="text-xs text-amber-700 dark:text-amber-200">
|
||||||
{warning}
|
{warning}
|
||||||
@@ -145,13 +136,6 @@ const PostImportRowSummary: FC<Props> = ({ row, onEdit }) => {
|
|||||||
<div className="flex flex-wrap gap-2">
|
<div className="flex flex-wrap gap-2">
|
||||||
<PostImportStatusBadge value={effectiveStatus}/>
|
<PostImportStatusBadge value={effectiveStatus}/>
|
||||||
</div>
|
</div>
|
||||||
{row.createdPostId && (
|
|
||||||
<PrefetchLink
|
|
||||||
to={`/posts/${ row.createdPostId }`}
|
|
||||||
className="inline-block text-xs text-sky-700 underline
|
|
||||||
dark:text-sky-300">
|
|
||||||
投稿 #{row.createdPostId}
|
|
||||||
</PrefetchLink>)}
|
|
||||||
{error && (
|
{error && (
|
||||||
<div className="text-xs text-red-700 dark:text-red-200">
|
<div className="text-xs text-red-700 dark:text-red-200">
|
||||||
{error}
|
{error}
|
||||||
|
|||||||
@@ -2,22 +2,12 @@ import { cn } from '@/lib/utils'
|
|||||||
|
|
||||||
import type { FC } from 'react'
|
import type { FC } from 'react'
|
||||||
|
|
||||||
import type { PostImportOrigin } from '@/lib/postImportSession'
|
import type { PostImportBadgeValue } from '@/components/posts/import/postImportRowStatus'
|
||||||
|
|
||||||
type BadgeValue =
|
|
||||||
'ready'
|
|
||||||
| 'warning'
|
|
||||||
| 'error'
|
|
||||||
| 'pending'
|
|
||||||
| 'created'
|
|
||||||
| 'skipped'
|
|
||||||
| 'failed'
|
|
||||||
| PostImportOrigin
|
|
||||||
|
|
||||||
type Props = {
|
type Props = {
|
||||||
value: BadgeValue }
|
value: PostImportBadgeValue }
|
||||||
|
|
||||||
const LABELS: Record<BadgeValue, string> = {
|
const LABELS: Record<PostImportBadgeValue, string> = {
|
||||||
ready: '登録可能',
|
ready: '登録可能',
|
||||||
warning: '警告',
|
warning: '警告',
|
||||||
error: '要修正',
|
error: '要修正',
|
||||||
@@ -28,7 +18,7 @@ const LABELS: Record<BadgeValue, string> = {
|
|||||||
automatic: '自動取得',
|
automatic: '自動取得',
|
||||||
manual: '手修正' }
|
manual: '手修正' }
|
||||||
|
|
||||||
const STYLES: Record<BadgeValue, string[]> = {
|
const STYLES: Record<PostImportBadgeValue, string[]> = {
|
||||||
ready: [
|
ready: [
|
||||||
'border-emerald-300 bg-emerald-50 text-emerald-700',
|
'border-emerald-300 bg-emerald-50 text-emerald-700',
|
||||||
'dark:border-emerald-900 dark:bg-emerald-950 dark:text-emerald-200'],
|
'dark:border-emerald-900 dark:bg-emerald-950 dark:text-emerald-200'],
|
||||||
|
|||||||
@@ -1,4 +1,5 @@
|
|||||||
import type { PostImportRow } from '@/lib/postImportSession'
|
import type { PostImportOrigin,
|
||||||
|
PostImportRow } from '@/lib/postImportSession'
|
||||||
|
|
||||||
export type PostImportEffectiveStatus =
|
export type PostImportEffectiveStatus =
|
||||||
'created'
|
'created'
|
||||||
@@ -8,10 +9,18 @@ export type PostImportEffectiveStatus =
|
|||||||
| 'warning'
|
| 'warning'
|
||||||
| 'ready'
|
| 'ready'
|
||||||
|
|
||||||
|
export type PostImportBadgeValue =
|
||||||
|
PostImportEffectiveStatus
|
||||||
|
| 'pending'
|
||||||
|
| PostImportOrigin
|
||||||
|
|
||||||
|
|
||||||
export const effectivePostImportStatus = (
|
export const effectivePostImportStatus = (
|
||||||
row: PostImportRow,
|
row: PostImportRow,
|
||||||
): PostImportEffectiveStatus => {
|
): PostImportEffectiveStatus => {
|
||||||
|
if (Object.keys (row.validationErrors ?? { }).length > 0)
|
||||||
|
return 'error'
|
||||||
|
|
||||||
switch (row.importStatus)
|
switch (row.importStatus)
|
||||||
{
|
{
|
||||||
case 'created':
|
case 'created':
|
||||||
|
|||||||
@@ -0,0 +1,196 @@
|
|||||||
|
import type { PostImportResultRow,
|
||||||
|
PostImportRow } from '@/lib/postImportTypes'
|
||||||
|
|
||||||
|
const hasSkipReason = (row: PostImportRow): boolean =>
|
||||||
|
row.skipReason === 'existing'
|
||||||
|
|
||||||
|
const hasValidationErrors = (row: PostImportRow): boolean =>
|
||||||
|
Object.keys (row.validationErrors ?? { }).length > 0
|
||||||
|
|
||||||
|
const rowChanged = (
|
||||||
|
previous: PostImportRow,
|
||||||
|
next: PostImportRow,
|
||||||
|
): boolean =>
|
||||||
|
previous.url !== next.url
|
||||||
|
|| JSON.stringify (previous.attributes) !== JSON.stringify (next.attributes)
|
||||||
|
|| JSON.stringify (previous.provenance) !== JSON.stringify (next.provenance)
|
||||||
|
|| JSON.stringify (previous.tagSources ?? { }) !== JSON.stringify (next.tagSources ?? { })
|
||||||
|
|
||||||
|
|
||||||
|
export const processableImportRows = (rows: PostImportRow[]): PostImportRow[] =>
|
||||||
|
rows.filter (row => {
|
||||||
|
if (row.importStatus === 'created')
|
||||||
|
return false
|
||||||
|
if (row.importStatus === 'skipped')
|
||||||
|
return false
|
||||||
|
if (row.importStatus === 'failed')
|
||||||
|
return false
|
||||||
|
if (hasValidationErrors (row))
|
||||||
|
return false
|
||||||
|
return row.importStatus == null || row.importStatus === 'pending'
|
||||||
|
})
|
||||||
|
|
||||||
|
export const creatableImportRows = (rows: PostImportRow[]): PostImportRow[] =>
|
||||||
|
processableImportRows (rows).filter (row => !(hasSkipReason (row)))
|
||||||
|
|
||||||
|
|
||||||
|
export const reviewSummaryCounts = (rows: PostImportRow[]) => ({
|
||||||
|
total: rows.length,
|
||||||
|
submittable: rows.filter (row =>
|
||||||
|
creatableImportRows ([row]).length > 0
|
||||||
|
&& !(hasValidationErrors (row))).length,
|
||||||
|
invalid: rows.filter (row => hasValidationErrors (row)).length,
|
||||||
|
skipPlanned: rows.filter (row =>
|
||||||
|
hasSkipReason (row)
|
||||||
|
&& !(hasValidationErrors (row))
|
||||||
|
&& row.importStatus !== 'created').length,
|
||||||
|
created: rows.filter (row => row.importStatus === 'created').length,
|
||||||
|
failed: rows.filter (row => row.importStatus === 'failed').length })
|
||||||
|
|
||||||
|
|
||||||
|
export const resultSummaryCounts = (rows: PostImportRow[]) =>
|
||||||
|
rows.reduce (
|
||||||
|
(counts, row) => {
|
||||||
|
if (hasValidationErrors (row))
|
||||||
|
{
|
||||||
|
++counts.invalid
|
||||||
|
return counts
|
||||||
|
}
|
||||||
|
if (row.importStatus === 'created')
|
||||||
|
{
|
||||||
|
++counts.created
|
||||||
|
return counts
|
||||||
|
}
|
||||||
|
if (row.importStatus === 'skipped')
|
||||||
|
{
|
||||||
|
++counts.skipped
|
||||||
|
return counts
|
||||||
|
}
|
||||||
|
if (row.importStatus === 'failed')
|
||||||
|
++counts.failed
|
||||||
|
return counts
|
||||||
|
},
|
||||||
|
{ created: 0, skipped: 0, failed: 0, invalid: 0 })
|
||||||
|
|
||||||
|
|
||||||
|
export const mergeValidatedImportRows = (
|
||||||
|
current: PostImportRow[],
|
||||||
|
validated: PostImportRow[],
|
||||||
|
): PostImportRow[] => {
|
||||||
|
const validatedMap = new Map (validated.map (row => [row.sourceRow, row]))
|
||||||
|
return current.map (previous => {
|
||||||
|
if (previous.importStatus === 'created')
|
||||||
|
return previous
|
||||||
|
|
||||||
|
const row = validatedMap.get (previous.sourceRow)
|
||||||
|
if (row == null)
|
||||||
|
return previous
|
||||||
|
|
||||||
|
const fieldWarnings =
|
||||||
|
Object.keys (row.fieldWarnings).length > 0 || row.metadataUrl !== previous.metadataUrl
|
||||||
|
? { ...row.fieldWarnings }
|
||||||
|
: { ...previous.fieldWarnings }
|
||||||
|
for (const [field, origin] of Object.entries (row.provenance))
|
||||||
|
{
|
||||||
|
if (origin === 'manual')
|
||||||
|
delete fieldWarnings[field]
|
||||||
|
}
|
||||||
|
|
||||||
|
return {
|
||||||
|
...previous,
|
||||||
|
url: row.url,
|
||||||
|
attributes: row.attributes,
|
||||||
|
provenance: row.provenance,
|
||||||
|
tagSources: row.tagSources,
|
||||||
|
skipReason: row.skipReason,
|
||||||
|
existingPostId: row.existingPostId,
|
||||||
|
fieldWarnings,
|
||||||
|
baseWarnings:
|
||||||
|
row.baseWarnings.length > 0 || row.metadataUrl !== previous.metadataUrl
|
||||||
|
? row.baseWarnings
|
||||||
|
: previous.baseWarnings,
|
||||||
|
validationErrors: row.validationErrors,
|
||||||
|
status: row.status,
|
||||||
|
metadataUrl: row.metadataUrl }
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
export const mergePreviewImportRows = (
|
||||||
|
current: PostImportRow[],
|
||||||
|
preview: PostImportRow[],
|
||||||
|
): PostImportRow[] => {
|
||||||
|
const currentMap = new Map (current.map (row => [row.sourceRow, row]))
|
||||||
|
return preview.map (row => {
|
||||||
|
const previous = currentMap.get (row.sourceRow)
|
||||||
|
if (previous == null)
|
||||||
|
return row
|
||||||
|
if (previous.importStatus === 'created')
|
||||||
|
return previous
|
||||||
|
|
||||||
|
const mergedAttributes = { ...row.attributes }
|
||||||
|
const mergedProvenance = { ...row.provenance }
|
||||||
|
|
||||||
|
for (const [field, origin] of Object.entries (previous.provenance))
|
||||||
|
{
|
||||||
|
if (origin !== 'manual' || field === 'url')
|
||||||
|
continue
|
||||||
|
if (field in previous.attributes)
|
||||||
|
mergedAttributes[field] = previous.attributes[field]
|
||||||
|
mergedProvenance[field] = 'manual'
|
||||||
|
}
|
||||||
|
|
||||||
|
const mergedTagSources =
|
||||||
|
previous.provenance.tags === 'manual'
|
||||||
|
? {
|
||||||
|
automatic: row.tagSources?.automatic ?? '',
|
||||||
|
manual: previous.tagSources?.manual ?? String (previous.attributes.tags ?? '') }
|
||||||
|
: row.tagSources
|
||||||
|
|
||||||
|
const nextRow = {
|
||||||
|
...row,
|
||||||
|
attributes: mergedAttributes,
|
||||||
|
provenance: mergedProvenance,
|
||||||
|
tagSources: mergedTagSources,
|
||||||
|
skipReason: row.skipReason,
|
||||||
|
existingPostId: row.existingPostId,
|
||||||
|
createdPostId: previous.createdPostId,
|
||||||
|
importStatus: previous.importStatus,
|
||||||
|
importErrors: previous.importErrors }
|
||||||
|
|
||||||
|
if (rowChanged (previous, nextRow))
|
||||||
|
{
|
||||||
|
nextRow.importStatus = 'pending'
|
||||||
|
nextRow.importErrors = undefined
|
||||||
|
}
|
||||||
|
return nextRow
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
export const mergeImportResults = (
|
||||||
|
rows: PostImportRow[],
|
||||||
|
results: PostImportResultRow[],
|
||||||
|
): PostImportRow[] => {
|
||||||
|
const resultMap = new Map (results.map (row => [row.sourceRow, row]))
|
||||||
|
return rows.map (row => {
|
||||||
|
const result = resultMap.get (row.sourceRow)
|
||||||
|
return result
|
||||||
|
? {
|
||||||
|
...row,
|
||||||
|
importStatus: result.status,
|
||||||
|
createdPostId: result.post?.id,
|
||||||
|
importErrors: result.errors }
|
||||||
|
: row
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
export const retryImportRow = (
|
||||||
|
rows: PostImportRow[],
|
||||||
|
sourceRow: number,
|
||||||
|
): PostImportRow[] =>
|
||||||
|
rows.map (row =>
|
||||||
|
row.sourceRow === sourceRow && row.importStatus === 'failed'
|
||||||
|
? { ...row, importStatus: 'pending', importErrors: undefined }
|
||||||
|
: row)
|
||||||
@@ -1,614 +1,4 @@
|
|||||||
export type PostImportOrigin = 'automatic' | 'manual'
|
export * from '@/lib/postImportTypes'
|
||||||
export type PostImportRepairMode = 'all' | 'failed'
|
export * from '@/lib/postImportStorage'
|
||||||
export type PostImportStatus =
|
export * from '@/lib/postImportSourceValidation'
|
||||||
'pending'
|
export * from '@/lib/postImportRows'
|
||||||
| 'created'
|
|
||||||
| 'skipped'
|
|
||||||
| 'failed'
|
|
||||||
export type PostImportResultStatus = 'created' | 'skipped' | 'failed'
|
|
||||||
export type PostImportAttributeValue = string | number
|
|
||||||
|
|
||||||
export type PostImportRow = {
|
|
||||||
sourceRow: number
|
|
||||||
url: string
|
|
||||||
attributes: Record<string, PostImportAttributeValue>
|
|
||||||
fieldWarnings: Record<string, string[]>
|
|
||||||
baseWarnings: string[]
|
|
||||||
validationErrors: Record<string, string[]>
|
|
||||||
importErrors?: Record<string, string[]>
|
|
||||||
provenance: Record<string, PostImportOrigin>
|
|
||||||
tagSources?: Record<PostImportOrigin, string>
|
|
||||||
status: 'ready' | 'warning' | 'error'
|
|
||||||
metadataUrl?: string
|
|
||||||
createdPostId?: number
|
|
||||||
importStatus?: PostImportStatus }
|
|
||||||
|
|
||||||
export type PostImportResultRow = {
|
|
||||||
sourceRow: number
|
|
||||||
status: PostImportResultStatus
|
|
||||||
post?: { id: number }
|
|
||||||
errors?: Record<string, string[]> }
|
|
||||||
|
|
||||||
export type PostImportSession = {
|
|
||||||
version: number
|
|
||||||
savedAt: string
|
|
||||||
source: string
|
|
||||||
rows: PostImportRow[]
|
|
||||||
repairMode: PostImportRepairMode }
|
|
||||||
|
|
||||||
export type PostImportSourceIssue = {
|
|
||||||
sourceRow: number
|
|
||||||
message: string
|
|
||||||
url: string }
|
|
||||||
|
|
||||||
type StorageErrorHandler = (message: string) => void
|
|
||||||
|
|
||||||
const SESSION_VERSION = 2
|
|
||||||
const SESSION_PREFIX = 'post-import-session:'
|
|
||||||
const SOURCE_DRAFT_KEY = 'post-import-source-draft'
|
|
||||||
const SESSION_MAX_AGE_MS = 24 * 60 * 60 * 1000
|
|
||||||
const MAX_ROWS = 100
|
|
||||||
const MAX_URL_BYTES = 20 * 1024
|
|
||||||
|
|
||||||
|
|
||||||
const isPlainObject = (value: unknown): value is Record<string, unknown> =>
|
|
||||||
typeof value === 'object' && value != null && !(Array.isArray (value))
|
|
||||||
|
|
||||||
|
|
||||||
const sessionKey = (sessionId: string): string => `${ SESSION_PREFIX }${ sessionId }`
|
|
||||||
|
|
||||||
|
|
||||||
const readStorage = (
|
|
||||||
key: string,
|
|
||||||
onError?: StorageErrorHandler,
|
|
||||||
): string | null => {
|
|
||||||
if (typeof window === 'undefined')
|
|
||||||
return null
|
|
||||||
|
|
||||||
try
|
|
||||||
{
|
|
||||||
return sessionStorage.getItem (key)
|
|
||||||
}
|
|
||||||
catch
|
|
||||||
{
|
|
||||||
onError?.('保存済みデータを読み込めませんでした.')
|
|
||||||
return null
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
const writeStorage = (
|
|
||||||
key: string,
|
|
||||||
value: string,
|
|
||||||
onError?: StorageErrorHandler,
|
|
||||||
): boolean => {
|
|
||||||
if (typeof window === 'undefined')
|
|
||||||
return false
|
|
||||||
|
|
||||||
try
|
|
||||||
{
|
|
||||||
sessionStorage.setItem (key, value)
|
|
||||||
return true
|
|
||||||
}
|
|
||||||
catch
|
|
||||||
{
|
|
||||||
onError?.('ブラウザへ保存できませんでした.')
|
|
||||||
return false
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
const removeStorage = (
|
|
||||||
key: string,
|
|
||||||
onError?: StorageErrorHandler,
|
|
||||||
) => {
|
|
||||||
if (typeof window === 'undefined')
|
|
||||||
return
|
|
||||||
|
|
||||||
try
|
|
||||||
{
|
|
||||||
sessionStorage.removeItem (key)
|
|
||||||
}
|
|
||||||
catch
|
|
||||||
{
|
|
||||||
onError?.('保存済みデータを削除できませんでした.')
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
const truncateUrl = (value: string): string =>
|
|
||||||
value.length > 120 ? `${ value.slice (0, 117) }…` : value
|
|
||||||
|
|
||||||
|
|
||||||
const bytesize = (value: string): number =>
|
|
||||||
new TextEncoder ().encode (value).length
|
|
||||||
|
|
||||||
|
|
||||||
const normaliseImportUrl = (value: string): string | null => {
|
|
||||||
const trimmed = value.trim ()
|
|
||||||
if (!(trimmed))
|
|
||||||
return null
|
|
||||||
|
|
||||||
try
|
|
||||||
{
|
|
||||||
const url = new URL (trimmed)
|
|
||||||
if (!(url.protocol === 'http:' || url.protocol === 'https:'))
|
|
||||||
return null
|
|
||||||
if (!(url.host))
|
|
||||||
return null
|
|
||||||
|
|
||||||
url.hostname = url.hostname.toLowerCase ()
|
|
||||||
if (url.pathname.endsWith ('/'))
|
|
||||||
url.pathname = url.pathname.replace (/\/+$/, '')
|
|
||||||
return url.toString ()
|
|
||||||
}
|
|
||||||
catch
|
|
||||||
{
|
|
||||||
return null
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
const ensureStringListRecord = (value: unknown): Record<string, string[]> | null => {
|
|
||||||
if (!(isPlainObject (value)))
|
|
||||||
return null
|
|
||||||
|
|
||||||
const result: Record<string, string[]> = { }
|
|
||||||
for (const [key, entry] of Object.entries (value))
|
|
||||||
{
|
|
||||||
if (!(Array.isArray (entry)) || !(entry.every (_1 => typeof _1 === 'string')))
|
|
||||||
return null
|
|
||||||
result[key] = entry
|
|
||||||
}
|
|
||||||
return result
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
const isValidStatus = (
|
|
||||||
value: unknown,
|
|
||||||
): value is PostImportRow['status'] =>
|
|
||||||
value === 'ready' || value === 'warning' || value === 'error'
|
|
||||||
|
|
||||||
|
|
||||||
const isValidImportStatus = (
|
|
||||||
value: unknown,
|
|
||||||
): value is PostImportStatus =>
|
|
||||||
value === 'pending'
|
|
||||||
|| value === 'created'
|
|
||||||
|| value === 'skipped'
|
|
||||||
|| value === 'failed'
|
|
||||||
|
|
||||||
|
|
||||||
const isValidOrigin = (
|
|
||||||
value: unknown,
|
|
||||||
): value is PostImportOrigin =>
|
|
||||||
value === 'automatic' || value === 'manual'
|
|
||||||
|
|
||||||
|
|
||||||
const sanitiseRow = (value: unknown): PostImportRow | null => {
|
|
||||||
if (!(isPlainObject (value)))
|
|
||||||
return null
|
|
||||||
if (!(Number.isInteger (value.sourceRow)) || Number (value.sourceRow) <= 0)
|
|
||||||
return null
|
|
||||||
if (typeof value.url !== 'string')
|
|
||||||
return null
|
|
||||||
if (!(isPlainObject (value.attributes)))
|
|
||||||
return null
|
|
||||||
if (!(isPlainObject (value.provenance)))
|
|
||||||
return null
|
|
||||||
if (!(isValidStatus (value.status)))
|
|
||||||
return null
|
|
||||||
if (value.importStatus != null && !(isValidImportStatus (value.importStatus)))
|
|
||||||
return null
|
|
||||||
|
|
||||||
const validationErrors = ensureStringListRecord (value.validationErrors)
|
|
||||||
const fieldWarnings = ensureStringListRecord (value.fieldWarnings)
|
|
||||||
if (validationErrors == null || fieldWarnings == null)
|
|
||||||
return null
|
|
||||||
|
|
||||||
const importErrors =
|
|
||||||
value.importErrors == null
|
|
||||||
? undefined
|
|
||||||
: ensureStringListRecord (value.importErrors)
|
|
||||||
if (value.importErrors != null && importErrors == null)
|
|
||||||
return null
|
|
||||||
if (!(Array.isArray (value.baseWarnings))
|
|
||||||
|| !(value.baseWarnings.every (_1 => typeof _1 === 'string')))
|
|
||||||
return null
|
|
||||||
|
|
||||||
const provenanceEntries = Object.entries (value.provenance)
|
|
||||||
if (!(provenanceEntries.every (([, origin]) => isValidOrigin (origin))))
|
|
||||||
return null
|
|
||||||
|
|
||||||
if (value.tagSources != null)
|
|
||||||
{
|
|
||||||
if (!(isPlainObject (value.tagSources)))
|
|
||||||
return null
|
|
||||||
if (!(Object.values (value.tagSources).every (_1 => typeof _1 === 'string')))
|
|
||||||
return null
|
|
||||||
}
|
|
||||||
|
|
||||||
return {
|
|
||||||
sourceRow: Number (value.sourceRow),
|
|
||||||
url: value.url,
|
|
||||||
attributes: value.attributes as Record<string, PostImportAttributeValue>,
|
|
||||||
fieldWarnings,
|
|
||||||
baseWarnings: value.baseWarnings,
|
|
||||||
validationErrors,
|
|
||||||
importErrors,
|
|
||||||
provenance: value.provenance as Record<string, PostImportOrigin>,
|
|
||||||
tagSources: value.tagSources as Record<PostImportOrigin, string> | undefined,
|
|
||||||
status: value.status,
|
|
||||||
metadataUrl: typeof value.metadataUrl === 'string' ? value.metadataUrl : undefined,
|
|
||||||
createdPostId:
|
|
||||||
Number.isInteger (value.createdPostId) ? Number (value.createdPostId) : undefined,
|
|
||||||
importStatus: value.importStatus }
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
const isExpiredSession = (savedAt: string): boolean => {
|
|
||||||
const value = Date.parse (savedAt)
|
|
||||||
return Number.isNaN (value) || Date.now () - value > SESSION_MAX_AGE_MS
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
export const createPostImportSessionId = (): string =>
|
|
||||||
typeof crypto !== 'undefined' && 'randomUUID' in crypto
|
|
||||||
? crypto.randomUUID ()
|
|
||||||
: `${ Date.now () }-${ Math.random ().toString (36).slice (2) }`
|
|
||||||
|
|
||||||
|
|
||||||
export const cleanupExpiredPostImportSessions = (
|
|
||||||
onError?: StorageErrorHandler,
|
|
||||||
) => {
|
|
||||||
if (typeof window === 'undefined')
|
|
||||||
return
|
|
||||||
|
|
||||||
try
|
|
||||||
{
|
|
||||||
for (let i = 0; i < sessionStorage.length; ++i)
|
|
||||||
{
|
|
||||||
const key = sessionStorage.key (i)
|
|
||||||
if (key == null || !(key.startsWith (SESSION_PREFIX)))
|
|
||||||
continue
|
|
||||||
|
|
||||||
const raw = sessionStorage.getItem (key)
|
|
||||||
if (raw == null)
|
|
||||||
continue
|
|
||||||
try
|
|
||||||
{
|
|
||||||
const value = JSON.parse (raw) as { savedAt?: string }
|
|
||||||
if (typeof value.savedAt !== 'string' || isExpiredSession (value.savedAt))
|
|
||||||
{
|
|
||||||
sessionStorage.removeItem (key)
|
|
||||||
--i
|
|
||||||
}
|
|
||||||
}
|
|
||||||
catch
|
|
||||||
{
|
|
||||||
sessionStorage.removeItem (key)
|
|
||||||
--i
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
catch
|
|
||||||
{
|
|
||||||
onError?.('保存済みデータを整理できませんでした.')
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
export const loadPostImportSourceDraft = (
|
|
||||||
onError?: StorageErrorHandler,
|
|
||||||
): { source: string } => {
|
|
||||||
const raw = readStorage (SOURCE_DRAFT_KEY, onError)
|
|
||||||
if (raw == null)
|
|
||||||
return { source: '' }
|
|
||||||
|
|
||||||
try
|
|
||||||
{
|
|
||||||
const value = JSON.parse (raw) as { source?: string }
|
|
||||||
return { source: typeof value.source === 'string' ? value.source : '' }
|
|
||||||
}
|
|
||||||
catch
|
|
||||||
{
|
|
||||||
return { source: '' }
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
export const savePostImportSourceDraft = (
|
|
||||||
source: string,
|
|
||||||
onError?: StorageErrorHandler,
|
|
||||||
): boolean =>
|
|
||||||
writeStorage (SOURCE_DRAFT_KEY, JSON.stringify ({ source }), onError)
|
|
||||||
|
|
||||||
|
|
||||||
export const clearPostImportSourceDraft = (
|
|
||||||
onError?: StorageErrorHandler,
|
|
||||||
) => {
|
|
||||||
removeStorage (SOURCE_DRAFT_KEY, onError)
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
export const savePostImportSession = (
|
|
||||||
sessionId: string,
|
|
||||||
session: Omit<PostImportSession, 'version' | 'savedAt'>,
|
|
||||||
onError?: StorageErrorHandler,
|
|
||||||
): boolean =>
|
|
||||||
writeStorage (
|
|
||||||
sessionKey (sessionId),
|
|
||||||
JSON.stringify ({
|
|
||||||
...session,
|
|
||||||
version: SESSION_VERSION,
|
|
||||||
savedAt: new Date ().toISOString () }),
|
|
||||||
onError,
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
export const loadPostImportSession = (
|
|
||||||
sessionId: string,
|
|
||||||
onError?: StorageErrorHandler,
|
|
||||||
): PostImportSession | null => {
|
|
||||||
const raw = readStorage (sessionKey (sessionId), onError)
|
|
||||||
if (raw == null)
|
|
||||||
return null
|
|
||||||
|
|
||||||
try
|
|
||||||
{
|
|
||||||
const value = JSON.parse (raw) as Partial<PostImportSession>
|
|
||||||
if (value.version !== SESSION_VERSION || !(Array.isArray (value.rows)))
|
|
||||||
return null
|
|
||||||
if (typeof value.savedAt !== 'string' || isExpiredSession (value.savedAt))
|
|
||||||
{
|
|
||||||
removeStorage (sessionKey (sessionId), onError)
|
|
||||||
return null
|
|
||||||
}
|
|
||||||
|
|
||||||
const rows = value.rows.map (sanitiseRow)
|
|
||||||
if (rows.some (_1 => _1 == null))
|
|
||||||
return null
|
|
||||||
|
|
||||||
return {
|
|
||||||
version: SESSION_VERSION,
|
|
||||||
savedAt: value.savedAt,
|
|
||||||
source: typeof value.source === 'string' ? value.source : '',
|
|
||||||
rows: rows as PostImportRow[],
|
|
||||||
repairMode: value.repairMode === 'failed' ? 'failed' : 'all' }
|
|
||||||
}
|
|
||||||
catch
|
|
||||||
{
|
|
||||||
return null
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
const rowChanged = (
|
|
||||||
previous: PostImportRow,
|
|
||||||
next: PostImportRow,
|
|
||||||
): boolean =>
|
|
||||||
previous.url !== next.url
|
|
||||||
|| JSON.stringify (previous.attributes) !== JSON.stringify (next.attributes)
|
|
||||||
|| JSON.stringify (previous.provenance) !== JSON.stringify (next.provenance)
|
|
||||||
|| JSON.stringify (previous.tagSources ?? { }) !== JSON.stringify (next.tagSources ?? { })
|
|
||||||
|
|
||||||
|
|
||||||
export const mergeValidatedImportRows = (
|
|
||||||
current: PostImportRow[],
|
|
||||||
validated: PostImportRow[],
|
|
||||||
): PostImportRow[] => {
|
|
||||||
const validatedMap = new Map (validated.map (row => [row.sourceRow, row]))
|
|
||||||
return current.map (previous => {
|
|
||||||
if (previous.importStatus === 'created')
|
|
||||||
return previous
|
|
||||||
|
|
||||||
const row = validatedMap.get (previous.sourceRow)
|
|
||||||
if (row == null)
|
|
||||||
return previous
|
|
||||||
|
|
||||||
const fieldWarnings =
|
|
||||||
Object.keys (row.fieldWarnings).length > 0 || row.metadataUrl !== previous.metadataUrl
|
|
||||||
? { ...row.fieldWarnings }
|
|
||||||
: { ...previous.fieldWarnings }
|
|
||||||
for (const [field, origin] of Object.entries (row.provenance))
|
|
||||||
{
|
|
||||||
if (origin === 'manual')
|
|
||||||
delete fieldWarnings[field]
|
|
||||||
}
|
|
||||||
|
|
||||||
return {
|
|
||||||
...previous,
|
|
||||||
url: row.url,
|
|
||||||
attributes: row.attributes,
|
|
||||||
provenance: row.provenance,
|
|
||||||
tagSources: row.tagSources,
|
|
||||||
fieldWarnings,
|
|
||||||
baseWarnings:
|
|
||||||
row.baseWarnings.length > 0 || row.metadataUrl !== previous.metadataUrl
|
|
||||||
? row.baseWarnings
|
|
||||||
: previous.baseWarnings,
|
|
||||||
validationErrors: row.validationErrors,
|
|
||||||
status: row.status,
|
|
||||||
metadataUrl: row.metadataUrl }
|
|
||||||
})
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
export const mergePreviewImportRows = (
|
|
||||||
current: PostImportRow[],
|
|
||||||
preview: PostImportRow[],
|
|
||||||
): PostImportRow[] => {
|
|
||||||
const currentMap = new Map (current.map (row => [row.sourceRow, row]))
|
|
||||||
return preview.map (row => {
|
|
||||||
const previous = currentMap.get (row.sourceRow)
|
|
||||||
if (previous == null)
|
|
||||||
return row
|
|
||||||
if (previous.importStatus === 'created')
|
|
||||||
return previous
|
|
||||||
|
|
||||||
const mergedAttributes = { ...row.attributes }
|
|
||||||
const mergedProvenance = { ...row.provenance }
|
|
||||||
for (const [field, origin] of Object.entries (previous.provenance))
|
|
||||||
{
|
|
||||||
if (origin !== 'manual' || field === 'url')
|
|
||||||
continue
|
|
||||||
if (field in previous.attributes)
|
|
||||||
mergedAttributes[field] = previous.attributes[field]
|
|
||||||
mergedProvenance[field] = 'manual'
|
|
||||||
}
|
|
||||||
|
|
||||||
const mergedTagSources =
|
|
||||||
previous.provenance.tags === 'manual'
|
|
||||||
? {
|
|
||||||
automatic: row.tagSources?.automatic ?? '',
|
|
||||||
manual: previous.tagSources?.manual ?? String (previous.attributes.tags ?? '') }
|
|
||||||
: row.tagSources
|
|
||||||
|
|
||||||
const nextRow = {
|
|
||||||
...row,
|
|
||||||
attributes: mergedAttributes,
|
|
||||||
provenance: mergedProvenance,
|
|
||||||
tagSources: mergedTagSources,
|
|
||||||
createdPostId: previous.createdPostId,
|
|
||||||
importStatus: previous.importStatus,
|
|
||||||
importErrors: previous.importErrors }
|
|
||||||
|
|
||||||
if (rowChanged (previous, nextRow))
|
|
||||||
{
|
|
||||||
nextRow.importStatus = 'pending'
|
|
||||||
nextRow.importErrors = undefined
|
|
||||||
}
|
|
||||||
return nextRow
|
|
||||||
})
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
export const mergeImportResults = (
|
|
||||||
rows: PostImportRow[],
|
|
||||||
results: PostImportResultRow[],
|
|
||||||
): PostImportRow[] => {
|
|
||||||
const resultMap = new Map (results.map (row => [row.sourceRow, row]))
|
|
||||||
return rows.map (row => {
|
|
||||||
const result = resultMap.get (row.sourceRow)
|
|
||||||
return result
|
|
||||||
? {
|
|
||||||
...row,
|
|
||||||
importStatus: result.status,
|
|
||||||
createdPostId: result.post?.id,
|
|
||||||
importErrors: result.errors }
|
|
||||||
: row
|
|
||||||
})
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
export const retryImportRow = (
|
|
||||||
rows: PostImportRow[],
|
|
||||||
sourceRow: number,
|
|
||||||
): PostImportRow[] =>
|
|
||||||
rows.map (row =>
|
|
||||||
row.sourceRow === sourceRow && row.importStatus === 'failed'
|
|
||||||
? { ...row, importStatus: 'pending', importErrors: undefined }
|
|
||||||
: row)
|
|
||||||
|
|
||||||
|
|
||||||
export const countImportSourceLines = (source: string): number =>
|
|
||||||
source
|
|
||||||
.split (/\r\n|\n|\r/)
|
|
||||||
.map (_1 => _1.trim ())
|
|
||||||
.filter (_1 => _1 !== '')
|
|
||||||
.length
|
|
||||||
|
|
||||||
|
|
||||||
export const validateImportSource = (
|
|
||||||
source: string,
|
|
||||||
): PostImportSourceIssue[] => {
|
|
||||||
const lines = source.split (/\r\n|\n|\r/)
|
|
||||||
const issues: PostImportSourceIssue[] = []
|
|
||||||
const seen = new Map<string, number> ()
|
|
||||||
let count = 0
|
|
||||||
|
|
||||||
lines.forEach ((rawLine, index) => {
|
|
||||||
const value = rawLine.trim ()
|
|
||||||
if (!(value))
|
|
||||||
return
|
|
||||||
|
|
||||||
++count
|
|
||||||
const sourceRow = index + 1
|
|
||||||
const displayUrl = truncateUrl (value)
|
|
||||||
const normalised = normaliseImportUrl (value)
|
|
||||||
if (count > MAX_ROWS)
|
|
||||||
{
|
|
||||||
issues.push ({
|
|
||||||
sourceRow,
|
|
||||||
message: `取込件数は ${ MAX_ROWS } 件までです.`,
|
|
||||||
url: displayUrl })
|
|
||||||
return
|
|
||||||
}
|
|
||||||
if (!(value.startsWith ('http://') || value.startsWith ('https://')))
|
|
||||||
{
|
|
||||||
issues.push ({
|
|
||||||
sourceRow,
|
|
||||||
message: 'HTTP または HTTPS の URL ではありません.',
|
|
||||||
url: displayUrl })
|
|
||||||
return
|
|
||||||
}
|
|
||||||
if (bytesize (value) > MAX_URL_BYTES)
|
|
||||||
{
|
|
||||||
issues.push ({
|
|
||||||
sourceRow,
|
|
||||||
message: 'URL が長すぎます.',
|
|
||||||
url: displayUrl })
|
|
||||||
return
|
|
||||||
}
|
|
||||||
if (normalised == null)
|
|
||||||
{
|
|
||||||
issues.push ({
|
|
||||||
sourceRow,
|
|
||||||
message: 'URL の形式が不正です.',
|
|
||||||
url: displayUrl })
|
|
||||||
return
|
|
||||||
}
|
|
||||||
const duplicateRow = seen.get (normalised)
|
|
||||||
if (duplicateRow != null)
|
|
||||||
{
|
|
||||||
issues.push ({
|
|
||||||
sourceRow,
|
|
||||||
message: `${ duplicateRow } 行目と同じ URL です.`,
|
|
||||||
url: displayUrl })
|
|
||||||
return
|
|
||||||
}
|
|
||||||
seen.set (normalised, sourceRow)
|
|
||||||
})
|
|
||||||
|
|
||||||
return issues
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
const hasSkipWarning = (row: PostImportRow): boolean =>
|
|
||||||
(row.fieldWarnings.url ?? []).includes ('既存投稿のためスキップします.')
|
|
||||||
|
|
||||||
|
|
||||||
export const submittableImportRows = (rows: PostImportRow[]): PostImportRow[] =>
|
|
||||||
rows.filter (row => {
|
|
||||||
if (row.importStatus === 'created')
|
|
||||||
return false
|
|
||||||
if (row.importStatus === 'skipped')
|
|
||||||
return false
|
|
||||||
if (hasSkipWarning (row))
|
|
||||||
return false
|
|
||||||
if (row.importStatus === 'failed')
|
|
||||||
return false
|
|
||||||
return row.importStatus == null || row.importStatus === 'pending'
|
|
||||||
})
|
|
||||||
|
|
||||||
|
|
||||||
export const reviewSummaryCounts = (rows: PostImportRow[]) => ({
|
|
||||||
total: rows.length,
|
|
||||||
submittable: rows.filter (row =>
|
|
||||||
submittableImportRows ([row]).length > 0
|
|
||||||
&& Object.keys (row.validationErrors ?? { }).length === 0).length,
|
|
||||||
invalid: rows.filter (row => Object.keys (row.validationErrors ?? { }).length > 0).length,
|
|
||||||
skipPlanned: rows.filter (row =>
|
|
||||||
hasSkipWarning (row) && row.importStatus !== 'created').length,
|
|
||||||
created: rows.filter (row => row.importStatus === 'created').length,
|
|
||||||
failed: rows.filter (row => row.importStatus === 'failed').length })
|
|
||||||
|
|||||||
@@ -0,0 +1,110 @@
|
|||||||
|
import type { PostImportSourceIssue } from '@/lib/postImportTypes'
|
||||||
|
|
||||||
|
const MAX_ROWS = 100
|
||||||
|
const MAX_URL_BYTES = 20 * 1024
|
||||||
|
|
||||||
|
|
||||||
|
const truncateUrl = (value: string): string =>
|
||||||
|
value.length > 120 ? `${ value.slice (0, 117) }…` : value
|
||||||
|
|
||||||
|
|
||||||
|
const bytesize = (value: string): number =>
|
||||||
|
new TextEncoder ().encode (value).length
|
||||||
|
|
||||||
|
|
||||||
|
const normaliseImportUrl = (value: string): string | null => {
|
||||||
|
const trimmed = value.trim ()
|
||||||
|
if (!(trimmed))
|
||||||
|
return null
|
||||||
|
|
||||||
|
try
|
||||||
|
{
|
||||||
|
const url = new URL (trimmed)
|
||||||
|
if (!(url.protocol === 'http:' || url.protocol === 'https:'))
|
||||||
|
return null
|
||||||
|
if (!(url.host))
|
||||||
|
return null
|
||||||
|
|
||||||
|
url.hostname = url.hostname.toLowerCase ()
|
||||||
|
if (url.pathname.endsWith ('/'))
|
||||||
|
url.pathname = url.pathname.replace (/\/+$/, '')
|
||||||
|
return url.toString ()
|
||||||
|
}
|
||||||
|
catch
|
||||||
|
{
|
||||||
|
return null
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
export const countImportSourceLines = (source: string): number =>
|
||||||
|
source
|
||||||
|
.split (/\r\n|\n|\r/)
|
||||||
|
.map (_1 => _1.trim ())
|
||||||
|
.filter (_1 => _1 !== '')
|
||||||
|
.length
|
||||||
|
|
||||||
|
|
||||||
|
export const validateImportSource = (
|
||||||
|
source: string,
|
||||||
|
): PostImportSourceIssue[] => {
|
||||||
|
const lines = source.split (/\r\n|\n|\r/)
|
||||||
|
const issues: PostImportSourceIssue[] = []
|
||||||
|
const seen = new Map<string, number> ()
|
||||||
|
let count = 0
|
||||||
|
|
||||||
|
lines.forEach ((rawLine, index) => {
|
||||||
|
const value = rawLine.trim ()
|
||||||
|
if (!(value))
|
||||||
|
return
|
||||||
|
|
||||||
|
++count
|
||||||
|
const sourceRow = index + 1
|
||||||
|
const displayUrl = truncateUrl (value)
|
||||||
|
const normalised = normaliseImportUrl (value)
|
||||||
|
if (count > MAX_ROWS)
|
||||||
|
{
|
||||||
|
issues.push ({
|
||||||
|
sourceRow,
|
||||||
|
message: `取込件数は ${ MAX_ROWS } 件までです.`,
|
||||||
|
url: displayUrl })
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if (!(value.startsWith ('http://') || value.startsWith ('https://')))
|
||||||
|
{
|
||||||
|
issues.push ({
|
||||||
|
sourceRow,
|
||||||
|
message: 'HTTP または HTTPS の URL ではありません.',
|
||||||
|
url: displayUrl })
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if (bytesize (value) > MAX_URL_BYTES)
|
||||||
|
{
|
||||||
|
issues.push ({
|
||||||
|
sourceRow,
|
||||||
|
message: 'URL が長すぎます.',
|
||||||
|
url: displayUrl })
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if (normalised == null)
|
||||||
|
{
|
||||||
|
issues.push ({
|
||||||
|
sourceRow,
|
||||||
|
message: 'URL の形式が不正です.',
|
||||||
|
url: displayUrl })
|
||||||
|
return
|
||||||
|
}
|
||||||
|
const duplicateRow = seen.get (normalised)
|
||||||
|
if (duplicateRow != null)
|
||||||
|
{
|
||||||
|
issues.push ({
|
||||||
|
sourceRow,
|
||||||
|
message: `${ duplicateRow } 行目と同じ URL です.`,
|
||||||
|
url: displayUrl })
|
||||||
|
return
|
||||||
|
}
|
||||||
|
seen.set (normalised, sourceRow)
|
||||||
|
})
|
||||||
|
|
||||||
|
return issues
|
||||||
|
}
|
||||||
@@ -0,0 +1,331 @@
|
|||||||
|
import type { PostImportOrigin,
|
||||||
|
PostImportRow,
|
||||||
|
PostImportSession,
|
||||||
|
PostImportStatus,
|
||||||
|
PostImportSkipReason,
|
||||||
|
StorageErrorHandler } from '@/lib/postImportTypes'
|
||||||
|
|
||||||
|
const SESSION_VERSION = 2
|
||||||
|
const SESSION_PREFIX = 'post-import-session:'
|
||||||
|
const SOURCE_DRAFT_KEY = 'post-import-source-draft'
|
||||||
|
const SESSION_MAX_AGE_MS = 24 * 60 * 60 * 1000
|
||||||
|
|
||||||
|
|
||||||
|
const isPlainObject = (value: unknown): value is Record<string, unknown> =>
|
||||||
|
typeof value === 'object' && value != null && !(Array.isArray (value))
|
||||||
|
|
||||||
|
|
||||||
|
const sessionKey = (sessionId: string): string => `${ SESSION_PREFIX }${ sessionId }`
|
||||||
|
|
||||||
|
|
||||||
|
const readStorage = (
|
||||||
|
key: string,
|
||||||
|
onError?: StorageErrorHandler,
|
||||||
|
): string | null => {
|
||||||
|
if (typeof window === 'undefined')
|
||||||
|
return null
|
||||||
|
|
||||||
|
try
|
||||||
|
{
|
||||||
|
return sessionStorage.getItem (key)
|
||||||
|
}
|
||||||
|
catch
|
||||||
|
{
|
||||||
|
onError?.('保存済みデータを読み込めませんでした.')
|
||||||
|
return null
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
const writeStorage = (
|
||||||
|
key: string,
|
||||||
|
value: string,
|
||||||
|
onError?: StorageErrorHandler,
|
||||||
|
): boolean => {
|
||||||
|
if (typeof window === 'undefined')
|
||||||
|
return false
|
||||||
|
|
||||||
|
try
|
||||||
|
{
|
||||||
|
sessionStorage.setItem (key, value)
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
catch
|
||||||
|
{
|
||||||
|
onError?.('ブラウザへ保存できませんでした.')
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
const removeStorage = (
|
||||||
|
key: string,
|
||||||
|
onError?: StorageErrorHandler,
|
||||||
|
) => {
|
||||||
|
if (typeof window === 'undefined')
|
||||||
|
return
|
||||||
|
|
||||||
|
try
|
||||||
|
{
|
||||||
|
sessionStorage.removeItem (key)
|
||||||
|
}
|
||||||
|
catch
|
||||||
|
{
|
||||||
|
onError?.('保存済みデータを削除できませんでした.')
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
const ensureStringListRecord = (value: unknown): Record<string, string[]> | null => {
|
||||||
|
if (!(isPlainObject (value)))
|
||||||
|
return null
|
||||||
|
|
||||||
|
const result: Record<string, string[]> = { }
|
||||||
|
for (const [key, entry] of Object.entries (value))
|
||||||
|
{
|
||||||
|
if (!(Array.isArray (entry)) || !(entry.every (_1 => typeof _1 === 'string')))
|
||||||
|
return null
|
||||||
|
result[key] = entry
|
||||||
|
}
|
||||||
|
return result
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
const isValidStatus = (
|
||||||
|
value: unknown,
|
||||||
|
): value is PostImportRow['status'] =>
|
||||||
|
value === 'ready' || value === 'warning' || value === 'error'
|
||||||
|
|
||||||
|
|
||||||
|
const isValidImportStatus = (
|
||||||
|
value: unknown,
|
||||||
|
): value is PostImportStatus =>
|
||||||
|
value === 'pending'
|
||||||
|
|| value === 'created'
|
||||||
|
|| value === 'skipped'
|
||||||
|
|| value === 'failed'
|
||||||
|
|
||||||
|
|
||||||
|
const isValidOrigin = (
|
||||||
|
value: unknown,
|
||||||
|
): value is PostImportOrigin =>
|
||||||
|
value === 'automatic' || value === 'manual'
|
||||||
|
|
||||||
|
|
||||||
|
const isValidSkipReason = (
|
||||||
|
value: unknown,
|
||||||
|
): value is PostImportSkipReason =>
|
||||||
|
value === 'existing'
|
||||||
|
|
||||||
|
const isPositiveInteger = (value: unknown): value is number =>
|
||||||
|
Number.isInteger (value) && Number (value) > 0
|
||||||
|
|
||||||
|
|
||||||
|
const sanitiseRow = (value: unknown): PostImportRow | null => {
|
||||||
|
if (!(isPlainObject (value)))
|
||||||
|
return null
|
||||||
|
if (!(Number.isInteger (value.sourceRow)) || Number (value.sourceRow) <= 0)
|
||||||
|
return null
|
||||||
|
if (typeof value.url !== 'string')
|
||||||
|
return null
|
||||||
|
if (!(isPlainObject (value.attributes)))
|
||||||
|
return null
|
||||||
|
if (!(isPlainObject (value.provenance)))
|
||||||
|
return null
|
||||||
|
if (!(isValidStatus (value.status)))
|
||||||
|
return null
|
||||||
|
if (value.importStatus != null && !(isValidImportStatus (value.importStatus)))
|
||||||
|
return null
|
||||||
|
if (value.skipReason != null && !(isValidSkipReason (value.skipReason)))
|
||||||
|
return null
|
||||||
|
if (value.skipReason === 'existing' && !(isPositiveInteger (value.existingPostId)))
|
||||||
|
return null
|
||||||
|
if (value.skipReason !== 'existing' && value.existingPostId != null)
|
||||||
|
return null
|
||||||
|
if (value.importStatus === 'created' && !(isPositiveInteger (value.createdPostId)))
|
||||||
|
return null
|
||||||
|
if (value.importStatus !== 'created' && value.createdPostId != null)
|
||||||
|
return null
|
||||||
|
|
||||||
|
const validationErrors = ensureStringListRecord (value.validationErrors)
|
||||||
|
const fieldWarnings = ensureStringListRecord (value.fieldWarnings)
|
||||||
|
if (validationErrors == null || fieldWarnings == null)
|
||||||
|
return null
|
||||||
|
|
||||||
|
const importErrors =
|
||||||
|
value.importErrors == null
|
||||||
|
? undefined
|
||||||
|
: ensureStringListRecord (value.importErrors)
|
||||||
|
if (value.importErrors != null && importErrors == null)
|
||||||
|
return null
|
||||||
|
if (!(Array.isArray (value.baseWarnings))
|
||||||
|
|| !(value.baseWarnings.every (_1 => typeof _1 === 'string')))
|
||||||
|
return null
|
||||||
|
|
||||||
|
const provenanceEntries = Object.entries (value.provenance)
|
||||||
|
if (!(provenanceEntries.every (([, origin]) => isValidOrigin (origin))))
|
||||||
|
return null
|
||||||
|
|
||||||
|
if (value.tagSources != null)
|
||||||
|
{
|
||||||
|
if (!(isPlainObject (value.tagSources)))
|
||||||
|
return null
|
||||||
|
if (!(Object.values (value.tagSources).every (_1 => typeof _1 === 'string')))
|
||||||
|
return null
|
||||||
|
}
|
||||||
|
|
||||||
|
return {
|
||||||
|
sourceRow: Number (value.sourceRow),
|
||||||
|
url: value.url,
|
||||||
|
attributes: value.attributes as Record<string, string | number>,
|
||||||
|
fieldWarnings,
|
||||||
|
baseWarnings: value.baseWarnings,
|
||||||
|
validationErrors,
|
||||||
|
importErrors,
|
||||||
|
provenance: value.provenance as Record<string, PostImportOrigin>,
|
||||||
|
tagSources: value.tagSources as Record<PostImportOrigin, string> | undefined,
|
||||||
|
status: value.status,
|
||||||
|
skipReason: value.skipReason,
|
||||||
|
existingPostId:
|
||||||
|
isPositiveInteger (value.existingPostId) ? Number (value.existingPostId) : undefined,
|
||||||
|
metadataUrl: typeof value.metadataUrl === 'string' ? value.metadataUrl : undefined,
|
||||||
|
createdPostId:
|
||||||
|
isPositiveInteger (value.createdPostId) ? Number (value.createdPostId) : undefined,
|
||||||
|
importStatus: value.importStatus }
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
const isExpiredSession = (savedAt: string): boolean => {
|
||||||
|
const value = Date.parse (savedAt)
|
||||||
|
return Number.isNaN (value) || Date.now () - value > SESSION_MAX_AGE_MS
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
export const createPostImportSessionId = (): string =>
|
||||||
|
typeof crypto !== 'undefined' && 'randomUUID' in crypto
|
||||||
|
? crypto.randomUUID ()
|
||||||
|
: `${ Date.now () }-${ Math.random ().toString (36).slice (2) }`
|
||||||
|
|
||||||
|
|
||||||
|
export const cleanupExpiredPostImportSessions = (
|
||||||
|
onError?: StorageErrorHandler,
|
||||||
|
) => {
|
||||||
|
if (typeof window === 'undefined')
|
||||||
|
return
|
||||||
|
|
||||||
|
try
|
||||||
|
{
|
||||||
|
for (let i = 0; i < sessionStorage.length; ++i)
|
||||||
|
{
|
||||||
|
const key = sessionStorage.key (i)
|
||||||
|
if (key == null || !(key.startsWith (SESSION_PREFIX)))
|
||||||
|
continue
|
||||||
|
|
||||||
|
const raw = sessionStorage.getItem (key)
|
||||||
|
if (raw == null)
|
||||||
|
continue
|
||||||
|
try
|
||||||
|
{
|
||||||
|
const value = JSON.parse (raw) as { savedAt?: string }
|
||||||
|
if (typeof value.savedAt !== 'string' || isExpiredSession (value.savedAt))
|
||||||
|
{
|
||||||
|
sessionStorage.removeItem (key)
|
||||||
|
--i
|
||||||
|
}
|
||||||
|
}
|
||||||
|
catch
|
||||||
|
{
|
||||||
|
sessionStorage.removeItem (key)
|
||||||
|
--i
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
catch
|
||||||
|
{
|
||||||
|
onError?.('保存済みデータを整理できませんでした.')
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
export const loadPostImportSourceDraft = (
|
||||||
|
onError?: StorageErrorHandler,
|
||||||
|
): { source: string } => {
|
||||||
|
const raw = readStorage (SOURCE_DRAFT_KEY, onError)
|
||||||
|
if (raw == null)
|
||||||
|
return { source: '' }
|
||||||
|
|
||||||
|
try
|
||||||
|
{
|
||||||
|
const value = JSON.parse (raw) as { source?: string }
|
||||||
|
return { source: typeof value.source === 'string' ? value.source : '' }
|
||||||
|
}
|
||||||
|
catch
|
||||||
|
{
|
||||||
|
return { source: '' }
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
export const savePostImportSourceDraft = (
|
||||||
|
source: string,
|
||||||
|
onError?: StorageErrorHandler,
|
||||||
|
): boolean =>
|
||||||
|
writeStorage (SOURCE_DRAFT_KEY, JSON.stringify ({ source }), onError)
|
||||||
|
|
||||||
|
|
||||||
|
export const clearPostImportSourceDraft = (
|
||||||
|
onError?: StorageErrorHandler,
|
||||||
|
) => {
|
||||||
|
removeStorage (SOURCE_DRAFT_KEY, onError)
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
export const savePostImportSession = (
|
||||||
|
sessionId: string,
|
||||||
|
session: Omit<PostImportSession, 'version' | 'savedAt'>,
|
||||||
|
onError?: StorageErrorHandler,
|
||||||
|
): boolean =>
|
||||||
|
writeStorage (
|
||||||
|
sessionKey (sessionId),
|
||||||
|
JSON.stringify ({
|
||||||
|
...session,
|
||||||
|
version: SESSION_VERSION,
|
||||||
|
savedAt: new Date ().toISOString () }),
|
||||||
|
onError)
|
||||||
|
|
||||||
|
|
||||||
|
export const loadPostImportSession = (
|
||||||
|
sessionId: string,
|
||||||
|
onError?: StorageErrorHandler,
|
||||||
|
): PostImportSession | null => {
|
||||||
|
const raw = readStorage (sessionKey (sessionId), onError)
|
||||||
|
if (raw == null)
|
||||||
|
return null
|
||||||
|
|
||||||
|
try
|
||||||
|
{
|
||||||
|
const value = JSON.parse (raw) as Partial<PostImportSession>
|
||||||
|
if (value.version !== SESSION_VERSION || !(Array.isArray (value.rows)))
|
||||||
|
return null
|
||||||
|
if (typeof value.savedAt !== 'string' || isExpiredSession (value.savedAt))
|
||||||
|
{
|
||||||
|
removeStorage (sessionKey (sessionId), onError)
|
||||||
|
return null
|
||||||
|
}
|
||||||
|
|
||||||
|
const rows = value.rows.map (sanitiseRow)
|
||||||
|
if (rows.some (_1 => _1 == null))
|
||||||
|
return null
|
||||||
|
|
||||||
|
return {
|
||||||
|
version: SESSION_VERSION,
|
||||||
|
savedAt: value.savedAt,
|
||||||
|
source: typeof value.source === 'string' ? value.source : '',
|
||||||
|
rows: rows as PostImportRow[],
|
||||||
|
repairMode: value.repairMode === 'failed' ? 'failed' : 'all' }
|
||||||
|
}
|
||||||
|
catch
|
||||||
|
{
|
||||||
|
return null
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,47 @@
|
|||||||
|
export type PostImportOrigin = 'automatic' | 'manual'
|
||||||
|
export type PostImportRepairMode = 'all' | 'failed'
|
||||||
|
export type PostImportStatus =
|
||||||
|
'pending'
|
||||||
|
| 'created'
|
||||||
|
| 'skipped'
|
||||||
|
| 'failed'
|
||||||
|
export type PostImportSkipReason = 'existing'
|
||||||
|
export type PostImportResultStatus = 'created' | 'skipped' | 'failed'
|
||||||
|
export type PostImportAttributeValue = string | number
|
||||||
|
|
||||||
|
export type PostImportRow = {
|
||||||
|
sourceRow: number
|
||||||
|
url: string
|
||||||
|
attributes: Record<string, PostImportAttributeValue>
|
||||||
|
fieldWarnings: Record<string, string[]>
|
||||||
|
baseWarnings: string[]
|
||||||
|
validationErrors: Record<string, string[]>
|
||||||
|
importErrors?: Record<string, string[]>
|
||||||
|
provenance: Record<string, PostImportOrigin>
|
||||||
|
tagSources?: Record<PostImportOrigin, string>
|
||||||
|
status: 'ready' | 'warning' | 'error'
|
||||||
|
skipReason?: PostImportSkipReason
|
||||||
|
existingPostId?: number
|
||||||
|
metadataUrl?: string
|
||||||
|
createdPostId?: number
|
||||||
|
importStatus?: PostImportStatus }
|
||||||
|
|
||||||
|
export type PostImportResultRow = {
|
||||||
|
sourceRow: number
|
||||||
|
status: PostImportResultStatus
|
||||||
|
post?: { id: number }
|
||||||
|
errors?: Record<string, string[]> }
|
||||||
|
|
||||||
|
export type PostImportSession = {
|
||||||
|
version: number
|
||||||
|
savedAt: string
|
||||||
|
source: string
|
||||||
|
rows: PostImportRow[]
|
||||||
|
repairMode: PostImportRepairMode }
|
||||||
|
|
||||||
|
export type PostImportSourceIssue = {
|
||||||
|
sourceRow: number
|
||||||
|
message: string
|
||||||
|
url: string }
|
||||||
|
|
||||||
|
export type StorageErrorHandler = (message: string) => void
|
||||||
@@ -8,6 +8,7 @@ import PageTitle from '@/components/common/PageTitle'
|
|||||||
import PrefetchLink from '@/components/PrefetchLink'
|
import PrefetchLink from '@/components/PrefetchLink'
|
||||||
import MainArea from '@/components/layout/MainArea'
|
import MainArea from '@/components/layout/MainArea'
|
||||||
import PostImportStatusBadge from '@/components/posts/import/PostImportStatusBadge'
|
import PostImportStatusBadge from '@/components/posts/import/PostImportStatusBadge'
|
||||||
|
import { effectivePostImportStatus } from '@/components/posts/import/postImportRowStatus'
|
||||||
import { Button } from '@/components/ui/button'
|
import { Button } from '@/components/ui/button'
|
||||||
import { toast } from '@/components/ui/use-toast'
|
import { toast } from '@/components/ui/use-toast'
|
||||||
import { SITE_TITLE } from '@/config'
|
import { SITE_TITLE } from '@/config'
|
||||||
@@ -16,9 +17,12 @@ import { canEditContent } from '@/lib/users'
|
|||||||
import { clearPostImportSourceDraft,
|
import { clearPostImportSourceDraft,
|
||||||
loadPostImportSession,
|
loadPostImportSession,
|
||||||
mergeImportResults,
|
mergeImportResults,
|
||||||
|
mergeValidatedImportRows,
|
||||||
|
resultSummaryCounts,
|
||||||
retryImportRow,
|
retryImportRow,
|
||||||
savePostImportSession,
|
savePostImportSession,
|
||||||
type PostImportResultRow,
|
type PostImportResultRow,
|
||||||
|
type PostImportRow,
|
||||||
type PostImportSession } from '@/lib/postImportSession'
|
type PostImportSession } from '@/lib/postImportSession'
|
||||||
import Forbidden from '@/pages/Forbidden'
|
import Forbidden from '@/pages/Forbidden'
|
||||||
|
|
||||||
@@ -28,7 +32,9 @@ import type { User } from '@/types'
|
|||||||
|
|
||||||
type Props = { user: User | null }
|
type Props = { user: User | null }
|
||||||
|
|
||||||
const resultToneClass = (status: string | undefined): string[] => {
|
const resultToneClass = (
|
||||||
|
status: ReturnType<typeof effectivePostImportStatus>,
|
||||||
|
): string[] => {
|
||||||
switch (status)
|
switch (status)
|
||||||
{
|
{
|
||||||
case 'created':
|
case 'created':
|
||||||
@@ -40,9 +46,14 @@ const resultToneClass = (status: string | undefined): string[] => {
|
|||||||
'border-stone-200 bg-stone-50',
|
'border-stone-200 bg-stone-50',
|
||||||
'dark:border-stone-800 dark:bg-stone-900/60']
|
'dark:border-stone-800 dark:bg-stone-900/60']
|
||||||
case 'failed':
|
case 'failed':
|
||||||
|
case 'error':
|
||||||
return [
|
return [
|
||||||
'border-rose-200 bg-rose-50',
|
'border-rose-200 bg-rose-50',
|
||||||
'dark:border-rose-900 dark:bg-rose-950/30']
|
'dark:border-rose-900 dark:bg-rose-950/30']
|
||||||
|
case 'warning':
|
||||||
|
return [
|
||||||
|
'border-amber-200 bg-amber-50',
|
||||||
|
'dark:border-amber-900 dark:bg-amber-950/30']
|
||||||
default:
|
default:
|
||||||
return [
|
return [
|
||||||
'border-border bg-white',
|
'border-border bg-white',
|
||||||
@@ -50,6 +61,10 @@ const resultToneClass = (status: string | undefined): string[] => {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
const rowMessages = (row: PostImportRow): string[] => [
|
||||||
|
...Object.values (row.validationErrors ?? { }).flat (),
|
||||||
|
...Object.values (row.importErrors ?? { }).flat ()]
|
||||||
|
|
||||||
|
|
||||||
const PostImportResultPage: FC<Props> = ({ user }) => {
|
const PostImportResultPage: FC<Props> = ({ user }) => {
|
||||||
const editable = canEditContent (user)
|
const editable = canEditContent (user)
|
||||||
@@ -61,7 +76,7 @@ const PostImportResultPage: FC<Props> = ({ user }) => {
|
|||||||
const [loadingRow, setLoadingRow] = useState<number | null> (null)
|
const [loadingRow, setLoadingRow] = useState<number | null> (null)
|
||||||
|
|
||||||
useEffect (() => {
|
useEffect (() => {
|
||||||
if (!(sessionId))
|
if (sessionId == null)
|
||||||
return
|
return
|
||||||
|
|
||||||
const loaded = loadPostImportSession (sessionId, message =>
|
const loaded = loadPostImportSession (sessionId, message =>
|
||||||
@@ -71,20 +86,19 @@ const PostImportResultPage: FC<Props> = ({ user }) => {
|
|||||||
}, [sessionId])
|
}, [sessionId])
|
||||||
|
|
||||||
useEffect (() => {
|
useEffect (() => {
|
||||||
if (!(sessionId) || !(session))
|
if (sessionId == null || session == null)
|
||||||
return
|
return
|
||||||
|
|
||||||
savePostImportSession (sessionId, session, message =>
|
savePostImportSession (sessionId, session, message =>
|
||||||
toast ({ title: '取込状態を保存できませんでした', description: message }))
|
toast ({ title: '取込状態を保存できませんでした', description: message }))
|
||||||
}, [session, sessionId])
|
}, [session, sessionId])
|
||||||
|
|
||||||
const counts = useMemo (() => ({
|
const counts = useMemo (
|
||||||
created: session?.rows.filter (_1 => _1.importStatus === 'created').length ?? 0,
|
() => resultSummaryCounts (session?.rows ?? []),
|
||||||
skipped: session?.rows.filter (_1 => _1.importStatus === 'skipped').length ?? 0,
|
[session])
|
||||||
failed: session?.rows.filter (_1 => _1.importStatus === 'failed').length ?? 0 }), [session])
|
|
||||||
|
|
||||||
const retry = async (sourceRow: number) => {
|
const retry = async (sourceRow: number) => {
|
||||||
if (!(session))
|
if (session == null)
|
||||||
return
|
return
|
||||||
|
|
||||||
setLoadingRow (sourceRow)
|
setLoadingRow (sourceRow)
|
||||||
@@ -92,9 +106,27 @@ const PostImportResultPage: FC<Props> = ({ user }) => {
|
|||||||
{
|
{
|
||||||
const pendingRows = retryImportRow (session.rows, sourceRow)
|
const pendingRows = retryImportRow (session.rows, sourceRow)
|
||||||
setSession ({ ...session, rows: pendingRows })
|
setSession ({ ...session, rows: pendingRows })
|
||||||
const target = pendingRows.find (_1 => _1.sourceRow === sourceRow)
|
const validated = await apiPost<{ rows: PostImportRow[] }> ('/posts/import/validate', {
|
||||||
|
rows: pendingRows
|
||||||
|
.filter (row => row.importStatus !== 'created')
|
||||||
|
.map (row => ({
|
||||||
|
sourceRow: row.sourceRow,
|
||||||
|
url: row.url,
|
||||||
|
attributes: row.attributes,
|
||||||
|
provenance: row.provenance,
|
||||||
|
tagSources: row.tagSources,
|
||||||
|
metadataUrl: row.metadataUrl })),
|
||||||
|
changed_row: -1 })
|
||||||
|
const validatedRows = mergeValidatedImportRows (pendingRows, validated.rows)
|
||||||
|
setSession (current =>
|
||||||
|
current
|
||||||
|
? { ...current, rows: validatedRows }
|
||||||
|
: current)
|
||||||
|
const target = validatedRows.find (_1 => _1.sourceRow === sourceRow)
|
||||||
if (target == null)
|
if (target == null)
|
||||||
return
|
return
|
||||||
|
if (Object.keys (target.validationErrors ?? { }).length > 0)
|
||||||
|
return
|
||||||
|
|
||||||
const result = await apiPost<{
|
const result = await apiPost<{
|
||||||
created: number
|
created: number
|
||||||
@@ -125,22 +157,22 @@ const PostImportResultPage: FC<Props> = ({ user }) => {
|
|||||||
}
|
}
|
||||||
|
|
||||||
const openRepair = (sourceRow: number) => {
|
const openRepair = (sourceRow: number) => {
|
||||||
|
if (session == null || sessionId == null)
|
||||||
|
return
|
||||||
|
|
||||||
const nextSession = { ...session, repairMode: 'failed' as const }
|
const nextSession = { ...session, repairMode: 'failed' as const }
|
||||||
setSession (nextSession)
|
setSession (nextSession)
|
||||||
if (sessionId)
|
const saved = savePostImportSession (sessionId, nextSession, message =>
|
||||||
{
|
toast ({ title: '取込状態を保存できませんでした', description: message }))
|
||||||
const saved = savePostImportSession (sessionId, nextSession, message =>
|
if (!(saved))
|
||||||
toast ({ title: '取込状態を保存できませんでした', description: message }))
|
return
|
||||||
if (!(saved))
|
|
||||||
return
|
|
||||||
}
|
|
||||||
navigate (`/posts/import/${ sessionId }/review?edit=${ sourceRow }`)
|
navigate (`/posts/import/${ sessionId }/review?edit=${ sourceRow }`)
|
||||||
}
|
}
|
||||||
|
|
||||||
if (!(editable))
|
if (!(editable))
|
||||||
return <Forbidden/>
|
return <Forbidden/>
|
||||||
|
|
||||||
if (missing || !(sessionId) || !(session))
|
if (missing || sessionId == null || session == null)
|
||||||
{
|
{
|
||||||
return (
|
return (
|
||||||
<MainArea>
|
<MainArea>
|
||||||
@@ -169,22 +201,25 @@ const PostImportResultPage: FC<Props> = ({ user }) => {
|
|||||||
<SummaryChip label="登録成功" value={counts.created} badge="created"/>
|
<SummaryChip label="登録成功" value={counts.created} badge="created"/>
|
||||||
<SummaryChip label="スキップ" value={counts.skipped} badge="skipped"/>
|
<SummaryChip label="スキップ" value={counts.skipped} badge="skipped"/>
|
||||||
<SummaryChip label="失敗" value={counts.failed} badge="failed"/>
|
<SummaryChip label="失敗" value={counts.failed} badge="failed"/>
|
||||||
|
<SummaryChip label="要修正" value={counts.invalid} badge="error"/>
|
||||||
</div>
|
</div>
|
||||||
</div>
|
</div>
|
||||||
|
|
||||||
<div className="space-y-3">
|
<div className="space-y-3">
|
||||||
{session.rows.map (row => (
|
{session.rows.map (row => {
|
||||||
|
const status = effectivePostImportStatus (row)
|
||||||
|
return (
|
||||||
<div
|
<div
|
||||||
key={row.sourceRow}
|
key={row.sourceRow}
|
||||||
className={[
|
className={[
|
||||||
'rounded-lg border p-4',
|
'rounded-lg border p-4',
|
||||||
...resultToneClass (row.importStatus)].join (' ')}>
|
...resultToneClass (status)].join (' ')}>
|
||||||
<div className="flex flex-col gap-3 md:flex-row md:items-start
|
<div className="flex flex-col gap-3 md:flex-row md:items-start
|
||||||
md:justify-between">
|
md:justify-between">
|
||||||
<div className="space-y-2">
|
<div className="space-y-2">
|
||||||
<div className="flex flex-wrap items-center gap-2">
|
<div className="flex flex-wrap items-center gap-2">
|
||||||
<span className="text-sm font-medium">行 {row.sourceRow}</span>
|
<span className="text-sm font-medium">行 {row.sourceRow}</span>
|
||||||
<PostImportStatusBadge value={row.importStatus ?? 'pending'}/>
|
<PostImportStatusBadge value={status}/>
|
||||||
</div>
|
</div>
|
||||||
<div className="text-sm text-neutral-700 dark:text-neutral-200">
|
<div className="text-sm text-neutral-700 dark:text-neutral-200">
|
||||||
{String (row.attributes.title ?? '') || row.url}
|
{String (row.attributes.title ?? '') || row.url}
|
||||||
@@ -192,31 +227,26 @@ const PostImportResultPage: FC<Props> = ({ user }) => {
|
|||||||
<div className="text-xs text-neutral-500 dark:text-neutral-400">
|
<div className="text-xs text-neutral-500 dark:text-neutral-400">
|
||||||
{row.url}
|
{row.url}
|
||||||
</div>
|
</div>
|
||||||
{row.createdPostId && (
|
<FieldError messages={rowMessages (row)}/>
|
||||||
<PrefetchLink
|
|
||||||
to={`/posts/${ row.createdPostId }`}
|
|
||||||
className="inline-block text-xs text-sky-700 underline
|
|
||||||
dark:text-sky-300">
|
|
||||||
投稿 #{row.createdPostId}
|
|
||||||
</PrefetchLink>)}
|
|
||||||
<FieldError messages={Object.values (row.importErrors ?? { }).flat ()}/>
|
|
||||||
</div>
|
</div>
|
||||||
|
|
||||||
<div className="flex flex-col gap-2 sm:flex-row">
|
<div className="flex flex-col gap-2 sm:flex-row">
|
||||||
{row.createdPostId && (
|
{(row.createdPostId != null || row.existingPostId != null) && (
|
||||||
<Button type="button" variant="outline" asChild>
|
<Button type="button" variant="outline" asChild>
|
||||||
<PrefetchLink to={`/posts/${ row.createdPostId }`}>
|
<PrefetchLink
|
||||||
|
to={`/posts/${ row.createdPostId ?? row.existingPostId }`}>
|
||||||
投稿を開く
|
投稿を開く
|
||||||
</PrefetchLink>
|
</PrefetchLink>
|
||||||
</Button>)}
|
</Button>)}
|
||||||
{row.importStatus === 'failed' && (
|
{(status === 'error' || row.importStatus === 'failed') && (
|
||||||
|
<Button
|
||||||
|
type="button"
|
||||||
|
variant="outline"
|
||||||
|
onClick={() => openRepair (row.sourceRow)}>
|
||||||
|
編輯
|
||||||
|
</Button>)}
|
||||||
|
{row.importStatus === 'failed' && status !== 'error' && (
|
||||||
<>
|
<>
|
||||||
<Button
|
|
||||||
type="button"
|
|
||||||
variant="outline"
|
|
||||||
onClick={() => openRepair (row.sourceRow)}>
|
|
||||||
編輯
|
|
||||||
</Button>
|
|
||||||
<Button
|
<Button
|
||||||
type="button"
|
type="button"
|
||||||
onClick={() => retry (row.sourceRow)}
|
onClick={() => retry (row.sourceRow)}
|
||||||
@@ -226,7 +256,7 @@ const PostImportResultPage: FC<Props> = ({ user }) => {
|
|||||||
</>)}
|
</>)}
|
||||||
</div>
|
</div>
|
||||||
</div>
|
</div>
|
||||||
</div>))}
|
</div>)})}
|
||||||
</div>
|
</div>
|
||||||
|
|
||||||
<div className="flex flex-col gap-2 sm:flex-row">
|
<div className="flex flex-col gap-2 sm:flex-row">
|
||||||
@@ -263,7 +293,7 @@ const SummaryChip = (
|
|||||||
{ label, value, badge }: {
|
{ label, value, badge }: {
|
||||||
label: string
|
label: string
|
||||||
value: number
|
value: number
|
||||||
badge: 'created' | 'skipped' | 'failed' },
|
badge: 'created' | 'skipped' | 'failed' | 'error' },
|
||||||
) => (
|
) => (
|
||||||
<div className="flex items-center gap-2 rounded-full border border-border
|
<div className="flex items-center gap-2 rounded-full border border-border
|
||||||
bg-background px-3 py-1 text-sm">
|
bg-background px-3 py-1 text-sm">
|
||||||
|
|||||||
@@ -15,11 +15,12 @@ import { SITE_TITLE } from '@/config'
|
|||||||
import { apiPost } from '@/lib/api'
|
import { apiPost } from '@/lib/api'
|
||||||
import { canEditContent } from '@/lib/users'
|
import { canEditContent } from '@/lib/users'
|
||||||
import { loadPostImportSession,
|
import { loadPostImportSession,
|
||||||
|
creatableImportRows,
|
||||||
mergeImportResults,
|
mergeImportResults,
|
||||||
mergeValidatedImportRows,
|
mergeValidatedImportRows,
|
||||||
|
processableImportRows,
|
||||||
reviewSummaryCounts,
|
reviewSummaryCounts,
|
||||||
savePostImportSession,
|
savePostImportSession,
|
||||||
submittableImportRows,
|
|
||||||
type PostImportResultRow,
|
type PostImportResultRow,
|
||||||
type PostImportRow,
|
type PostImportRow,
|
||||||
type PostImportSession } from '@/lib/postImportSession'
|
type PostImportSession } from '@/lib/postImportSession'
|
||||||
@@ -54,7 +55,7 @@ const PostImportReviewPage: FC<Props> = ({ user }) => {
|
|||||||
const [savingRow, setSavingRow] = useState<number | null> (null)
|
const [savingRow, setSavingRow] = useState<number | null> (null)
|
||||||
|
|
||||||
useEffect (() => {
|
useEffect (() => {
|
||||||
if (!(sessionId))
|
if (sessionId == null)
|
||||||
return
|
return
|
||||||
|
|
||||||
const loaded = loadPostImportSession (sessionId, message =>
|
const loaded = loadPostImportSession (sessionId, message =>
|
||||||
@@ -64,7 +65,7 @@ const PostImportReviewPage: FC<Props> = ({ user }) => {
|
|||||||
}, [sessionId])
|
}, [sessionId])
|
||||||
|
|
||||||
useEffect (() => {
|
useEffect (() => {
|
||||||
if (!(sessionId) || !(session))
|
if (sessionId == null || session == null)
|
||||||
return
|
return
|
||||||
|
|
||||||
savePostImportSession (sessionId, session, message =>
|
savePostImportSession (sessionId, session, message =>
|
||||||
@@ -78,11 +79,12 @@ const PostImportReviewPage: FC<Props> = ({ user }) => {
|
|||||||
? rows.find (_1 => _1.sourceRow === editingSourceRow) ?? null
|
? rows.find (_1 => _1.sourceRow === editingSourceRow) ?? null
|
||||||
: null
|
: null
|
||||||
const counts = useMemo (() => reviewSummaryCounts (rows), [rows])
|
const counts = useMemo (() => reviewSummaryCounts (rows), [rows])
|
||||||
const submittable = useMemo (
|
const processable = useMemo (
|
||||||
() => submittableImportRows (rows).filter (row =>
|
() => processableImportRows (rows),
|
||||||
Object.keys (row.validationErrors ?? { }).length === 0),
|
[rows])
|
||||||
[rows],
|
const creatable = useMemo (
|
||||||
)
|
() => creatableImportRows (rows),
|
||||||
|
[rows])
|
||||||
const reviewRows =
|
const reviewRows =
|
||||||
session?.repairMode === 'failed'
|
session?.repairMode === 'failed'
|
||||||
? [...rows].sort ((a, b) => {
|
? [...rows].sort ((a, b) => {
|
||||||
@@ -105,7 +107,7 @@ const PostImportReviewPage: FC<Props> = ({ user }) => {
|
|||||||
current ? { ...current, rows: nextRows } : current)
|
current ? { ...current, rows: nextRows } : current)
|
||||||
|
|
||||||
const saveDraft = async (draft: Draft): Promise<boolean> => {
|
const saveDraft = async (draft: Draft): Promise<boolean> => {
|
||||||
if (!(session))
|
if (session == null)
|
||||||
return false
|
return false
|
||||||
if (editingRow == null)
|
if (editingRow == null)
|
||||||
return false
|
return false
|
||||||
@@ -122,8 +124,7 @@ const PostImportReviewPage: FC<Props> = ({ user }) => {
|
|||||||
['originalCreatedFrom', draft.originalCreatedFrom],
|
['originalCreatedFrom', draft.originalCreatedFrom],
|
||||||
['originalCreatedBefore', draft.originalCreatedBefore],
|
['originalCreatedBefore', draft.originalCreatedBefore],
|
||||||
['duration', draft.duration],
|
['duration', draft.duration],
|
||||||
['parentPostIds', draft.parentPostIds],
|
['parentPostIds', draft.parentPostIds]] as const
|
||||||
] as const
|
|
||||||
draftFields.forEach (([field, value]) => {
|
draftFields.forEach (([field, value]) => {
|
||||||
nextAttributes[field] = value
|
nextAttributes[field] = value
|
||||||
nextProvenance[field] =
|
nextProvenance[field] =
|
||||||
@@ -186,7 +187,7 @@ const PostImportReviewPage: FC<Props> = ({ user }) => {
|
|||||||
}
|
}
|
||||||
|
|
||||||
const submit = async () => {
|
const submit = async () => {
|
||||||
if (!(sessionId) || !(session) || submittable.length === 0)
|
if (sessionId == null || session == null || processable.length === 0)
|
||||||
return
|
return
|
||||||
|
|
||||||
setLoading (true)
|
setLoading (true)
|
||||||
@@ -197,7 +198,7 @@ const PostImportReviewPage: FC<Props> = ({ user }) => {
|
|||||||
skipped: number
|
skipped: number
|
||||||
failed: number
|
failed: number
|
||||||
rows: PostImportResultRow[] }> ('/posts/import', {
|
rows: PostImportResultRow[] }> ('/posts/import', {
|
||||||
rows: submittable.map (row => ({
|
rows: processable.map (row => ({
|
||||||
sourceRow: row.sourceRow,
|
sourceRow: row.sourceRow,
|
||||||
url: row.url,
|
url: row.url,
|
||||||
attributes: row.attributes,
|
attributes: row.attributes,
|
||||||
@@ -226,7 +227,7 @@ const PostImportReviewPage: FC<Props> = ({ user }) => {
|
|||||||
if (!(editable))
|
if (!(editable))
|
||||||
return <Forbidden/>
|
return <Forbidden/>
|
||||||
|
|
||||||
if (missing || !(sessionId) || !(session))
|
if (missing || sessionId == null || session == null)
|
||||||
{
|
{
|
||||||
return (
|
return (
|
||||||
<MainArea>
|
<MainArea>
|
||||||
@@ -278,7 +279,9 @@ const PostImportReviewPage: FC<Props> = ({ user }) => {
|
|||||||
<PostImportFooter
|
<PostImportFooter
|
||||||
loading={loading}
|
loading={loading}
|
||||||
invalidCount={counts.invalid}
|
invalidCount={counts.invalid}
|
||||||
submittableCount={submittable.length}
|
processableCount={processable.length}
|
||||||
|
creatableCount={creatable.length}
|
||||||
|
skipPlannedCount={counts.skipPlanned}
|
||||||
onBack={() => navigate ('/posts/import')}
|
onBack={() => navigate ('/posts/import')}
|
||||||
onSubmit={submit}/>
|
onSubmit={submit}/>
|
||||||
|
|
||||||
@@ -295,12 +298,20 @@ const PostImportReviewPage: FC<Props> = ({ user }) => {
|
|||||||
}
|
}
|
||||||
|
|
||||||
const PostImportFooter = (
|
const PostImportFooter = (
|
||||||
{ loading, invalidCount, submittableCount, onBack, onSubmit }: {
|
{ loading,
|
||||||
loading: boolean
|
invalidCount,
|
||||||
invalidCount: number
|
processableCount,
|
||||||
submittableCount: number
|
creatableCount,
|
||||||
onBack: () => void
|
skipPlannedCount,
|
||||||
onSubmit: () => void },
|
onBack,
|
||||||
|
onSubmit }: {
|
||||||
|
loading: boolean
|
||||||
|
invalidCount: number
|
||||||
|
processableCount: number
|
||||||
|
creatableCount: number
|
||||||
|
skipPlannedCount: number
|
||||||
|
onBack: () => void
|
||||||
|
onSubmit: () => void },
|
||||||
) => (
|
) => (
|
||||||
<div
|
<div
|
||||||
className="shrink-0 border-t bg-white/95 p-4 backdrop-blur
|
className="shrink-0 border-t bg-white/95 p-4 backdrop-blur
|
||||||
@@ -309,7 +320,11 @@ const PostImportFooter = (
|
|||||||
md:items-center md:justify-between">
|
md:items-center md:justify-between">
|
||||||
<div className="flex flex-wrap items-center gap-2 text-sm">
|
<div className="flex flex-wrap items-center gap-2 text-sm">
|
||||||
<PostImportStatusBadge value="pending"/>
|
<PostImportStatusBadge value="pending"/>
|
||||||
<span>登録対象 {submittableCount} 件</span>
|
<span>処理対象 {processableCount} 件</span>
|
||||||
|
<PostImportStatusBadge value="ready"/>
|
||||||
|
<span>登録対象 {creatableCount} 件</span>
|
||||||
|
<PostImportStatusBadge value="skipped"/>
|
||||||
|
<span>スキップ予定 {skipPlannedCount} 件</span>
|
||||||
<PostImportStatusBadge value="error"/>
|
<PostImportStatusBadge value="error"/>
|
||||||
<span>要修正 {invalidCount} 件</span>
|
<span>要修正 {invalidCount} 件</span>
|
||||||
</div>
|
</div>
|
||||||
@@ -320,8 +335,8 @@ const PostImportFooter = (
|
|||||||
<Button
|
<Button
|
||||||
type="button"
|
type="button"
|
||||||
onClick={onSubmit}
|
onClick={onSubmit}
|
||||||
disabled={loading || submittableCount === 0}>
|
disabled={loading || processableCount === 0}>
|
||||||
有効な投稿を一括登録
|
取込を実行
|
||||||
</Button>
|
</Button>
|
||||||
</div>
|
</div>
|
||||||
</div>
|
</div>
|
||||||
|
|||||||
@@ -29,6 +29,8 @@ import type { User } from '@/types'
|
|||||||
type Props = { user: User | null }
|
type Props = { user: User | null }
|
||||||
|
|
||||||
const MAX_ROWS = 100
|
const MAX_ROWS = 100
|
||||||
|
const SOURCE_ERROR_ID = 'post-import-source-error'
|
||||||
|
const SOURCE_ISSUES_ID = 'post-import-source-issues'
|
||||||
|
|
||||||
|
|
||||||
const PostImportSourcePage: FC<Props> = ({ user }) => {
|
const PostImportSourcePage: FC<Props> = ({ user }) => {
|
||||||
@@ -45,6 +47,11 @@ const PostImportSourcePage: FC<Props> = ({ user }) => {
|
|||||||
|
|
||||||
const lineCount = countImportSourceLines (source)
|
const lineCount = countImportSourceLines (source)
|
||||||
const messages = sourceError ? [sourceError] : []
|
const messages = sourceError ? [sourceError] : []
|
||||||
|
const sourceDescribedBy = [
|
||||||
|
sourceError ? SOURCE_ERROR_ID : null,
|
||||||
|
sourceIssues.length > 0 ? SOURCE_ISSUES_ID : null]
|
||||||
|
.filter (_1 => _1 != null)
|
||||||
|
.join (' ')
|
||||||
|
|
||||||
useEffect (() => {
|
useEffect (() => {
|
||||||
cleanupExpiredPostImportSessions (message =>
|
cleanupExpiredPostImportSessions (message =>
|
||||||
@@ -134,6 +141,8 @@ const PostImportSourcePage: FC<Props> = ({ user }) => {
|
|||||||
<textarea
|
<textarea
|
||||||
value={source}
|
value={source}
|
||||||
rows={14}
|
rows={14}
|
||||||
|
aria-invalid={messages.length > 0 || sourceIssues.length > 0}
|
||||||
|
aria-describedby={sourceDescribedBy || undefined}
|
||||||
className={inputClass (
|
className={inputClass (
|
||||||
messages.length > 0 || sourceIssues.length > 0,
|
messages.length > 0 || sourceIssues.length > 0,
|
||||||
'font-mono text-sm',
|
'font-mono text-sm',
|
||||||
@@ -150,8 +159,12 @@ const PostImportSourcePage: FC<Props> = ({ user }) => {
|
|||||||
setSourceIssues ([])
|
setSourceIssues ([])
|
||||||
}}/>)}
|
}}/>)}
|
||||||
</FormField>
|
</FormField>
|
||||||
<FieldError messages={messages}/>
|
<div id={messages.length > 0 ? SOURCE_ERROR_ID : undefined}>
|
||||||
<ul className="space-y-2 text-sm text-red-700 dark:text-red-300">
|
<FieldError messages={messages}/>
|
||||||
|
</div>
|
||||||
|
<ul
|
||||||
|
id={SOURCE_ISSUES_ID}
|
||||||
|
className="space-y-2 text-sm text-red-700 dark:text-red-300">
|
||||||
{sourceIssues.map (issue => (
|
{sourceIssues.map (issue => (
|
||||||
<li key={`${ issue.sourceRow }-${ issue.message }-${ issue.url }`}>
|
<li key={`${ issue.sourceRow }-${ issue.message }-${ issue.url }`}>
|
||||||
<div>{issue.sourceRow} 行目: {issue.message}</div>
|
<div>{issue.sourceRow} 行目: {issue.message}</div>
|
||||||
|
|||||||
@@ -42,10 +42,8 @@ const PostNewPage: FC<Props> = ({ user }) => {
|
|||||||
const { baseErrors, fieldErrors, clearValidationErrors, applyValidationError } =
|
const { baseErrors, fieldErrors, clearValidationErrors, applyValidationError } =
|
||||||
useValidationErrors<PostFormField> ()
|
useValidationErrors<PostFormField> ()
|
||||||
|
|
||||||
const [originalCreatedBefore, setOriginalCreatedBefore] =
|
const [originalCreatedBefore, setOriginalCreatedBefore] = useState<string | null> (null)
|
||||||
useState<string | null> (null)
|
const [originalCreatedFrom, setOriginalCreatedFrom] = useState<string | null> (null)
|
||||||
const [originalCreatedFrom, setOriginalCreatedFrom] =
|
|
||||||
useState<string | null> (null)
|
|
||||||
const [parentPostIds, setParentPostIds] = useState ('')
|
const [parentPostIds, setParentPostIds] = useState ('')
|
||||||
const [tags, setTags] = useState ('')
|
const [tags, setTags] = useState ('')
|
||||||
const [duration, setDuration] = useState ('')
|
const [duration, setDuration] = useState ('')
|
||||||
@@ -82,8 +80,7 @@ const PostNewPage: FC<Props> = ({ user }) => {
|
|||||||
|
|
||||||
try
|
try
|
||||||
{
|
{
|
||||||
await apiPost ('/posts', formData,
|
await apiPost ('/posts', formData, { headers: { 'Content-Type': 'multipart/form-data' } })
|
||||||
{ headers: { 'Content-Type': 'multipart/form-data' } })
|
|
||||||
toast ({ title: '投稿成功!' })
|
toast ({ title: '投稿成功!' })
|
||||||
navigate ('/posts')
|
navigate ('/posts')
|
||||||
}
|
}
|
||||||
@@ -98,8 +95,7 @@ const PostNewPage: FC<Props> = ({ user }) => {
|
|||||||
setTitleLoading (true)
|
setTitleLoading (true)
|
||||||
try
|
try
|
||||||
{
|
{
|
||||||
const data = await apiGet<{ title: string }> ('/preview/title',
|
const data = await apiGet<{ title: string }> ('/preview/title', { params: { url } })
|
||||||
{ params: { url } })
|
|
||||||
setTitle (data.title || '')
|
setTitle (data.title || '')
|
||||||
}
|
}
|
||||||
finally
|
finally
|
||||||
@@ -176,7 +172,7 @@ const PostNewPage: FC<Props> = ({ user }) => {
|
|||||||
<Button
|
<Button
|
||||||
type="button"
|
type="button"
|
||||||
variant="outline"
|
variant="outline"
|
||||||
onClick={fetchTitle}
|
onClick={() => void fetchTitle ()}
|
||||||
disabled={!(url) || titleLoading}>
|
disabled={!(url) || titleLoading}>
|
||||||
取得
|
取得
|
||||||
</Button>
|
</Button>
|
||||||
@@ -193,7 +189,7 @@ const PostNewPage: FC<Props> = ({ user }) => {
|
|||||||
<Button
|
<Button
|
||||||
type="button"
|
type="button"
|
||||||
variant="outline"
|
variant="outline"
|
||||||
onClick={fetchThumbnail}
|
onClick={() => void fetchThumbnail ()}
|
||||||
disabled={!(url) || thumbnailLoading}>
|
disabled={!(url) || thumbnailLoading}>
|
||||||
取得
|
取得
|
||||||
</Button>
|
</Button>
|
||||||
|
|||||||
新しい課題から参照
ユーザをブロックする