diff --git a/AGENTS.md b/AGENTS.md index 5917b9c..1b27081 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -114,28 +114,242 @@ npm run preview - Ruby: `render` 系メソッド呼び出しでは、keyword 引数付きでも括弧を書かない。 - Ruby: never put a line break immediately before `)`. - Ruby: do not use `%w` or `%i`. +- Ruby: TypeScript / TSX の associative syntax に近い感覚で読むこと。 + Hash の `}` は block の `}` ではない。call の `)` も function 定義の + parameter-list `)` ではない。delimiter の役割を見誤らないこと。 - In Ruby, when an `if` condition is split across multiple lines and combines clauses with `&&` or `||`, wrap the whole condition in parentheses. -- Ruby hashes are not blocks; keep `}` on the same line as the final pair. -- Ruby hashes keep the first pair on the same line as `{` unless line length - requires a break. -- Short Ruby hashes may stay visually compact across two lines with the first - pair kept on the opening line and aligned continuation pairs below it. -- Ruby blocks use separate `{ ... }` rules from hashes, with 2-space body - indentation. +- Ruby hash / block / call delimiter の詳細は、下の + `Ruby delimiter and wrapping rules` を正本として扱ふこと。 - For arrays, never put whitespace or a line break immediately before `]`. - Keep the first element on the same line as `[` by default. - If an array would exceed the line limit, break after `[` and indent - elements by 4 spaces. + elements 4 spaces deeper than the statement's base indentation. + +### Ruby delimiter and wrapping rules + +- Ruby でも、closing delimiter は glyph ではなく syntax role で判定する。 + `}` が block close なのか hash close なのか、`)` が method call なのか + grouping なのかを区別してから配置すること。 +- Ruby の multi-line method call では、receiver / method と opening `(` を + 同じ行に保ち、closing `)` を単独行へ落とさない。 +- Ruby の multi-line hash literal / keyword-like argument hash では、opening + `{` を最初の pair と同じ行に置き、closing `}` を最後の pair と同じ行に + 置く。Prettier 的な縦開き・縦閉じをしない。 +- Ruby の guard 条件は、1 行で収まるなら modifier 形式を優先する。 + 99 文字を超えるなら block 形式へ切り替へるか、message 定数化などで縮める。 +- Ruby の method chain や call argument を折り返す際、call-site の `)` を + block close のやうに独立させない。 +- Ruby では、行末の `\` を用途を問はず一切使用しない。 +- Ruby では、文字列連結、logger message、method call、条件式、SQL 断片、 + 正規表現その他すべての式で、行末バックスラッシュによる継続を禁止する。 +- Ruby の block body は、その基準位置から 2 空白深くする。 +- Ruby の wrapped expression、method argument、array element、hash pair などの + continuation indentation は、その statement の基準位置から 4 空白深くする。 +- Ruby の continuation indentation を、行頭からの絶対空白数として扱はない。 +- Ruby では、暗黙的に継続可能な構文を優先し、method call、array、Hash 及び + 括弧内ではバックスラッシュなしで改行する。 +- Ruby では、行長制限を守るために行末バックスラッシュを導入してはならない。 +- Ruby で行末バックスラッシュが必要に見える場合は、括弧内で自然に改行する、 + 一つの文字列補間へまとめる、中間変数へ分ける、`format` を使ふ、heredoc を + 使ふ、array 又は Hash を組み立ててから処理する、条件式全体を括弧で囲む、 + method へ抽出する、のいづれかへ書き換へる。 +- Ruby では、一つの文字列を、改行をまたいだ隣接文字列 literal として記述しない。 +- Ruby では、method argument 内でも、複数の文字列 literal を区切りなしで縦に + 並べない。 +- RSpec の `describe`、`context`、`it` 等の description が長い場合は、意味を + 保ったまま一行へ収まる文言へ短縮する。 +- 文字列を短縮できない場合は、用途に応じて `format`、heredoc 又は中間変数を + 検討する。 +- ただし RSpec description では、原則として簡潔な一行の文字列を使ふ。 +- formatter 又は自動修正にも、Ruby の行末バックスラッシュを生成させない。 +- 新規 code だけでなく、今回触れる Ruby code にも行末バックスラッシュを残さない。 +- 例へば class body 内の array は、class body の 2 空白を基準に、更に 4 空白 + 深くするため、結果として行頭から 6 空白になる。 + +Bad: + +```rb +response = Example.fetch( + value, + option: option, +) +``` + +Good: + +```rb +response = Example.fetch( + value, + option: option) +``` + +Bad: + +```rb +payload = { + title: title, + url: url, +} +``` + +Good: + +```rb +payload = { + title: title, + url: url } +``` + +Bad: + +```rb +raise ArgumentError, 'URL が長すぎます．' if url.bytesize > MAX_URL_BYTES && flag.present? +``` + +Good: + +```rb +if url.bytesize > MAX_URL_BYTES && flag.present? + raise ArgumentError, 'URL が長すぎます．' +end +``` + +Bad: + +```rb +Rails.logger.info( + "post_import_metadata_fetch_failure "\ + "#{ payload.to_json }") +``` + +Good: + +```rb +payload = { + error: e.class.name, + message: e.message } + +Rails.logger.info( + "post_import_metadata_fetch_failure #{ payload.to_json }") +``` + +Bad: + +```rb +message = "first "\ + "second" +``` + +Good: + +```rb +message = format( + '%s %s', + first: 'first', + second: 'second') +``` + +Bad: + +```rb +result = first_value + \ + second_value +``` + +Bad: + +```rb +it( + 'returns 409 when stale changes ' + 'do not conflict' +) do +``` + +Good: + +```rb +it 'returns mergeable 409 for stale non-conflicting changes' do +``` + +Bad: + +```rb +records.each { + do_work(_1) } +``` + +Good: + +```rb +records.each { + do_work(_1) +} +``` - TypeScript and Python: use GNU-style spacing before parentheses where syntactically valid. - Never write Ruby, TypeScript, or TSX lines longer than 99 characters. - Aim to keep Ruby, TypeScript, and TSX lines within 79 characters where practical. -- TypeScript and TSX use 4-space logical indentation. -- In TypeScript and TSX only, replace every leading run of 8 spaces with a tab. -- Tabs are only for leading indentation, never for spaces after non-space text. +- In TypeScript and TSX, use 2-space block indentation. +- In TypeScript and TSX, block bodies for components, functions, callbacks, + `if`, `try`, `catch`, `finally`, loops, and JSX nesting use 2 spaces per + level. +- In TypeScript and TSX, put the opening brace of `try`, `catch`, and + `finally` blocks on the next line at the same indentation as the keyword. +- Do not indent the opening `{` one level deeper than `try`, `catch`, or + `finally`. +- Indent the block body 2 spaces deeper than the keyword and opening brace. +- Put the closing `}` on its own line at the same indentation as the keyword. +- Do not write `try {`, `catch {`, or `finally {`. +- In TypeScript and TSX, use 4-space continuation indentation for wrapped + expressions, arguments, conditions, arrays, object literals, JSX + attributes, and similar continuations. +- Treat the user's `PostImportSourcePage.tsx` and `PostImportReviewPage.tsx` + formatting as the local reference shape: component and callback bodies use + 2-space block indentation, single-line bodies do not gain unnecessary + braces, wrapped expressions use 4-space continuation indentation, multi- + stage ternaries use explicit parentheses, and TypeScript / TSX indentation + and alignment whitespace compress every complete run of 8 spaces to tabs. +- A tab does not represent one indentation level. +- In TypeScript and TSX, first determine visible indentation using 2-space + block indentation and 4-space continuation indentation, then compress every + complete run of 8 spaces used for indentation or column alignment to tabs. +- In TypeScript and TSX, 8-space compression is mandatory, not optional. +- In TypeScript and TSX, this applies both to leading indentation and to + alignment whitespace after non-space text, such as aligned inline type + columns. +- In TypeScript and TSX, keep residual 2, 4, or 6 spaces after each tab + compression. +- In TypeScript and TSX, do not treat a tab as one logical indentation level. +- In TypeScript and TSX, do not alter string literals, template-literal + contents, regular expressions, or user-facing text merely to apply tab + compression. +- Examples: 2 columns = 2 spaces, 4 columns = 4 spaces, 6 columns = 6 + spaces, 8 columns = 1 tab, 10 columns = 1 tab + 2 spaces, 12 columns = 1 + tab + 4 spaces, 14 columns = 1 tab + 6 spaces, 16 columns = 2 tabs. +- When TypeScript or TSX code already uses column alignment, apply the same + 8-space compression rule to that alignment whitespace. +- Example: + +```ts +type Props = { + row: PostImportRow + displayNumber?: number + onEdit?: () => void + onRetry?: () => void + onToggleSkip?: (checked: boolean) => void } +``` - TypeScript and TSX imports may stay on one line if they remain within the line limit; do not expand short type-only imports mechanically. +- Keep runtime value imports and type imports in separate declarations. +- Do not write `import { value, type TypeName } from ...`. +- Type-only declarations must use `import type`. +- Do not merge type imports into value imports merely to save lines. +- In TypeScript and TSX, order imports as four groups with a blank line + between groups: external value imports, `@/...` value imports, external + type imports, `@/...` type imports. +- Do not mechanically split a short value import from one module across + multiple lines when it still fits within 99 characters. - In TypeScript and TSX, when breaking a line at an operator, break before the operator and put the operator at the beginning of the next line. A trailing operator at end of line is unacceptable. This rule does not apply to Ruby, @@ -167,6 +381,17 @@ case 'no': - In TypeScript and TSX, use `value == null` and `value != null` as the default nullish checks. Do not use `=== null`, `=== undefined`, `!== null`, or `!== undefined`. +- In JavaScript, JSX, TypeScript, and TSX, never use `_1`, `_2`, or similar + Ruby-style numbered parameter names. Reserve numbered parameters for Ruby. + Use a meaningful callback parameter name such as `row`, `item`, `value`, + `entry`, or `result`. +- In JavaScript, JSX, TypeScript, and TSX, use `cn` from `@/lib/utils` + whenever `className` combines multiple values, conditional classes, or a + caller-provided `className` prop. +- Do not construct `className` with template literals, `${ ... }`, string + concatenation, arrays joined with spaces, or feature-local class-merging + helpers. +- A static `className="..."` containing only fixed classes does not need `cn`. - If code appears to need a distinction between `null` and `undefined`, treat that as a design smell and revise the logic to avoid the distinction. External library APIs that explicitly require distinguishing the two are the @@ -190,6 +415,151 @@ const value = value selection. Do not replace a clear ternary with `if` statements, and do not introduce immediately invoked functions just to avoid or reformat a ternary expression. +- In TypeScript and TSX, multi-stage ternary expressions must make branch + boundaries explicit. Wrap each condition group in parentheses instead of + relying on indentation alone to show which `?` matches which `:`. +- In TypeScript and TSX, wrap nested ternary expressions in parentheses. Do + not write flat vertical chains of `?` and `:` without explicit grouping. +- In TypeScript and TSX, when a ternary condition contains `&&` or `||`, wrap + the whole condition in parentheses before `?`. +- In TypeScript and TSX, if a ternary reaches three or more stages and still + reads poorly after explicit grouping, extract a helper function or use `if` + statements instead of keeping a flat multi-stage ternary. + +Bad: + +```ts +row.skipReason === 'existing' || row.importStatus === 'skipped' +? 'skipped' +: Object.keys (row.validationErrors ?? { }).length > 0 + || row.importStatus === 'failed' + || row.importStatus === 'created' +? null +: hasWarnings (row) || row.status === 'warning' +? 'warning' +: 'ready' +``` + +Good: + +```ts +(row.skipReason === 'existing' || row.importStatus === 'skipped') +? 'skipped' +: ((Object.keys (row.validationErrors ?? { }).length > 0 + || row.importStatus === 'failed' + || row.importStatus === 'created') + ? null + : ((hasWarnings (row) || row.status === 'warning') + ? 'warning' + : 'ready')) +``` + +## Shared-system discovery and reuse + +Before creating a new component, hook, service, helper, utility, concern, +representation, normaliser, validator, parser, fetcher, store, context, event +bus, API client, query key, permission helper, dialogue, toast, form field, or +version recorder, search the existing repository first. + +Do not search by name alone. Search by responsibility, behaviour, and usage +intent as well. Typical search themes include: + +- dialogue, modal, confirm, alert, choice +- validation error, field error, unprocessable entity +- permission, role, member, admin, editable +- URL normalise, sanitise, canonicalise +- API call, query key, prefetch, cache invalidation +- version, snapshot, history, restore +- thumbnail, metadata, HTTP fetch, URL safety +- form, field, input, textarea, warning, status badge +- file storage, Active Storage, ZIP, export +- transaction, locking, race, idempotency + +Before deciding that something new is needed, confirm at least: + +1. the existing definition +2. its public API +3. representative call sites +4. other implementations with similar responsibility +5. the nearest directory-level `AGENTS.md` + +Do not reject an existing implementation by name alone. Read the code and its +usage first. + +When new behaviour is needed, make the decision in this order: + +1. use the existing common API as-is +2. use the existing extension points of that API +3. extend the existing common API minimally +4. keep the implementation local when the meaning is feature-specific +5. add a new common system only when multiple real users and a stable contract + are already clear + +Do not create a parallel foundation merely because the existing one feels +slightly awkward, because a new one seems faster, or because the current task +looks special. + +Do not create a second common platform with names such as `CommonFoo`, +`SharedFoo`, `BaseFoo`, `FooManager`, `FooService`, `FooProvider`, +`FooWrapper`, `FooUtils`, or `useFoo` when an existing system already owns the +same responsibility. Judge by responsibility, not by spelling. + +Wrapper aliases, barrel exports, and re-export shims do not count as reuse. +Using a different import path that merely forwards to the existing system does +not satisfy a reuse requirement. + +Keep this boundary explicit: + +- business meaning stays in feature code +- visual and mechanical shell stays in common code + +Common code may hold generic primitives, interaction shells, API transport, +auth and permission helpers, query-key and prefetch conventions, validation +error conversion, shared normalisation, version-recording mechanisms, storage +helpers, HTTP safety, and other contracts that carry the same meaning across +multiple features. + +Feature code should keep feature-specific states, labels, input fields, +workflow steps, payload shapes, validation rules, and business decisions. + +Do not commonise business logic merely because the visuals look similar or two +code fragments resemble each other. + +Create a new common system only when all of the following are true: + +- no existing implementation already owns the responsibility +- there are multiple real consumers, or a clearly defined platform contract +- the differences between consumers do not require option bloat +- feature-specific vocabulary does not leak into common types, props, tones, or + state names +- the location matches the existing directory structure +- the new abstraction does not compete with an existing common system +- it is not just a one-off wrapper for a single task + +Do not add unused tones, variants, options, callbacks, states, or abstractions +for speculative future use. + +Before finishing work that touches shared systems, verify: + +- you searched for an existing implementation with the same responsibility +- you are using the canonical import path or entrypoint +- feature code is not reaching directly for a low-level primitive that already + has a higher-level common API +- you did not bypass an existing common API +- any new wrapper or shim is genuinely necessary +- feature-specific vocabulary did not leak into common code +- the feature did not reimplement a common shell +- existing unrelated consumers were not changed without need +- one task did not trigger a needless redesign of the whole foundation +- any newly introduced common system really has multiple consumers + +For backend-specific discovery, Rails structure, services, representation +selection, versioning, normalisation, and HTTP safety, follow +`backend/AGENTS.md`. + +For frontend-specific component hierarchy, low-level primitive reuse, dialogue +entrypoints, API/query helpers, validation/form infrastructure, state/storage, +and layout reuse, follow `frontend/AGENTS.md`. - In TypeScript and TSX, do not write `let` followed by later `if` assignments when the value can be expressed as a single `const` initializer. Prefer `const` because it prevents accidental later reassignment. @@ -197,9 +567,39 @@ const value = structure, control flow, or variable mutability unless the requested style explicitly requires it. - Do not add production dependencies without explicit approval. +- Do not add user-facing copy, helper text, descriptions, notes, tooltips, + placeholders, empty-state messages, loading messages, or explanatory text + unless the user explicitly specified the wording. +- When new user-facing wording appears necessary, ask the user for the exact + wording and placement before implementing it. +- Do not invent replacement copy when removing unrequested wording. - Do not create, modify, or run tests unless the user explicitly asks for - test work. When the user asks for tests, keep working and rerun them until - they pass or the remaining failure is clearly blocked. + test work. When the user asks for tests, keep working within the permitted + test-file scope and rerun them until they pass or the remaining failure is + clearly blocked. +- Test-only work includes adding, updating, deleting, reorganising, or fixing + SyntaxError in tests. During test-only work, do not modify production code. +- During test-only work, do not change production constants, behaviour, API + contracts, validation, routes, authentication, permissions, UI, copy, + dependencies, limits, thresholds, defaults, migrations, schema, or + environment settings to satisfy tests. +- Do not make production code match failing tests, mock assumptions, fixtures, + snapshots, old expectations, or stale setup. This includes changing + production constants merely because a test expects a different value. +- If test work reveals a production bug, spec mismatch, or missing behaviour, + stop without modifying production code and report: the failing test or + discovered issue, the related production file, the actual behaviour, the + expected behaviour, and why a production change appears necessary. +- Modify production code for test failures only when the user explicitly asks + for that production change. Do not expand a test task into a production task + on your own authority. +- If the user explicitly asks for both production implementation and test + updates, implement production code to the confirmed specification first, + then add or update tests to verify that specification. Never roll production + behaviour back to satisfy old tests. +- If it is unclear whether the test or the production implementation is stale, + or a test cannot be corrected without changing production code, ask the user + instead of guessing. ## Backend rules @@ -253,6 +653,11 @@ const value = - Mobile UI must be checked as a first-class layout. Avoid wide fixed content, make dense controls wrap or scroll intentionally, and keep tag/filter controls usable without horizontal page overflow. +- Frontend のスマホ／PC表示境界は原則 `md` とする。 +- button stack、footer action、dialogue action は `md` 未満で縦並び、 + `md` 以上で横並びとする。 +- 同じ画面内で `sm` と `md` を混在させて中間 layout を作らない。 +- 明確に別の responsive 要件がある component だけを例外とする。 - For mobile horizontal scrollers, make the scroll direction and item sizing explicit, and ensure chip text remains readable in both light and dark modes. - In TypeScript and TSX, prefer direct comparison operators such as `===` and @@ -265,6 +670,20 @@ const value = - For user-facing Japanese kanji spelling, do not normalize to 《当用漢字による書きかえ》; prefer original forms such as `編輯`. - For user-facing Japanese ellipses, prefer `……` over ASCII `...`. +- Frontend dialogue work must use `@/lib/dialogues/useDialogue` as the + feature-facing entrypoint. +- Reuse the existing common dialogue API and common dialogue component + instead of building feature-local overlay, close button, header, footer, + focus handling, outside click handling, Escape handling, or confirmation + flows. +- Do not import `@/components/ui/dialog` directly in feature code to build a + one-off dialogue, and do not evade this rule with aliases such as + `Dialog as Dialogue`. +- Keep business-specific form content in feature code, and keep the visual + and behavioural dialogue shell in common code. +- Use British spelling `Dialogue` for project-defined dialogue identifiers. + Keep an exact third-party API spelling only at the external boundary where + compatibility requires it. ### Frontend TypeScript and TSX style @@ -303,15 +722,34 @@ const value = beginning of a line. - The TSX-specific self-review must confirm JSX closing markers and closing parentheses keep the surrounding compact style. -- The TypeScript/TSX self-review must confirm leading indentation follows - 4-space logical indentation with tabs only as leading 8-space compression. +- The TypeScript/TSX self-review must confirm block indentation uses 2 spaces + per level, wrapped continuations use the repository's 4-space continuation + alignment, and every complete run of 8 spaces used for indentation or + alignment has been compressed to tabs. - Prefer `const` arrow functions for TypeScript/TSX component and helper declarations. - Put two blank lines before and after top-level `const` function declarations, unless imports, exports, or file boundaries make that awkward. -- In TSX, indent with 4-space logical indentation. -- In TypeScript and TSX, convert every leading run of 8 spaces to a tab - character. -- A leading tab is exactly equivalent to 8 leading spaces. +- In TSX, use 2-space block indentation and 4-space continuation + indentation. +- In TypeScript and TSX, put the opening brace of `try`, `catch`, and + `finally` blocks on the next line at the same indentation as the keyword. +- Do not indent the opening `{` one level deeper than `try`, `catch`, or + `finally`. +- Indent the block body 2 spaces deeper than the keyword and opening brace. +- Put the closing `}` on its own line at the same indentation as the keyword. +- Do not write `try {`, `catch {`, or `finally {`. +- In TypeScript and TSX, convert every complete run of 8 spaces used for + indentation or alignment to a tab character. +- In TypeScript and TSX, a tab is exactly equivalent to 8 columns, whether it + appears at the beginning of a line or in alignment whitespace after + non-space text. +- In TSX, JSX nesting uses 2-space block indentation and wrapped JSX + attributes use 4-space continuation indentation; after visible columns are + determined, compress every complete run of 8 spaces in the resulting + indentation or alignment to tabs. +- In TSX, do not leave JSX subtree indentation at 8, 10, 12, 14, or 16 + columns as spaces alone; convert each complete run of 8 spaces to tabs and + keep only the residual 2, 4, or 6 spaces. - In TypeScript and TSX function declarations, including `const` arrow function declarations, classify the parameter list before placing the closing `)`. @@ -359,6 +797,22 @@ const value = single physical line. - Always add braces around `if`, `else`, or `for` bodies when the body spans two or more physical lines, even if it is one statement. +- `try` / `catch` / `finally` brace placement example: + +```ts +try +{ + doWork () +} +catch +{ + recover () +} +finally +{ + cleanUp () +} +``` - Do not use a leading semicolon for expression statements such as `;([...]).forEach(...)`; rewrite the expression to avoid ASI hazards explicitly, for example with `void`. @@ -938,8 +1392,9 @@ to `.ts` and `.tsx`: 7. JSX `>` and `/>` stay with the final prop unless nearby code proves otherwise. 8. JSX closing parentheses keep the compact local style. -9. Leading indentation is 4-space logical indentation with tabs used only as - leading 8-space compression. +9. Block indentation uses 2 spaces per level, wrapped continuations use the + repository's 4-space continuation alignment, and every complete run of 8 + spaces used for indentation or alignment has been compressed to tabs. 10. No line has trailing whitespace. Preferred: diff --git a/backend/AGENTS.md b/backend/AGENTS.md index 2953f22..e0f304e 100644 --- a/backend/AGENTS.md +++ b/backend/AGENTS.md @@ -67,6 +67,192 @@ pass or the remaining failure is clearly blocked. Before changing behavior, inspect the matching route, controller, model, service, representation, and spec. +## Shared backend systems + +Before adding backend behaviour, search the existing backend first. At minimum, +check these locations: + +- `app/controllers` +- `app/controllers/concerns` +- `app/models` +- `app/models/concerns` +- `app/representations` +- `app/services` +- `app/services/*` +- `app/jobs` +- `lib` +- `lib/tasks` +- `config/initializers` + +Do not infer commonality from directory names alone. Read the actual +responsibility and representative usage sites. + +### Controller reuse + +Before adding logic to a controller, inspect: + +- `ApplicationController` authentication, authorization, BAN, and IP BAN +- existing render and validation-error helpers +- existing param parsing +- controller concerns +- the controller for the same resource +- existing services +- existing representations + +Keep controllers focused on: + +- authentication and authorization +- parameter intake +- service and model invocation +- HTTP status selection +- representation selection + +Do not reimplement these per controller when an existing path already owns +them: + +- authentication and role checks +- validation error JSON +- URL normalisation +- tag normalisation +- thumbnail handling +- version recording +- complex transactions +- external HTTP fetching +- response representation assembly + +### Authentication, authorization, and BAN + +Treat these as the canonical backend entrypoints: + +- `ApplicationController#authenticate_user` +- `current_user` +- `X-Transfer-Code` +- `reject_banned_ip_address!` +- `reject_banned_user!` +- `gte_member?` +- `admin?` + +Do not create feature-local permission services, role comparisons, or header +parsing when the existing authentication boundary already owns the behaviour. +If the current boundary is insufficient, extend it minimally instead of adding +another permission path. + +### Representations + +If an endpoint for the same resource already uses `app/representations`, do not +assemble a separate JSON shape directly inside the controller without first +checking the existing representation contract. + +Inspect at least: + +- `PostRepr` +- `TagRepr` +- `MaterialRepr` +- `TheatreRepr` +- `UserRepr` +- `WikiPageRepr` +- `DeerjikistRepr` + +When a lightweight response is genuinely different in purpose, keep it +deliberate and compatible with the surrounding contracts. Do not force every +identifier list into a large representation, but do not fork the same resource +shape casually either. + +### Domain services + +When work touches multiple models, transactions, external APIs, file handling, +history creation, or multi-step workflow, search `app/services` first. + +At minimum, search for existing services in these responsibility areas: + +- version recorder and versioning +- wiki commit +- YouTube or Google Drive API client +- material sync or ZIP export +- similarity calculation +- theatre selection or skip finalisation +- metadata, thumbnail, or file processing +- URL normaliser or sanitisation +- import or export +- preview safety or HTTP fetch + +Do not create a same-responsibility service under another namespace or another +name. If an existing service is close, extend that API minimally instead of +wrapping it in a feature-local service. + +### Versioning + +When a feature writes history, snapshots, or restore roots, search the existing +versioning path first. At minimum, inspect: + +- `VersionRecorder` +- `PostVersionRecorder` +- `TagVersionRecorder` +- `TagVersioning` +- `MaterialVersionRecorder` +- `NicoTagVersionRecorder` +- `WikiVersionRecorder` + +Do not implement history writes in controllers, callbacks, or ad hoc feature +services when the recorder layer already owns the transaction boundary and +meaning. + +### Normalisation, sanitisation, and parsing + +For URLs, tag names, times, video durations, identifiers, and paths, search the +existing normaliser, sanitisation rule, parser, and model-callback path first. + +Do not let frontend, controller, service, and model each invent different rules +for the same value. Use one canonical normalisation path and keep input +validation distinct from pre-persistence normalisation. + +### External HTTP and URL safety + +When fetching external URLs, reuse the existing preview-safety stack. Search at +least for: + +- URL safety +- redirect validation +- response size limits +- timeouts +- network failure mapping +- HTML metadata extraction +- known-site extraction +- thumbnail fetching + +Do not add direct `Net::HTTP`, `Faraday`, or equivalent feature-local HTTP code +that reimplements SSRF checks, redirect restrictions, size limits, or timeouts. +If the current fetcher is insufficient, extend its existing safety contract. + +### Storage, files, and Active Storage + +When handling files, thumbnails, ZIP output, object storage, or Active Storage +blobs, inspect existing storage helpers, exporters, thumbnail generators, and +checksum helpers first. Do not reimplement the same attach, export path, +download, resize, or checksum flow in a controller or one-off service. + +### Concerns + +Do not create controller or model concerns merely because some code is shared. +Use a concern only when multiple classes share the same lifecycle, macro, +callback, or tightly cohesive behaviour. Utility collections belong in explicit +objects or services, not in `CommonConcern`, `SharedMethods`, or `Utils`. + +### Model boundaries + +Model-specific invariants, associations, validations, and normalisation may +live in the model. Multi-model workflow, external access, complex transaction +flow, and feature orchestration belong in services. Do not hide feature +workflow in model callbacks. + +### Transactions, locking, and race handling + +If transactions, locking, idempotency, or race recovery already exist in a +service or model method, do not add a second implementation in a controller or +new service. Inspect the existing transaction boundary first, avoid wrapping +the same operation in needless nested transactions, and handle unique-constraint +races according to the target constraint's business meaning. + ## Ruby style - Prefer precise, minimal changes. diff --git a/backend/app/controllers/posts_controller.rb b/backend/app/controllers/posts_controller.rb index 339519b..fb3c61a 100644 --- a/backend/app/controllers/posts_controller.rb +++ b/backend/app/controllers/posts_controller.rb @@ -1,5 +1,6 @@ class PostsController < ApplicationController Event = Struct.new(:post, :tag, :user, :change_type, :timestamp, keyword_init: true) + MAX_BULK_REQUEST_BYTES = 40 * 1024 * 1024 class VideoMsParseError < ArgumentError ; @@ -114,6 +115,77 @@ class PostsController < ApplicationController render json: PostRepr.base(post, current_user) end + def metadata + return head :unauthorized unless current_user + return head :forbidden unless current_user.gte_member? + return render_bad_request('URL は必須です．') if params[:url].blank? + + normal_url = PostUrlNormaliser.normalise(params[:url].to_s) + return render_validation_error(fields: { url: ['URL が不正です．'] }) if normal_url.blank? + + Preview::UrlSafety.validate(normal_url) + existing_post = Post.with_attached_thumbnail.find_by(url: normal_url) + if existing_post.present? + return render json: { + url: normal_url, + title: nil, + thumbnail_base: nil, + tags: nil, + display_tags: [], + original_created_from: nil, + original_created_before: nil, + duration: nil, + video_ms: nil, + field_warnings: { }, + base_warnings: [], + existing_post_id: existing_post.id, + existing_post: compact_post(existing_post.id) } + end + + metadata = PostMetadataFetcher.fetch(normal_url) + field_warnings = { } + field_warnings[:title] = ['タイトルを取得できませんでした．'] if metadata[:title].blank? + if metadata[:thumbnail_base].blank? + field_warnings[:thumbnail_base] = ['サムネールを取得できませんでした．'] + end + + render json: { + url: normal_url, + title: metadata[:title], + thumbnail_base: metadata[:thumbnail_base], + tags: metadata[:tags], + display_tags: metadata[:display_tags], + original_created_from: metadata[:original_created_from], + original_created_before: metadata[:original_created_before], + duration: metadata[:duration], + video_ms: metadata[:video_ms], + field_warnings: field_warnings, + base_warnings: [], + existing_post_id: nil, + existing_post: nil } + rescue ArgumentError => e + render_bad_request e.message + rescue Preview::UrlSafety::UnsafeUrl => e + render_validation_error fields: { url: [e.message] } + rescue Preview::HttpFetcher::FetchFailed, + Preview::HttpFetcher::FetchTimeout, + Preview::HttpFetcher::ResponseTooLarge + render json: { + url: normal_url, + title: nil, + thumbnail_base: nil, + tags: nil, + display_tags: [], + original_created_from: nil, + original_created_before: nil, + duration: nil, + video_ms: nil, + field_warnings: { url: ['自動取得に失敗しました．'] }, + base_warnings: [], + existing_post_id: nil, + existing_post: nil } + end + def show post = Post @@ -142,48 +214,51 @@ class PostsController < ApplicationController return head :unauthorized unless current_user return head :forbidden unless current_user.gte_member? - # TODO: サイトに応じて thumbnail_base 設定 - title = params[:title].presence - url = params[:url] - thumbnail = params[:thumbnail] - tag_names = params[:tags].to_s.split - original_created_from = params[:original_created_from] - original_created_before = params[:original_created_before] - parent_post_ids = parse_parent_post_ids - resized_thumbnail = thumbnail.present? ? Post.resized_thumbnail_attachment(thumbnail) : nil - - post = Post.new(title:, url:, thumbnail_base: nil, uploaded_user: current_user, - original_created_from:, original_created_before:) - post.thumbnail.attach(resized_thumbnail) if resized_thumbnail - - ApplicationRecord.transaction do - post.save! - - Tag.normalise_tags!(tag_names, deny_deprecated: true, with_sections: true) => - { tags:, sections: } - TagVersioning.record_tag_snapshots!(tags, created_by_user: current_user) - - tags = Tag.expand_parent_tags(tags).reject(&:deprecated?) - post.video_ms = normalise_video_ms(tags) - validate_video_sections!(post.video_ms, sections) - post.save! - sync_post_tags!(post, tags, sections) - - sync_parent_posts!(post, parent_post_ids) - - PostVersionRecorder.record!(post:, event_type: :create, created_by_user: current_user) + preflight = PostCreatePreflight.new( + attributes: post_create_attributes, + thumbnail: params[:thumbnail], + host: request.base_url).run + return render json: dry_run_json(preflight) if bool?(:dry) + if preflight[:existing_post_id].present? + post = Post.new(url: preflight[:url]) + post.errors.add :url, :taken + return render_post_form_record_invalid post end + post = PostCreator.new(actor: current_user, + attributes: post_create_attributes.merge( + preflight.slice( + :url, + :title, + :thumbnail_base, + :tags, + :parent_post_ids, + :original_created_from, + :original_created_before, + :duration, + :video_ms, + :direct_tag_specs, + :default_tag_specs, + :snapshot_tag_specs, + :post_tag_specs, + :tag_sections, + :normalised_parent_post_ids).symbolize_keys).merge( + thumbnail: params[:thumbnail])).create! + post.reload render json: PostRepr.base(post), status: :created + rescue PostCreatePreflight::ValidationFailed => e + render_validation_error fields: e.fields, base: e.base_errors rescue Tag::NicoTagNormalisationError render_validation_error fields: { tags: 'ニコニコ・タグは直接指定できません．' } rescue Tag::DeprecatedTagNormalisationError render_unprocessable_entity '廃止済みタグは付与できません．', field: :tags rescue Tag::SectionLiteralParseError render_validation_error fields: { tags: ['タグ区間の記法が不正です．'] } - rescue VideoMsParseError + rescue PostCreator::VideoMsParseError render_validation_error fields: { video_ms: ['動画時間の記法が不正です．'] } + rescue Post::RemoteThumbnailFetchFailed + render_validation_error fields: { thumbnail_base: ['サムネイル画像の取得に失敗しました．'] } rescue MiniMagick::Error render_validation_error fields: { thumbnail: ['サムネイル画像の変換に失敗しました．'] } rescue ArgumentError => e @@ -192,6 +267,25 @@ class PostsController < ApplicationController render_post_form_record_invalid e.record end + def bulk + return head :unauthorized unless current_user + return head :forbidden unless current_user.gte_member? + return head :unsupported_media_type unless request.content_mime_type == Mime[:multipart_form] + return head :payload_too_large if request.content_length.to_i > MAX_BULK_REQUEST_BYTES + posts = parse_bulk_posts_manifest + thumbnails = parse_bulk_thumbnails(posts.length) + result = PostBulkCreator.new( + actor: current_user, + posts:, + thumbnails:, + host: request.base_url).run + render json: result + rescue JSON::ParserError + render_bad_request 'posts manifest の JSON が不正です．' + rescue ArgumentError => e + render_validation_error base: [e.message] + end + def viewed return head :unauthorized unless current_user @@ -486,6 +580,79 @@ class PostsController < ApplicationController }.uniq end + def post_create_attributes + { title: params[:title], + url: params[:url], + thumbnail_base: params[:thumbnail_base], + tags: params[:tags], + original_created_from: params[:original_created_from], + original_created_before: params[:original_created_before], + parent_post_ids: parse_parent_post_ids, + video_ms: params[:video_ms], + duration: params[:duration] } + end + + def parse_bulk_posts_manifest + manifest = params[:posts] + raise ArgumentError, 'posts は必須です．' if manifest.blank? + raise ArgumentError, 'posts は JSON 文字列で指定してください．' unless manifest.is_a?(String) + + posts = JSON.parse(manifest) + raise ArgumentError, 'posts は配列で指定してください．' unless posts.is_a?(Array) + raise ArgumentError, '投稿件数は 1 件以上必要です．' if posts.empty? + raise ArgumentError, '投稿件数が多すぎます．' if posts.length > 100 + raise ArgumentError, 'posts 要素の形式が不正です．' unless posts.all? { _1.is_a?(Hash) } + + posts + end + + def parse_bulk_thumbnails post_count + thumbnails = { } + raw = params[:thumbnails] + return thumbnails if raw.blank? + raise ArgumentError, 'thumbnail key が不正です．' unless raw.respond_to?(:to_unsafe_h) + + raw.to_unsafe_h.each do |key, value| + raise ArgumentError, 'thumbnail key が不正です．' unless key.to_s.match?(/\A\d+\z/) + + index = Integer(key, 10) + raise ArgumentError, 'thumbnail index が範囲外です．' if index.negative? || index >= post_count + raise ArgumentError, 'thumbnail index が重複しています．' if thumbnails.key?(index) + unless value.is_a?(ActionDispatch::Http::UploadedFile) + raise ArgumentError, 'thumbnail upload が不正です．' + end + + thumbnails[index] = value + end + + thumbnails + end + + def compact_post post_id + return nil if post_id.blank? + + post = Post.with_attached_thumbnail.find_by(id: post_id) + PostCompactRepr.base(post, host: request.base_url) + end + + def dry_run_json preflight + preflight.slice( + :url, + :title, + :thumbnail_base, + :tags, + :display_tags, + :parent_post_ids, + :original_created_from, + :original_created_before, + :duration, + :video_ms, + :field_warnings, + :base_warnings, + :existing_post_id, + :existing_post) + end + def sync_parent_posts! post, parent_post_ids if parent_post_ids.include?(post.id) post.errors.add :parent_post_ids, '自分自身を親投稿にはできません．' @@ -565,6 +732,7 @@ class PostsController < ApplicationController def post_incoming_snapshot title:, original_created_from:, original_created_before:, tag_names:, video_ms_param:, duration_param:, parent_post_ids: + validate_original_created_values!(original_created_from, original_created_before) Tag.normalise_tags!(tag_names, with_tagme: false, deny_deprecated: true, with_sections: true) => { tags:, sections: } @@ -602,6 +770,23 @@ class PostsController < ApplicationController value.to_s end + def validate_original_created_values! original_created_from, original_created_before + candidate = Post.new( + url: 'https://example.invalid/original-created-validation', + original_created_from:, + original_created_before:) + candidate.valid? + fields = [:original_created_from, :original_created_before, :original_created_at] + relevant_errors = candidate.errors.select { fields.include?(_1.attribute) } + return if relevant_errors.empty? + + invalid_post = Post.new + relevant_errors.each { |error| + invalid_post.errors.add(error.attribute, error.message) + } + raise ActiveRecord::RecordInvalid, invalid_post + end + def section_literal section "[#{ Post.ms_to_time(section[0]) }-#{ section[1] ? Post.ms_to_time(section[1]) : '' }]" end diff --git a/backend/app/controllers/preview_controller.rb b/backend/app/controllers/preview_controller.rb index 4829e2b..74fe1a9 100644 --- a/backend/app/controllers/preview_controller.rb +++ b/backend/app/controllers/preview_controller.rb @@ -18,14 +18,12 @@ class PreviewController < ApplicationController def thumbnail return render_bad_request('URL は必須です．') if params[:url].blank? - image = MiniMagick::Image.read(Preview::ThumbnailFetcher.fetch(params[:url])) - image.auto_orient - image.resize '180x180>' - image.format 'png' - width, height = image.dimensions - raise Preview::ThumbnailFetcher::GenerationFailed, 'サムネール画像の変換に失敗しました．' if width > 180 || height > 180 - - send_data image.to_blob, type: 'image/png', disposition: 'inline' + attachment = + Post.resized_thumbnail_attachment( + StringIO.new(Preview::ThumbnailFetcher.fetch(params[:url]))) + send_data attachment[:io].read, + type: attachment[:content_type], + disposition: 'inline' rescue Preview::UrlSafety::UnsafeUrl => e render_bad_request(e.message) rescue Preview::HttpFetcher::FetchTimeout => e diff --git a/backend/app/models/post.rb b/backend/app/models/post.rb index d22dd8d..4f5b09d 100644 --- a/backend/app/models/post.rb +++ b/backend/app/models/post.rb @@ -1,20 +1,57 @@ class Post < ApplicationRecord + require 'date' require 'mini_magick' + require 'nokogiri' require 'stringio' + require 'timeout' - def self.resized_thumbnail_attachment(upload) + class RemoteThumbnailFetchFailed < StandardError; end + + ORIGINAL_CREATED_INVALID_MESSAGE = 'オリジナルの作成日時の形式が不正です．'.freeze + ORIGINAL_CREATED_MINUTE_PRECISION_MESSAGE = + 'オリジナルの作成日時は分単位で入力してください．'.freeze + ORIGINAL_CREATED_ORDER_MESSAGE = 'オリジナルの作成日時の順番がをかしぃです．'.freeze + ORIGINAL_CREATED_MINIMUM_RANGE_MESSAGE = + 'オリジナルの作成日時の範囲は1分以上必要です．'.freeze + REMOTE_SVG_CONTENT_TYPE = 'image/svg+xml'.freeze + MAX_SVG_DIMENSION = 4_096 + MAX_SVG_PIXELS = 16_777_216 + THUMBNAIL_PROCESS_TIMEOUT = 5.seconds + def self.resized_thumbnail_attachment(upload, content_type: nil) upload.rewind - image = MiniMagick::Image.read(upload.read) - image.resize '180x180' - image.format 'jpg' + bytes = upload.read + blob = Timeout.timeout(THUMBNAIL_PROCESS_TIMEOUT) do + image = image_for_thumbnail_upload(bytes, content_type:) + image.auto_orient + image.resize '180x180' + image.format 'jpg' + image.to_blob + end - { io: StringIO.new(image.to_blob), + { io: StringIO.new(blob), filename: 'resized_thumbnail.jpg', content_type: 'image/jpeg' } + rescue Timeout::Error + raise MiniMagick::Error, 'サムネイル画像の変換に失敗しました．' ensure upload.rewind end + def self.remote_thumbnail_attachment(raw_url) + response = Preview::ThumbnailFetcher.fetch_image_response(raw_url) + resized_thumbnail_attachment( + StringIO.new(response.body), + content_type: response.content_type) + rescue Preview::UrlSafety::UnsafeUrl, + Preview::ThumbnailFetcher::GenerationFailed, + Preview::HttpFetcher::FetchFailed, + Preview::HttpFetcher::FetchTimeout, + Preview::HttpFetcher::ResponseTooLarge, + Timeout::Error, + MiniMagick::Error => e + raise RemoteThumbnailFetchFailed, e.message + end + belongs_to :uploaded_user, class_name: 'User', optional: true has_many :post_tags, dependent: :destroy, inverse_of: :post @@ -101,7 +138,10 @@ class Post < ApplicationRecord end def self.section_literal section - "[#{ Post.ms_to_time(section.begin_ms) }-#{ section.end_ms ? Post.ms_to_time(section.end_ms) : '' }]" + end_ms = + section.end_ms ? Post.ms_to_time(section.end_ms) : '' + + "[#{ Post.ms_to_time(section.begin_ms) }-#{ end_ms }]" end def self.ms_to_time ms @@ -118,7 +158,11 @@ class Post < ApplicationRecord '%d:%02d' % [min, s] end - remainder_ms.positive? ? "#{ base }.#{ remainder_ms.to_s.rjust(3, '0') }" : base + if remainder_ms.positive? + "#{ base }.#{ remainder_ms.to_s.rjust(3, '0') }" + else + base + end end def snapshot_parent_post_ids = parents.order(:id).pluck(:id) @@ -140,19 +184,26 @@ class Post < ApplicationRecord thumbnail.attach(self.class.resized_thumbnail_attachment(StringIO.new(thumbnail.download))) end + def attach_thumbnail_from_url! raw_url + thumbnail.attach(self.class.remote_thumbnail_attachment(raw_url)) + end + private def validate_original_created_range - f = original_created_from - b = original_created_before - return if f.blank? || b.blank? + return if skip_original_created_validation? - f = Time.zone.parse(f) if String === f - b = Time.zone.parse(b) if String === b - return if !(f) || !(b) + f = parse_original_created_value(:original_created_from) + b = parse_original_created_value(:original_created_before) + return if f.nil? || b.nil? - if f >= b - errors.add :original_created_at, 'オリジナルの作成日時の順番がをかしぃです．' + if b <= f + errors.add :original_created_at, ORIGINAL_CREATED_ORDER_MESSAGE + return + end + + if b - f < 1.minute + errors.add :original_created_at, ORIGINAL_CREATED_MINIMUM_RANGE_MESSAGE end end @@ -173,15 +224,307 @@ class Post < ApplicationRecord def normalise_url return if url.blank? - self.url = url.strip + self.url = PostUrlNormaliser.normalise(url) || url.strip + end - u = URI.parse(url) - return unless u in URI::HTTP + def self.image_for_thumbnail_upload(bytes, content_type: nil) + if svg_content_type?(content_type) || svg_document_bytes?(bytes) + return decode_svg_thumbnail(bytes) + end - u.host = u.host.downcase if u.host - u.path = u.path.sub(/\/\Z/, '') if u.path.present? - self.url = PostUrlSanitisationRule.sanitise(u.to_s) - rescue URI::InvalidURIError - ; + raise MiniMagick::Error, 'サムネイル画像の形式が不正です．' unless raster_thumbnail_bytes?(bytes) + + decode_raster_thumbnail(bytes) + end + + def self.raster_thumbnail_bytes?(bytes) + raster_thumbnail_format(bytes).present? + end + + def self.remote_thumbnail_image_bytes?(bytes, content_type: nil) + return true if svg_content_type?(content_type) || svg_document_bytes?(bytes) + + raster_thumbnail_bytes?(bytes) + end + + def self.svg_content_type?(content_type) + content_type.to_s.split(';', 2).first.to_s.downcase.strip == REMOTE_SVG_CONTENT_TYPE + end + + def self.svg_document_bytes?(bytes) + document = Nokogiri::XML( + bytes, + nil, + nil, + Nokogiri::XML::ParseOptions::STRICT | + Nokogiri::XML::ParseOptions::NONET) + document.root&.name == 'svg' + rescue Nokogiri::XML::SyntaxError + false + end + + def self.decode_raster_thumbnail(bytes) + MiniMagick::Image.read(bytes) + end + + def self.decode_svg_thumbnail(bytes) + MiniMagick::Image.read(sanitised_svg_bytes(bytes)) + end + + def self.raster_thumbnail_format(bytes) + binary = bytes.to_s.b + return 'jpeg' if binary.start_with?("\xFF\xD8\xFF".b) + return 'png' if binary.start_with?("\x89PNG\r\n\x1A\n".b) + return 'gif' if binary.start_with?('GIF87a'.b) || binary.start_with?('GIF89a'.b) + return 'webp' if binary.bytesize >= 12 && + binary.start_with?('RIFF'.b) && + binary.byteslice(8, 4) == 'WEBP' + + nil + end + + def self.sanitised_svg_bytes(bytes) + parse_options = + Nokogiri::XML::ParseOptions::STRICT | + Nokogiri::XML::ParseOptions::NONET + document = Nokogiri::XML( + bytes, + nil, + nil, + parse_options) + root = document.root + raise MiniMagick::Error, 'SVG が不正です．' if root == nil || root.name != 'svg' + raise MiniMagick::Error, 'SVG が不正です．' if document.internal_subset != nil + raise MiniMagick::Error, 'SVG が不正です．' if svg_uses_disallowed_features?(document) + + width, height = svg_dimensions(root) + raise MiniMagick::Error, 'SVG が大きすぎます．' if width == nil || height == nil + if width > MAX_SVG_DIMENSION || height > MAX_SVG_DIMENSION || width * height > MAX_SVG_PIXELS + raise MiniMagick::Error, 'SVG が大きすぎます．' + end + + document.to_xml + rescue Nokogiri::XML::SyntaxError + raise MiniMagick::Error, 'SVG が不正です．' + end + + def self.svg_uses_disallowed_features?(document) + document.traverse.any? do |node| + next false unless node.element? + + name = node.name.to_s.downcase + next true if name == 'script' || name == 'foreignobject' + next style_contains_disallowed_urls?(node.text.to_s) if name == 'style' + + node.attribute_nodes.any? do |attribute| + attribute_name = attribute.name.to_s.downcase + attribute_value = attribute.value.to_s + attribute_name.start_with?('on') || + external_svg_reference?(attribute_name, attribute_value) + end + end + end + + def self.external_svg_reference?(attribute_name, attribute_value) + if ['href', 'xlink:href', 'src'].include?(attribute_name) + return external_svg_url?(attribute_value) + end + return style_contains_disallowed_urls?(attribute_value) if attribute_name == 'style' + return svg_url_function_disallowed?(attribute_value) if attribute_value.match?(/url\s*\(/i) + + false + end + + def self.external_svg_url?(value) + stripped = value.to_s.strip + return false if stripped.blank? || stripped.start_with?('#') + + true + end + + def self.style_contains_disallowed_urls?(value) + text = value.to_s + text.match?(/@import/i) || svg_url_function_disallowed?(text) + end + + def self.svg_url_function_disallowed?(value) + value.to_s.scan(/url\s*\(([^)]*)\)/i).flatten.any? do |entry| + reference = + entry.to_s.strip + .delete_prefix("'") + .delete_prefix('"') + .delete_suffix("'") + .delete_suffix('"') + reference.present? && !(reference.start_with?('#')) + end + end + + def self.svg_dimensions(root) + width = svg_length_to_pixels(root['width']) + height = svg_length_to_pixels(root['height']) + return [width, height] if width && height + + view_box = root['viewBox'].to_s.strip.split(/\s+/).map { Float(_1) rescue nil } + return [nil, nil] if view_box.length != 4 || view_box.any?(&:nil?) + return [nil, nil] unless view_box[2].finite? && view_box[2].positive? + return [nil, nil] unless view_box[3].finite? && view_box[3].positive? + + [view_box[2], view_box[3]] + end + + def self.svg_length_to_pixels(value) + return nil if value.blank? + + matched = /\A([0-9]+(?:\.[0-9]+)?)(px)?\z/i.match(value.to_s.strip) + return nil if matched == nil + + pixels = Float(matched[1]) + return nil unless pixels.finite? && pixels.positive? + + pixels + rescue ArgumentError + nil + end + + private_class_method :image_for_thumbnail_upload, + :svg_content_type?, + :decode_raster_thumbnail, + :decode_svg_thumbnail, + :raster_thumbnail_format, + :sanitised_svg_bytes, + :svg_uses_disallowed_features?, + :external_svg_reference?, + :external_svg_url?, + :style_contains_disallowed_urls?, + :svg_url_function_disallowed?, + :svg_dimensions, + :svg_length_to_pixels + + def parse_original_created_value field + raw_value = public_send("#{ field }_before_type_cast") + value = public_send(field) + return nil if raw_value.blank? && value.blank? + + time = + case raw_value + when String + parse_original_created_string(raw_value) + when Time, ActiveSupport::TimeWithZone + raw_value.in_time_zone + else + value&.in_time_zone + end + if time.nil? + errors.add field, ORIGINAL_CREATED_INVALID_MESSAGE + return nil + end + unless minute_precision_time?(time) + errors.add field, ORIGINAL_CREATED_MINUTE_PRECISION_MESSAGE + return nil + end + time + end + + def parse_original_created_string raw_value + value = raw_value.to_s.strip + return nil if value.blank? + + match = value.match(/\A(\d{4})-(\d{2})-(\d{2})T(\d{2}):(\d{2})\z/) + if match + year = match[1].to_i + month = match[2].to_i + day = match[3].to_i + hour = match[4].to_i + minute = match[5].to_i + return nil unless valid_original_created_components?(year, month, day, hour, minute, 0) + + return Time.zone.local(year, month, day, hour, minute) + end + + match = + value.match( + / + \A(\d{4})-(\d{2})-(\d{2})T(\d{2}):(\d{2}) + (?::(\d{2})(?:\.(\d+))?)? + (Z|[+-]\d{2}:?\d{2})? + \z + /x) + return nil if match.nil? + + year = match[1].to_i + month = match[2].to_i + day = match[3].to_i + hour = match[4].to_i + minute = match[5].to_i + second = match[6]&.to_i || 0 + fraction = match[7] + offset = match[8] + return nil unless valid_original_created_components?(year, month, day, hour, minute, second) + return nil if offset.present? && !(valid_original_created_offset?(offset)) + + if offset.present? + return Time.new( + year, + month, + day, + hour, + minute, + second + Rational(parse_original_created_nanoseconds(fraction), 1_000_000_000), + normalise_original_created_offset(offset)).in_time_zone + end + + Time.zone.local( + year, + month, + day, + hour, + minute, + second).change(nsec: parse_original_created_nanoseconds(fraction)) + rescue ArgumentError, TypeError + nil + end + + def minute_precision_time? value + value.sec.zero? && value.nsec.zero? + end + + def valid_original_created_components? year, month, day, hour, minute, second + return false unless Date.valid_date?(year, month, day) + return false unless hour.between?(0, 23) + return false unless minute.between?(0, 59) + return false unless second.between?(0, 59) + + true + end + + def valid_original_created_offset? value + match = value.match(/\A([+-])(\d{2}):?(\d{2})\z/) + return true if value == 'Z' + return false if match.nil? + + hours = match[2].to_i + minutes = match[3].to_i + hours.between?(0, 23) && minutes.between?(0, 59) + end + + def parse_original_created_nanoseconds value + return 0 if value.blank? + + digits = value[0, 9].ljust(9, '0') + Integer(digits, 10) + end + + def normalise_original_created_offset value + return '+00:00' if value == 'Z' + + value.match?(/\A[+-]\d{2}:\d{2}\z/) ? value : "#{ value[0, 3] }:#{ value[3, 2] }" + end + + def skip_original_created_validation? + return false if new_record? + return false if will_save_change_to_original_created_from? + return false if will_save_change_to_original_created_before? + + true end end diff --git a/backend/app/representations/post_compact_repr.rb b/backend/app/representations/post_compact_repr.rb new file mode 100644 index 0000000..94fcdc7 --- /dev/null +++ b/backend/app/representations/post_compact_repr.rb @@ -0,0 +1,19 @@ +# frozen_string_literal: true + + +module PostCompactRepr + module_function + + def base post, host: nil + return nil if post.nil? + + PostRepr + .common(post, host:) + .slice( + 'id', + 'title', + 'url', + 'thumbnail', + 'thumbnail_base') + end +end diff --git a/backend/app/representations/post_repr.rb b/backend/app/representations/post_repr.rb index c878291..0dc8a9b 100644 --- a/backend/app/representations/post_repr.rb +++ b/backend/app/representations/post_repr.rb @@ -17,8 +17,13 @@ module PostRepr module_function - def base post, current_user = nil - json = common(post) + def base post, current_user = nil, host: nil + json = + if host.present? + common(post, host:) + else + common(post) + end json['tags'] = tag_json(post) json['uploaded_user'] = post.uploaded_user && UserRepr.base(post.uploaded_user) json['viewed'] = current_user ? current_user.viewed?(post) : false @@ -26,31 +31,59 @@ module PostRepr end def detail post, current_user = nil, parent_posts: [], child_posts: [], - sibling_posts: { }, related: [] - base(post, current_user).merge( - 'parent_posts' => cards(parent_posts), - 'child_posts' => cards(child_posts), - 'sibling_posts' => sibling_posts.transform_keys(&:to_s).transform_values { |posts| - cards(posts) - }, - 'related' => cards(related)) + sibling_posts: { }, related: [], host: nil + if host.present? + base(post, current_user, host:).merge( + 'parent_posts' => cards(parent_posts, host:), + 'child_posts' => cards(child_posts, host:), + 'sibling_posts' => sibling_posts.transform_keys(&:to_s).transform_values { |posts| + cards(posts, host:) + }, + 'related' => cards(related, host:)) + else + base(post, current_user).merge( + 'parent_posts' => cards(parent_posts), + 'child_posts' => cards(child_posts), + 'sibling_posts' => sibling_posts.transform_keys(&:to_s).transform_values { |posts| + cards(posts) + }, + 'related' => cards(related)) + end end - def card post - common(post).merge('parent_posts' => [], 'child_posts' => []) + def card post, host: nil + if host.present? + common(post, host:).merge('parent_posts' => [], 'child_posts' => []) + else + common(post).merge('parent_posts' => [], 'child_posts' => []) + end end - def cards posts - posts.map { |post| card(post) } + def cards posts, host: nil + if host.present? + posts.map { |post| card(post, host:) } + else + posts.map { |post| card(post) } + end end - def many posts, current_user = nil - posts.map { |p| base(p, current_user) } + def many posts, current_user = nil, host: nil + if host.present? + posts.map { |p| base(p, current_user, host:) } + else + posts.map { |p| base(p, current_user) } + end end - def common post + def common post, host: nil BASE_FIELDS.to_h { |field| [field.to_s, post.public_send(field)] } - .merge('thumbnail' => thumbnail_url(post)) + .merge( + 'thumbnail' => + if host.present? + thumbnail_url(post, host:) + else + thumbnail_url(post) + end) end def tag_json post @@ -65,11 +98,22 @@ module PostRepr } end - def thumbnail_url post + def thumbnail_url post, host: nil return nil unless post.thumbnail.attached? - Rails.application.routes.url_helpers.rails_blob_url(post.thumbnail, only_path: false) - rescue + options = { only_path: false } + options[:host] = host if host.present? + + Rails.application.routes.url_helpers.rails_storage_proxy_url(post.thumbnail, **options) + rescue ActionController::UrlGenerationError, ArgumentError, URI::InvalidURIError => e + payload = { + post_id: post.id, + attachment_id: post.thumbnail.attachment&.id, + blob_id: post.thumbnail.blob&.id, + error_class: e.class, + message: e.message } + + Rails.logger.warn("PostRepr.thumbnail_url failed #{ payload.to_json }") nil end end diff --git a/backend/app/services/post_bulk_creator.rb b/backend/app/services/post_bulk_creator.rb new file mode 100644 index 0000000..1e75e24 --- /dev/null +++ b/backend/app/services/post_bulk_creator.rb @@ -0,0 +1,202 @@ +class PostBulkCreator + def initialize actor:, posts:, thumbnails:, host: nil + @actor_id = actor.id + @posts = posts + @thumbnails = thumbnails + @host = host + end + + def run + results = Array.new(@posts.length) + mutex = Mutex.new + next_index = 0 + + workers = Array.new(2) do + Thread.new do + Rails.application.executor.wrap do + ActiveRecord::Base.connection_pool.with_connection do + actor = User.find(@actor_id) + loop do + index = nil + begin + index = mutex.synchronize do + current = next_index + next_index += 1 + current + end + break if index >= @posts.length + + attributes = @posts[index] + results[index] = create_row(actor, attributes, index) + rescue StandardError => e + Rails.logger.error( + "post_bulk_creator_worker_failure #{ { error: e.class.name, + message: e.message, + index: }.to_json }") + results[index] = { + status: 'failed', + recoverable: false, + errors: { base: ['登録中にエラーが発生しました．'] }, + base_errors: [] } + end + end + end + end + end + end + workers.each(&:join) + + results.each_index do |index| + next if results[index].present? + + results[index] = { + status: 'failed', + recoverable: false, + errors: { base: ['登録中にエラーが発生しました．'] }, + base_errors: [] } + end + + { results: } + end + + private + + def create_row actor, attributes, index + preflight = + PostCreatePreflight.new( + attributes: attributes, + thumbnail: thumbnail_for(index, attributes), + host: @host).run + if preflight[:existing_post_id].present? + return { + status: 'skipped', + existing_post_id: preflight[:existing_post_id], + existing_post: preflight[:existing_post] } + end + + post = PostCreator.new( + actor: actor, + attributes: normalised_attributes(attributes, preflight, index)).create! + result = { + status: 'created', + post: { id: post.id } } + result[:field_warnings] = preflight[:field_warnings] if preflight[:field_warnings].present? + result[:base_warnings] = preflight[:base_warnings] if preflight[:base_warnings].present? + result + rescue PostCreatePreflight::ValidationFailed => e + { + status: 'failed', + recoverable: true, + errors: e.fields, + base_errors: e.base_errors } + rescue ActiveRecord::RecordInvalid => e + existing_post = existing_post_for_race(attributes, e.record) + if existing_post.present? + return { + status: 'skipped', + existing_post_id: existing_post[:id], + existing_post: existing_post } + end + + { + status: 'failed', + recoverable: true, + errors: e.record.errors.to_hash, + base_errors: e.record.errors[:base] } + rescue ActiveRecord::RecordNotUnique => e + if e.message.include?('index_posts_on_url') + existing_post = existing_post_for_race(attributes) + return { + status: 'skipped', + existing_post_id: existing_post[:id], + existing_post: existing_post } if existing_post.present? + end + + Rails.logger.error( + "post_bulk_creator_record_not_unique #{ { error: e.class.name, + message: e.message }.to_json }") + { + status: 'failed', + recoverable: false, + errors: { base: ['登録中にエラーが発生しました．'] }, + base_errors: [] } + rescue Tag::NicoTagNormalisationError + { + status: 'failed', + recoverable: true, + errors: { tags: ['ニコニコ・タグは直接指定できません．'] }, + base_errors: [] } + rescue Tag::DeprecatedTagNormalisationError + { + status: 'failed', + recoverable: true, + errors: { tags: ['廃止済みタグは付与できません．'] }, + base_errors: [] } + rescue PostCreator::VideoMsParseError + { + status: 'failed', + recoverable: true, + errors: { video_ms: ['動画時間の記法が不正です．'] }, + base_errors: [] } + rescue Post::RemoteThumbnailFetchFailed + { + status: 'failed', + recoverable: true, + errors: { thumbnail_base: ['サムネイル画像の取得に失敗しました．'] }, + base_errors: [] } + rescue ArgumentError => e + { + status: 'failed', + recoverable: true, + errors: { base: [e.message] }, + base_errors: [] } + rescue StandardError => e + Rails.logger.error( + "post_bulk_creator_failure #{ { error: e.class.name, + message: e.message }.to_json }") + { + status: 'failed', + recoverable: false, + errors: { base: ['登録中にエラーが発生しました．'] }, + base_errors: [] } + end + + def normalised_attributes attributes, preflight, index + { + url: preflight[:url], + title: preflight[:title], + thumbnail_base: preflight[:thumbnail_base], + thumbnail: thumbnail_for(index, attributes), + tags: preflight[:tags], + parent_post_ids: preflight[:parent_post_ids], + original_created_from: preflight[:original_created_from], + original_created_before: preflight[:original_created_before], + duration: preflight[:duration], + video_ms: preflight[:video_ms], + direct_tag_specs: preflight[:direct_tag_specs], + default_tag_specs: preflight[:default_tag_specs], + snapshot_tag_specs: preflight[:snapshot_tag_specs], + post_tag_specs: preflight[:post_tag_specs], + tag_sections: preflight[:tag_sections], + normalised_parent_post_ids: preflight[:normalised_parent_post_ids] } + end + + def thumbnail_for index, attributes + return nil if attributes['thumbnail_base'].present? || attributes[:thumbnail_base].present? + + @thumbnails[index] + end + + def existing_post_for_race attributes, record = nil + return nil if record.present? && !(record.errors.of_kind?(:url, :taken)) + + normal_url = PostUrlNormaliser.normalise(attributes['url'] || attributes[:url]) + return nil if normal_url.blank? + + compact_existing_post(Post.with_attached_thumbnail.find_by(url: normal_url)) + end + + def compact_existing_post post + PostCompactRepr.base(post, host: @host) + end +end diff --git a/backend/app/services/post_create_plan.rb b/backend/app/services/post_create_plan.rb new file mode 100644 index 0000000..a6f38b7 --- /dev/null +++ b/backend/app/services/post_create_plan.rb @@ -0,0 +1,271 @@ +class PostCreatePlan + VIDEO_TAG_NAME = '動画'.freeze + TAGME_TAG_NAME = 'タグ希望'.freeze + NO_DEERJIKIST_TAG_NAME = 'ニジラー情報不詳'.freeze + + def initialize attributes: + @attributes = attributes.symbolize_keys + @existing_tags_by_name = nil + end + + def build! + direct_tag_specs, tag_sections = parse_direct_tag_specs + default_tag_specs = build_default_tag_specs(direct_tag_specs) + snapshot_tag_specs = merge_tag_specs(direct_tag_specs + default_tag_specs) + preload_existing_tags_by_name!(snapshot_tag_specs.map { _1[:name] }) + validate_new_tag_specs!(snapshot_tag_specs) + post_tag_specs = expand_parent_tag_specs(snapshot_tag_specs) + video_ms = normalise_video_ms(snapshot_tag_specs) + validate_video_sections!(video_ms, tag_sections) + parent_post_ids = normalise_parent_post_ids + validate_parent_post_ids!(parent_post_ids) + + { + url: @attributes[:url], + title: @attributes[:title].to_s, + thumbnail_base: @attributes[:thumbnail_base].presence, + original_created_from: @attributes[:original_created_from].presence, + original_created_before: @attributes[:original_created_before].presence, + tags: serialised_tags(direct_tag_specs, tag_sections), + display_tags: display_tags(direct_tag_specs, tag_sections), + duration: @attributes[:duration].to_s, + video_ms: video_ms, + parent_post_ids: parent_post_ids.join(' '), + direct_tag_specs: direct_tag_specs, + default_tag_specs: default_tag_specs, + snapshot_tag_specs: snapshot_tag_specs, + post_tag_specs: post_tag_specs, + tag_sections: tag_sections, + normalised_parent_post_ids: parent_post_ids } + end + + private + + def tag_names = @attributes[:tags].to_s.split + + def parse_direct_tag_specs + tag_sections = { } + direct_tag_specs = [] + + tag_names.each do |raw_name| + tag_name, category, sections = parse_raw_tag_name(raw_name) + existing_tag = existing_tags_by_name[tag_name] + raise Tag::NicoTagNormalisationError if existing_tag&.nico? + raise Tag::DeprecatedTagNormalisationError, [existing_tag.name] if existing_tag&.deprecated? + + direct_tag_specs << { + name: tag_name, + category: (category || existing_tag&.category || 'general').to_sym } + if sections.present? + tag_sections[tag_name] ||= [] + tag_sections[tag_name].concat(sections) + tag_sections[tag_name] = Tag.merge_section_ranges(tag_sections[tag_name]) + tag_sections.delete(tag_name) if tag_sections[tag_name] == [[0, nil]] + end + end + + [merge_tag_specs(direct_tag_specs), tag_sections] + end + + def parse_raw_tag_name raw_name + name = raw_name.to_s + prefix, category = + Tag::CATEGORY_PREFIXES.find { + name.downcase.start_with?(_1[0]) + } || ['', nil] + name = name.sub(/\A#{ prefix }/i, '') + + sections = [] + while (match = name.match(/\A(\S*?)\[([^\[\]\s]*)-([^\[\]\s]*)\](\S*)\z/)) + name = "#{ match[1] }#{ match[4] }" + next if match[2].empty? && match[3].empty? + + sections << Tag.normalise_section_range!( + begin_raw: match[2], + end_raw: match[3], + tag_name: name) + end + if name.include?('[') || name.include?(']') + raise Tag::SectionLiteralParseError.new(raw_name, raw_name) + end + + [resolved_tag_name(name), category&.to_sym, sections] + end + + def build_default_tag_specs direct_tag_specs + default_tag_specs = [] + if direct_tag_specs.length < 10 && direct_tag_specs.none? { _1[:name] == TAGME_TAG_NAME } + default_tag_specs << { + name: TAGME_TAG_NAME, + category: :meta } + end + if direct_tag_specs.none? { deerjikist_tag_spec?(_1) } + default_tag_specs << { + name: NO_DEERJIKIST_TAG_NAME, + category: :meta } + end + + default_tag_specs + end + + def validate_new_tag_specs! specs + Array(specs).each do |spec| + next if existing_tags_by_name.key?(spec[:name]) + + validate_new_tag_spec!(spec) + end + end + + def validate_new_tag_spec! spec + tag_name = TagName.new(name: spec[:name]) + tag = Tag.new(category: spec[:category], tag_name:) + return if tag_name.valid? && tag.valid? + + post = Post.new + tag_name.errors[:name].each do |message| + post.errors.add :tags, "#{ spec[:name] }: #{ message }" + end + tag.errors.each do |error| + next if error.attribute == :tag_name + + post.errors.add :tags, "#{ spec[:name] }: #{ error.message }" + end + raise ActiveRecord::RecordInvalid, post + end + + def expand_parent_tag_specs snapshot_tag_specs + existing_snapshot_tags = snapshot_tag_specs.filter_map { existing_tags_by_name[_1[:name]] } + expanded_parent_specs = + Tag.expand_parent_tags(existing_snapshot_tags) + .reject(&:deprecated?) + .map { |tag| + { + name: tag.name, + category: tag.category.to_sym } } + merge_tag_specs(snapshot_tag_specs + expanded_parent_specs) + end + + def merge_tag_specs specs + specs.each_with_object({ }) do |spec, merged| + merged[spec[:name]] = + if merged.key?(spec[:name]) && merged[spec[:name]][:category] != :general + merged[spec[:name]] + else + { + name: spec[:name], + category: spec[:category] } + end + end.values.sort_by { _1[:name] } + end + + def existing_tags_by_name + @existing_tags_by_name ||= begin + names = tag_names.map { canonical_tag_name_without_sections(_1) }.uniq + Tag.joins(:tag_name).where(tag_names: { name: names }).index_by(&:name) + end + end + + def preload_existing_tags_by_name! names + wanted_names = Array(names).map { _1.to_s }.reject(&:blank?).uniq + missing_names = wanted_names - existing_tags_by_name.keys + return if missing_names.empty? + + existing_tags_by_name.merge!( + Tag.joins(:tag_name) + .where(tag_names: { name: missing_names }) + .index_by(&:name)) + end + + def canonical_tag_name_without_sections raw_name + name, = parse_raw_tag_name(raw_name) + name + end + + def deerjikist_tag_spec? spec + return true if spec[:category] == :deerjikist + + existing_tags_by_name[spec[:name]]&.deerjikist? + end + + def normalise_parent_post_ids + Array(@attributes[:parent_post_ids]).flat_map { _1.to_s.split }.map { |token| + id = Integer(token, exception: false) + raise ArgumentError, "親投稿 Id. が不正です: #{ token }" if id.nil? || id <= 0 + + id + }.uniq.sort + end + + def validate_parent_post_ids! ids + missing = ids - Post.where(id: ids).pluck(:id) + raise ArgumentError, "存在しない親投稿 Id. があります: #{ missing.join(' ') }" if missing.present? + end + + def serialised_tags direct_tag_specs, tag_sections + direct_tag_specs.map { |spec| + "#{ spec[:name] }#{ tag_sections[spec[:name]].to_a.map { section_literal(_1) }.join }" + }.sort.join(' ') + end + + def display_tags direct_tag_specs, tag_sections + direct_tag_specs.map { |spec| + { + name: spec[:name], + category: spec[:category].to_s, + section_literals: tag_sections[spec[:name]].to_a.map { section_literal(_1) } } + }.sort_by { _1[:name] } + end + + def section_literal range + begin_ms, end_ms = range + "[#{ Post.ms_to_time(begin_ms) }-#{ end_ms ? Post.ms_to_time(end_ms) : '' }]" + end + + def normalise_video_ms snapshot_tag_specs + return nil unless snapshot_tag_specs.any? { _1[:name] == VIDEO_TAG_NAME } + + video_ms = @attributes[:video_ms] + if video_ms.present? + value = Integer(video_ms, exception: false) + raise PostCreator::VideoMsParseError unless value&.positive? + + return value + end + + duration = @attributes[:duration] + return nil if duration.blank? + + value = Tag.time_to_ms!(duration.to_s, tag_name: '動画時間') + raise PostCreator::VideoMsParseError unless value.positive? + + value + rescue Tag::SectionLiteralParseError + raise PostCreator::VideoMsParseError + end + + def validate_video_sections! video_ms, tag_sections + return unless video_ms + + tag_sections.each_value do |ranges| + ranges.each do |begin_ms, end_ms| + if begin_ms >= video_ms + post = Post.new + post.errors.add :video_ms, 'タグ区間の開始が動画時間以上です．' + raise ActiveRecord::RecordInvalid, post + end + if end_ms && end_ms > video_ms + post = Post.new + post.errors.add :video_ms, 'タグ区間の終端が動画時間を超えてゐます．' + raise ActiveRecord::RecordInvalid, post + end + end + end + end + + def resolved_tag_name name + tag_name = TagName.includes(:canonical).find_by(name:) + return name if tag_name.nil? + + (tag_name.canonical || tag_name).name + end +end diff --git a/backend/app/services/post_create_preflight.rb b/backend/app/services/post_create_preflight.rb new file mode 100644 index 0000000..04d3492 --- /dev/null +++ b/backend/app/services/post_create_preflight.rb @@ -0,0 +1,141 @@ +class PostCreatePreflight + class ValidationFailed < StandardError + attr_reader :fields, :base_errors + + def initialize fields: { }, base_errors: [] + super('入力内容を確認してください．') + @fields = fields + @base_errors = base_errors + end + end + + def initialize attributes:, thumbnail: nil, host: nil + @attributes = attributes.symbolize_keys + @thumbnail = thumbnail + @host = host + end + + def run + preview = PostImportPreviewer.new.preview_rows( + rows: [preview_row], + fetch_metadata: false).first + if preview[:existing_post_id].present? + return { + url: preview[:url], + title: preview[:attributes]['title'], + thumbnail_base: preview[:attributes]['thumbnail_base'], + tags: preview[:attributes]['tags'], + parent_post_ids: preview[:attributes]['parent_post_ids'], + original_created_from: preview[:attributes]['original_created_from'], + original_created_before: preview[:attributes]['original_created_before'], + duration: preview[:attributes]['duration'], + video_ms: preview[:attributes]['video_ms'], + display_tags: preview[:display_tags] || [], + field_warnings: final_field_warnings(preview[:field_warnings] || { }), + base_warnings: preview[:base_warnings], + existing_post_id: preview[:existing_post_id], + existing_post: existing_post_compact(preview[:existing_post_id]) } + end + + if preview[:validation_errors].present? + raise ValidationFailed.new(fields: preview[:validation_errors]) + end + + validate_thumbnail_upload! + + plan = PostCreatePlan.new( + attributes: { + url: preview[:url], + title: preview[:attributes]['title'], + thumbnail_base: preview[:attributes]['thumbnail_base'], + tags: preview[:attributes]['tags'], + parent_post_ids: preview[:attributes]['parent_post_ids'], + original_created_from: preview[:attributes]['original_created_from'], + original_created_before: preview[:attributes]['original_created_before'], + duration: preview[:attributes]['duration'], + video_ms: preview[:attributes]['video_ms'] }).build! + + { + url: plan[:url], + title: plan[:title], + thumbnail_base: plan[:thumbnail_base], + tags: plan[:tags], + parent_post_ids: plan[:parent_post_ids], + original_created_from: plan[:original_created_from], + original_created_before: plan[:original_created_before], + duration: plan[:duration], + video_ms: plan[:video_ms], + display_tags: plan[:display_tags], + direct_tag_specs: plan[:direct_tag_specs], + default_tag_specs: plan[:default_tag_specs], + snapshot_tag_specs: plan[:snapshot_tag_specs], + post_tag_specs: plan[:post_tag_specs], + tag_sections: plan[:tag_sections], + normalised_parent_post_ids: plan[:normalised_parent_post_ids], + field_warnings: final_field_warnings(preview[:field_warnings] || { }), + base_warnings: preview[:base_warnings], + existing_post_id: preview[:existing_post_id], + existing_post: existing_post_compact(preview[:existing_post_id]) } + end + + private + + def preview_row + { + source_row: 1, + url: @attributes[:url].to_s, + attributes: { + 'title' => @attributes[:title].to_s, + 'thumbnail_base' => @attributes[:thumbnail_base].to_s, + 'original_created_from' => @attributes[:original_created_from].to_s, + 'original_created_before' => @attributes[:original_created_before].to_s, + 'duration' => @attributes[:duration].to_s, + 'video_ms' => @attributes[:video_ms], + 'tags' => @attributes[:tags].to_s, + 'parent_post_ids' => parent_post_ids_text }, + provenance: { + 'url' => 'manual', + 'title' => 'manual', + 'thumbnail_base' => 'manual', + 'original_created_from' => 'manual', + 'original_created_before' => 'manual', + 'duration' => 'manual', + 'video_ms' => 'manual', + 'tags' => 'manual', + 'parent_post_ids' => 'manual' }, + tag_sources: { + 'automatic' => '', + 'manual' => @attributes[:tags].to_s } } + end + + def parent_post_ids_text + Array(@attributes[:parent_post_ids]).flat_map { _1.to_s.split }.join(' ') + end + + def validate_thumbnail_upload! + return if @attributes[:thumbnail_base].present? + return if @thumbnail.blank? + + PostThumbnailUploadValidator.validate!(@thumbnail) + rescue PostThumbnailUploadValidator::InvalidUpload => e + raise ValidationFailed.new(fields: { thumbnail: [e.message] }) + end + + def existing_post_compact post_id + return nil if post_id.blank? + + post = Post.with_attached_thumbnail.find_by(id: post_id) + PostCompactRepr.base(post, host: @host) + end + + def final_field_warnings field_warnings + thumbnail_warnings = (field_warnings['thumbnail_base'] || []).reject { _1 == 'サムネールなし' } + if @attributes[:thumbnail_base].blank? && @thumbnail.blank? + thumbnail_warnings = (thumbnail_warnings + ['サムネールなし']).uniq + end + + next_warnings = field_warnings.except('thumbnail_base') + next_warnings['thumbnail_base'] = thumbnail_warnings if thumbnail_warnings.present? + next_warnings + end +end diff --git a/backend/app/services/post_creator.rb b/backend/app/services/post_creator.rb new file mode 100644 index 0000000..8078991 --- /dev/null +++ b/backend/app/services/post_creator.rb @@ -0,0 +1,154 @@ +class PostCreator + class VideoMsParseError < ArgumentError; end + + attr_reader :field_warnings + + def initialize actor:, attributes: + @actor = actor + @attributes = attributes.symbolize_keys + @field_warnings = { } + end + + def create! + thumbnail_attachment = prepare_thumbnail_attachment + post = Post.new(title: @attributes[:title].presence, + url: @attributes[:url], + thumbnail_base: @attributes[:thumbnail_base].presence, + uploaded_user: @actor, + original_created_from: @attributes[:original_created_from].presence, + original_created_before: @attributes[:original_created_before].presence) + + ApplicationRecord.transaction do + post.save! + post.thumbnail.attach(thumbnail_attachment) if thumbnail_attachment.present? + snapshot_tags = planned_snapshot_tags + post_tags = planned_post_tags + sections = planned_sections + TagVersioning.record_tag_snapshots!(snapshot_tags, created_by_user: @actor) + post.video_ms = planned_video_ms + post.save! + sync_post_tags!(post, post_tags, sections) + sync_parent_posts!(post, planned_parent_post_ids) + PostVersionRecorder.record!(post:, event_type: :create, created_by_user: @actor) + end + post + rescue StandardError + post&.thumbnail&.purge if post&.thumbnail&.attached? + raise + end + + private + + def prepare_thumbnail_attachment + PostThumbnailAttachmentBuilder.build( + thumbnail: @attributes[:thumbnail], + thumbnail_base: @attributes[:thumbnail_base].presence) + end + + def planned_snapshot_tags = planned_create_attributes[:snapshot_tags] + + def planned_post_tags = planned_create_attributes[:post_tags] + + def planned_sections = planned_create_attributes[:tag_sections] + + def planned_parent_post_ids = planned_create_attributes[:normalised_parent_post_ids] + + def planned_video_ms + planned_create_attributes[:video_ms] + end + + def planned_create_attributes + @planned_create_attributes ||= begin + if @attributes.key?(:snapshot_tag_specs) + snapshot_tags = materialise_tags(@attributes[:snapshot_tag_specs] || []) + post_tags = materialise_tags(@attributes[:post_tag_specs] || []) + { + snapshot_tags: snapshot_tags, + post_tags: post_tags, + tag_sections: materialise_sections( + @attributes[:tag_sections] || { }, + snapshot_tags, + post_tags), + normalised_parent_post_ids: @attributes[:normalised_parent_post_ids] || [], + video_ms: @attributes[:video_ms] } + else + build_materialised_plan + end + end + end + + def build_materialised_plan + plan = PostCreatePlan.new(attributes: @attributes).build! + snapshot_tags = materialise_tags(plan[:snapshot_tag_specs] || []) + post_tags = materialise_tags(plan[:post_tag_specs] || []) + { + snapshot_tags: snapshot_tags, + post_tags: post_tags, + tag_sections: materialise_sections( + plan[:tag_sections] || { }, + snapshot_tags, + post_tags), + normalised_parent_post_ids: plan[:normalised_parent_post_ids] || [], + video_ms: plan[:video_ms] } + end + + def materialise_tags specs + Array(specs).each_with_object({ }) do |spec, tags| + name = spec[:name] || spec['name'] + category = spec[:category] || spec['category'] + next if name.blank? || category.blank? + + tag = Tag.find_or_create_by_tag_name!(name, category:) + tag.update!(category:) if tag.category.to_sym != category.to_sym + tags[name] ||= tag + end.values + end + + def materialise_sections sections_by_name, snapshot_tags, post_tags + tags_by_name = post_tags.index_by(&:name) + snapshot_tags.each do |tag| + tags_by_name[tag.name] ||= tag + end + + sections_by_name.each_with_object({ }) do |(tag_name, ranges), sections| + tag = tags_by_name[tag_name.to_s] + next if tag.nil? + + sections[tag.id] = Array(ranges).map { |range| [range[0], range[1]] } + end + end + + def sync_post_tags! post, desired_tags, sections + desired_ids = desired_tags.map(&:id).to_set + current_ids = post.tags.pluck(:id).to_set + Tag.where(id: desired_ids - current_ids).find_each do |tag| + PostTag.create_or_find_by!(post:, tag:, created_user: @actor) + end + PostTagSection.where(post_id: post.id).destroy_all + sections.each do |tag_id, ranges| + ranges.each do |begin_ms, end_ms| + PostTagSection.create!(post_id: post.id, + tag_id:, + begin_ms:, + end_ms:) + end + end + PostTag.where(post_id: post.id, + tag_id: (current_ids - desired_ids).to_a).kept.find_each do |post_tag| + post_tag.discard_by!(@actor) + end + end + + def sync_parent_posts! post, ids + if ids.include?(post.id) + post.errors.add :parent_post_ids, '自分自身を親投稿にはできません．' + raise ActiveRecord::RecordInvalid, post + end + missing = ids - Post.where(id: ids).pluck(:id) + if missing.present? + post.errors.add :parent_post_ids, "存在しない親投稿 Id. があります: #{ missing.join(' ') }" + raise ActiveRecord::RecordInvalid, post + end + ids.each { |parent_post_id| PostImplication.create_or_find_by!(post:, parent_post_id:) } + end +end diff --git a/backend/app/services/post_import_previewer.rb b/backend/app/services/post_import_previewer.rb new file mode 100644 index 0000000..0a812f5 --- /dev/null +++ b/backend/app/services/post_import_previewer.rb @@ -0,0 +1,522 @@ +require 'time' +require 'timeout' + +class PostImportPreviewer + FIELDS = [ + 'title', + 'thumbnail_base', + 'original_created_from', + 'original_created_before', + 'video_ms', + 'duration', + 'tags', + 'parent_post_ids'].freeze + FETCH_WARNING_FIELDS = ['url', 'title', 'thumbnail_base'].freeze + TITLE_FETCH_WARNING = 'タイトルを取得できませんでした．'.freeze + THUMBNAIL_FETCH_WARNING = 'サムネールを取得できませんでした．'.freeze + METADATA_FETCH_WARNING = '自動取得に失敗しました．'.freeze + + def preview_rows rows:, fetch_metadata: true, metadata_cache: { } + prepared_rows = rows.map { prepare_row(_1) } + url_counts = prepared_rows.filter_map { _1[:normal_url] }.tally + existing_posts = + Post.where(url: prepared_rows.map { _1[:normal_url] }.compact.uniq).index_by(&:url) + existing_parent_ids = preload_parent_ids(prepared_rows) + preload_metadata!( + prepared_rows, + fetch_metadata, + metadata_cache, + existing_posts, + url_counts) + known_tags = + preload_known_tags( + prepared_rows, + fetch_metadata, + metadata_cache, + existing_posts, + url_counts) + prepared_rows.map { |row| + preview_row(row, + fetch_metadata:, + metadata_cache:, + existing_posts:, + url_counts:, + known_tags:, + existing_parent_ids:) + } + end + + def normalised_url value + PostUrlNormaliser.normalise(value) + end + + private + + def prepare_row row + source = row.symbolize_keys + url = source[:url].to_s.strip + normal_url = normalised_url(url) + source.merge(url_text: url, normal_url:, url_error: validate_url_safety(normal_url)) + end + + def preview_row row, + fetch_metadata:, + metadata_cache:, + existing_posts:, + url_counts:, + known_tags:, + existing_parent_ids: + attributes = initial_attributes(row) + provenance = initial_provenance(row) + tag_sources = initial_tag_sources(row, attributes, provenance) + field_warnings = initial_field_warnings(row) + base_warnings = initial_base_warnings(row) + url = row[:url_text] + provenance['url'] = 'manual' + normal_url = row[:normal_url] + url_for_metadata = normal_url || url + existing_post = normal_url.present? ? existing_posts[normal_url] : nil + + validation_errors = {} + validation_errors[:url] = ['URL が不正です．'] if normal_url.blank? + if row[:url_error].present? + validation_errors[:url] = [row[:url_error]] + end + if normal_url.present? && url_counts[normal_url].to_i > 1 + validation_errors[:url] = ['URL が重複しています．'] + end + + if row[:metadata_url].present? && row[:metadata_url] != url_for_metadata + clear_automatic_values!(attributes, provenance, tag_sources) + field_warnings = { } + base_warnings = [ ] + end + + if validation_errors.blank? && normal_url.present? && existing_post + attributes['tags'] = merged_tags(tag_sources, provenance['tags']) + warnings_present = field_warnings.values.any?(&:present?) || base_warnings.present? + return { source_row: row[:source_row], + url: normal_url, + attributes:, + provenance:, + tag_sources:, + metadata_url: url_for_metadata, + skip_reason: 'existing', + existing_post_id: existing_post.id, + field_warnings:, + base_warnings:, + validation_errors:, + status: warnings_present ? 'warning' : 'ready' } + end + + should_fetch = validation_errors.blank? + should_fetch &&= should_fetch_metadata?(fetch_metadata, row[:source_row].to_i) + if should_fetch + clear_fetch_warnings!(field_warnings) + metadata = metadata_for(url_for_metadata, metadata_cache) + if metadata[:validation_errors].present? + validation_errors.merge!(metadata[:validation_errors]) + else + apply_metadata!(attributes, provenance, tag_sources, metadata[:data]) + apply_fetch_warnings!(field_warnings, metadata[:warnings]) + end + end + + attributes['url'] = url + validate_basic_data(attributes, validation_errors) + validate_preview_tags(merged_tags(tag_sources, provenance['tags']), + validation_errors, + known_tags) + validate_parents(attributes['parent_post_ids'], validation_errors, existing_parent_ids) + attributes.delete('url') + attributes['tags'] = merged_tags(tag_sources, provenance['tags']) + + warnings_present = field_warnings.values.any?(&:present?) || base_warnings.present? + { source_row: row[:source_row], + url: validation_errors[:url].present? ? url : (normal_url || url), + attributes:, + provenance:, + tag_sources:, + metadata_url: url_for_metadata, + skip_reason: nil, + existing_post_id: nil, + field_warnings:, + base_warnings:, + validation_errors:, + status: validation_errors.present? ? 'error' : (warnings_present ? 'warning' : 'ready') } + end + + def should_fetch_metadata? fetch_metadata, source_row + case fetch_metadata + when true then true + when Integer then fetch_metadata == source_row + when false, nil then false + else raise ArgumentError, '取得対象が不正です．' + end + end + + def initial_attributes row + attributes = row[:attributes]&.stringify_keys || { } + FIELDS.to_h { |field| + value = attributes[field] + normalised = + if field == 'duration' + normalise_duration_attribute(value) + elsif field == 'video_ms' + value.nil? ? '' : value.to_s + else + value.to_s + end + [field, normalised] } + end + + def initial_provenance row + provenance = row[:provenance]&.stringify_keys || { } + FIELDS.to_h { |field| [field, provenance[field].presence || 'automatic'] } + .merge('url' => 'manual') + end + + def initial_tag_sources row, attributes, provenance + sources = row[:tag_sources]&.stringify_keys || { 'automatic' => '', 'manual' => '' } + sources['automatic'] = sources['automatic'].to_s + sources['manual'] = + provenance['tags'] == 'manual' ? attributes['tags'].to_s : sources['manual'].to_s + sources + end + + def initial_field_warnings row + (row[:field_warnings] || { }) + .stringify_keys + .transform_values { |value| Array(value).map(&:to_s) } + end + + def initial_base_warnings row + Array(row[:base_warnings]).map(&:to_s) + end + + def clear_automatic_values! attributes, provenance, tag_sources + ['title', 'thumbnail_base', 'original_created_from', + 'original_created_before', 'video_ms', 'duration'].each do |field| + attributes[field] = '' if provenance[field] == 'automatic' + end + tag_sources['automatic'] = '' + attributes['tags'] = merged_tags(tag_sources, provenance['tags']) + end + + def clear_fetch_warnings! field_warnings + FETCH_WARNING_FIELDS.each do |field| + field_warnings.delete(field) + end + end + + def metadata_for url, cache + cache[url] ||= fetch_metadata(url) + end + + def fetch_metadata url + return { data: { }, warnings: { 'url' => ['URL が空です．'] }, + validation_errors: { } } if url.blank? + + data = sanitise_metadata(PostMetadataFetcher.fetch(url).stringify_keys.compact) + warnings = { } + add_field_warning!(warnings, 'title', TITLE_FETCH_WARNING) if data['title'].blank? + if data['thumbnail_base'].blank? + add_field_warning!(warnings, 'thumbnail_base', THUMBNAIL_FETCH_WARNING) + end + { data:, warnings:, validation_errors: { } } + rescue Preview::UrlSafety::UnsafeUrl => e + payload = { error: e.class.name, message: e.message } + Rails.logger.info( + "post_import_metadata_fetch_unsafe_url #{ payload.to_json }") + { data: { }, warnings: { }, validation_errors: { url: [e.message] } } + rescue Preview::HttpFetcher::FetchFailed, + Preview::HttpFetcher::ResponseTooLarge => e + payload = { error: e.class.name, message: e.message } + Rails.logger.info( + "post_import_metadata_fetch_failure #{ payload.to_json }") + { data: { }, warnings: { 'url' => [METADATA_FETCH_WARNING] }, validation_errors: { } } + end + + def sanitise_metadata metadata + { + 'title' => sanitise_metadata_title(metadata['title']), + 'thumbnail_base' => sanitise_metadata_url(metadata['thumbnail_base']), + 'original_created_from' => sanitise_metadata_time(metadata['original_created_from']), + 'original_created_before' => sanitise_metadata_time(metadata['original_created_before']), + 'duration' => sanitise_metadata_duration(metadata['duration']), + 'tags' => metadata['tags'].to_s.presence }.compact + end + + def sanitise_metadata_title value + value.is_a?(String) ? value.presence : nil + end + + def sanitise_metadata_url value + return nil unless value.is_a?(String) + + stripped = value.strip + return nil if stripped.blank? + + uri = URI.parse(stripped) + return nil unless uri.is_a?(URI::HTTP) && uri.host.present? + + stripped + rescue URI::InvalidURIError + nil + end + + def sanitise_metadata_time value + return nil unless value.is_a?(String) + + Time.iso8601(value).in_time_zone.change(sec: 0, nsec: 0).iso8601 + rescue ArgumentError, TypeError + nil + end + + def sanitise_metadata_duration value + return nil unless value.is_a?(String) + + value.presence + end + + def normalise_duration_attribute value + return '' if value.nil? + return value if value.is_a?(String) + + milliseconds = Integer(value, exception: false) + return value.to_s if milliseconds.nil? || milliseconds <= 0 + + seconds_string = (milliseconds / 1_000.0).to_s + seconds_string.end_with?('.0') ? seconds_string.delete_suffix('.0') : seconds_string + end + + def preload_metadata! prepared_rows, fetch_metadata, metadata_cache, existing_posts, url_counts + urls = prepared_rows.filter_map { |row| + next unless row[:normal_url].present? + next if row[:url_error].present? + next unless should_fetch_metadata?(fetch_metadata, row[:source_row].to_i) + next if url_counts[row[:normal_url]].to_i > 1 + next if existing_posts.key?(row[:normal_url]) + + row[:normal_url] + }.uniq + urls = urls.reject { metadata_cache.key?(_1) } + return if urls.empty? + + url_queue = Queue.new + result_queue = Queue.new + urls.each { url_queue << _1 } + workers = [urls.length, 4].min.times.map { + Thread.new { + Rails.application.executor.wrap do + loop do + url = url_queue.pop(true) + result_queue << [url, safe_fetch_metadata(url)] + rescue ThreadError + break + end + end + } + } + Timeout.timeout(15) { workers.each(&:join) } + rescue Timeout::Error + workers&.each(&:kill) + ensure + workers&.each(&:join) + while result_queue&.size.to_i.positive? + url, result = result_queue.pop + metadata_cache[url] = result + end + urls&.each do |url| + metadata_cache[url] ||= { data: { }, warnings: { 'url' => [METADATA_FETCH_WARNING] } } + end + end + + def safe_fetch_metadata url + fetch_metadata(url) + rescue Preview::UrlSafety::UnsafeUrl => e + payload = { error: e.class.name, message: e.message } + Rails.logger.info( + "post_import_metadata_fetch_unsafe_url #{ payload.to_json }") + { data: { }, warnings: { }, validation_errors: { url: [e.message] } } + rescue StandardError => e + payload = { error: e.class.name, message: e.message } + Rails.logger.error( + "post_import_metadata_fetch_unexpected_failure #{ payload.to_json }") + { data: { }, warnings: { 'url' => [METADATA_FETCH_WARNING] }, validation_errors: { } } + end + + def preload_known_tags prepared_rows, fetch_metadata, metadata_cache, existing_posts, url_counts + names = prepared_rows.flat_map { |row| + attributes = initial_attributes(row) + provenance = initial_provenance(row) + tag_sources = initial_tag_sources(row, attributes, provenance) + if metadata_url_changed?(row) + clear_automatic_values!(attributes, provenance, tag_sources) + end + if should_apply_metadata_to_row?(row, fetch_metadata, existing_posts, url_counts) + metadata = metadata_for(row[:normal_url], metadata_cache) + apply_metadata!(attributes, provenance, tag_sources, metadata[:data]) + end + preview_tag_names(merged_tags(tag_sources, provenance['tags'])) + }.compact.uniq + return { } if names.empty? + + Tag.joins(:tag_name) + .where(tag_names: { name: names }) + .includes(:tag_name) + .to_a + .index_by(&:name) + end + + def preload_parent_ids prepared_rows + ids = prepared_rows.flat_map { |row| + attributes = initial_attributes(row) + preview_parent_ids(attributes['parent_post_ids']) + }.uniq + return { } if ids.empty? + + Post.where(id: ids).pluck(:id).to_h { [_1, true] } + end + + def metadata_url_changed? row + row[:metadata_url].present? && row[:metadata_url] != (row[:normal_url] || row[:url_text]) + end + + def should_apply_metadata_to_row? row, fetch_metadata, existing_posts, url_counts + normal_url = row[:normal_url] + return false if normal_url.blank? + return false if row[:url_error].present? + return false if url_counts[normal_url].to_i > 1 + return false if existing_posts.key?(normal_url) + + should_fetch_metadata?(fetch_metadata, row[:source_row].to_i) + end + + def validate_url_safety normal_url + return nil if normal_url.blank? + + Preview::UrlSafety.validate(normal_url) + nil + rescue Preview::UrlSafety::UnsafeUrl => e + e.message + end + + def apply_metadata! attributes, provenance, tag_sources, metadata + metadata.each do |field, value| + if field == 'tags' + next if provenance['tags'] == 'manual' + + tag_sources['automatic'] = value.to_s + attributes['tags'] = merged_tags(tag_sources) + next + end + next unless provenance[field] == 'automatic' + + attributes[field] = value + provenance[field] = 'automatic' + end + end + + def apply_fetch_warnings! field_warnings, warnings + warnings.each do |field, values| + values.each do |value| + add_field_warning!(field_warnings, field, value) + end + end + end + + def add_field_warning! field_warnings, field, message + field_warnings[field] ||= [] + field_warnings[field] << message unless field_warnings[field].include?(message) + end + + def merged_tags sources, origin = nil + return sources['manual'].to_s if origin == 'manual' + + sources['automatic'].to_s + end + + def preview_tag_names raw + names = raw.to_s.split + return [] if names.empty? + if names.any? { _1.downcase.start_with?('nico:') } + return [] + end + + names.map { |name| TagName.canonicalise(name.sub(/\[.*\]\z/, '')).first } + rescue Tag::SectionLiteralParseError + [] + end + + def validate_preview_tags raw, errors, known_tags + names = raw.to_s.split + return if names.empty? + if names.any? { _1.downcase.start_with?('nico:') } + errors[:tags] = ['ニコニコ・タグは直接指定できません．'] + return + end + + parsed = names.map { |name| TagName.canonicalise(name.sub(/\[.*\]\z/, '')).first } + existing = parsed.filter_map { known_tags[_1] } + deprecated = existing.select(&:deprecated?).map(&:name) + errors[:tags] = ["廃止済みタグがあります: #{ deprecated.join(' ') }"] if deprecated.present? + rescue Tag::SectionLiteralParseError + errors[:tags] = ['タグ区間の記法が不正です．'] + end + + def validate_basic_data attributes, errors + post = Post.new(title: attributes['title'].presence, + url: attributes['url'], + thumbnail_base: attributes['thumbnail_base'].presence, + original_created_from: attributes['original_created_from'].presence, + original_created_before: attributes['original_created_before'].presence, + video_ms: parse_video_ms(attributes, errors)) + post.valid? + post.errors.each do |error| + next if error.attribute == :url && error.type == :taken + + (errors[error.attribute] ||= []) << error.message + end + end + + def parse_video_ms attributes, errors + return nil unless attributes['tags'].to_s.split.include?('動画') + + video_ms = attributes['video_ms'] + if video_ms.present? + value = Integer(video_ms, exception: false) + if value&.positive? + return value + end + + errors[:video_ms] = ['動画時間の記法が不正です．'] + return nil + end + + value = attributes['duration'] + return nil if value.blank? + + Tag.time_to_ms!(value.to_s, tag_name: '動画時間') + rescue Tag::SectionLiteralParseError + errors[:video_ms] = ['動画時間の記法が不正です．'] + nil + end + + def preview_parent_ids raw + raw.to_s.split.map { Integer(_1, exception: false) }.compact + end + + def validate_parents raw, errors, existing_parent_ids + ids = raw.to_s.split.map { Integer(_1, exception: false) } + return if ids.compact.empty? && raw.to_s.blank? + + if ids.any? { _1.nil? || _1 <= 0 } + errors[:parent_post_ids] = ['親投稿 Id. が不正です．'] + return + end + if ids.uniq.any? { !existing_parent_ids[_1] } + errors[:parent_post_ids] = ['存在しない親投稿 Id. があります．'] + end + end +end diff --git a/backend/app/services/post_metadata_fetcher.rb b/backend/app/services/post_metadata_fetcher.rb new file mode 100644 index 0000000..a8dcb6d --- /dev/null +++ b/backend/app/services/post_metadata_fetcher.rb @@ -0,0 +1,198 @@ +require 'time' +require 'date' + +class PostMetadataFetcher + TIMESTAMP_PATTERN = + /\A(\d{4})-(\d{2})-(\d{2})T(\d{2}) + (?::(\d{2})(?::(\d{2})(?:\.(\d+))?)?)? + (Z|[+-]\d{2}:?\d{2})?\z/x + + def self.fetch raw_url + uri, = Preview::UrlSafety.validate(raw_url) + response = Preview::HttpFetcher.fetch( + uri.to_s, + max_bytes: Preview::ThumbnailFetcher::HTML_MAX_BYTES) + metadata = Preview::HtmlMetadataExtractor.extract(response) + document = Nokogiri::HTML.parse(response.body) + content = lambda { |name| + document + .at_css("meta[property='#{ name }'], meta[name='#{ name }']") + &.[]('content') + &.strip + &.presence + } + duration = content.call('og:video:duration') || content.call('video:duration') + published = content.call('article:published_time') || content.call('date') + created_range = original_created_range(published) + platform_tags = platform_tags(uri) + { title: metadata[:title], + thumbnail_base: + Preview::KnownSiteExtractor.thumbnail_url(uri) || metadata[:image_url], + original_created_from: serialise_time(created_range&.first), + original_created_before: serialise_time(created_range&.last), + duration: serialise_duration(duration), + display_tags: display_tags(platform_tags), + tags: platform_tags.join(' ') } + end + + def self.platform_tags uri + return ['動画', 'YouTube'] if Preview::KnownSiteExtractor.youtube_video_id(uri) + return ['動画', 'ニコニコ'] if Preview::KnownSiteExtractor.niconico_video_id(uri) + + [] + end + + def self.display_tags names + return [] if names.blank? + + existing_tags = + Tag + .joins(:tag_name) + .where(tag_names: { name: names }) + .index_by(&:name) + + names.map { |name| + tag = existing_tags[name] + { + name: name, + category: (tag&.category || 'meta'), + section_literals: [] } + } + end + + def self.original_created_range value + return nil if value.blank? + + raw = value.to_s.strip + from, before = + case raw + when /\A(\d{4})\z/ + year = Regexp.last_match(1).to_i + from = Time.zone.local(year, 1, 1) + [from, from + 1.year] + when /\A(\d{4})-(\d{2})\z/ + year = Regexp.last_match(1).to_i + month = Regexp.last_match(2).to_i + return nil unless month.between?(1, 12) + + from = Time.zone.local(year, month, 1) + [from, from + 1.month] + when /\A(\d{4})-(\d{2})-(\d{2})\z/ + year = Regexp.last_match(1).to_i + month = Regexp.last_match(2).to_i + day = Regexp.last_match(3).to_i + return nil unless Date.valid_date?(year, month, day) + + from = Time.zone.local(year, month, day) + [from, from + 1.day] + else + parse_timestamp_range(raw) + end + return nil if from.nil? || before.nil? + + [from, before] + rescue ArgumentError, TypeError + nil + end + + def self.parse_timestamp_range raw + match = raw.match(TIMESTAMP_PATTERN) + return nil unless match + + year = match[1].to_i + month = match[2].to_i + day = match[3].to_i + hour = match[4].to_i + minute = match[5]&.to_i || 0 + second = match[6]&.to_i || 0 + fraction = match[7] + offset = match[8] + return nil unless valid_timestamp_components?(year, month, day, hour, minute, second) + return nil unless valid_offset?(offset) + + nanoseconds = parse_nanoseconds(fraction) + timestamp = + if offset.present? + Time.new( + year, + month, + day, + hour, + minute, + second + Rational(nanoseconds, 1_000_000_000), + parse_offset(offset)).in_time_zone + else + Time.zone.local(year, month, day, hour, minute, second).change(nsec: nanoseconds) + end + + from = timestamp.change(sec: 0, nsec: 0) + before = + if match[5].nil? + from + 1.hour + else + from + 1.minute + end + [from, before] + end + + def self.parse_nanoseconds value + return 0 if value.blank? + + digits = value[0, 9].ljust(9, '0') + Integer(digits, 10) + end + + def self.parse_offset value + return '+00:00' if value == 'Z' + + value.match?(/\A[+-]\d{2}:\d{2}\z/) ? value : "#{ value[0, 3] }:#{ value[3, 2] }" + end + + def self.serialise_duration value + seconds = Float(value) + return nil unless seconds.positive? + + milliseconds = (seconds * 1_000).round + seconds_string = (milliseconds / 1_000.0).to_s + seconds_string.end_with?('.0') ? seconds_string.delete_suffix('.0') : seconds_string + rescue ArgumentError, TypeError + nil + end + + def self.valid_timestamp_components? year, month, day, hour, minute, second + return false unless Date.valid_date?(year, month, day) + return false unless hour.between?(0, 23) + return false unless minute.between?(0, 59) + return false unless second.between?(0, 59) + + true + end + + def self.valid_offset? value + return true if value.nil? || value == 'Z' + + match = value.match(/\A([+-])(\d{2}):?(\d{2})\z/) + return false if match.nil? + + hours = match[2].to_i + minutes = match[3].to_i + hours.between?(0, 23) && minutes.between?(0, 59) + end + + def self.serialise_time value + return nil if value.nil? + + value.nsec.zero? ? value.iso8601 : value.iso8601(9) + end + + private_class_method :platform_tags, + :display_tags, + :original_created_range, + :parse_timestamp_range, + :parse_nanoseconds, + :parse_offset, + :serialise_duration, + :valid_timestamp_components?, + :valid_offset?, + :serialise_time +end diff --git a/backend/app/services/post_thumbnail_attachment_builder.rb b/backend/app/services/post_thumbnail_attachment_builder.rb new file mode 100644 index 0000000..ad71c91 --- /dev/null +++ b/backend/app/services/post_thumbnail_attachment_builder.rb @@ -0,0 +1,11 @@ +class PostThumbnailAttachmentBuilder + def self.build thumbnail:, thumbnail_base: + if thumbnail_base.present? + return Post.remote_thumbnail_attachment(thumbnail_base) + end + + return nil if thumbnail.blank? + + Post.resized_thumbnail_attachment(thumbnail) + end +end diff --git a/backend/app/services/post_thumbnail_upload_validator.rb b/backend/app/services/post_thumbnail_upload_validator.rb new file mode 100644 index 0000000..ccbab3a --- /dev/null +++ b/backend/app/services/post_thumbnail_upload_validator.rb @@ -0,0 +1,34 @@ +class PostThumbnailUploadValidator + MAX_THUMBNAIL_BYTES = 20 * 1024 * 1024 + ALLOWED_CONTENT_TYPES = Preview::ThumbnailFetcher::RASTER_IMAGE_CONTENT_TYPES.freeze + + class InvalidUpload < StandardError; end + + def self.validate! thumbnail + return if thumbnail.blank? + unless thumbnail.is_a?(ActionDispatch::Http::UploadedFile) + raise InvalidUpload, 'thumbnail upload が不正です．' + end + raise InvalidUpload, 'thumbnail file size が大きすぎます．' if thumbnail.size > MAX_THUMBNAIL_BYTES + raise InvalidUpload, 'サムネイル画像の形式が不正です．' unless allowed_content_type?(thumbnail.content_type) + bytes = thumbnail.read + raise InvalidUpload, 'サムネイル画像の形式が不正です．' if Post.svg_document_bytes?(bytes) + unless Post.raster_thumbnail_bytes?(bytes) + raise InvalidUpload, 'サムネイル画像の形式が不正です．' + end + + attachment = Post.resized_thumbnail_attachment( + StringIO.new(bytes), + content_type: thumbnail.content_type) + attachment[:io].close if attachment[:io].respond_to?(:close) + rescue MiniMagick::Error, Timeout::Error + raise InvalidUpload, 'サムネイル画像の変換に失敗しました．' + ensure + thumbnail&.rewind if thumbnail.respond_to?(:rewind) + end + + def self.allowed_content_type? content_type + mime_type = content_type.to_s.split(';', 2).first.to_s.downcase.strip + ALLOWED_CONTENT_TYPES.include?(mime_type) + end +end diff --git a/backend/app/services/post_url_normaliser.rb b/backend/app/services/post_url_normaliser.rb new file mode 100644 index 0000000..a8d4251 --- /dev/null +++ b/backend/app/services/post_url_normaliser.rb @@ -0,0 +1,13 @@ +class PostUrlNormaliser + def self.normalise raw_url + value = raw_url.to_s.strip + uri = URI.parse(value) + return nil unless uri.is_a?(URI::HTTP) && uri.host.present? + + uri.host = uri.host.downcase + uri.path = uri.path.sub(/\/\z/, '') if uri.path.present? + PostUrlSanitisationRule.sanitise(uri.to_s) + rescue URI::InvalidURIError + nil + end +end diff --git a/backend/app/services/preview/http_fetcher.rb b/backend/app/services/preview/http_fetcher.rb index 30c1644..ce4b701 100644 --- a/backend/app/services/preview/http_fetcher.rb +++ b/backend/app/services/preview/http_fetcher.rb @@ -67,7 +67,8 @@ module Preview log_failure(:timeout, url: uri&.to_s || raw_url, error: e.class.name, message: e.message) raise FetchTimeout, e.message rescue SocketError, SystemCallError, OpenSSL::SSL::SSLError, EOFError => e - log_failure(:network_error, url: uri&.to_s || raw_url, error: e.class.name, message: e.message) + log_failure(:network_error, url: uri&.to_s || raw_url, error: e.class.name, + message: e.message) raise FetchFailed, e.message end diff --git a/backend/app/services/preview/known_site_extractor.rb b/backend/app/services/preview/known_site_extractor.rb index b36629c..07bb4ba 100644 --- a/backend/app/services/preview/known_site_extractor.rb +++ b/backend/app/services/preview/known_site_extractor.rb @@ -4,6 +4,15 @@ module Preview youtube_thumbnail(uri) end + def self.youtube_video_id(uri) + case uri.host&.downcase + when 'youtu.be' + uri.path.split('/').reject(&:blank?).first + when 'www.youtube.com', 'youtube.com', 'm.youtube.com' + uri.path == '/watch' ? URI.decode_www_form(uri.query.to_s).to_h['v'] : nil + end&.then { _1 if _1.match?(/\A[A-Za-z0-9_-]{6,20}\z/) } + end + def self.niconico_video_id(uri) case uri.host&.downcase when 'www.nicovideo.jp', 'nicovideo.jp' @@ -14,14 +23,8 @@ module Preview end def self.youtube_thumbnail(uri) - id = - case uri.host&.downcase - when 'youtu.be' - uri.path.split('/').reject(&:blank?).first - when 'www.youtube.com', 'youtube.com', 'm.youtube.com' - uri.path == '/watch' ? URI.decode_www_form(uri.query.to_s).to_h['v'] : nil - end - return unless id&.match?(/\A[A-Za-z0-9_-]{6,20}\z/) + id = youtube_video_id(uri) + return unless id "https://i.ytimg.com/vi/#{ id }/hqdefault.jpg" end diff --git a/backend/app/services/preview/thumbnail_fetcher.rb b/backend/app/services/preview/thumbnail_fetcher.rb index 746aa13..f829fb7 100644 --- a/backend/app/services/preview/thumbnail_fetcher.rb +++ b/backend/app/services/preview/thumbnail_fetcher.rb @@ -1,7 +1,7 @@ module Preview class ThumbnailFetcher class GenerationFailed < StandardError; end - ALLOWED_IMAGE_CONTENT_TYPES = [ + RASTER_IMAGE_CONTENT_TYPES = [ 'image/jpeg', 'image/png', 'image/gif', 'image/webp' ].freeze HTML_MAX_BYTES = 1.megabyte @@ -25,6 +25,24 @@ module Preview fetch_image!(metadata[:image_url]) end + def self.fetch_image_response(raw_url) + uri, = UrlSafety.validate(raw_url) + response = HttpFetcher.fetch(uri.to_s) + unless Post.remote_thumbnail_image_bytes?( + response.body, + content_type: response.content_type) + raise GenerationFailed, 'サムネール画像が見つかりませんでした．' + end + + response + rescue HttpFetcher::FetchTimeout + raise + rescue HttpFetcher::ResponseTooLarge + raise + rescue HttpFetcher::FetchFailed + raise GenerationFailed, 'サムネール画像を取得できませんでした．' + end + def self.title(raw_url) uri, = UrlSafety.validate(raw_url) HtmlMetadataExtractor.extract( @@ -35,7 +53,9 @@ module Preview return nil if url.blank? response = HttpFetcher.fetch(url) - return nil unless allowed_image_content_type?(response.content_type) + return nil unless Post.remote_thumbnail_image_bytes?( + response.body, + content_type: response.content_type) response.body rescue HttpFetcher::FetchTimeout @@ -45,18 +65,7 @@ module Preview end def self.fetch_image!(url) - response = HttpFetcher.fetch(url) - unless allowed_image_content_type?(response.content_type) - raise GenerationFailed, 'サムネール画像が見つかりませんでした．' - end - - response.body - rescue HttpFetcher::FetchTimeout - raise - rescue HttpFetcher::ResponseTooLarge - raise - rescue HttpFetcher::FetchFailed - raise GenerationFailed, 'サムネール画像を取得できませんでした．' + fetch_image_response(url).body end def self.niconico_thumbnail_url(uri) @@ -70,26 +79,24 @@ module Preview xml.at_xpath('//thumbnail_url')&.text&.strip.presence rescue HttpFetcher::FetchFailed, HttpFetcher::FetchTimeout => e - Rails.logger.info("preview_niconico_getthumbinfo_fallback #{ { url: uri.to_s, - video_id:, - error: e.class.name, - message: e.message }.to_json }") + payload = { + url: uri.to_s, + video_id:, + error: e.class.name, + message: e.message } + Rails.logger.info("preview_niconico_getthumbinfo_fallback #{ payload.to_json }") nil rescue Nokogiri::XML::SyntaxError => e - Rails.logger.info("preview_niconico_getthumbinfo_fallback #{ { url: uri.to_s, - video_id:, - error: e.class.name, - message: e.message }.to_json }") + payload = { + url: uri.to_s, + video_id:, + error: e.class.name, + message: e.message } + Rails.logger.info("preview_niconico_getthumbinfo_fallback #{ payload.to_json }") nil end - def self.allowed_image_content_type?(content_type) - mime_type = content_type.to_s.split(';', 2).first.downcase.strip - ALLOWED_IMAGE_CONTENT_TYPES.include?(mime_type) - end - private_class_method :fetch_image_or_nil, :fetch_image!, - :niconico_thumbnail_url, - :allowed_image_content_type? + :niconico_thumbnail_url end end diff --git a/backend/app/services/youtube/sync.rb b/backend/app/services/youtube/sync.rb index 2056dc2..7fe46f0 100644 --- a/backend/app/services/youtube/sync.rb +++ b/backend/app/services/youtube/sync.rb @@ -1,4 +1,3 @@ -require 'open-uri' require 'set' require 'time' @@ -127,12 +126,12 @@ module Youtube return if post.thumbnail.attached? return if thumbnail_url.blank? - post.thumbnail.attach( - io: URI.open(thumbnail_url), - filename: File.basename(URI.parse(thumbnail_url).path), - content_type: 'image/jpeg') - - post.resized_thumbnail! + post.attach_thumbnail_from_url!(thumbnail_url) + rescue Post::RemoteThumbnailFetchFailed => e + Rails.logger.info("youtube_sync_thumbnail_fetch_failed #{ { post_id: post.id, + thumbnail_url:, + error: e.class.name, + message: e.message }.to_json }") end def youtube_url_regexp id diff --git a/backend/config/environments/development.rb b/backend/config/environments/development.rb index e7722fc..8890673 100644 --- a/backend/config/environments/development.rb +++ b/backend/config/environments/development.rb @@ -1,4 +1,4 @@ -require "active_support/core_ext/integer/time" +require 'active_support/core_ext/integer/time' Rails.application.configure do # Settings specified here will take precedence over those in config/application.rb. @@ -17,8 +17,8 @@ Rails.application.configure do # Enable/disable Action Controller caching. By default Action Controller caching is disabled. # Run rails dev:cache to toggle Action Controller caching. - if Rails.root.join("tmp/caching-dev.txt").exist? - config.public_file_server.headers = { "cache-control" => "public, max-age=#{2.days.to_i}" } + if Rails.root.join('tmp/caching-dev.txt').exist? + config.public_file_server.headers = { 'cache-control' => "public, max-age=#{2.days.to_i}" } else config.action_controller.perform_caching = false end @@ -36,7 +36,11 @@ Rails.application.configure do config.action_mailer.perform_caching = false # Set localhost to be used by links generated in mailer templates. - config.action_mailer.default_url_options = { host: "localhost", port: 3000 } + config.action_mailer.default_url_options = { host: 'localhost', port: 3000 } + Rails.application.routes.default_url_options.merge!( + host: 'localhost', + port: 3002, + protocol: 'http') # Print deprecation notices to the Rails logger. config.active_support.deprecation = :log diff --git a/backend/config/routes.rb b/backend/config/routes.rb index 223332a..30cc5bd 100644 --- a/backend/config/routes.rb +++ b/backend/config/routes.rb @@ -52,6 +52,8 @@ Rails.application.routes.draw do resources :posts, only: [:index, :show, :create, :update] do collection do + get :metadata + post :bulk get :random get :changes get :versions, to: 'post_versions#index' diff --git a/backend/lib/tasks/sync_nico.rake b/backend/lib/tasks/sync_nico.rake index f7c760c..7aca540 100644 --- a/backend/lib/tasks/sync_nico.rake +++ b/backend/lib/tasks/sync_nico.rake @@ -70,11 +70,17 @@ namespace :nico do unless post.thumbnail.attached? thumbnail_base = fetch_thumbnail.(post.url) rescue nil if thumbnail_base.present? - post.thumbnail.attach( - io: URI.open(thumbnail_base), - filename: File.basename(URI.parse(thumbnail_base).path), - content_type: 'image/jpeg') attrs[:thumbnail_base] = thumbnail_base + begin + post.attach_thumbnail_from_url!(thumbnail_base) + rescue Post::RemoteThumbnailFetchFailed => e + payload = { + post_id: post.id, + thumbnail_base:, + error: e.class.name, + message: e.message } + Rails.logger.info("nico_sync_thumbnail_fetch_failed #{ payload.to_json }") + end end end @@ -82,7 +88,6 @@ namespace :nico do post_changed = post.changed? if post_changed post.save! - post.resized_thumbnail! if post.thumbnail.attached? end else post_created = true @@ -91,13 +96,18 @@ namespace :nico do post = Post.new(title:, url:, thumbnail_base:, uploaded_user: nil, original_created_from:, original_created_before:) if thumbnail_base.present? - post.thumbnail.attach( - io: URI.open(thumbnail_base), - filename: File.basename(URI.parse(thumbnail_base).path), - content_type: 'image/jpeg') + begin + post.attach_thumbnail_from_url!(thumbnail_base) + rescue Post::RemoteThumbnailFetchFailed => e + payload = { + post_id: nil, + thumbnail_base:, + error: e.class.name, + message: e.message } + Rails.logger.info("nico_sync_thumbnail_fetch_failed #{ payload.to_json }") + end end post.save! - post.resized_thumbnail! sync_post_tags!(post, [Tag.tagme.id, Tag.bot.id, Tag.niconico.id, Tag.video.id]) end diff --git a/backend/spec/models/post_spec.rb b/backend/spec/models/post_spec.rb index 627b28f..c097fdf 100644 --- a/backend/spec/models/post_spec.rb +++ b/backend/spec/models/post_spec.rb @@ -1,4 +1,5 @@ require 'rails_helper' +require 'tempfile' RSpec.describe Post, type: :model do before do @@ -65,4 +66,296 @@ RSpec.describe Post, type: :model do expect(post.errors.details.fetch(:url)).to include(error: :taken, value: post.url) end end + + describe 'thumbnail processing' do + def image_blob(width:, height:, background:, draw: nil) + Tempfile.create(['post-thumbnail', '.png']) do |file| + MiniMagick::Tool::Convert.new do |convert| + convert.size "#{ width }x#{ height }" + convert.xc background + draw&.call(convert) + convert << file.path + end + File.binread(file.path) + end + end + + def upload_for(blob) + StringIO.new(blob).tap(&:rewind) + end + + def read_image(attachment) + blob = + attachment.is_a?(Hash) ? attachment.fetch(:io).read : attachment.download + MiniMagick::Image.read(blob) + end + + def colour_at(image, x, y) + image.get_pixels.fetch(y).fetch(x) + end + + def expect_green(pixel) + expect(pixel[1]).to be > pixel[0] + 40 + expect(pixel[1]).to be > pixel[2] + 40 + end + + def expect_red(pixel) + expect(pixel[0]).to be > pixel[1] + 40 + expect(pixel[0]).to be > pixel[2] + 40 + end + + def expect_blue(pixel) + expect(pixel[2]).to be > pixel[0] + 40 + expect(pixel[2]).to be > pixel[1] + 40 + end + + describe '.resized_thumbnail_attachment' do + it 'fits a wide image within 180x180 without distorting it' do + blob = image_blob( + width: 360, + height: 180, + background: 'red', + draw: -> convert { + convert.fill 'green' + convert.draw 'rectangle 90,0 269,179' + }) + + resized = described_class.resized_thumbnail_attachment(upload_for(blob)) + image = read_image(resized) + + expect(image.dimensions).to eq([180, 90]) + expect_red(colour_at(image, 0, 45)) + expect_green(colour_at(image, 90, 45)) + expect_red(colour_at(image, 179, 45)) + end + + it 'fits a tall image within 180x180 without distorting it' do + blob = image_blob( + width: 180, + height: 360, + background: 'red', + draw: -> convert { + convert.fill 'green' + convert.draw 'rectangle 0,90 179,269' + }) + + resized = described_class.resized_thumbnail_attachment(upload_for(blob)) + image = read_image(resized) + + expect(image.dimensions).to eq([90, 180]) + expect_red(colour_at(image, 45, 0)) + expect_green(colour_at(image, 45, 90)) + expect_red(colour_at(image, 45, 179)) + end + + it 'keeps a square image square without distortion' do + blob = image_blob( + width: 180, + height: 180, + background: 'red', + draw: -> convert { + convert.fill 'blue' + convert.draw 'rectangle 90,0 179,179' + }) + + resized = described_class.resized_thumbnail_attachment(upload_for(blob)) + image = read_image(resized) + + expect(image.dimensions).to eq([180, 180]) + expect_red(colour_at(image, 20, 90)) + expect_blue(colour_at(image, 160, 90)) + end + end + + describe '#attach_thumbnail_from_url!' do + it 'attaches a fetched remote image through the common resize path' do + blob = image_blob( + width: 240, + height: 180, + background: 'red', + draw: -> convert { + convert.fill 'green' + convert.draw 'rectangle 30,0 209,179' + }) + response = Preview::HttpFetcher::Response.new( + blob, + 'image/png', + 'https://example.com/thumb.png') + allow(Preview::ThumbnailFetcher).to receive(:fetch_image_response) + .with('https://example.com/thumb.png') + .and_return(response) + + post = described_class.create!(title: 'title', url: 'https://example.com/post') + + expect(post.thumbnail).to receive(:attach).once.and_call_original + post.attach_thumbnail_from_url!('https://example.com/thumb.png') + + expect(post.thumbnail).to be_attached + image = read_image(post.thumbnail) + expect(image.dimensions).to eq([180, 135]) + end + + it 'does not attach anything when thumbnail conversion fails' do + response = Preview::HttpFetcher::Response.new( + 'not-an-image', + 'image/png', + 'https://example.com/thumb.png') + allow(Preview::ThumbnailFetcher).to receive(:fetch_image_response) + .with('https://example.com/thumb.png') + .and_return(response) + allow(described_class).to receive(:resized_thumbnail_attachment) + .and_raise(MiniMagick::Error, 'convert failed') + + post = described_class.create!(title: 'title', url: 'https://example.com/post') + + expect { + post.attach_thumbnail_from_url!('https://example.com/thumb.png') + }.to raise_error(Post::RemoteThumbnailFetchFailed, 'convert failed') + + expect(post.thumbnail).not_to be_attached + end + + it 'keeps an existing thumbnail when remote conversion fails' do + existing = described_class.create!(title: 'title', url: 'https://example.com/post') + existing.thumbnail.attach( + io: StringIO.new('existing'), + filename: 'existing.jpg', + content_type: 'image/jpeg') + blob_id = existing.thumbnail.blob.id + response = Preview::HttpFetcher::Response.new( + 'not-an-image', + 'image/png', + 'https://example.com/thumb.png') + allow(Preview::ThumbnailFetcher).to receive(:fetch_image_response) + .with('https://example.com/thumb.png') + .and_return(response) + allow(described_class).to receive(:resized_thumbnail_attachment) + .and_raise(MiniMagick::Error, 'convert failed') + + expect { + existing.attach_thumbnail_from_url!('https://example.com/thumb.png') + }.to raise_error(Post::RemoteThumbnailFetchFailed, 'convert failed') + + existing.reload + expect(existing.thumbnail).to be_attached + expect(existing.thumbnail.blob.id).to eq(blob_id) + end + end + end + + describe 'original created datetime validation' do + it 'allows unrelated updates on persisted posts with second-bearing datetimes' do + post = described_class.create!(title: 'title', url: 'https://example.com/post') + post.update_columns( + original_created_from: Time.zone.parse('2024-01-01T12:34:30Z'), + original_created_before: Time.zone.parse('2024-01-01T12:35:30Z') + ) + + post.title = 'updated title' + + expect(post).to be_valid + expect { post.save! }.not_to raise_error + end + + it 'rejects second-bearing updates when the datetime field changes' do + post = described_class.create!(title: 'title', url: 'https://example.com/post') + + post.original_created_from = '2024-01-01T12:34:30Z' + + expect(post).to be_invalid + expect(post.errors[:original_created_from]).to eq( + [described_class::ORIGINAL_CREATED_MINUTE_PRECISION_MESSAGE] + ) + end + + it 'accepts fixing persisted datetimes to minute precision' do + post = described_class.create!(title: 'title', url: 'https://example.com/post') + post.update_columns( + original_created_from: Time.zone.parse('2024-01-01T12:34:30Z'), + original_created_before: Time.zone.parse('2024-01-01T12:35:30Z') + ) + + post.original_created_from = '2024-01-01T12:34Z' + post.original_created_before = '2024-01-01T12:35Z' + + expect(post).to be_valid + expect { post.save! }.not_to raise_error + end + + it 'adds only the minute-precision error for second precision values' do + post = described_class.new( + title: 'title', + url: 'https://example.com/post', + original_created_from: '2024-01-01T12:34:30', + original_created_before: '2024-01-01T12:35' + ) + + expect(post).to be_invalid + expect(post.errors[:original_created_from]).to eq( + [described_class::ORIGINAL_CREATED_MINUTE_PRECISION_MESSAGE] + ) + expect(post.errors[:original_created_at]).to be_empty + end + + it 'adds only the minute-precision error for fractional-second values' do + post = described_class.new( + title: 'title', + url: 'https://example.com/post', + original_created_from: '2024-01-01T12:34:00.123', + original_created_before: '2024-01-01T12:35' + ) + + expect(post).to be_invalid + expect(post.errors[:original_created_from]).to eq( + [described_class::ORIGINAL_CREATED_MINUTE_PRECISION_MESSAGE] + ) + expect(post.errors[:original_created_at]).to be_empty + end + + it 'checks range rules only for valid minute-precision endpoints' do + post = described_class.new( + title: 'title', + url: 'https://example.com/post', + original_created_from: '2024-01-01T12:34', + original_created_before: '2024-01-01T12:34' + ) + + expect(post).to be_invalid + expect(post.errors[:original_created_at]).to eq( + [described_class::ORIGINAL_CREATED_ORDER_MESSAGE] + ) + end + + it 'accepts a one-minute range at minute precision' do + post = described_class.new( + title: 'title', + url: 'https://example.com/post', + original_created_from: '2024-01-01T12:34', + original_created_before: '2024-01-01T12:35' + ) + + expect(post).to be_valid + end + + it 'rejects invalid calendar dates and invalid hours' do + invalid_dates = [ + '2024-02-31T12:00', + '2023-02-29T12:00', + '2024-02-29T24:00' + ] + + invalid_dates.each do |value| + post = described_class.new( + title: 'title', + url: 'https://example.com/post', + original_created_from: value + ) + + expect(post).to be_invalid + expect(post.errors[:original_created_from]).to eq( + [described_class::ORIGINAL_CREATED_INVALID_MESSAGE] + ) + end + end + end end diff --git a/backend/spec/requests/posts_spec.rb b/backend/spec/requests/posts_spec.rb index 752ac6a..e1a0ee4 100644 --- a/backend/spec/requests/posts_spec.rb +++ b/backend/spec/requests/posts_spec.rb @@ -9,11 +9,11 @@ RSpec.describe 'Posts API', type: :request do # resized_thumbnail! が MiniMagick 依存でコケやすいので request spec ではスタブしとくのが無難。 before do allow_any_instance_of(Post).to receive(:resized_thumbnail!).and_return(true) - allow(Post).to receive(:resized_thumbnail_attachment).and_return( - io: StringIO.new('dummy'), - filename: 'resized_thumbnail.jpg', - content_type: 'image/jpeg' - ) + allow(Post).to receive(:resized_thumbnail_attachment) do + { io: StringIO.new('dummy'), + filename: 'resized_thumbnail.jpg', + content_type: 'image/jpeg' } + end end def create_nico_tag!(name) @@ -21,8 +21,7 @@ RSpec.describe 'Posts API', type: :request do end def dummy_upload - # 中身は何でもいい（加工処理はスタブしてる） - Rack::Test::UploadedFile.new(StringIO.new('dummy'), 'image/jpeg', original_filename: 'dummy.jpg') + real_thumbnail_upload end def real_thumbnail_upload @@ -102,34 +101,34 @@ RSpec.describe 'Posts API', type: :request do end end - describe "GET /posts" do + describe 'GET /posts' do let!(:user) { create_member_user! } - let!(:tag_name) { TagName.create!(name: "spec_tag") } + let!(:tag_name) { TagName.create!(name: 'spec_tag') } let!(:tag) { Tag.create!(tag_name:, category: :general) } let!(:tag_name2) { TagName.create!(name: 'unko') } let!(:tag2) { Tag.create!(tag_name: tag_name2, category: :deerjikist) } let!(:alias_tag_name) { TagName.create!(name: 'manko', canonical: tag_name) } let!(:hit_post) do - Post.create!(uploaded_user: user, title: "hello spec world", + Post.create!(uploaded_user: user, title: 'hello spec world', url: 'https://example.com/spec2').tap do |p| PostTag.create!(post: p, tag:) end end let!(:miss_post) do - Post.create!(uploaded_user: user, title: "unrelated title", + Post.create!(uploaded_user: user, title: 'unrelated title', url: 'https://example.com/spec3').tap do |p| PostTag.create!(post: p, tag: tag2) end end - it "returns posts with tag name in JSON" do - get "/posts" + it 'returns posts with tag name in JSON' do + get '/posts' expect(response).to have_http_status(:ok) - posts = json.fetch("posts") + posts = json.fetch('posts') # 全postの全tagが name を含むこと expect(posts).not_to be_empty @@ -142,8 +141,8 @@ RSpec.describe 'Posts API', type: :request do expect(json['count']).to be_an(Integer) # spec_tag を含む投稿が存在すること - all_tag_names = posts.flat_map { |p| p["tags"].map { |t| t["name"] } } - expect(all_tag_names).to include("spec_tag") + all_tag_names = posts.flat_map { |p| p['tags'].map { |t| t['name'] } } + expect(all_tag_names).to include('spec_tag') end it 'keeps children and sections keys in non-detail tag responses' do @@ -162,9 +161,9 @@ RSpec.describe 'Posts API', type: :request do ]) end - context "when q is provided" do - it "filters posts by q (hit case)" do - get "/posts", params: { tags: "spec_tag" } + context 'when q is provided' do + it 'filters posts by q (hit case)' do + get '/posts', params: { tags: 'spec_tag' } expect(response).to have_http_status(:ok) posts = json.fetch('posts') @@ -182,8 +181,8 @@ RSpec.describe 'Posts API', type: :request do end end - it "filters posts by q (hit case by alias)" do - get "/posts", params: { tags: "manko" } + it 'filters posts by q (hit case by alias)' do + get '/posts', params: { tags: 'manko' } expect(response).to have_http_status(:ok) posts = json.fetch('posts') @@ -201,11 +200,11 @@ RSpec.describe 'Posts API', type: :request do end end - it "returns empty posts when nothing matches" do - get "/posts", params: { tags: "no_such_keyword_12345" } + it 'returns empty posts when nothing matches' do + get '/posts', params: { tags: 'no_such_keyword_12345' } expect(response).to have_http_status(:ok) - expect(json.fetch("posts")).to eq([]) + expect(json.fetch('posts')).to eq([]) expect(json.fetch('count')).to eq(0) end end @@ -693,6 +692,73 @@ RSpec.describe 'Posts API', type: :request do end end + describe 'GET /posts/metadata' do + let(:member) { create(:user, :member) } + + it 'returns compact existing post data without fetching external metadata' do + sign_in_as(member) + existing = create( + :post, + title: 'existing post', + url: 'https://example.com/existing') + existing.thumbnail.attach( + io: StringIO.new('thumbnail'), + filename: 'thumbnail.jpg', + content_type: 'image/jpeg') + expect(PostMetadataFetcher).not_to receive(:fetch) + + get '/posts/metadata', params: { url: 'https://example.com/existing' } + + expect(response).to have_http_status(:ok) + expect(json).to include( + 'url' => existing.url, + 'existing_post_id' => existing.id, + 'field_warnings' => { }) + expect(json.fetch('existing_post')).to include( + 'id' => existing.id, + 'title' => existing.title, + 'url' => existing.url, + 'thumbnail_base' => existing.thumbnail_base) + expect(json.dig('existing_post', 'thumbnail')) + .to include('/rails/active_storage/blobs/proxy/') + end + + it 'returns fetched metadata and structured display tags' do + sign_in_as(member) + allow(Preview::UrlSafety).to receive(:validate) + allow(PostMetadataFetcher).to receive(:fetch).and_return( + title: 'fetched title', + thumbnail_base: 'https://example.com/thumbnail.jpg', + tags: 'character:虹夏', + display_tags: [{ name: '虹夏', category: 'character' }], + original_created_from: nil, + original_created_before: nil, + duration: '1:00', + video_ms: 60_000) + + get '/posts/metadata', params: { url: 'https://example.com/new' } + + expect(response).to have_http_status(:ok) + expect(json).to include( + 'url' => 'https://example.com/new', + 'title' => 'fetched title', + 'duration' => '1:00', + 'video_ms' => 60_000, + 'field_warnings' => { }) + expect(json.fetch('display_tags')).to eq( + [{ 'name' => '虹夏', 'category' => 'character' }]) + end + + it 'returns URL validation errors as 422' do + sign_in_as(member) + + get '/posts/metadata', params: { url: 'file:///etc/passwd' } + + expect(response).to have_http_status(:unprocessable_entity) + expect(json.fetch('errors')).to have_key('url') + end + end + describe 'POST /posts' do let(:member) { create(:user, :member) } let!(:alias_tag_name) { TagName.create!(name: 'manko', canonical: tag_name) } @@ -712,6 +778,46 @@ RSpec.describe 'Posts API', type: :request do expect(response).to have_http_status(:forbidden) end + it 'dry-runs without persisting posts or new tags' do + sign_in_as(member) + counts = [Post.count, Tag.count, TagName.count] + + post '/posts?dry=1', params: post_write_params( + title: 'dry-run post', + url: 'https://example.com/dry-run', + tags: 'character:new_dry_run_tag') + + expect(response).to have_http_status(:ok) + expect(json).to include( + 'url' => 'https://example.com/dry-run', + 'tags' => 'new_dry_run_tag', + 'existing_post_id' => nil) + expect(json.fetch('display_tags')).to eq( + [{ 'name' => 'new_dry_run_tag', + 'category' => 'character', + 'section_literals' => [] }]) + expect([Post.count, Tag.count, TagName.count]).to eq(counts) + end + + it 'dry-runs an existing URL without validating its upload' do + sign_in_as(member) + existing = create(:post, url: 'https://example.com/dry-existing') + invalid_upload = Rack::Test::UploadedFile.new( + StringIO.new(''), + 'image/png', + original_filename: 'thumbnail.png') + + post '/posts?dry=1', params: post_write_params( + title: '', + url: existing.url, + tags: '', + thumbnail: invalid_upload) + + expect(response).to have_http_status(:ok) + expect(json).to include('existing_post_id' => existing.id) + expect(json.fetch('field_warnings')).not_to have_key('thumbnail_base') + end + it '201 and creates post + tags when member' do sign_in_as(member) @@ -769,7 +875,8 @@ RSpec.describe 'Posts API', type: :request do ) expect(response).to have_http_status(:created) - expect(open_transactions).to eq([baseline_open_transactions]) + expect(open_transactions).to eq( + [baseline_open_transactions, baseline_open_transactions]) end it 'returns 422 and does not create a post when thumbnail resize fails' do @@ -827,7 +934,7 @@ RSpec.describe 'Posts API', type: :request do expect(response).to have_http_status(:unprocessable_entity) expect(json.fetch('errors')).to include( - 'tags' => ['廃止済みタグは付与できません．'] + 'tags' => ['廃止済みタグがあります: deprecated_direct_tag'] ) end @@ -1054,7 +1161,7 @@ RSpec.describe 'Posts API', type: :request do ) end - context "when nico tag already exists in tags" do + context 'when nico tag already exists in tags' do before do Tag.find_undiscard_or_create_by!( tag_name: TagName.find_undiscard_or_create_by!(name: 'nico:nico_tag'), @@ -1251,6 +1358,47 @@ RSpec.describe 'Posts API', type: :request do end end + describe 'POST /posts/bulk' do + let(:member) { create(:user, :member) } + + it 'parses the manifest and indexed thumbnail parts' do + sign_in_as(member) + manifest = [{ 'url' => 'https://example.com/bulk', + 'title' => 'bulk post', + 'tags' => 'spec_tag', + 'parent_post_ids' => '' }] + creator = instance_double( + PostBulkCreator, + run: { results: [{ status: 'created', post: { id: 123 } }] }) + allow(PostBulkCreator).to receive(:new).and_return(creator) + + post '/posts/bulk', params: { + posts: JSON.generate(manifest), + thumbnails: { '0' => real_thumbnail_upload } } + + expect(response).to have_http_status(:ok) + expect(json.fetch('results')).to eq( + [{ 'status' => 'created', 'post' => { 'id' => 123 } }]) + expect(PostBulkCreator).to have_received(:new) do |arguments| + expect(arguments[:actor]).to eq(member) + expect(arguments[:posts]).to eq(manifest) + expect(arguments[:thumbnails].keys).to eq([0]) + expect(arguments[:host]).to eq('http://www.example.com') + end + end + + it 'rejects malformed manifests as a request-level error' do + sign_in_as(member) + + post '/posts/bulk', params: { + posts: '{', + thumbnails: { '0' => real_thumbnail_upload } } + + expect(response).to have_http_status(:bad_request) + expect(json.fetch('message')).to eq('posts manifest の JSON が不正です．') + end + end + describe 'PUT /posts/:id' do let(:member) { create(:user, :member) } @@ -1307,7 +1455,7 @@ RSpec.describe 'Posts API', type: :request do ) end - context "when nico tag already exists in tags" do + context 'when nico tag already exists in tags' do before do Tag.find_undiscard_or_create_by!( tag_name: TagName.find_undiscard_or_create_by!(name: 'nico:nico_tag'), @@ -1564,7 +1712,7 @@ RSpec.describe 'Posts API', type: :request do expect(post_record.reload.title).to eq('updated by other user') end - it 'returns 409 with mergeable true when stale tag changes do not conflict but merge is not requested' do + it 'returns mergeable 409 for stale non-conflicting tag changes without merge' do sign_in_as(member) base_version = create_post_version_for!(post_record.reload) @@ -2206,6 +2354,91 @@ RSpec.describe 'Posts API', type: :request do expect(response).to have_http_status(:unprocessable_entity) end + + it 'rejects second-bearing original created timestamps on POST /posts' do + sign_in_as(member) + + post '/posts', params: post_write_params( + title: 'invalid original created from', + url: 'https://example.com/invalid-original-created-from', + tags: 'spec_tag', + thumbnail: dummy_upload, + original_created_from: '2020-01-01T00:00:01Z') + + expect(response).to have_http_status(:unprocessable_entity) + expect(json.fetch('errors')).to include( + 'original_created_from' => ['オリジナルの作成日時は分単位で入力してください．'] + ) + expect(json.fetch('errors')).not_to have_key('original_created_at') + end + + it 'rejects fractional-second original created timestamps on POST /posts' do + sign_in_as(member) + + post '/posts', params: post_write_params( + title: 'invalid original created before', + url: 'https://example.com/invalid-original-created-before', + tags: 'spec_tag', + thumbnail: dummy_upload, + original_created_before: '2020-01-01T00:00:00.123Z') + + expect(response).to have_http_status(:unprocessable_entity) + expect(json.fetch('errors')).to include( + 'original_created_before' => ['オリジナルの作成日時は分単位で入力してください．'] + ) + expect(json.fetch('errors')).not_to have_key('original_created_at') + end + + it 'rejects non-increasing original created ranges on POST /posts' do + sign_in_as(member) + + post '/posts', params: post_write_params( + title: 'non-increasing original created range', + url: 'https://example.com/non-increasing-original-created-range', + tags: 'spec_tag', + thumbnail: dummy_upload, + original_created_from: '2020-01-01T00:00Z', + original_created_before: '2020-01-01T00:00Z') + + expect(response).to have_http_status(:unprocessable_entity) + expect(json.fetch('errors')).to include( + 'original_created_at' => ['オリジナルの作成日時の順番がをかしぃです．'] + ) + end + + it 'accepts original created ranges that are exactly one minute on POST /posts' do + sign_in_as(member) + + post '/posts', params: post_write_params( + title: 'valid original created range', + url: 'https://example.com/valid-original-created-range', + tags: 'spec_tag', + thumbnail: dummy_upload, + original_created_from: '2020-01-01T00:00Z', + original_created_before: '2020-01-01T00:01Z') + + expect(response).to have_http_status(:created) + expect(Time.iso8601(json.fetch('original_created_from'))) + .to eq(Time.utc(2020, 1, 1, 0, 0)) + expect(Time.iso8601(json.fetch('original_created_before'))) + .to eq(Time.utc(2020, 1, 1, 0, 1)) + end + + it 'rejects unparseable original created timestamps on PUT /posts/:id' do + sign_in_as(member) + base_version = create_post_version_for!(post_record) + + put "/posts/#{post_record.id}", params: post_write_params( + base_version_no: base_version.version_no, + title: 'updated title', + tags: 'spec_tag', + original_created_from: 'not-a-time') + + expect(response).to have_http_status(:unprocessable_entity) + expect(json.fetch('errors')).to include( + 'original_created_from' => ['オリジナルの作成日時の形式が不正です．'] + ) + end end describe 'tag versioning from post write actions' do diff --git a/backend/spec/requests/wiki_body_search_pending_spec.rb b/backend/spec/requests/wiki_body_search_pending_spec.rb deleted file mode 100644 index 98bd6af..0000000 --- a/backend/spec/requests/wiki_body_search_pending_spec.rb +++ /dev/null @@ -1,27 +0,0 @@ -require 'rails_helper' - -RSpec.describe 'Wiki body search', type: :request do - let!(:user) { create_member_user! } - - it 'searches wiki pages by body text' do - pending '#336 で対応予定' - - Wiki::Commit.create_content!( - tag_name: TagName.create!(name: 'wiki_body_search_hit'), - body: 'unique body keyword for wiki search', - created_by_user: user, - message: 'init') - - Wiki::Commit.create_content!( - tag_name: TagName.create!(name: 'wiki_body_search_miss'), - body: 'ordinary body', - created_by_user: user, - message: 'init') - - get '/wiki/search', params: { body: 'unique body keyword' } - - expect(response).to have_http_status(:ok) - expect(json.map { |page| page['title'] }).to include('wiki_body_search_hit') - expect(json.map { |page| page['title'] }).not_to include('wiki_body_search_miss') - end -end diff --git a/backend/spec/requests/wiki_restore_pending_spec.rb b/backend/spec/requests/wiki_restore_pending_spec.rb deleted file mode 100644 index 33b6a63..0000000 --- a/backend/spec/requests/wiki_restore_pending_spec.rb +++ /dev/null @@ -1,37 +0,0 @@ -require 'rails_helper' - -RSpec.describe 'Wiki restore', type: :request do - let!(:user) { create_member_user! } - - def auth_headers user - { 'X-Transfer-Code' => user.inheritance_code } - end - - it 'restores wiki page to previous version' do - pending '#337 で対応予定' - - page = - Wiki::Commit.create_content!( - tag_name: TagName.create!(name: 'wiki_restore_page'), - body: 'v1', - created_by_user: user, - message: 'init') - - v1 = page.wiki_versions.order(:version_no).last - - Wiki::Commit.content!( - page:, - body: 'v2', - created_user: user, - message: 'edit', - base_revision_id: page.current_revision.id) - - post "/wiki/#{ page.id }/restore", - params: { version_no: v1.version_no }, - headers: auth_headers(user) - - expect(response).to have_http_status(:ok) - expect(page.reload.body).to eq('v1') - expect(page.wiki_versions.order(:version_no).last.event_type).to eq('restore') - end -end diff --git a/backend/spec/services/post_bulk_creator_spec.rb b/backend/spec/services/post_bulk_creator_spec.rb new file mode 100644 index 0000000..0779661 --- /dev/null +++ b/backend/spec/services/post_bulk_creator_spec.rb @@ -0,0 +1,59 @@ +require 'rails_helper' + +RSpec.describe PostBulkCreator do + it 'limits workers to two and keeps failures in their request slots' do + actor = instance_double(User, id: 123) + allow(User).to receive(:find).with(123) { + instance_double(User, id: 123) + } + mutex = Mutex.new + active = 0 + maximum_active = 0 + + allow(PostCreatePreflight).to receive(:new) do |attributes:, **| + preflight = instance_double(PostCreatePreflight) + allow(preflight).to receive(:run) do + mutex.synchronize do + active += 1 + maximum_active = [maximum_active, active].max + end + sleep 0.02 + mutex.synchronize { active -= 1 } + attributes.symbolize_keys.merge( + existing_post_id: nil, + field_warnings: { }, + base_warnings: []) + end + preflight + end + allow(PostCreator).to receive(:new) do |attributes:, **| + creator = instance_double(PostCreator) + if attributes[:title] == 'broken' + allow(creator).to receive(:create!).and_raise(StandardError, 'broken') + else + post = instance_double(Post, id: attributes[:title].delete_prefix('post ').to_i) + allow(creator).to receive(:create!).and_return(post) + end + creator + end + posts = [ + { 'title' => 'post 1', 'url' => 'https://example.com/1' }, + { 'title' => 'broken', 'url' => 'https://example.com/2' }, + { 'title' => 'post 3', 'url' => 'https://example.com/3' }, + { 'title' => 'post 4', 'url' => 'https://example.com/4' }] + + results = described_class.new( + actor:, + posts:, + thumbnails: { }).run.fetch(:results) + + expect(maximum_active).to eq(2) + expect(results.length).to eq(posts.length) + expect(results.map { _1[:status] }).to eq( + ['created', 'failed', 'created', 'created']) + expect(results[0].dig(:post, :id)).to eq(1) + expect(results[1]).to include(status: 'failed', recoverable: false) + expect(results[2].dig(:post, :id)).to eq(3) + expect(results[3].dig(:post, :id)).to eq(4) + end +end diff --git a/backend/spec/services/post_create_plan_spec.rb b/backend/spec/services/post_create_plan_spec.rb new file mode 100644 index 0000000..5558d81 --- /dev/null +++ b/backend/spec/services/post_create_plan_spec.rb @@ -0,0 +1,82 @@ +require 'rails_helper' + +RSpec.describe PostCreatePlan do + def create_tag! name, category + Tag.create!(name:, category:) + end + + before do + create_tag!('タグ希望', :meta) + create_tag!('ニジラー情報不詳', :meta) + end + + it 'plans direct and existing default tags without persisting records' do + counts = [TagName.count, Tag.count] + + plan = described_class.new( + attributes: { + url: 'https://example.com/post', + title: 'title', + tags: 'character:new_character', + parent_post_ids: '' }).build! + + expect(plan[:tags]).to eq('new_character') + expect(plan[:direct_tag_specs]).to eq( + [{ name: 'new_character', category: :character }]) + expect(plan[:default_tag_specs]).to include( + { name: 'タグ希望', category: :meta }, + { name: 'ニジラー情報不詳', category: :meta }) + expect([TagName.count, Tag.count]).to eq(counts) + end + + it 'resolves aliases and keeps tag sections separate from canonical names' do + canonical = create_tag!('虹夏', :character) + TagName.create!(name: 'にじか', canonical: canonical.tag_name) + create_tag!('動画', :meta) + + plan = described_class.new( + attributes: { + url: 'https://example.com/video', + title: 'video', + tags: '動画 にじか[0:10-0:20]', + duration: '1:00', + parent_post_ids: '' }).build! + + expect(plan[:tags].split).to include('動画', '虹夏[0:10-0:20]') + expect(plan[:display_tags]).to include( + { name: '虹夏', + category: 'character', + section_literals: ['[0:10-0:20]'] }) + expect(plan[:video_ms]).to eq(60_000) + end + + it 'validates a new tag name without persisting it' do + long_name = 'a' * 256 + counts = [TagName.count, Tag.count] + + expect { + described_class.new( + attributes: { + url: 'https://example.com/post', + title: 'title', + tags: long_name, + parent_post_ids: '' }).build! + }.to raise_error(ActiveRecord::RecordInvalid) { |error| + expect(error.record.errors[:tags]).not_to be_empty + } + expect([TagName.count, Tag.count]).to eq(counts) + end + + it 'ignores duration when the planned tags do not include video' do + plan = described_class.new( + attributes: { + url: 'https://example.com/post', + title: 'title', + tags: 'ordinary_tag', + duration: 'invalid', + parent_post_ids: '' }).build! + + expect(plan[:duration]).to eq('invalid') + expect(plan[:video_ms]).to be_nil + end +end diff --git a/backend/spec/services/post_creator_spec.rb b/backend/spec/services/post_creator_spec.rb new file mode 100644 index 0000000..cadf8a1 --- /dev/null +++ b/backend/spec/services/post_creator_spec.rb @@ -0,0 +1,78 @@ +require 'rails_helper' +require 'base64' + +RSpec.describe PostCreator do + let(:actor) { create(:user, :member) } + + def real_thumbnail_upload + gif = Base64.decode64('R0lGODdhAQABAIABAP///wAAACwAAAAAAQABAAACAkQBADs=') + + Rack::Test::UploadedFile.new( + StringIO.new(gif), + 'image/gif', + original_filename: 'thumbnail.gif') + end + + before do + allow(Tag).to receive(:normalise_tags!).and_return({ tags: [], sections: {} }) + allow(TagVersioning).to receive(:record_tag_snapshots!) + allow(Tag).to receive(:expand_parent_tags).and_return([]) + allow(PostVersionRecorder).to receive(:record!) + end + + it 'prefers thumbnail_base over an explicit upload' do + expect(Post).not_to receive(:resized_thumbnail_attachment) + allow(Post).to receive(:remote_thumbnail_attachment).and_return( + io: StringIO.new('remote'), + filename: 'resized_thumbnail.jpg', + content_type: 'image/jpeg') + + post = described_class.new( + actor:, + attributes: { + title: 'title', + url: 'https://example.com/post', + thumbnail: real_thumbnail_upload, + thumbnail_base: 'https://example.com/thumb.jpg', + tags: '' }).create! + + expect(post.thumbnail).to be_attached + expect(post.thumbnail_base).to eq('https://example.com/thumb.jpg') + end + + it 'uses the common remote thumbnail attach path when thumbnail_base is given' do + expect(Post).to receive(:remote_thumbnail_attachment) + .with('https://example.com/thumb.jpg') + .and_return( + io: StringIO.new('thumbnail'), + filename: 'thumbnail.jpg', + content_type: 'image/jpeg') + + post = described_class.new( + actor:, + attributes: { + title: 'title', + url: 'https://example.com/post', + thumbnail_base: 'https://example.com/thumb.jpg', + tags: '' }).create! + + expect(post.thumbnail_base).to eq('https://example.com/thumb.jpg') + expect(post.thumbnail).to be_attached + end + + it 'does not create a post when remote thumbnail fetch fails' do + allow(Post).to receive(:remote_thumbnail_attachment) + .and_raise(Post::RemoteThumbnailFetchFailed, 'サムネール画像を取得できませんでした．') + creator = described_class.new( + actor:, + attributes: { + title: 'title', + url: 'https://example.com/post', + thumbnail_base: 'https://example.com/thumb.jpg', + tags: '' }) + + post_count = Post.count + expect { creator.create! }.to raise_error(Post::RemoteThumbnailFetchFailed) + expect(Post.count).to eq(post_count) + end +end diff --git a/backend/spec/services/post_import_previewer_spec.rb b/backend/spec/services/post_import_previewer_spec.rb new file mode 100644 index 0000000..bd6e267 --- /dev/null +++ b/backend/spec/services/post_import_previewer_spec.rb @@ -0,0 +1,187 @@ +require 'rails_helper' + +RSpec.describe PostImportPreviewer do + before do + allow(Preview::UrlSafety).to receive(:validate) do |url| + [URI.parse(url), ['8.8.8.8']] + end + end + + def row(source_row:, url:, attributes: { }, provenance: { }, **values) + { + source_row:, + url:, + attributes:, + provenance: + }.merge(values) + end + + describe '#preview_rows' do + it 'marks an existing post for skipping without fetching metadata' do + existing = create(:post, url: 'https://example.com/existing') + expect(PostMetadataFetcher).not_to receive(:fetch) + + result = described_class.new.preview_rows(rows: [ + row(source_row: 1, url: 'https://EXAMPLE.com/existing/') + ]).first + + expect(result).to include( + url: 'https://example.com/existing', + skip_reason: 'existing', + existing_post_id: existing.id, + validation_errors: {} + ) + expect(result.fetch(:field_warnings)).to eq({}) + end + + it 'reports list duplicates as URL errors instead of existing skips' do + create(:post, url: 'https://example.com/duplicate') + expect(PostMetadataFetcher).not_to receive(:fetch) + + results = described_class.new.preview_rows(rows: [ + row(source_row: 1, url: 'https://example.com/duplicate'), + row(source_row: 2, url: 'https://EXAMPLE.com/duplicate/') + ]) + + expect(results.map { _1[:validation_errors] }).to all( + include(url: ['URL が重複しています．']) + ) + expect(results).to all(include(skip_reason: nil, existing_post_id: nil)) + end + + it 'does not fetch metadata for an unsafe URL' do + allow(Preview::UrlSafety).to receive(:validate) + .and_raise(Preview::UrlSafety::UnsafeUrl, '安全でない接続先は使用できません．') + expect(PostMetadataFetcher).not_to receive(:fetch) + + result = described_class.new.preview_rows(rows: [ + row(source_row: 1, url: 'https://unsafe.example/post') + ]).first + + expect(result.fetch(:validation_errors)).to include( + url: ['安全でない接続先は使用できません．'] + ) + end + + it 'applies metadata to automatic fields and recognises metadata tags' do + Tag.create!(name: 'known-tag', category: :general) + allow(PostMetadataFetcher).to receive(:fetch).and_return( + title: 'metadata title', + thumbnail_base: 'https://example.com/thumb.jpg', + duration: '2', + tags: 'known-tag' + ) + + result = described_class.new.preview_rows(rows: [ + row(source_row: 1, url: 'https://example.com/new') + ]).first + + expect(result.fetch(:attributes)).to include( + 'title' => 'metadata title', + 'thumbnail_base' => 'https://example.com/thumb.jpg', + 'duration' => '2', + 'tags' => 'known-tag' + ) + expect(result.fetch(:field_warnings)).not_to have_key('tags') + expect(PostMetadataFetcher).to have_received(:fetch).once + end + + it 'preserves manual empty values when metadata is available' do + allow(PostMetadataFetcher).to receive(:fetch).and_return( + title: 'metadata title', + tags: 'metadata-tag' + ) + + result = described_class.new.preview_rows(rows: [ + row( + source_row: 1, + url: 'https://example.com/manual', + attributes: { title: '', tags: '' }, + provenance: { title: 'manual', tags: 'manual' }, + tag_sources: { automatic: 'old-tag', manual: '' } + ) + ]).first + + expect(result.fetch(:attributes)).to include('title' => '', 'tags' => '') + expect(result.fetch(:tag_sources)).to include('manual' => '') + end + + it 'clears warnings derived from an old metadata URL' do + allow(PostMetadataFetcher).to receive(:fetch).and_return( + title: 'new title', + thumbnail_base: 'https://example.com/new-thumb.jpg', + tags: '' + ) + + result = described_class.new.preview_rows(rows: [ + row( + source_row: 1, + url: 'https://example.com/new-url', + attributes: { title: 'old title', tags: 'old-tag' }, + provenance: { title: 'automatic', tags: 'automatic' }, + tag_sources: { automatic: 'old-tag', manual: '' }, + metadata_url: 'https://example.com/old-url', + field_warnings: { tags: ['old tag warning'], url: ['old fetch warning'] }, + base_warnings: ['old base warning'] + ) + ]).first + + expect(result.fetch(:attributes)).to include('title' => 'new title', 'tags' => '') + expect(result.fetch(:base_warnings)).to eq([]) + expect(result.fetch(:field_warnings).values.flatten) + .not_to include('old tag warning', 'old fetch warning') + end + + it 'isolates an unexpected metadata failure to the affected URL' do + allow(PostMetadataFetcher).to receive(:fetch) do |url| + raise 'fetch failure' if url.include?('failure') + + { title: 'successful title', tags: '' } + end + allow(Rails.logger).to receive(:error) + + results = described_class.new.preview_rows(rows: [ + row(source_row: 1, url: 'https://example.com/failure'), + row(source_row: 2, url: 'https://example.com/success') + ]) + + expect(results[0].fetch(:field_warnings)).to include( + 'url' => [described_class::METADATA_FETCH_WARNING] + ) + expect(results[1].fetch(:attributes)).to include('title' => 'successful title') + end + + it 'turns an unsafe URL detected during metadata fetching into a URL error' do + allow(PostMetadataFetcher).to receive(:fetch) + .and_raise(Preview::UrlSafety::UnsafeUrl, '安全でない接続先です．') + allow(Rails.logger).to receive(:info) + + result = described_class.new.preview_rows(rows: [ + row(source_row: 1, url: 'https://example.com/redirects-to-private') + ]).first + + expect(result.fetch(:validation_errors)).to include( + url: ['安全でない接続先です．'] + ) + end + + it 'validates parent IDs from the preloaded set' do + parent = create(:post) + + results = described_class.new.preview_rows( + rows: [ + row(source_row: 1, url: 'https://example.com/valid-parent', + attributes: { parent_post_ids: parent.id.to_s }), + row(source_row: 2, url: 'https://example.com/missing-parent', + attributes: { parent_post_ids: '999999' }) + ], + fetch_metadata: false + ) + + expect(results[0].fetch(:validation_errors)).not_to have_key(:parent_post_ids) + expect(results[1].fetch(:validation_errors)).to include( + parent_post_ids: ['存在しない親投稿 Id. があります．'] + ) + end + end +end diff --git a/backend/spec/services/post_metadata_fetcher_spec.rb b/backend/spec/services/post_metadata_fetcher_spec.rb new file mode 100644 index 0000000..94da2bd --- /dev/null +++ b/backend/spec/services/post_metadata_fetcher_spec.rb @@ -0,0 +1,123 @@ +require 'rails_helper' + +RSpec.describe PostMetadataFetcher do + Response = Struct.new(:body) + + def fetch_with_published_time(value, url: 'https://example.com/video') + html = <<~HTML + + + + HTML + uri = URI.parse(url) + allow(Preview::UrlSafety).to receive(:validate).with(url).and_return([uri, ['8.8.8.8']]) + allow(Preview::HttpFetcher).to receive(:fetch).and_return(Response.new(html)) + allow(Preview::HtmlMetadataExtractor).to receive(:extract) + .and_return(title: 'title', image_url: nil) + + described_class.fetch(url) + end + + it 'builds ranges explicitly for year, month, and day precision' do + year = fetch_with_published_time('2024') + month = fetch_with_published_time('2024-02') + day = fetch_with_published_time('2024-02-03') + + expect(year).to include( + original_created_from: Time.zone.local(2024, 1, 1).iso8601, + original_created_before: Time.zone.local(2025, 1, 1).iso8601 + ) + expect(month).to include( + original_created_from: Time.zone.local(2024, 2, 1).iso8601, + original_created_before: Time.zone.local(2024, 3, 1).iso8601 + ) + expect(day).to include( + original_created_from: Time.zone.local(2024, 2, 3).iso8601, + original_created_before: Time.zone.local(2024, 2, 4).iso8601 + ) + end + + it 'rounds second and fractional-second timestamps down to one-minute ranges' do + result = fetch_with_published_time('2024-02-03T12:34:56.123+02:30') + + expect(Time.iso8601(result.fetch(:original_created_from))) + .to eq(Time.iso8601('2024-02-03T12:34:00+02:30')) + expect( + Time.iso8601(result.fetch(:original_created_before)) - + Time.iso8601(result.fetch(:original_created_from)) + ).to eq(60) + expect(result.fetch(:original_created_from)).not_to include('.') + end + + it 'treats hour-precision timestamps as one-hour ranges with and without offsets' do + offset_result = fetch_with_published_time('2024-02-03T12+02:30') + local_result = fetch_with_published_time('2024-02-03T12') + + expect(Time.iso8601(offset_result.fetch(:original_created_from))) + .to eq(Time.iso8601('2024-02-03T12:00:00+02:30')) + expect( + Time.iso8601(offset_result.fetch(:original_created_before)) - + Time.iso8601(offset_result.fetch(:original_created_from)) + ).to eq(3600) + expect(local_result.fetch(:original_created_from)).to eq('2024-02-03T12:00:00Z') + expect(local_result.fetch(:original_created_before)).to eq('2024-02-03T13:00:00Z') + end + + it 'adds platform tags for known video URLs' do + result = fetch_with_published_time('2024', url: 'https://youtu.be/abc123') + + expect(result.fetch(:tags)).to eq('動画 YouTube') + end + + it 'returns nil dates for an invalid timestamp' do + result = fetch_with_published_time('not-a-time') + + expect(result.fetch(:original_created_from)).to be_nil + expect(result.fetch(:original_created_before)).to be_nil + end + + it 'returns nil dates for an invalid timestamp second value' do + result = fetch_with_published_time('2024-02-03T12:34:99Z') + + expect(result.fetch(:original_created_from)).to be_nil + expect(result.fetch(:original_created_before)).to be_nil + end + + it 'returns nil dates for invalid calendar dates and invalid hour values' do + invalid_day = fetch_with_published_time('2024-02-31T12:00Z') + invalid_leap = fetch_with_published_time('2023-02-29T12:00Z') + invalid_hour = fetch_with_published_time('2024-02-29T24:00Z') + + [invalid_day, invalid_leap, invalid_hour].each do |result| + expect(result.fetch(:original_created_from)).to be_nil + expect(result.fetch(:original_created_before)).to be_nil + end + end + + it 'accepts a valid leap-day timestamp at minute precision' do + result = fetch_with_published_time('2024-02-29T12:34Z') + + expect(result.fetch(:original_created_from)).to eq('2024-02-29T12:34:00Z') + expect(result.fetch(:original_created_before)).to eq('2024-02-29T12:35:00Z') + end + + it 'serialises metadata duration as the same seconds string contract used by forms' do + html = <<~HTML + + + + + HTML + uri = URI.parse('https://example.com/video') + allow(Preview::UrlSafety).to receive(:validate) + .with('https://example.com/video') + .and_return([uri, ['8.8.8.8']]) + allow(Preview::HttpFetcher).to receive(:fetch).and_return(Response.new(html)) + allow(Preview::HtmlMetadataExtractor).to receive(:extract) + .and_return(title: 'title', image_url: nil) + + result = described_class.fetch('https://example.com/video') + + expect(result.fetch(:duration)).to eq('2.5') + end +end diff --git a/backend/spec/services/post_thumbnail_upload_validator_spec.rb b/backend/spec/services/post_thumbnail_upload_validator_spec.rb new file mode 100644 index 0000000..ddf495b --- /dev/null +++ b/backend/spec/services/post_thumbnail_upload_validator_spec.rb @@ -0,0 +1,52 @@ +require 'rails_helper' +require 'base64' +require 'tempfile' + +RSpec.describe PostThumbnailUploadValidator do + def with_upload bytes, content_type:, filename: + tempfile = Tempfile.new(['thumbnail-upload', File.extname(filename)]) + tempfile.binmode + tempfile.write(bytes) + tempfile.rewind + upload = ActionDispatch::Http::UploadedFile.new( + tempfile:, + filename:, + type: content_type) + yield upload + ensure + tempfile&.close! + end + + it 'accepts a raster upload after decoding and rewinds it' do + gif = Base64.decode64('R0lGODdhAQABAIABAP///wAAACwAAAAAAQABAAACAkQBADs=') + + with_upload(gif, content_type: 'image/gif', filename: 'thumbnail.gif') do |upload| + expect { described_class.validate!(upload) }.not_to raise_error + expect(upload.read(6)).to eq('GIF87a') + end + end + + it 'rejects SVG content disguised as a raster MIME type' do + with_upload( + '', + content_type: 'image/png', + filename: 'thumbnail.png') do |upload| + expect { described_class.validate!(upload) } + .to raise_error( + described_class::InvalidUpload, + 'サムネイル画像の形式が不正です．') + end + end + + it 'rejects non-raster content disguised as an image' do + with_upload( + '%PDF-1.7', + content_type: 'image/png', + filename: 'thumbnail.png') do |upload| + expect { described_class.validate!(upload) } + .to raise_error( + described_class::InvalidUpload, + 'サムネイル画像の形式が不正です．') + end + end +end diff --git a/backend/spec/services/post_url_normaliser_spec.rb b/backend/spec/services/post_url_normaliser_spec.rb new file mode 100644 index 0000000..dafcdb2 --- /dev/null +++ b/backend/spec/services/post_url_normaliser_spec.rb @@ -0,0 +1,27 @@ +require 'rails_helper' + +RSpec.describe PostUrlNormaliser do + before do + PostUrlSanitisationRule.unscoped.delete_all + end + + describe '.normalise' do + it 'normalises the host and trailing slash before applying sanitisation rules' do + PostUrlSanitisationRule.create!( + priority: 10, + source_pattern: '\\Ahttps://example\\.com/source\\z', + replacement: 'https://example.com/canonical' + ) + + result = described_class.normalise(' https://EXAMPLE.com/source/ ') + + expect(result).to eq('https://example.com/canonical') + end + + it 'returns nil for unsupported or malformed URLs' do + expect(described_class.normalise('ftp://example.com/file')).to be_nil + expect(described_class.normalise('https://[')).to be_nil + expect(described_class.normalise('https:/path')).to be_nil + end + end +end diff --git a/backend/spec/services/preview/thumbnail_fetcher_spec.rb b/backend/spec/services/preview/thumbnail_fetcher_spec.rb index 3088e0d..1856865 100644 --- a/backend/spec/services/preview/thumbnail_fetcher_spec.rb +++ b/backend/spec/services/preview/thumbnail_fetcher_spec.rb @@ -2,7 +2,7 @@ require 'rails_helper' RSpec.describe Preview::ThumbnailFetcher do describe '.fetch' do - it 'rejects svg thumbnails' do + it 'accepts svg thumbnails for the common safe rasterisation path' do page = Preview::HttpFetcher::Response.new( '', 'text/html', @@ -12,8 +12,9 @@ RSpec.describe Preview::ThumbnailFetcher do 'image/svg+xml', 'https://example.com/thumb.svg') - allow(Preview::UrlSafety).to receive(:validate) - .and_return([URI.parse('https://example.com/page'), ['203.0.113.10']]) + allow(Preview::UrlSafety).to receive(:validate) do |url| + [URI.parse(url), ['203.0.113.10']] + end allow(Preview::HttpFetcher).to receive(:fetch) .with('https://example.com/page', max_bytes: described_class::HTML_MAX_BYTES) .and_return(page) @@ -21,9 +22,7 @@ RSpec.describe Preview::ThumbnailFetcher do .with('https://example.com/thumb.svg') .and_return(svg) - expect { - described_class.fetch('https://example.com/page') - }.to raise_error(Preview::ThumbnailFetcher::GenerationFailed) + expect(described_class.fetch('https://example.com/page')).to eq('') end it 'accepts allowed image content type with parameters' do @@ -32,12 +31,13 @@ RSpec.describe Preview::ThumbnailFetcher do 'text/html', 'https://example.com/page') image = Preview::HttpFetcher::Response.new( - 'jpeg-bytes', + "\xFF\xD8\xFFjpeg-bytes".b, 'image/jpeg; charset=binary', 'https://example.com/thumb.jpg') - allow(Preview::UrlSafety).to receive(:validate) - .and_return([URI.parse('https://example.com/page'), ['203.0.113.10']]) + allow(Preview::UrlSafety).to receive(:validate) do |url| + [URI.parse(url), ['203.0.113.10']] + end allow(Preview::HttpFetcher).to receive(:fetch) .with('https://example.com/page', max_bytes: described_class::HTML_MAX_BYTES) .and_return(page) @@ -45,7 +45,60 @@ RSpec.describe Preview::ThumbnailFetcher do .with('https://example.com/thumb.jpg') .and_return(image) - expect(described_class.fetch('https://example.com/page')).to eq('jpeg-bytes') + expect(described_class.fetch('https://example.com/page')) + .to eq("\xFF\xD8\xFFjpeg-bytes".b) + end + end + + describe '.fetch_image_response' do + it 'rejects an unsafe input URL before HTTP fetch' do + allow(Preview::UrlSafety).to receive(:validate) + .and_raise(Preview::UrlSafety::UnsafeUrl, '安全でない接続先は使用できません．') + expect(Preview::HttpFetcher).not_to receive(:fetch) + + expect { + described_class.fetch_image_response('https://unsafe.example.com/thumb.jpg') + }.to raise_error(Preview::UrlSafety::UnsafeUrl, '安全でない接続先は使用できません．') + end + + it 'rejects an unsafe redirect target' do + allow(Preview::UrlSafety).to receive(:validate) + .and_return([URI.parse('https://example.com/thumb.jpg'), ['8.8.8.8']]) + allow(Preview::HttpFetcher).to receive(:fetch) + .with('https://example.com/thumb.jpg') + .and_raise(Preview::UrlSafety::UnsafeUrl, '安全でない接続先は使用できません．') + + expect { + described_class.fetch_image_response('https://example.com/thumb.jpg') + }.to raise_error(Preview::UrlSafety::UnsafeUrl, '安全でない接続先は使用できません．') + end + + it 'rejects an oversized image response' do + allow(Preview::UrlSafety).to receive(:validate) + .and_return([URI.parse('https://example.com/thumb.jpg'), ['8.8.8.8']]) + allow(Preview::HttpFetcher).to receive(:fetch) + .with('https://example.com/thumb.jpg') + .and_raise(Preview::HttpFetcher::ResponseTooLarge, 'too large') + + expect { + described_class.fetch_image_response('https://example.com/thumb.jpg') + }.to raise_error(Preview::HttpFetcher::ResponseTooLarge) + end + + it 'rejects a non-image content type' do + allow(Preview::UrlSafety).to receive(:validate) + .and_return([URI.parse('https://example.com/thumb.jpg'), ['8.8.8.8']]) + allow(Preview::HttpFetcher).to receive(:fetch) + .with('https://example.com/thumb.jpg') + .and_return( + Preview::HttpFetcher::Response.new( + '', + 'text/html', + 'https://example.com/thumb.jpg')) + + expect { + described_class.fetch_image_response('https://example.com/thumb.jpg') + }.to raise_error(Preview::ThumbnailFetcher::GenerationFailed) end end end diff --git a/backend/spec/services/youtube/sync_spec.rb b/backend/spec/services/youtube/sync_spec.rb index df8009a..9e8f2c0 100644 --- a/backend/spec/services/youtube/sync_spec.rb +++ b/backend/spec/services/youtube/sync_spec.rb @@ -10,6 +10,37 @@ RSpec.describe Youtube::Sync do allow(sync).to receive(:attach_thumbnail_if_needed!) end + describe '#attach_thumbnail_if_needed!' do + it 'uses the common remote thumbnail attach path' do + post = create(:post, thumbnail_base: nil) + allow(sync).to receive(:attach_thumbnail_if_needed!).and_call_original + + expect(post).to receive(:attach_thumbnail_from_url!) + .with('https://example.com/thumb.jpg') + + sync.send( + :attach_thumbnail_if_needed!, + post, + 'https://example.com/thumb.jpg') + end + + it 'retries on a later sync when the previous remote attach failed' do + post = create(:post, thumbnail_base: nil) + allow(sync).to receive(:attach_thumbnail_if_needed!).and_call_original + expect(post).to receive(:attach_thumbnail_from_url!) + .with('https://example.com/thumb.jpg') + .twice + .and_raise(Post::RemoteThumbnailFetchFailed, 'failed') + + 2.times do + sync.send( + :attach_thumbnail_if_needed!, + post, + 'https://example.com/thumb.jpg') + end + end + end + describe '#sync!' do it 'returns without fetching video details when no video ids are discovered' do allow(sync).to receive(:query_terms).and_return([]) @@ -76,7 +107,7 @@ RSpec.describe Youtube::Sync do sync.sync! end - it 'creates a YouTube post with default tags and no_deerjikist when no deerjikist mapping exists' do + it 'creates a YouTube post with default tags when no deerjikist mapping exists' do Tag.tagme Tag.bot Tag.youtube diff --git a/backend/spec/tasks/nico_sync_spec.rb b/backend/spec/tasks/nico_sync_spec.rb index 87b27d7..75e531f 100644 --- a/backend/spec/tasks/nico_sync_spec.rb +++ b/backend/spec/tasks/nico_sync_spec.rb @@ -1,10 +1,10 @@ -require "rails_helper" +require 'rails_helper' -RSpec.describe "nico:sync" do +RSpec.describe 'nico:sync' do def stub_python(json_array) status = instance_double(Process::Status, success?: true) - allow(Open3).to receive(:capture3).and_return([json_array.to_json, "", status]) + allow(Open3).to receive(:capture3).and_return([json_array.to_json, '', status]) end def create_tag!(name, category:) @@ -16,17 +16,21 @@ RSpec.describe "nico:sync" do NicoTagRelation.create!(nico_tag_id: nico_tag.id, tag_id: tag.id) end - it "既存 post を見つけて、nico tag と linked tag を追加し、差分が出たら bot を付ける" do + it '既存 post を見つけて、nico tag と linked tag を追加し、差分が出たら bot を付ける' do # 既存 post（正規表現で拾われるURL） - post = Post.create!(title: "old", url: "https://www.nicovideo.jp/watch/sm9", uploaded_user: nil) + post = Post.create!( + title: 'old', + url: 'https://www.nicovideo.jp/watch/sm9', + uploaded_user: nil + ) # 既存の非nicoタグ（kept_non_nico_ids） - kept_general = create_tag!("spec_kept", category: "general") + kept_general = create_tag!('spec_kept', category: 'general') PostTag.create!(post: post, tag: kept_general) # 追加される linked tag を準備（nico tag に紐付く一般タグ） - linked = create_tag!("spec_linked", category: "general") - nico = create_tag!("nico:AAA", category: "nico") + linked = create_tag!('spec_linked', category: 'general') + nico = create_tag!('nico:AAA', category: 'nico') link_nico_to_tag!(nico, linked) # bot / tagme は task 内で使うので作っておく（Tag.bot/tagme がある前提） @@ -42,43 +46,91 @@ RSpec.describe "nico:sync" do 'deleted_at' => '2026-01-31 00:00:00' }]) # 外部HTTPは今回「既存 post なので呼ばれない」はずだが、念のため塞ぐ - allow(URI).to receive(:open).and_return(StringIO.new("")) + allow(URI).to receive(:open).and_return(StringIO.new('')) - run_rake_task("nico:sync") + run_rake_task('nico:sync') post.reload - active_tag_names = post.tags.joins(:tag_name).pluck("tag_names.name") + active_tag_names = post.tags.joins(:tag_name).pluck('tag_names.name') - expect(active_tag_names).to include("spec_kept") - expect(active_tag_names).to include("nico:AAA") - expect(active_tag_names).to include("spec_linked") + expect(active_tag_names).to include('spec_kept') + expect(active_tag_names).to include('nico:AAA') + expect(active_tag_names).to include('spec_linked') expect(post.original_created_from).to eq(Time.iso8601('2026-01-01T03:34:00Z')) expect(post.original_created_before).to eq(Time.iso8601('2026-01-01T03:35:00Z')) # 差分が出るので bot が付く（kept_non_nico_ids != desired_non_nico_ids） - expect(active_tag_names).to include("bot操作") + expect(active_tag_names).to include('bot操作') end - it "既存 post にあった古い nico tag は active から外され、履歴として discard される" do - post = Post.create!(title: "old", url: "https://www.nicovideo.jp/watch/sm9", uploaded_user: nil) + it '既存 post のサムネール取得に共通 attach 経路を使ふ' do + post = Post.create!( + title: 'old', + url: 'https://www.nicovideo.jp/watch/sm9', + uploaded_user: nil) + Tag.bot + Tag.tagme + + stub_python([{ 'code' => 'sm9', 'title' => 't', 'tags' => [] }]) + allow(URI).to receive(:open) do + StringIO.new( + '') + end + expect_any_instance_of(Post).to receive(:attach_thumbnail_from_url!) + .with('https://example.com/thumb.jpg') + + run_rake_task('nico:sync') + end + + it 'サムネール取得失敗後も次回同期で再試行できる' do + post = Post.create!( + title: 'old', + url: 'https://www.nicovideo.jp/watch/sm9', + uploaded_user: nil) + Tag.bot + Tag.tagme + + stub_python([{ 'code' => 'sm9', 'title' => 't', 'tags' => [] }]) + allow(URI).to receive(:open) do + StringIO.new( + '') + end + calls = 0 + allow_any_instance_of(Post).to receive(:attach_thumbnail_from_url!) do + calls += 1 + raise Post::RemoteThumbnailFetchFailed, 'failed' + end + + 2.times do + run_rake_task('nico:sync') + end + expect(calls).to eq(2) + end + + it '既存 post にあった古い nico tag は active から外され、履歴として discard される' do + post = Post.create!( + title: 'old', + url: 'https://www.nicovideo.jp/watch/sm9', + uploaded_user: nil + ) # 旧nicoタグ（今回の同期結果に含まれない） - old_nico = create_tag!("nico:OLD", category: "nico") + old_nico = create_tag!('nico:OLD', category: 'nico') old_pt = PostTag.create!(post: post, tag: old_nico) expect(old_pt.discarded_at).to be_nil # 今回は NEW のみ欲しい - new_nico = create_tag!("nico:NEW", category: "nico") + new_nico = create_tag!('nico:NEW', category: 'nico') # bot/tagme 念のため Tag.bot Tag.tagme - stub_python([{ "code" => "sm9", "title" => "t", "tags" => ["NEW"] }]) - allow(URI).to receive(:open).and_return(StringIO.new("")) + stub_python([{ 'code' => 'sm9', 'title' => 't', 'tags' => ['NEW'] }]) + allow(URI).to receive(:open).and_return(StringIO.new('')) - run_rake_task("nico:sync") + run_rake_task('nico:sync') # OLD は active から外れる（discarded_at が入る） old_pts = PostTag.where(post_id: post.id, tag_id: old_nico.id).order(:id).to_a @@ -86,9 +138,9 @@ RSpec.describe "nico:sync" do # NEW は active にいる post.reload - active_names = post.tags.joins(:tag_name).pluck("tag_names.name") - expect(active_names).to include("nico:NEW") - expect(active_names).not_to include("nico:OLD") + active_names = post.tags.joins(:tag_name).pluck('tag_names.name') + expect(active_names).to include('nico:NEW') + expect(active_names).not_to include('nico:OLD') end def snapshot_tags(post) diff --git a/frontend/AGENTS.md b/frontend/AGENTS.md index cddc89b..ae68412 100644 --- a/frontend/AGENTS.md +++ b/frontend/AGENTS.md @@ -50,16 +50,27 @@ pass or the remaining failure is clearly blocked. - Prefer single quotes for strings unless interpolation or escaping makes double quotes better. - Never write a TypeScript or TSX line longer than 99 characters. - Aim to keep TypeScript and TSX lines within 79 characters where practical. -- Use 4-space logical indentation in TypeScript and TSX. +- Use 2-space block indentation in TypeScript and TSX. +- Use 4-space continuation indentation for wrapped expressions, arguments, + ternary branches, method chains, object pairs, arrays, and JSX attributes. +- Treat the user's `PostImportSourcePage.tsx` and + `PostImportReviewPage.tsx` formatting as the local reference shape. - For arrays, never put whitespace or a line break immediately before `]`. - Keep the first element on the same line as `[` by default. - If an array would exceed the line limit, break after `[` and indent elements by 4 spaces. -- In TypeScript and TSX only, replace every leading run of 8 spaces with a tab - to reduce bytes. +- In TypeScript and TSX only, use tabs for leading 8-column compression only. +- A tab does not represent one indentation level. +- Determine visible indentation with 2-space block indentation and 4-space + continuation indentation first, then compress only complete leading runs of + 8 spaces into tabs. - Treat one leading tab as exactly equivalent to 8 leading spaces. - Use tabs only for leading indentation. Never replace spaces that occur after a non-space character on the same line. +- Keep residual leading 2, 4, or 6 spaces after any tab compression. +- Examples: 2 columns = 2 spaces, 4 columns = 4 spaces, 6 columns = 6 + spaces, 8 columns = 1 tab, 10 columns = 1 tab + 2 spaces, 12 columns = 1 + tab + 4 spaces. ## React @@ -97,17 +108,35 @@ pass or the remaining failure is clearly blocked. third-party request outside the Rails API. - For blob responses, pass `responseType: 'blob'` so the wrapper does not camelCase the body. +## Dialogues + +- Dialogue work follows the shared-frontend reuse rules below. + ## Imports and aliases - The `@` alias points to `frontend/src`. - Prefer `@/...` imports for app code instead of long relative paths. - Keep type imports separate with `import type`. -- Match existing import grouping: external packages, app modules, then type imports. +- Do not mix runtime values and `type` specifiers in one named import + declaration. +- Do not write `import { value, type TypeName } from ...`. +- Keep short value imports from one module on one line when they fit within + 99 characters. +- Order imports as four groups with a blank line between groups: external + value imports, `@/...` value imports, external type imports, `@/...` + type imports. ## Tailwind and UI - Tailwind scans `src/**/*.{html,js,ts,jsx,tsx,mdx}`. - Use `cn` from `src/lib/utils.ts` for conditional class names and class merging. +- In JavaScript, JSX, TypeScript, and TSX, use `cn` from `@/lib/utils` + whenever `className` combines multiple values, conditional classes, or a + caller-provided `className` prop. +- Do not construct `className` with template literals, `${ ... }`, string + concatenation, arrays joined with spaces, or feature-local class-merging + helpers. +- A static `className="..."` containing only fixed classes does not need `cn`. - Reuse components from `src/components/common`, `src/components/layout`, and `src/components/ui` before adding new primitives. - Keep Tailwind classes consistent with nearby components. @@ -118,6 +147,15 @@ pass or the remaining failure is clearly blocked. short Japanese labels that fit the control. - Preserve existing Japanese tone and orthography in nearby UI text, including old-kana wording where the file already uses it. +- Do not add user-facing copy, helper text, descriptions, notes, tooltips, + placeholders, empty-state messages, loading messages, or explanatory text + unless the user explicitly specified the wording. +- When new user-facing wording appears necessary, ask the user for the exact + wording and placement before implementing it. +- Do not invent replacement copy when removing unrequested wording. +- Do not use `タグなし` as user-facing copy for an empty tag state. When the + tag state is empty, show no copy. If actual data contains the tag name + `タグなし`, treat it as ordinary data and display it normally. - When adding dynamic tag colour classes, update `tailwind.config.js` safelist if the class cannot be statically detected. - Do not introduce new UI libraries or production dependencies without approval. @@ -129,6 +167,41 @@ pass or the remaining failure is clearly blocked. it is JSX- or React-specific. - Preserve compact TSX expression shapes such as inline ternary branches and closing `)` forms when nearby code uses them. +- Block bodies for components, functions, callbacks, `if`, `try`, `catch`, + `finally`, loops, and JSX nesting use 2 spaces per level. +- Put the opening brace of `try`, `catch`, and `finally` blocks on the next + line at the same indentation as the keyword. +- Do not indent the opening `{` one level deeper than `try`, `catch`, or + `finally`. +- Indent the block body 2 spaces deeper than the keyword and opening brace. +- Put the closing `}` on its own line at the same indentation as the keyword. +- Do not write `try {`, `catch {`, or `finally {`. +- Wrapped expressions, arguments, ternary branches, method chains, and object + pairs use 4-space continuation indentation relative to the owning + expression. Do not confuse this with 2-space block indentation. +- Tabs are leading 8-column compression only. They do not represent one + nesting level. Decide visible indentation first, then compress only + complete leading runs of 8 spaces into tabs. +- Do not add braces around a single-line `if` body merely for formatting. +- Use braces for multi-line `if`, `else`, and loop bodies. +- Multi-stage ternary expressions must use explicit parentheses for each + condition group and nested branch. Do not rely on indentation alone to show + `?` / `:` pairing. +- Keep short inline props types local when they remain readable and within the + line limit; do not mechanically extract a named type with no reuse benefit. +- In JavaScript, JSX, TypeScript, and TSX, never use `_1`, `_2`, or similar + Ruby-style numbered parameter names. Reserve numbered parameters for Ruby. + Use a meaningful callback parameter name such as `row`, `item`, `value`, + `entry`, or `result`. +- In multi-line object literals, keep the opening `{` with the first pair when + the line length allows it; do not mechanically explode short objects into + Prettier-style vertical blocks. +- Method chains should align as a continuation under the receiver expression; + do not indent chains more deeply than the normal continuation depth. +- `PostImportSourcePage.tsx` and `PostImportReviewPage.tsx` are the current + canonical examples for block indentation, continuation indentation, import + grouping, ternary grouping, method-chain placement, and local inline props + types. - Treat TypeScript and TSX formatting rules as hard constraints, not preferences. Before finishing a TypeScript or TSX edit, inspect the edited hunks for closing `)`, `]`, and `}` placement and fix violations instead of @@ -158,14 +231,93 @@ pass or the remaining failure is clearly blocked. beginning of a line. - The TSX-specific self-review must confirm JSX closing markers and closing parentheses keep the surrounding compact style. -- The TypeScript/TSX self-review must confirm leading indentation follows - 4-space logical indentation with tabs only as leading 8-space compression. +- The TypeScript/TSX self-review must confirm leading block indentation uses + 2 spaces per level, wrapped continuations use the repository's 4-space + continuation alignment, and complete leading runs of 8 spaces may be + compressed to tabs. - For long Tailwind `className` strings, wrap across lines only when needed. -- Keep continuation indentation aligned with the 4-space logical indentation - rule, using tabs only as leading 8-space compression. +- Keep continuation indentation aligned with the repository's 4-space + continuation rule while keeping block indentation at 2 spaces. +- Keep short value imports from one module on one line when they fit within + 99 characters. - In TypeScript and TSX function declarations, including `const` arrow function declarations, classify the parameter list before placing the closing `)`. +- Block indentation example: + +```ts +const Component = () => { + const value = loadValue () + + useEffect (() => { + if (value != null) + useValue (value) + }, [value]) +} +``` + +- `try` / `catch` / `finally` brace placement example: + +```ts +try +{ + doWork () +} +catch +{ + recover () +} +finally +{ + cleanUp () +} +``` + +- Continuation indentation example: + +```ts +const editingRow = + Number.isFinite (editingSourceRow) + ? rows.find (row => row.sourceRow === editingSourceRow) ?? null + : null +``` + +- Import grouping example: + +```ts +import { useNavigate, useParams, useSearchParams } from 'react-router-dom' + +import { loadPostImportSession } from '@/lib/postImportSession' + +import type { FC } from 'react' + +import type { PostImportRow } from '@/lib/postImportSession' +``` + +- Inline props type example: + +```ts +const Footer = ( + { loading, + onSubmit }: { loading: boolean + onSubmit: () => void }, +) => null +``` + +- Ternary grouping example: + +```ts +const rows = + repairMode === 'failed' + ? ( + [...source].sort ((a, b) => { + const aFailed = a.failed ? 0 : 1 + const bFailed = b.failed ? 0 : 1 + return aFailed - bFailed + })) + : source +``` + - If the parameter list itself is given its own multi-line block after the function's opening `(`, put the closing parameter `)` at the beginning of its own line before the return type or `=>`. @@ -219,6 +371,243 @@ pass or the remaining failure is clearly blocked. `BehaviorSettingsSection.tsx`. - Avoid reformatting unrelated JSX. +## Shared frontend systems + +Before creating a new component, hook, helper, store, context, or other +frontend abstraction, search at least: + +- `src/components/common` +- `src/components/layout` +- `src/components/ui` +- `src/components/dialogues` +- `src/lib` +- `src/lib/dialogues` +- `src/stores` +- `src/types.ts` + +Also inspect the existing pages and components in the same feature. + +Search by responsibility, not by filename alone. Check display, interaction, +state, communication, validation, and permission behaviour before deciding that +an existing implementation is unsuitable. + +### Component placement and reuse order + +When adding UI, use this order: + +1. reuse an existing feature component +2. reuse an existing component from `components/common` +3. reuse an existing layout component from `components/layout` +4. use an existing primitive from `components/ui` through the established + common API +5. extend an existing component minimally +6. add a feature-local component in the feature area +7. add a new common component only when multiple features clearly share a + stable visual contract + +Do not place a one-screen component in a common directory merely because its +name starts with `Common`. + +### Low-level primitives + +Treat `components/ui` as low-level primitives. If a higher-level common API +already exists for dialogues, toast, form validation, navigation, or similar +behaviour, feature code must use that API instead of assembling primitives +directly. + +Examples of existing preferred entrypoints include: + +- dialogue: `@/lib/dialogues/useDialogue` +- toast: the existing toast API +- internal navigation: `PrefetchLink` +- form errors: `FieldError`, `FieldWarning`, `FormField` +- buttons: `Button` +- conditional class merge: `cn` + +Do not evade the rule with aliases or thin wrappers around the low-level +primitive. + +### Dialogues + +Feature-facing dialogue work must use `@/lib/dialogues/useDialogue`. + +Reuse the existing common dialogue API and common dialogue component. Do not +import `@/components/ui/dialog` directly in feature code to assemble bespoke +dialogue shells, and do not evade this rule with aliases such as +`Dialog as Dialogue`. + +Do not reimplement overlay, portal, close button, header, footer, focus +handling, Escape handling, outside-click handling, or confirmation flow in +feature code. + +Keep business-specific form content in feature code, and keep the visual and +behavioural dialogue shell in common code. + +Use British spelling `Dialogue` for project-defined dialogue identifiers. Keep +exact third-party spellings only at the external boundary where compatibility +requires them. + +### API calls + +Rails API calls must use `src/lib/api.ts`. + +Do not create feature-local Axios instances, fetch wrappers, header injectors, +camelCase converters, or generic error converters. If blob or other special +transport behaviour is already supported by the common API, use the existing +options instead of bypassing the wrapper. + +### Query keys, server state, and prefetch + +Before adding query state, inspect: + +- `src/lib/queryKeys.ts` +- existing domain helpers +- existing prefetchers +- the root query-key hierarchy +- current mutation invalidation patterns +- the app-wide `QueryClient` + +Do not write ad hoc query-key arrays in feature code. Do not duplicate fetcher, +prefetcher, or invalidation helpers for the same resource. + +### Domain helpers + +For posts, tags, wiki, materials, and other domain work, inspect the existing +helpers in `src/lib/*.ts` before adding logic to a page component. + +Do not accumulate these in page components when an existing helper layer should +own them: + +- API request construction +- response-shape conversion +- query-key construction +- canonical URL generation +- permission calculation +- storage serialisation +- domain-specific parsing + +Keep purely local one-screen display shaping local when that is the clearest +place for it. + +### Permission helpers + +Use the existing permission helpers such as `src/lib/users.ts` when deciding +editability, role checks, admin/member visibility, and similar UI behaviour. + +Do not scatter `user?.role`, numeric role comparisons, or string comparisons +through components. Frontend visibility control should be consistent even though +backend authorization remains the final gate. + +### Validation errors + +Before adding feature-local validation-error handling, inspect: + +- `useValidationErrors` +- `apiErrors` +- `FieldError` +- `FieldWarning` +- `FormField` +- `inputClass` + +Do not create a new generic hook, field-error state shape, or error-rendering +component for a pattern the shared error stack already covers. Keep only +genuinely feature-specific business errors local. + +### Forms and fields + +Before creating a new input, textarea, date/time field, tag input, label, or +error layout, inspect at least: + +- `Form` +- `FormField` +- `FieldError` +- `FieldWarning` +- `DateTimeField` +- `TagInput` +- `TextArea` +- `Label` +- `Button` + +Do not create a same-function field component merely because the spacing or +surface styling is slightly different. Prefer feature-level composition over +bloated common-field option lists. + +### Navigation and prefetch + +Use `PrefetchLink` and existing router helpers for internal navigation. Do not +introduce feature-local ``, `window.location`, or custom prefetch logic for +internal routes. Keep path-segment encoding aligned with the existing rules. + +### State management + +Before adding state, decide whether the source of truth should be: + +- component-local state +- URL search params +- TanStack Query server state +- an existing Zustand store +- an existing event bus +- an existing storage helper + +Do not create a new global store, context, or event bus for one screen when +local state or an existing mechanism is enough. Do not create a second store +for the same responsibility. + +### Storage and settings + +When touching localStorage, sessionStorage, or user settings, inspect existing +settings helpers, storage helpers, expiry handling, versioning, and sanitisers. + +Do not reimplement per-component key naming, JSON parsing and serialisation, +expiry, or schema checks when a shared helper already owns the pattern. + +### Hooks + +Before creating a custom hook, search existing `src/lib/use*.ts` and +`src/lib/use*.tsx`. + +Hooks are for shared stateful behaviour or React lifecycle integration. Do not +turn a pure function, one-off helper, or mere re-export shim into `useFoo`. + +### Stores, contexts, and event buses + +Add a new store, context, or event bus only when the current mechanisms cannot +express the requirement and there are multiple genuinely separate consumers. + +Do not hold the same information redundantly across URL state, query cache, +local component state, Zustand, and an event bus. Keep one source of truth. + +### Types + +If a domain type already exists in `src/types.ts` or a domain helper, reuse it +instead of redefining the same shape in a feature file. + +Small local props and draft types may stay local. Do not create giant +catch-all type files such as `CommonTypes.ts`. + +### Styling utilities + +Use existing styling utilities such as `cn` and `inputClass`. + +Do not add feature-local class-merge helpers, generic status-colour mappers, or +responsive wrapper helpers when a shared utility already exists. Keep common +tone names visual only; feature-specific state names stay in feature code. + +### Layout + +Before adding page shells, padding rules, viewport-height handling, sidebar +offsets, or footer offsets, inspect existing layout components such as +`MainArea`, top navigation, sidebar, page title, and section-title patterns. + +Do not create a second layout shell before checking whether the current layout +can be reused or minimally extended. + +- Frontend のスマホ／PC表示境界は原則 `md` とする。 +- button stack、footer action、dialogue action は `md` 未満で縦並び、 + `md` 以上で横並びとする。 +- 同じ画面内で `sm` と `md` を混在させて中間 layout を作らない。 +- 明確に別の responsive 要件がある component だけを例外とする。 + ### Delimiter decision table Use this table before accepting any edited TypeScript or TSX hunk. The table is @@ -563,8 +952,9 @@ hunks line by line: 7. JSX `>` and `/>` stay with the final prop unless nearby code proves otherwise. 8. JSX closing parentheses keep the compact local style. -9. Leading indentation is 4-space logical indentation with tabs used only as - leading 8-space compression. +9. Leading block indentation uses 2 spaces per level, wrapped continuations + use the repository's 4-space continuation alignment, and complete leading + runs of 8 spaces may be compressed to tabs. 10. No line has trailing whitespace. ## Lint and build constraints diff --git a/frontend/src/App.tsx b/frontend/src/App.tsx index c2aa89b..c6dae14 100644 --- a/frontend/src/App.tsx +++ b/frontend/src/App.tsx @@ -1,8 +1,9 @@ import { AnimatePresence, LayoutGroup, MotionConfig, motion } from 'framer-motion' import { Fragment, useEffect, useMemo, useState } from 'react' -import { BrowserRouter, +import { createBrowserRouter, Navigate, Route, + RouterProvider, Routes, useLocation } from 'react-router-dom' @@ -41,8 +42,8 @@ import NotFound from '@/pages/NotFound' import TOSPage from '@/pages/TOSPage.mdx' import PostDetailPage from '@/pages/posts/PostDetailPage' import PostHistoryPage from '@/pages/posts/PostHistoryPage' -import PostListPage from '@/pages/posts/PostListPage' import PostNewPage from '@/pages/posts/PostNewPage' +import PostListPage from '@/pages/posts/PostListPage' import PostSearchPage from '@/pages/posts/PostSearchPage' import ServiceUnavailable from '@/pages/ServiceUnavailable' import SettingPage from '@/pages/users/SettingPage' @@ -154,7 +155,7 @@ const PostDetailRoute = ({ user }: { user: User | null }) => { } -const App: FC = () => { +const RoutedApp: FC = () => { const [user, setUser] = useState (null) const [status, setStatus] = useState (200) const behaviourSettings = useClientBehaviourSettings () @@ -253,42 +254,46 @@ const App: FC = () => { } return ( - <> - - {import.meta.env.DEV && } + + + + + + + + + + + - - - - - - - - - - - - - - - - - - - ) + + + + ) } +const router = createBrowserRouter ([{ + path: '*', + element: }]) + +const App: FC = () => ( + <> + + {import.meta.env.DEV && } + + ) + export default App diff --git a/frontend/src/components/PostEditForm.test.tsx b/frontend/src/components/PostEditForm.test.tsx index 2ff28b1..349c753 100644 --- a/frontend/src/components/PostEditForm.test.tsx +++ b/frontend/src/components/PostEditForm.test.tsx @@ -19,8 +19,8 @@ const toastApi = vi.hoisted (() => ({ vi.mock ('@/lib/posts', () => postsApi) vi.mock ('@/lib/api', () => api) vi.mock ('@/components/ui/use-toast', () => toastApi) -vi.mock ('@/components/dialogues/DialogueProvider', () => ({ - useDialogue: () => ({ +vi.mock ('@/lib/dialogues/useDialogue', () => ({ + default: () => ({ choice: vi.fn (), }), })) @@ -78,15 +78,43 @@ describe ('PostEditForm', () => { render () - expect (screen.getByRole ('spinbutton')).toHaveValue (180.5) + expect (screen.getByText ('動画時間').parentElement?.querySelector ('input')) + .toHaveValue ('180.5') const tags = screen.getAllByRole ('textbox')[2] fireEvent.change (tags, { target: { value: 'general-tag' } }) - expect (screen.queryByRole ('spinbutton')).not.toBeInTheDocument () + expect (screen.queryByText ('動画時間')).not.toBeInTheDocument () fireEvent.change (tags, { target: { value: '動画 general-tag' }, }) - expect (screen.getByRole ('spinbutton')).toHaveValue (180.5) + expect (screen.getByText ('動画時間').parentElement?.querySelector ('input')) + .toHaveValue ('180.5') }) + + it ( + 'shows deduplicated original-created endpoint errors on the shared datetime field', + async () => { + const post = buildPost () + api.isApiError.mockReturnValue (true) + postsApi.updatePost.mockRejectedValueOnce ({ + response: { + status: 422, + data: { + type: 'validation_error', + errors: { + original_created_at: ['日時を確認してください．'], + original_created_from: ['日時を確認してください．'], + original_created_before: ['終了を確認してください．'] }, + }, + }, + }) + + render () + fireEvent.submit (screen.getByRole ('button', { name: '更新' }).closest ('form')!) + + expect (await screen.findByText ('日時を確認してください．')).toBeInTheDocument () + expect (screen.getByText ('終了を確認してください．')).toBeInTheDocument () + expect (screen.getAllByText ('日時を確認してください．')).toHaveLength (1) + }) }) diff --git a/frontend/src/components/PostEditForm.tsx b/frontend/src/components/PostEditForm.tsx index 744c132..6d3fe3f 100644 --- a/frontend/src/components/PostEditForm.tsx +++ b/frontend/src/components/PostEditForm.tsx @@ -1,15 +1,16 @@ import { useEffect, useMemo, useState } from 'react' -import PostFormTagsArea from '@/components/PostFormTagsArea' import PostOriginalCreatedTimeField from '@/components/PostOriginalCreatedTimeField' import FieldError from '@/components/common/FieldError' -import FormField from '@/components/common/FormField' -import { useDialogue } from '@/components/dialogues/DialogueProvider' +import PostDurationField from '@/components/posts/PostDurationField' +import PostTagsField from '@/components/posts/PostTagsField' +import PostTextField from '@/components/posts/PostTextField' import { Button } from '@/components/ui/button' import { toast } from '@/components/ui/use-toast' import { isApiError } from '@/lib/api' +import useDialogue from '@/lib/dialogues/useDialogue' import { updatePost } from '@/lib/posts' -import { inputClass, msToTime } from '@/lib/utils' +import { msToTime } from '@/lib/utils' import { useValidationErrors } from '@/lib/useValidationErrors' import type { FC, FormEvent } from 'react' @@ -17,7 +18,11 @@ import type { FC, FormEvent } from 'react' import type { Post, TagWithSections } from '@/types' type PostFormField = - 'parentPostIds' | 'tags' | 'videoMs' | 'originalCreatedAt' + 'parentPostIds' | 'tags' | 'videoMs' + | 'originalCreatedAt' | 'originalCreatedFrom' | 'originalCreatedBefore' + +const groupedMessages = (...values: (string[] | undefined)[]): string[] => + [...new Set (values.flatMap (value => value ?? []))] const videoMsToDurationValue = (videoMs: number | null): string => videoMs == null ? '' : String (videoMs / 1_000) @@ -157,32 +162,20 @@ const PostEditForm: FC = ({ post, onSave }) => {
- {/* タイトル */} - - {({ invalid }) => ( - setTitle (e.target.value)}/>)} - + - {/* 親投稿 */} - - {({ describedBy, invalid }) => ( - setParentPostIds (e.target.value)} - aria-describedby={describedBy} - aria-invalid={invalid} - className={inputClass (invalid)}/>)} - + - {/* タグ */} - = ({ post, onSave }) => { setOriginalCreatedFrom={setOriginalCreatedFrom} originalCreatedBefore={originalCreatedBefore} setOriginalCreatedBefore={setOriginalCreatedBefore} - errors={fieldErrors.originalCreatedAt}/> + errors={groupedMessages ( + fieldErrors.originalCreatedAt, + fieldErrors.originalCreatedFrom, + fieldErrors.originalCreatedBefore)}/> - {/* 動画時間 */} {videoFlg && ( - - {({ invalid }) => ( - setDuration (e.target.value)}/>)} - )} + )} {/* 送信 */} - - + const d = new Date (v) + if (d.getMinutes () === 0 && d.getHours () === 0) + d.setDate (d.getDate () + 1) + else + d.setMinutes (d.getMinutes () + 1) + setOriginalCreatedBefore (toMinutePrecisionIsoUtc (d.toISOString ())) + }}/> + 以降 + +
+ +
+ -
-
- - より前 -
-
- -
-
- )} - ) +
+
+ + より前 +
+
+ +
+
+ )} + ) +} export default PostOriginalCreatedTimeField diff --git a/frontend/src/components/PrefetchLink.tsx b/frontend/src/components/PrefetchLink.tsx index 3651b5b..be533a4 100644 --- a/frontend/src/components/PrefetchLink.tsx +++ b/frontend/src/components/PrefetchLink.tsx @@ -6,7 +6,6 @@ import { createPath, useNavigate } from 'react-router-dom' import { useOverlayStore } from '@/components/RouteBlockerOverlay' import { prefetchForURL } from '@/lib/prefetchers' import { useClientBehaviourSettings } from '@/lib/useClientBehaviourSettings' -import { useUnsavedChangesGuard } from '@/lib/useUnsavedChangesGuard' import { cn } from '@/lib/utils' import type { AnchorHTMLAttributes, MouseEvent, TouchEvent } from 'react' @@ -36,7 +35,6 @@ export default forwardRef (({ const navigate = useNavigate () const qc = useQueryClient () const behaviourSettings = useClientBehaviourSettings () - const { confirmDiscardNavigation } = useUnsavedChangesGuard () const linkPreloadMode = behaviourSettings.linkPreload ?? 'intent' const path = useMemo ( () => typeof to === 'string' ? to : createPath (to), @@ -45,10 +43,6 @@ export default forwardRef (({ const url = useMemo (() => { return (new URL (path, window.location.origin)).toString () }, [path]) - const nextPathname = useMemo ( - () => (new URL (path, window.location.origin)).pathname, - [path], - ) const setOverlay = useOverlayStore (s => s.setActive) const doPrefetch = async () => { @@ -93,13 +87,6 @@ export default forwardRef (({ ev.preventDefault () - if (nextPathname !== window.location.pathname) - { - const confirmed = await confirmDiscardNavigation () - if (!(confirmed)) - return - } - flushSync (() => { setOverlay (true) }) diff --git a/frontend/src/components/TagLink.tsx b/frontend/src/components/TagLink.tsx index 615f7d1..455add6 100644 --- a/frontend/src/components/TagLink.tsx +++ b/frontend/src/components/TagLink.tsx @@ -4,9 +4,13 @@ import { cn } from '@/lib/utils' import type { ComponentProps, CSSProperties, FC, HTMLAttributes } from 'react' -import type { Tag } from '@/types' +import type { Category, Tag } from '@/types' -type CommonProps = { +type LightweightTag = { + name: string + category: Category } + +type FullCommonProps = { tag: Tag nestLevel?: number truncateOnMobile?: boolean @@ -14,18 +18,43 @@ type CommonProps = { withCount?: boolean } type PropsWithLink = - & CommonProps + & FullCommonProps & { linkFlg?: true } & Partial> type PropsWithoutLink = - & CommonProps + & FullCommonProps & { linkFlg: false } & Partial> +type LightweightPropsWithLink = + & { + tag: LightweightTag + nestLevel?: number + truncateOnMobile?: boolean + withWiki: false + withCount: false + linkFlg?: true } + & Partial> + +type LightweightPropsWithoutLink = + & { + tag: LightweightTag + nestLevel?: number + truncateOnMobile?: boolean + withWiki: false + withCount: false + linkFlg: false } + & Partial> + type Props = | PropsWithLink | PropsWithoutLink + | LightweightPropsWithLink + | LightweightPropsWithoutLink + +const isFullTag = (tag: Tag | LightweightTag): tag is Tag => + 'id' in tag const TagLink: FC = ({ tag, @@ -46,16 +75,18 @@ const TagLink: FC = ({ tag, const spanClass = 'tag-link-colour' const linkClass = 'tag-link-colour tag-link-hover-colour' const textClass = 'group min-w-0 max-w-full overflow-hidden align-bottom' - const rootClass = - 'inline-flex min-w-0 max-w-full flex-nowrap items-stretch align-baseline gap-x-1 md:items-baseline' const markerWrapClass = 'shrink-0 self-start md:self-auto' const countClass = 'shrink-0 self-end md:self-auto' + const matchedAlias = isFullTag (tag) ? tag.matchedAlias : null const textTitle = title - ?? (tag.matchedAlias == null ? tag.name : `${ tag.matchedAlias } → ${ tag.name }`) + ?? (matchedAlias == null ? tag.name : `${ matchedAlias } → ${ tag.name }`) return ( - - {(linkFlg && withWiki) && ( + + {(linkFlg && withWiki && isFullTag (tag)) && ( {(tag.materialId != null || tag.hasWiki || tag.hasDeerjikists) ? ( @@ -118,7 +149,7 @@ const TagLink: FC = ({ tag, style={{ paddingLeft: `${ (nestLevel - 1) }rem` }}> ↳ )} - {tag.matchedAlias != null && ( + {matchedAlias != null && ( <> = ({ tag, style={colourStyle} {...props}> @@ -156,7 +187,7 @@ const TagLink: FC = ({ tag, title={textTitle} truncateOnMobile={truncateOnMobile}/> )} - {withCount && ( + {(withCount && isFullTag (tag)) && ( {tag.postCount})} ) } diff --git a/frontend/src/components/TopNav.test.tsx b/frontend/src/components/TopNav.test.tsx new file mode 100644 index 0000000..de954eb --- /dev/null +++ b/frontend/src/components/TopNav.test.tsx @@ -0,0 +1,39 @@ +import { describe, expect, it } from 'vitest' + +import { menuOutline } from '@/components/TopNav' +import { buildUser } from '@/test/factories' + +const submenuItem = (role: 'guest' | 'member' | 'admin', section: string, item: string) => { + const menu = menuOutline ({ + user: buildUser ({ role }), + wikiId: section === 'Wiki' ? 10 : null, + pathName: section === 'Wiki' ? '/wiki/page' : '/posts' }) + return menu.find (entry => entry.name === section)?.subMenu.find ( + subMenuItem => subMenuItem.name === item) +} + +describe ('menuOutline', () => { + it ('uses content-edit permission for post, material, and Wiki actions', () => { + for (const role of ['member', 'admin'] as const) + { + expect (submenuItem (role, '広場', '追加')?.visible).toBe (true) + expect (submenuItem (role, '素材', '追加')?.visible).toBe (true) + expect (submenuItem (role, 'Wiki', '新規')?.visible).toBe (true) + expect (submenuItem (role, 'Wiki', '編輯')?.visible).toBe (true) + } + + expect (submenuItem ('guest', '広場', '追加')?.visible).toBe (false) + expect (submenuItem ('guest', '素材', '追加')?.visible).toBe (false) + expect (submenuItem ('guest', 'Wiki', '新規')?.visible).toBe (false) + expect (submenuItem ('guest', 'Wiki', '編輯')?.visible).toBe (false) + }) + + it ('uses /posts/new as the post creation entrypoint', () => { + expect (submenuItem ('member', '広場', '追加')?.to).toBe ('/posts/new') + }) + + it ('keeps material suppression admin-only', () => { + expect (submenuItem ('member', '素材', '抑止')?.visible).toBe (false) + expect (submenuItem ('admin', '素材', '抑止')?.visible).toBe (true) + }) +}) diff --git a/frontend/src/components/TopNav.tsx b/frontend/src/components/TopNav.tsx index 034d5c5..755651a 100644 --- a/frontend/src/components/TopNav.tsx +++ b/frontend/src/components/TopNav.tsx @@ -9,6 +9,7 @@ import TopNavUser from '@/components/TopNavUser' import { WikiIdBus } from '@/lib/eventBus/WikiIdBus' import { materialsKeys, tagsKeys, wikiKeys } from '@/lib/queryKeys' import { useClientBehaviourSettings } from '@/lib/useClientBehaviourSettings' +import { canEditContent } from '@/lib/users' import { fetchTag, fetchTagByName } from '@/lib/tags' import { fetchMaterial } from '@/lib/materials' import { cn } from '@/lib/utils' @@ -26,10 +27,11 @@ export const menuOutline = ( tag?: Tag | null material?: Material | null wikiId: number | null - user: User | null, + user: User | null pathName: string }, ): Menu => { const postCount = tag?.postCount ?? material?.tag?.postCount ?? 0 + const editable = canEditContent (user) const wikiPageFlg = Boolean (/^\/wiki\/(?!new|changes)[^/]+/.test (pathName) && wikiId) const wikiTitle = pathName.split ('/')[2] ?? '' @@ -42,7 +44,7 @@ export const menuOutline = ( { name: '広場', to: '/posts', subMenu: [ { name: '一覧', to: '/posts' }, { name: '検索', to: '/posts/search' }, - { name: '追加', to: '/posts/new' }, + { name: '追加', to: '/posts/new', visible: editable }, { name: '全体履歴', to: '/posts/changes' }, { name: 'ヘルプ', to: '/wiki/ヘルプ:広場' }] }, { name: 'タグ', to: '/tags', subMenu: [ @@ -58,8 +60,9 @@ export const menuOutline = ( visible: tagFlg && tag?.category !== 'nico' }] }, { name: '素材', to: '/materials', visible: true, subMenu: [ { name: '一覧', to: '/materials' }, - { name: '追加', to: '/materials/new' }, - { name: '抑止', to: '/materials/suppressions' }, + { name: '追加', to: '/materials/new', visible: editable }, + { name: '抑止', to: '/materials/suppressions', + visible: user?.role === 'admin' }, { name: '全体履歴', to: '/materials/changes' }, { name: 'ヘルプ', to: '/wiki/ヘルプ:素材管理' }, { component: , visible: materialFlg }, @@ -70,14 +73,15 @@ export const menuOutline = ( visible: materialFlg }] }, { name: 'Wiki', to: '/wiki/ヘルプ:ホーム', base: '/wiki', subMenu: [ { name: '検索', to: '/wiki' }, - { name: '新規', to: '/wiki/new' }, + { name: '新規', to: '/wiki/new', visible: editable }, { name: '全体履歴', to: '/wiki/changes' }, { name: 'ヘルプ', to: '/wiki/ヘルプ:Wiki' }, { component: , visible: wikiPageFlg }, { name: `広場 (${ postCount || 0 })`, to: `/posts?tags=${ wikiTitle }`, visible: wikiPageFlg }, { name: '履歴', to: `/wiki/changes?id=${ wikiId }`, visible: wikiPageFlg }, - { name: '編輯', to: `/wiki/${ wikiId || wikiTitle }/edit`, visible: wikiPageFlg }] }, + { name: '編輯', to: `/wiki/${ wikiId || wikiTitle }/edit`, + visible: wikiPageFlg && editable }] }, { name: 'おたのしみ', visible: false, subMenu: [ { name: '上映会 (β)', to: '/theatres/1' }, { name: 'グカネータ (β)', to: '/gekanator' }] }, diff --git a/frontend/src/components/common/DateTimeField.test.tsx b/frontend/src/components/common/DateTimeField.test.tsx index 19605c7..2bf9b75 100644 --- a/frontend/src/components/common/DateTimeField.test.tsx +++ b/frontend/src/components/common/DateTimeField.test.tsx @@ -21,7 +21,8 @@ describe ('DateTimeField', () => { fireEvent.change (input, { target: { value: '' } }) const first = handleChange.mock.calls[0]?.[0] - expect (new Date (first).getFullYear ()).toBe (2026) + expect (first).toMatch (/^\d{4}-\d{2}-\d{2}T\d{2}:\d{2}Z$/) + expect (new Date (first).getUTCSeconds ()).toBe (0) expect (handleChange).toHaveBeenLastCalledWith (null) }) }) diff --git a/frontend/src/components/common/DateTimeField.tsx b/frontend/src/components/common/DateTimeField.tsx index 0cb2210..74fbec8 100644 --- a/frontend/src/components/common/DateTimeField.tsx +++ b/frontend/src/components/common/DateTimeField.tsx @@ -8,13 +8,27 @@ import type { ComponentPropsWithoutRef, FC, FocusEvent } from 'react' const pad = (n: number): string => n.toString ().padStart (2, '0') -const toDateTimeLocalValue = (d: Date) => { - const y = d.getFullYear () - const m = pad (d.getMonth () + 1) - const day = pad (d.getDate ()) - const h = pad (d.getHours ()) - const min = pad (d.getMinutes ()) - return `${ y }-${ m }-${ day }T${ h }:${ min }:00` +const toDateTimeLocalValue = (value: Date) => { + const y = value.getFullYear () + const m = pad (value.getMonth () + 1) + const day = pad (value.getDate ()) + const h = pad (value.getHours ()) + const min = pad (value.getMinutes ()) + return `${ y }-${ m }-${ day }T${ h }:${ min }` +} + + +const toMinutePrecisionIsoUtc = (value: string) => { + const date = new Date (value) + if (Number.isNaN (date.getTime ())) + return value + + const y = date.getUTCFullYear () + const m = pad (date.getUTCMonth () + 1) + const day = pad (date.getUTCDate ()) + const h = pad (date.getUTCHours ()) + const min = pad (date.getUTCMinutes ()) + return `${ y }-${ m }-${ day }T${ h }:${ min }Z` } @@ -47,14 +61,16 @@ const DateTimeField: FC = ({ value, onChange, className, onBlur, invalid, 'focus:ring-2 focus:ring-blue-200']), className)} type="datetime-local" + step={60} value={local} aria-invalid={invalid} onChange={ev => { const v = ev.target.value setLocal (v) - onChange?.(v ? (new Date (v)).toISOString () : null) + onChange?.(v ? toMinutePrecisionIsoUtc (v) : null) }} onBlur={onBlur}/>) } export default DateTimeField +export { toMinutePrecisionIsoUtc } diff --git a/frontend/src/components/common/FieldWarning.tsx b/frontend/src/components/common/FieldWarning.tsx new file mode 100644 index 0000000..1f42ed0 --- /dev/null +++ b/frontend/src/components/common/FieldWarning.tsx @@ -0,0 +1,18 @@ +import type { FC } from 'react' + +type Props = { id?: string + messages?: string[] } + + +export const FieldWarning: FC = ({ id, messages }: Props) => { + if (messages == null || messages.length === 0) + return null + + return ( +
    + {messages.map ((message, i) =>
  • {message}
    • )} +
) +} + + +export default FieldWarning diff --git a/frontend/src/components/common/Form.tsx b/frontend/src/components/common/Form.tsx index 2a3c294..f97d669 100644 --- a/frontend/src/components/common/Form.tsx +++ b/frontend/src/components/common/Form.tsx @@ -1,11 +1,15 @@ +import { cn } from '@/lib/utils' + import type { FC, ReactNode } from 'react' -type Props = { children: ReactNode } +type Props = { + children: ReactNode + className?: string } -const Form: FC = ({ children }) => ( -
+const Form: FC = ({ children, className }) => ( +
{children}
) -export default Form \ No newline at end of file +export default Form diff --git a/frontend/src/components/common/StatusBadge.tsx b/frontend/src/components/common/StatusBadge.tsx new file mode 100644 index 0000000..fd14c46 --- /dev/null +++ b/frontend/src/components/common/StatusBadge.tsx @@ -0,0 +1,34 @@ +import { cn } from '@/lib/utils' + +import type { FC, ReactNode } from 'react' + +export type StatusBadgeTone = + 'success' + | 'neutral' + | 'warning' + +type Props = { + children: ReactNode + tone: StatusBadgeTone } + +const TONES: Record = { + success: [ + 'border-emerald-300 bg-emerald-50 text-emerald-700', + 'dark:border-emerald-900 dark:bg-emerald-950 dark:text-emerald-200'], + warning: [ + 'border-amber-300 bg-amber-50 text-amber-700', + 'dark:border-amber-900 dark:bg-amber-950 dark:text-amber-200'], + neutral: [ + 'border-stone-300 bg-stone-50 text-stone-700', + 'dark:border-stone-700 dark:bg-stone-900 dark:text-stone-200'] } + + +const StatusBadge: FC = ({ children, tone }) => ( + + {children} + ) + +export default StatusBadge diff --git a/frontend/src/components/dialogues/DialogueProvider.test.tsx b/frontend/src/components/dialogues/DialogueProvider.test.tsx new file mode 100644 index 0000000..171631f --- /dev/null +++ b/frontend/src/components/dialogues/DialogueProvider.test.tsx @@ -0,0 +1,116 @@ +import { useCallback, useEffect } from 'react' +import { fireEvent, render, screen, waitFor } from '@testing-library/react' +import { describe, expect, it, vi } from 'vitest' + +import DialogueProvider from '@/components/dialogues/DialogueProvider' +import useDialogue from '@/lib/dialogues/useDialogue' + +import type { DialogueFormControls } from '@/lib/dialogues/useDialogue' + +const FormBody = ( + { controls, + onSelect }: { controls: DialogueFormControls + onSelect: () => Promise | boolean }, +) => { + useEffect (() => { + controls.setActions ([{ + label: '左操作', + placement: 'start', + onSelect }, { + label: '保存', + onSelect: () => true }]) + }, [controls, onSelect]) + + return
長いフォーム本文
+} + +const NestedConfirmFormBody = ({ controls }: { controls: DialogueFormControls }) => { + const reset = useCallback (async () => { + await controls.confirm ({ + title: '変更をリセットしますか?', + confirmText: 'リセット' }) + return false + }, [controls]) + + return +} + +describe ('DialogueProvider', () => { + it ('keeps ordinary dialogues in FIFO order', async () => { + const Launcher = () => { + const dialogue = useDialogue () + return ( + ) + } + + render () + fireEvent.click (screen.getByRole ('button', { name: '開く' })) + + expect (screen.getByText ('一件目')).toBeInTheDocument () + expect (screen.queryByText ('二件目')).not.toBeInTheDocument () + fireEvent.click (screen.getByRole ('button', { name: '確定' })) + expect (await screen.findByText ('二件目')).toBeInTheDocument () + }) + + it ('keeps a large form open when an action returns false', async () => { + const action = vi.fn ().mockResolvedValue (false) + const Launcher = () => { + const dialogue = useDialogue () + return ( + ) + } + + render () + fireEvent.click (screen.getByRole ('button', { name: '開く' })) + + const dialogue = screen.getByRole ('dialog') + expect (dialogue).toHaveClass ('max-h-[calc(100dvh-1rem)]', 'flex-col', 'max-w-3xl') + await waitFor (() => expect (screen.getByRole ('button', { name: '左操作' })) + .toBeInTheDocument ()) + expect (screen.getByRole ('button', { name: '左操作' })).toHaveClass ( + 'w-full', + 'md:w-auto') + fireEvent.click (screen.getByRole ('button', { name: '左操作' })) + + await waitFor (() => expect (action).toHaveBeenCalledTimes (1)) + expect (screen.getByText ('投稿を編輯')).toBeInTheDocument () + }) + + it ('opens a nested confirmation over a form and returns to the same form', async () => { + const Launcher = () => { + const dialogue = useDialogue () + return ( + ) + } + + render () + fireEvent.click (screen.getByRole ('button', { name: '開く' })) + const action = await screen.findByRole ('button', { name: '左操作' }) + fireEvent.click (action) + + expect (await screen.findByText ('変更をリセットしますか?')).toBeInTheDocument () + fireEvent.click (screen.getByRole ('button', { name: 'リセット' })) + await waitFor (() => { + expect (screen.queryByText ('変更をリセットしますか?')).not.toBeInTheDocument () + }) + expect (screen.getByText ('投稿を編輯')).toBeInTheDocument () + }) +}) diff --git a/frontend/src/components/dialogues/DialogueProvider.tsx b/frontend/src/components/dialogues/DialogueProvider.tsx index fb77ecf..01ff21e 100644 --- a/frontend/src/components/dialogues/DialogueProvider.tsx +++ b/frontend/src/components/dialogues/DialogueProvider.tsx @@ -1,4 +1,4 @@ -import { createContext, useCallback, useContext, useMemo, useState } from 'react' +import { useCallback, useMemo, useRef, useState } from 'react' import { Button } from '@/components/ui/button' import { Dialog, @@ -7,29 +7,16 @@ import { Dialog, DialogFooter, DialogHeader, DialogTitle } from '@/components/ui/dialog' +import { DialogueContext } from '@/lib/dialogues/useDialogue' import type { FC, ReactNode } from 'react' - -type DialogueVariant = 'default' | 'danger' - -type ConfirmOptions = { title: string - description?: ReactNode - confirmText?: string - cancelText?: string - variant?: DialogueVariant } - -type AlertOptions = { title: string - description?: ReactNode - okText?: string } - -type Choice = { value: T - label: string - variant?: DialogueVariant } - -type ChoiceOptions = { title: string - description?: ReactNode - choices: Choice[] - cancelText?: string } +import type { AlertOptions, + ChoiceOptions, + ConfirmOptions, + DialogueAPI, + DialogueFormAction, + DialogueFormControls, + DialogueFormOptions } from '@/lib/dialogues/useDialogue' type DialogueRequest = | { id: number @@ -44,13 +31,10 @@ type DialogueRequest = kind: 'choice' options: ChoiceOptions resolve: (value: string | null) => void } - -type DialogueAPI = - { confirm: (options: ConfirmOptions) => Promise - alert: (options: AlertOptions) => Promise - choice: (options: ChoiceOptions) => Promise } - -const DialogueContext = createContext (null) + | { id: number + kind: 'form' + options: DialogueFormOptions + resolve: () => void } let nextDialogueId = 1 @@ -59,20 +43,27 @@ type Props = { children: ReactNode } const DialogueProvider: FC = ({ children }) => { const [queue, setQueue] = useState ([]) + const [pendingIds, setPendingIds] = useState ([]) + const [formActions, setFormActions] = useState> ({ }) + const formControls = useRef> ({ }) + const [nestedConfirm, setNestedConfirm] = useState<{ + parentId: number + options: ConfirmOptions + resolve: (value: boolean) => void } | null> (null) const push = useCallback ((request: Omit) => { const id = nextDialogueId ++nextDialogueId - setQueue (q => [...q, { ...request, id } as DialogueRequest]) + setQueue (current => [...current, { ...request, id } as DialogueRequest]) }, []) - const closeActive = useCallback ((result?: unknown) => { - setQueue (q => { - const [active, ...rest] = q + const closeRequest = useCallback ((id: number, result?: unknown) => { + setQueue (current => { + const active = current.find (request => request.id === id) - if (!(active)) - return rest + if (active == null) + return current switch (active.kind) { @@ -87,9 +78,42 @@ const DialogueProvider: FC = ({ children }) => { case 'choice': active.resolve ((result ?? null) as string | null) break + + case 'form': + active.resolve () + break } - return rest + return current.filter (request => request.id !== id) + }) + setPendingIds (current => current.filter (pendingId => pendingId !== id)) + setFormActions (current => { + const { [id]: _, ...rest } = current + return rest + }) + delete formControls.current[id] + }, []) + + const setRequestActions = useCallback ( + (id: number, actions: DialogueFormAction[]) => { + setFormActions (current => ({ ...current, [id]: actions })) + }, + []) + + const openNestedConfirm = useCallback ( + (parentId: number, options: ConfirmOptions) => + new Promise (resolve => { + setNestedConfirm ({ parentId, options, resolve }) + }), + []) + + const closeNestedConfirm = useCallback ((result: boolean) => { + setNestedConfirm (current => { + if (current == null) + return current + + current.resolve (result) + return null }) }, []) @@ -103,86 +127,229 @@ const DialogueProvider: FC = ({ children }) => { choice: options => new Promise (resolve => { push ({ kind: 'choice', options: options as ChoiceOptions, - resolve: resolve as (value: string | null) => void })}) }), [push]) + resolve: resolve as (value: string | null) => void }) + }), + form: options => new Promise (resolve => { + push ({ kind: 'form', options, resolve }) + }) }), [push]) + + const handleFormAction = useCallback ( + async (id: number, action: DialogueFormAction) => { + if (pendingIds.includes (id)) + return + + setPendingIds (current => [...current, id]) + try + { + const shouldClose = await action.onSelect () + if (shouldClose !== false) + closeRequest (id) + } + finally + { + setPendingIds (current => current.filter (pendingId => pendingId !== id)) + } + }, + [closeRequest, pendingIds]) const active = queue[0] + const startActions = + active?.kind === 'form' + ? (formActions[active.id] ?? []).filter (action => action.placement === 'start') + : [] + const endActions = + active?.kind === 'form' + ? (formActions[active.id] ?? []).filter (action => + action.placement == null || action.placement === 'end') + : [] return ( {children} - { - if (!(open)) - closeActive (active?.kind !== 'confirm' && null) - }}> - {active && ( + {active && ( + { + const blocked = + nestedConfirm?.parentId === active.id + || pendingIds.includes (active.id) + if (!(open) && !(blocked)) + closeRequest (active.id, active.kind !== 'confirm' && null) + }}> + { + if (nestedConfirm?.parentId === active.id || pendingIds.includes (active.id)) + event.preventDefault () + }} + onPointerDownOutside={event => { + if (nestedConfirm?.parentId === active.id || pendingIds.includes (active.id)) + event.preventDefault () + }}> + {active.kind === 'form' + ? ( + <> + + {active.options.title} + + {active.options.description && ( + +
{active.options.description}
+ )} + + +
+ {active.options.body ( + formControls.current[active.id] ??= { + close: () => closeRequest (active.id), + setActions: actions => setRequestActions (active.id, actions), + confirm: options => openNestedConfirm (active.id, options) })} +
+ + +
+ {startActions.map (action => ( + ))} +
+ +
+ + + {endActions.map (action => ( + ))} +
+ + ) + : ( + <> + + {active.options.title} + + {active.options.description && ( + +
{active.options.description}
+ )} + + + + {active.kind === 'confirm' && ( + <> + + + + )} + + {active.kind === 'alert' && ( + )} + + {active.kind === 'choice' && ( + <> + + + {active.options.choices.map (choice => ( + ))} + )} + + )} + + )} + + {nestedConfirm && ( + { + if (!(open)) + closeNestedConfirm (false) + }}> - {active.options.title} + {nestedConfirm.options.title} - {active.options.description && ( + {nestedConfirm.options.description && ( -
{active.options.description}
+
{nestedConfirm.options.description}
)} - {active.kind === 'confirm' && ( - <> - + - - )} - - {active.kind === 'alert' && ( - )} - - {active.kind === 'choice' && ( - <> - - - {active.options.choices.map (choice => ( - ))} - )} + - )} - + + )} ) } - - -export const useDialogue = () => { - const dialogue = useContext (DialogueContext) - - if (!(dialogue)) - throw new Error ('useDialogue must be used inside DialogueProvider') - - return dialogue -} - +export { useDialogue } from '@/lib/dialogues/useDialogue' export default DialogueProvider diff --git a/frontend/src/components/posts/PostCoreDataFields.tsx b/frontend/src/components/posts/PostCoreDataFields.tsx new file mode 100644 index 0000000..25b0378 --- /dev/null +++ b/frontend/src/components/posts/PostCoreDataFields.tsx @@ -0,0 +1,94 @@ +import PostOriginalCreatedTimeField from '@/components/PostOriginalCreatedTimeField' +import PostTagsField from '@/components/posts/PostTagsField' +import PostTextField from '@/components/posts/PostTextField' + +import type { FC, ReactNode } from 'react' + +type TextMessages = string[] | undefined + +type CoreField = { + value: string + onChange: (value: string) => void + errors?: TextMessages + warnings?: TextMessages + disabled?: boolean } + +type OriginalCreatedField = { + originalCreatedAt?: TextMessages + originalCreatedFrom?: TextMessages + originalCreatedBefore?: TextMessages } + +type PostCoreDataFieldsProps = { + title: { + value: string + onChange: (value: string) => void + errors?: TextMessages + warnings?: TextMessages + disabled?: boolean + after?: ReactNode } + originalCreated: { + disabled?: boolean + originalCreatedFrom: string | null + setOriginalCreatedFrom: (value: string | null) => void + originalCreatedBefore: string | null + setOriginalCreatedBefore: (value: string | null) => void + errors?: OriginalCreatedField } + tags: { + value: string + onChange: (value: string) => void + errors?: TextMessages + warnings?: TextMessages + disabled?: boolean + rows?: number } + parentPostIds: CoreField } + +const groupedMessages = (...values: (TextMessages | null | undefined)[]): string[] => + [...new Set (values.flatMap (value => value ?? []))] + + +const PostCoreDataFields: FC = ( + { title, + originalCreated, + tags, + parentPostIds }, +) => ( + <> + + + + + + + + ) + +export default PostCoreDataFields +export type { PostCoreDataFieldsProps } diff --git a/frontend/src/components/posts/PostCreationDataFields.tsx b/frontend/src/components/posts/PostCreationDataFields.tsx new file mode 100644 index 0000000..7348c5d --- /dev/null +++ b/frontend/src/components/posts/PostCreationDataFields.tsx @@ -0,0 +1,48 @@ +import PostCoreDataFields from '@/components/posts/PostCoreDataFields' +import PostTextField from '@/components/posts/PostTextField' + +import type { FC, ReactNode } from 'react' + +import type { PostCoreDataFieldsProps } from '@/components/posts/PostCoreDataFields' + +type TextMessages = string[] | undefined + +type Props = { + url: { + value: string + onChange: (value: string) => void + errors?: TextMessages + warnings?: TextMessages + disabled?: boolean + type?: string + placeholder?: string } + thumbnailField: ReactNode + core: PostCoreDataFieldsProps + extraFields?: ReactNode } + + +const PostCreationDataFields: FC = ( + { url, + thumbnailField, + core, + extraFields }, +) => ( + <> + + + {thumbnailField} + + + + {extraFields} + ) + +export default PostCreationDataFields diff --git a/frontend/src/components/posts/PostCreationDataFieldsUsage.test.tsx b/frontend/src/components/posts/PostCreationDataFieldsUsage.test.tsx new file mode 100644 index 0000000..d617eb9 --- /dev/null +++ b/frontend/src/components/posts/PostCreationDataFieldsUsage.test.tsx @@ -0,0 +1,31 @@ +import { render, screen } from '@testing-library/react' +import { describe, expect, it, vi } from 'vitest' + +import { buildPostImportRow } from '@/test/postImportFactories' + +import type { DialogueFormControls } from '@/lib/dialogues/useDialogue' + +const sharedFieldsSpy = vi.hoisted (() => vi.fn (() =>
)) + +vi.mock ('@/components/posts/PostCreationDataFields', () => ({ + default: sharedFieldsSpy, +})) + +describe ('PostCreationDataFields usage', () => { + it ('is used by PostImportRowForm', async () => { + const { default: PostImportRowForm } = await import ( + '@/components/posts/import/PostImportRowForm') + + render ( + ) + + expect (screen.getByTestId ('shared-fields')).toBeInTheDocument () + }) +}) diff --git a/frontend/src/components/posts/PostDurationField.tsx b/frontend/src/components/posts/PostDurationField.tsx new file mode 100644 index 0000000..5dcda0b --- /dev/null +++ b/frontend/src/components/posts/PostDurationField.tsx @@ -0,0 +1,27 @@ +import PostTextField from '@/components/posts/PostTextField' + +import type { FC } from 'react' + +type Props = { + value: string + onChange: (value: string) => void + errors?: string[] + disabled?: boolean } + + +const PostDurationField: FC = ( + { value, + onChange, + errors, + disabled }, +) => ( + +) + +export default PostDurationField diff --git a/frontend/src/components/posts/PostTagsField.tsx b/frontend/src/components/posts/PostTagsField.tsx new file mode 100644 index 0000000..59d520a --- /dev/null +++ b/frontend/src/components/posts/PostTagsField.tsx @@ -0,0 +1,29 @@ +import PostFormTagsArea from '@/components/PostFormTagsArea' +import FieldWarning from '@/components/common/FieldWarning' + +import type { ComponentPropsWithoutRef, FC } from 'react' + +type Props = Omit, 'value' | 'onChange'> & { + tags: string + setTags: (tags: string) => void + warnings?: string[] + errors?: string[] } + + +const PostTagsField: FC = ( + { tags, + setTags, + warnings, + errors, + ...rest }, +) => ( +
+ + +
) + +export default PostTagsField diff --git a/frontend/src/components/posts/PostTextField.tsx b/frontend/src/components/posts/PostTextField.tsx new file mode 100644 index 0000000..bdc6b0d --- /dev/null +++ b/frontend/src/components/posts/PostTextField.tsx @@ -0,0 +1,52 @@ +import FieldWarning from '@/components/common/FieldWarning' +import FormField from '@/components/common/FormField' +import { inputClass } from '@/lib/utils' + +import type { FC, ReactNode } from 'react' + +type Props = { + label: string + value: string + onChange: (value: string) => void + warnings?: string[] + errors?: string[] + disabled?: boolean + type?: string + placeholder?: string + className?: string + after?: ReactNode + onBlur?: () => void } + + +const PostTextField: FC = ( + { label, + value, + onChange, + warnings, + errors, + disabled, + type = 'text', + placeholder, + className, + after, + onBlur }, +) => ( + + {({ describedBy, invalid }) => ( + <> + onChange (ev.target.value)} + aria-describedby={describedBy} + aria-invalid={invalid} + className={inputClass (invalid, className)}/> + + {after} + )} + ) + +export default PostTextField diff --git a/frontend/src/components/posts/PostThumbnailPreview.test.tsx b/frontend/src/components/posts/PostThumbnailPreview.test.tsx new file mode 100644 index 0000000..bb9a17e --- /dev/null +++ b/frontend/src/components/posts/PostThumbnailPreview.test.tsx @@ -0,0 +1,36 @@ +import { fireEvent, render, screen } from '@testing-library/react' +import { describe, expect, it } from 'vitest' + +import PostThumbnailPreview from '@/components/posts/PostThumbnailPreview' + +describe ('PostThumbnailPreview', () => { + it ('keeps an existing blob preview URL unchanged for normal post forms', () => { + render () + + expect (screen.getByRole ('img')).toHaveAttribute ('src', 'blob:preview') + }) + + it ('renders an empty thumbnail frame without text when the URL is empty', () => { + const { container } = render ( + ) + + expect (screen.queryByRole ('img')).toBeNull () + expect (screen.queryByText ('サムネールを表示できません')).toBeNull () + expect (screen.queryByText ('なし')).toBeNull () + expect (container.querySelector ('div.rounded.border.bg-muted')).not.toBeNull () + expect (container.textContent).toBe ('') + }) + + it ('renders an empty thumbnail frame without text when image loading fails', () => { + const { container } = render ( + ) + + fireEvent.error (screen.getByRole ('img')) + + expect (screen.queryByRole ('img')).toBeNull () + expect (screen.queryByText ('サムネールを表示できません')).toBeNull () + expect (screen.queryByText ('なし')).toBeNull () + expect (container.querySelector ('div.rounded.border.bg-muted')).not.toBeNull () + expect (container.textContent).toBe ('') + }) +}) diff --git a/frontend/src/components/posts/PostThumbnailPreview.tsx b/frontend/src/components/posts/PostThumbnailPreview.tsx new file mode 100644 index 0000000..5383f13 --- /dev/null +++ b/frontend/src/components/posts/PostThumbnailPreview.tsx @@ -0,0 +1,64 @@ +import { useEffect, useState } from 'react' + +import { cn } from '@/lib/utils' + +import type { FC } from 'react' + +type Props = { + url: string + file?: File + alt?: string + className?: string + referrerPolicy?: 'no-referrer' } + + +const PostThumbnailPreview: FC = ( + { url, + file, + alt = 'サムネール', + className = 'h-16 w-16', + referrerPolicy }, +) => { + const [failed, setFailed] = useState (false) + const [fileUrl, setFileUrl] = useState (null) + + useEffect (() => { + setFailed (false) + }, [file, url]) + + useEffect (() => { + if (file == null) + { + setFileUrl (null) + return + } + + const nextUrl = URL.createObjectURL (file) + setFileUrl (nextUrl) + + return () => { + URL.revokeObjectURL (nextUrl) + } + }, [file]) + + const resolvedUrl = url.trim () !== '' ? url : (fileUrl ?? '') + + if (resolvedUrl === '' || failed) + { + return ( +
) + } + + return ( + {alt} setFailed (true)}/>) +} + +export default PostThumbnailPreview diff --git a/frontend/src/components/posts/import/PostImportRowForm.test.tsx b/frontend/src/components/posts/import/PostImportRowForm.test.tsx new file mode 100644 index 0000000..ad33957 --- /dev/null +++ b/frontend/src/components/posts/import/PostImportRowForm.test.tsx @@ -0,0 +1,278 @@ +import { act, render, screen, waitFor } from '@testing-library/react' +import { beforeEach, describe, expect, it, vi } from 'vitest' + +import PostImportRowForm from '@/components/posts/import/PostImportRowForm' +import { buildPostImportRow } from '@/test/postImportFactories' + +import type { DialogueFormAction, DialogueFormControls } from '@/lib/dialogues/useDialogue' + +const api = vi.hoisted (() => ({ + apiGet: vi.fn (), +})) + +vi.mock ('@/lib/api', () => api) + +describe ('PostImportRowForm', () => { + beforeEach (() => { + vi.clearAllMocks () + globalThis.URL.createObjectURL = vi.fn (() => 'blob:preview') + globalThis.URL.revokeObjectURL = vi.fn () + api.apiGet.mockResolvedValue (new Blob (['img'], { type: 'image/png' })) + }) + + it ('resets only the draft, then saves with resetRequested', async () => { + const row = buildPostImportRow () + row.attributes.title = 'manual title' + row.provenance.title = 'manual' + const actions: DialogueFormAction[][] = [] + const controls: DialogueFormControls = { + close: vi.fn (), + confirm: vi.fn ().mockResolvedValue (true), + setActions: next => actions.push (next) } + const invalidRow = buildPostImportRow ({ + validationErrors: { title: ['タイトルを確認してください．'] } }) + const onSave = vi.fn ().mockResolvedValue ({ saved: false, row: invalidRow }) + + render () + const titleInput = screen.getByDisplayValue ('manual title') + + await waitFor (() => expect (actions.at (-1)?.length).toBe (2)) + const reset = actions.at (-1)?.find (action => action.label === '変更をリセット') + expect (reset).toMatchObject ({ placement: 'start', variant: 'danger', disabled: false }) + await act (async () => { + await reset?.onSelect () + }) + + expect (controls.confirm).toHaveBeenCalled () + expect (titleInput).toHaveValue ('') + expect (onSave).not.toHaveBeenCalled () + + const save = actions.at (-1)?.find (action => action.label === '編輯内容を保存') + await act (async () => { + await save?.onSelect () + }) + + expect (onSave).toHaveBeenCalledWith ({ + draft: expect.objectContaining ({ title: '' }), + resetRequested: true }) + expect (screen.getByText ('タイトルを確認してください．')).toBeInTheDocument () + }) + + it ('does not reset the draft when confirmation is cancelled', async () => { + const row = buildPostImportRow ({ attributes: { title: 'manual title' } }) + let actions: DialogueFormAction[] = [] + const controls: DialogueFormControls = { + close: vi.fn (), + confirm: vi.fn ().mockResolvedValue (false), + setActions: next => { + actions = next + } } + + render ( + ) + await waitFor (() => expect (actions.length).toBe (2)) + + await act (async () => { + await actions.find (action => action.label === '変更をリセット')?.onSelect () + }) + + expect (screen.getByDisplayValue ('manual title')).toBeInTheDocument () + }) + + it ('marks edited fields and areas invalid from field errors', () => { + const row = buildPostImportRow ({ + validationErrors: { url: ['URL error'], tags: ['tag error'] }, + importErrors: { title: ['title error'] }, + fieldWarnings: { title: ['title warning'] } }) + + render ( + ) + + expect (screen.getByText ('URL error')).toBeInTheDocument () + expect (screen.getByText ('tag error')).toBeInTheDocument () + expect (screen.getByText ('title error')).toBeInTheDocument () + expect (screen.getByText ('title warning')).toBeInTheDocument () + expect (screen.getAllByRole ('textbox').filter ( + textbox => textbox.getAttribute ('aria-invalid') === 'true')).toHaveLength (3) + }) + + it ('keeps untouched original created values unchanged in the save payload', async () => { + let actions: DialogueFormAction[] = [] + const row = buildPostImportRow ({ + attributes: { + originalCreatedFrom: '2024-01-01T12:34+09:00', + originalCreatedBefore: '2024-01-01T12:35+09:00' } }) + const controls: DialogueFormControls = { + close: vi.fn (), + confirm: vi.fn (), + setActions: next => { + actions = next + } } + const onSave = vi.fn ().mockResolvedValue ({ saved: true, row: null }) + + render ( + ) + await waitFor (() => expect (actions.length).toBe (2)) + + await act (async () => { + void actions.find (action => action.label === '編輯内容を保存')?.onSelect () + }) + + expect (onSave).toHaveBeenCalledWith ({ + draft: expect.objectContaining ({ + originalCreatedFrom: '2024-01-01T12:34+09:00', + originalCreatedBefore: '2024-01-01T12:35+09:00' }), + resetRequested: false }) + }) + + it ( + 'shows upload input only when the thumbnail URL is blank', + async () => { + let actions: DialogueFormAction[] = [] + const controls: DialogueFormControls = { + close: vi.fn (), + confirm: vi.fn (), + setActions: next => { + actions = next + } } + const onSave = vi.fn ().mockResolvedValue ({ saved: true, row: null }) + + const { container } = render ( + ) + await waitFor (() => expect (actions.length).toBe (2)) + + const labels = Array.from (container.querySelectorAll ('label')) + .map (node => node.textContent?.trim ()) + + expect (labels.slice (0, 6)).toEqual ([ + 'URL', + 'サムネール', + 'タイトル', + 'オリジナルの作成日時', + 'タグ', + '親投稿']) + expect (container.querySelector ('input[type="file"]')).toHaveAttribute ( + 'accept', + 'image/*') + expect (screen.queryByPlaceholderText ('例: 2 / 2.5 / 1:23')).not.toBeInTheDocument () + expect (screen.getByDisplayValue ('tag1')).toBeInTheDocument () + + await act (async () => { + await actions.find (action => action.label === '編輯内容を保存')?.onSelect () + }) + + expect (onSave).toHaveBeenCalledWith ({ + draft: expect.objectContaining ({ + tags: 'tag1' }), + resetRequested: false }) + }) + + it ('keeps reset enabled when the value matches but provenance still differs', async () => { + const row = buildPostImportRow ({ + attributes: { title: 'same title' }, + provenance: { title: 'manual' }, + resetSnapshot: { + url: 'https://example.com/post', + attributes: { + title: 'same title', + thumbnailBase: '', + originalCreatedFrom: '', + originalCreatedBefore: '', + tags: '', + parentPostIds: '' }, + provenance: { + url: 'manual', + title: 'automatic', + thumbnailBase: 'automatic', + originalCreatedFrom: 'automatic', + originalCreatedBefore: 'automatic', + tags: 'automatic', + parentPostIds: 'automatic' }, + tagSources: { automatic: '', manual: '' }, + fieldWarnings: { }, + baseWarnings: [] } }) + let actions: DialogueFormAction[] = [] + + render ( + { + actions = next + } }} + onSave={vi.fn ()}/>) + await waitFor (() => expect (actions.length).toBe (2)) + + expect (actions.find (action => action.label === '変更をリセット')?.disabled).toBe (false) + }) + + it ( + 'disables every field while save validation is pending and re-enables them afterwards', + async () => { + let actions: DialogueFormAction[] = [] + let resolveSave: + ((value: { saved: boolean + row: ReturnType | null }) => void) | null + = null + const onSave = vi.fn (() => + new Promise<{ saved: boolean + row: ReturnType | null }> (resolve => { + resolveSave = resolve + })) + + render ( + { + actions = next + } }} + onSave={onSave}/>) + await waitFor (() => expect (actions.length).toBe (2)) + + let savePromise: Promise | undefined + await act (async () => { + savePromise = actions.find (action => action.label === '編輯内容を保存')?.onSelect () + }) + + await waitFor (() => { + screen.getAllByRole ('textbox').forEach (textbox => { + expect (textbox).toBeDisabled () + }) + }) + + resolveSave?.({ + saved: false, + row: buildPostImportRow ({ + attributes: { title: 'draft title' }, + validationErrors: { title: ['タイトルを確認してください．'] } }) }) + await act (async () => { + await savePromise + }) + + await waitFor (() => { + screen.getAllByRole ('textbox').forEach (textbox => { + expect (textbox).not.toBeDisabled () + }) + }) + expect (screen.getByDisplayValue ('draft title')).toBeInTheDocument () + expect (screen.getByText ('タイトルを確認してください．')).toBeInTheDocument () + }) +}) diff --git a/frontend/src/components/posts/import/PostImportRowForm.tsx b/frontend/src/components/posts/import/PostImportRowForm.tsx new file mode 100644 index 0000000..641fe4a --- /dev/null +++ b/frontend/src/components/posts/import/PostImportRowForm.tsx @@ -0,0 +1,308 @@ +import { useCallback, useEffect, useMemo, useState } from 'react' + +import FieldError from '@/components/common/FieldError' +import FieldWarning from '@/components/common/FieldWarning' +import PostCreationDataFields from '@/components/posts/PostCreationDataFields' +import PostDurationField from '@/components/posts/PostDurationField' +import PostTextField from '@/components/posts/PostTextField' +import PostImportThumbnailPreview from '@/components/posts/import/PostImportThumbnailPreview' +import { hasThumbnailBaseValue, hasVideoTag } from '@/lib/postImportRows' + +import type { FC } from 'react' + +import type { DialogueFormControls } from '@/lib/dialogues/useDialogue' +import type { PostImportEditableDraft, PostImportRow } from '@/lib/postImportTypes' + +type Draft = PostImportEditableDraft + +type Props = { + row: PostImportRow + controls: DialogueFormControls + onSave: (args: { draft: Draft + resetRequested: boolean }) => Promise<{ + saved: boolean + row: PostImportRow | null }> } + +const THUMBNAIL_MISSING_WARNING = 'サムネールなし' + +const buildDraft = (row: PostImportRow): Draft => ({ + url: row.url, + title: String (row.attributes.title ?? ''), + thumbnailBase: String (row.attributes.thumbnailBase ?? ''), + originalCreatedFrom: String (row.attributes.originalCreatedFrom ?? ''), + originalCreatedBefore: String (row.attributes.originalCreatedBefore ?? ''), + tags: String (row.attributes.tags ?? ''), + parentPostIds: String (row.attributes.parentPostIds ?? ''), + duration: String (row.attributes.duration ?? ''), + thumbnailFile: row.thumbnailFile }) + +const buildResetDraft = (row: PostImportRow): Draft => ({ + url: row.resetSnapshot.url, + title: String (row.resetSnapshot.attributes.title ?? ''), + thumbnailBase: String (row.resetSnapshot.attributes.thumbnailBase ?? ''), + originalCreatedFrom: String (row.resetSnapshot.attributes.originalCreatedFrom ?? ''), + originalCreatedBefore: String (row.resetSnapshot.attributes.originalCreatedBefore ?? ''), + tags: String (row.resetSnapshot.attributes.tags ?? ''), + parentPostIds: String (row.resetSnapshot.attributes.parentPostIds ?? ''), + duration: String (row.resetSnapshot.attributes.duration ?? ''), + thumbnailFile: undefined }) + +const groupedMessages = (...values: (string[] | undefined)[]): string[] => + [...new Set (values.flatMap (value => value ?? []))] + +const sameDraft = (left: Draft, right: Draft): boolean => + left.url === right.url + && left.title === right.title + && left.thumbnailBase === right.thumbnailBase + && left.originalCreatedFrom === right.originalCreatedFrom + && left.originalCreatedBefore === right.originalCreatedBefore + && left.tags === right.tags + && left.parentPostIds === right.parentPostIds + && left.duration === right.duration + && left.thumbnailFile === right.thumbnailFile + +const sameProvenance = ( + current: PostImportRow['provenance'], + reset: PostImportRow['resetSnapshot']['provenance'], +): boolean => + Object.keys (reset).every (field => current[field] === reset[field]) + +const sameTagSources = ( + current: PostImportRow['tagSources'], + reset: PostImportRow['resetSnapshot']['tagSources'], +): boolean => + (current?.automatic ?? '') === reset.automatic + && (current?.manual ?? '') === reset.manual + +const sameWarnings = ( + current: PostImportRow, + reset: PostImportRow['resetSnapshot'], +): boolean => + JSON.stringify (current.fieldWarnings) === JSON.stringify (reset.fieldWarnings) + && JSON.stringify (current.baseWarnings) === JSON.stringify (reset.baseWarnings) + +const thumbnailWarnings = ( + messages: string[] | undefined, + thumbnailBase: string, + thumbnailFile: File | undefined, +): string[] => { + const others = (messages ?? []).filter (message => message !== THUMBNAIL_MISSING_WARNING) + return hasThumbnailBaseValue (thumbnailBase) || thumbnailFile != null + ? others + : [...new Set ([...others, THUMBNAIL_MISSING_WARNING])] +} + + +const PostImportRowForm: FC = ( + { row, + controls, + onSave }, +) => { + const [draft, setDraft] = useState (() => buildDraft (row)) + const [messageRow, setMessageRow] = useState (null) + const [saving, setSaving] = useState (false) + const [resetRequested, setResetRequested] = useState (false) + const [committedThumbnailBase, setCommittedThumbnailBase] = useState ( + () => String (row.attributes.thumbnailBase ?? '')) + + useEffect (() => { + const nextDraft = buildDraft (row) + setDraft (nextDraft) + setMessageRow (null) + setResetRequested (false) + setCommittedThumbnailBase (String (row.attributes.thumbnailBase ?? '')) + }, [row]) + + const displayRow = messageRow ?? row + const resetDraft = useMemo ( + () => buildResetDraft (row), + [row]) + const durationVisible = hasVideoTag (draft.tags) + const currentThumbnailWarnings = thumbnailWarnings ( + displayRow.fieldWarnings.thumbnailBase, + draft.thumbnailBase, + draft.thumbnailFile) + const resetDisabled = + saving + || (sameDraft (draft, resetDraft) + && sameProvenance (row.provenance, row.resetSnapshot.provenance) + && sameTagSources (row.tagSources, row.resetSnapshot.tagSources) + && row.metadataUrl === row.resetSnapshot.metadataUrl + && sameWarnings (displayRow, row.resetSnapshot)) + + const update = ( + key: Key, + value: Draft[Key], + ) => { + if (messageRow != null) + setMessageRow (null) + setDraft (current => ({ ...current, [key]: value })) + } + + const reset = useCallback (async (): Promise => { + if (resetDisabled) + return false + + const confirmed = await controls.confirm ({ + title: '変更をリセットしますか?', + confirmText: 'リセット', + cancelText: '取消', + variant: 'danger' }) + if (!(confirmed)) + return false + + setDraft (resetDraft) + setResetRequested (true) + setMessageRow (null) + setCommittedThumbnailBase (resetDraft.thumbnailBase) + return false + }, [controls, resetDisabled, resetDraft]) + + const save = useCallback (async (): Promise => { + setSaving (true) + try + { + const result = await onSave ({ draft, resetRequested }) + if (result.saved) + return true + + if (result.row != null) + setMessageRow (result.row) + return false + } + finally + { + setSaving (false) + } + }, [draft, onSave, resetRequested]) + + useEffect (() => { + controls.setActions ([{ + label: '変更をリセット', + placement: 'start', + variant: 'danger', + disabled: resetDisabled, + onSelect: reset }, + { + label: '編輯内容を保存', + disabled: saving, + onSelect: save }]) + }, [controls, resetDisabled, reset, save, saving]) + + return ( + <> +
+
+
+
+ +
+ +
+ update ('url', value), + disabled: saving, + warnings: displayRow.fieldWarnings.url, + errors: groupedMessages ( + displayRow.validationErrors.url, + displayRow.importErrors?.url) }} + thumbnailField={ + <> + { + if (draft.thumbnailBase.trim () !== committedThumbnailBase.trim ()) + setCommittedThumbnailBase (draft.thumbnailBase) + }} + onChange={value => update ('thumbnailBase', value)}/> + {!(hasThumbnailBaseValue (draft.thumbnailBase)) && ( + { + const file = event.target.files?.[0] + update ('thumbnailFile', file) + }}/>)} + } + core={{ + title: { + value: draft.title, + onChange: value => update ('title', value), + disabled: saving, + warnings: displayRow.fieldWarnings.title, + errors: groupedMessages ( + displayRow.validationErrors.title, + displayRow.importErrors?.title) }, + originalCreated: { + disabled: saving, + originalCreatedFrom: draft.originalCreatedFrom || null, + setOriginalCreatedFrom: value => + update ('originalCreatedFrom', value ?? ''), + originalCreatedBefore: draft.originalCreatedBefore || null, + setOriginalCreatedBefore: value => + update ('originalCreatedBefore', value ?? ''), + errors: { + originalCreatedAt: groupedMessages ( + displayRow.validationErrors.originalCreatedAt, + displayRow.importErrors?.originalCreatedAt), + originalCreatedFrom: groupedMessages ( + displayRow.validationErrors.originalCreatedFrom, + displayRow.importErrors?.originalCreatedFrom), + originalCreatedBefore: groupedMessages ( + displayRow.validationErrors.originalCreatedBefore, + displayRow.importErrors?.originalCreatedBefore) } }, + tags: { + value: draft.tags, + onChange: value => update ('tags', value), + disabled: saving, + warnings: displayRow.fieldWarnings.tags, + errors: groupedMessages ( + displayRow.validationErrors.tags, + displayRow.importErrors?.tags), + rows: 4 }, + parentPostIds: { + value: draft.parentPostIds, + onChange: value => update ('parentPostIds', value), + disabled: saving, + errors: groupedMessages ( + displayRow.validationErrors.parentPostIds, + displayRow.importErrors?.parentPostIds) } }} + extraFields={ + durationVisible + ? ( + update ('duration', value)} + disabled={saving} + errors={groupedMessages ( + displayRow.validationErrors.videoMs, + displayRow.importErrors?.videoMs)}/>) + : null}/> + + + +
+
+
+
+ ) +} + +export default PostImportRowForm +export { buildDraft } +export type { Draft as PostImportRowDraft } diff --git a/frontend/src/components/posts/import/PostImportRowSummary.tsx b/frontend/src/components/posts/import/PostImportRowSummary.tsx new file mode 100644 index 0000000..4cbfbf1 --- /dev/null +++ b/frontend/src/components/posts/import/PostImportRowSummary.tsx @@ -0,0 +1,198 @@ +import FieldError from '@/components/common/FieldError' +import PostImportTagLinks from '@/components/posts/import/PostImportTagLinks' +import { Button } from '@/components/ui/button' +import PostImportThumbnailPreview from '@/components/posts/import/PostImportThumbnailPreview' +import PostImportStatusBadge from '@/components/posts/import/PostImportStatusBadge' +import { displayPostImportStatus } from '@/components/posts/import/postImportRowStatus' +import { + canEditReviewRow, + canRetryResultRow, + hasVideoTag, +} from '@/lib/postImportRows' +import { cn, originalCreatedAtString } from '@/lib/utils' + +import type { FC } from 'react' + +import type { PostImportRow } from '@/lib/postImportTypes' + +type Props = { + row: PostImportRow + displayNumber?: number + onEdit?: () => void + onRetry?: () => void + onToggleSkip?: (checked: boolean) => void + rowMessages?: string[] + editDisabled?: boolean + retryDisabled?: boolean + skipDisabled?: boolean + showActions?: boolean + showSkipToggle?: boolean } + +const summaryWarning = (row: PostImportRow): string | null => + Object.values (row.fieldWarnings ?? { }).flat ()[0] + ?? row.baseWarnings?.[0] + ?? null + +const summaryDate = (row: PostImportRow): string => + originalCreatedAtString ( + row.attributes.originalCreatedFrom?.toString () ?? null, + row.attributes.originalCreatedBefore?.toString () ?? null) + + +const PostImportRowSummary: FC = ( + { row, + displayNumber, + onEdit, + onRetry, + onToggleSkip, + rowMessages, + editDisabled, + retryDisabled, + skipDisabled, + showActions = true, + showSkipToggle = false }, +) => { + const warning = summaryWarning (row) + const displayStatus = displayPostImportStatus (row) + const editVisible = onEdit != null + const editAllowed = editVisible && canEditReviewRow (row) + const retryAllowed = onRetry != null && canRetryResultRow (row) + const skipChecked = row.skipReason === 'manual' + const rowNumber = displayNumber ?? row.sourceRow + const duration = String (row.attributes.duration ?? '') + const showDuration = hasVideoTag (row.attributes.tags) && duration !== '' + const skipControl = showSkipToggle + ? ( + ) + : null + + return ( + <> +
+
+
#{rowNumber}
+
+ +
+
+ {String (row.attributes.title ?? '')} +
+
+ {row.url} +
+ +
+ {summaryDate (row)} +
+ {showDuration && ( +
+ 動画時間 {duration} +
)} + {warning && ( +
+ {warning} +
)} + +
+
+ {displayStatus != null && } +
+
+
+ {skipControl} + {showActions && editVisible && ( + )} + {showActions && retryAllowed && ( + )} +
+
+
+ +
+
#{rowNumber}
+
+ +
+
+ {String (row.attributes.title ?? '')} +
+
+ {row.url} +
+
+ {displayStatus != null && } +
+ +
+ {summaryDate (row)} +
+ {showDuration && ( +
+ 動画時間 {duration} +
)} + {warning && ( +
+ {warning} +
)} + + {skipControl} +
+
+ {showActions && (editVisible || retryAllowed) && ( +
+ {editVisible && ( + )} + {retryAllowed && ( + )} +
)} +
+ ) +} + +export default PostImportRowSummary diff --git a/frontend/src/components/posts/import/PostImportStatusBadge.tsx b/frontend/src/components/posts/import/PostImportStatusBadge.tsx new file mode 100644 index 0000000..6c66990 --- /dev/null +++ b/frontend/src/components/posts/import/PostImportStatusBadge.tsx @@ -0,0 +1,33 @@ +import StatusBadge from '@/components/common/StatusBadge' + +import type { FC } from 'react' + +import type { StatusBadgeTone } from '@/components/common/StatusBadge' +import type { PostImportBadgeValue } from '@/components/posts/import/postImportRowStatus' + +type Props = { + value: PostImportBadgeValue } + +const LABELS: Record = { + ready: '登録可能', + error: '登録不可', + warning: '警告', + skipped: 'スキップ', + created: '登録済み', + failed: '登録失敗' } + +const TONES: Record = { + ready: 'success', + error: 'warning', + warning: 'warning', + skipped: 'neutral', + created: 'success', + failed: 'warning' } + + +const PostImportStatusBadge: FC = ({ value }) => ( + + {LABELS[value]} + ) + +export default PostImportStatusBadge diff --git a/frontend/src/components/posts/import/PostImportTagLinks.tsx b/frontend/src/components/posts/import/PostImportTagLinks.tsx new file mode 100644 index 0000000..2900d29 --- /dev/null +++ b/frontend/src/components/posts/import/PostImportTagLinks.tsx @@ -0,0 +1,36 @@ +import TagLink from '@/components/TagLink' + +import type { FC } from 'react' + +import type { PostImportDisplayTag } from '@/lib/postImportTypes' + +type Props = { + tags: PostImportDisplayTag[] | undefined } + +const PostImportTagLinks: FC = ({ tags }) => { + if (tags == null || tags.length === 0) + return null + + return ( +
+ {tags.map (tag => { + const key = `${ tag.category }:${ tag.name }:${ tag.sectionLiterals?.join ('|') ?? '' }` + return ( + + + {tag.sectionLiterals?.map (literal => ( + + {literal} + ))} + ) + })} +
) +} + +export default PostImportTagLinks diff --git a/frontend/src/components/posts/import/PostImportThumbnailPreview.test.tsx b/frontend/src/components/posts/import/PostImportThumbnailPreview.test.tsx new file mode 100644 index 0000000..63879a7 --- /dev/null +++ b/frontend/src/components/posts/import/PostImportThumbnailPreview.test.tsx @@ -0,0 +1,59 @@ +import { fireEvent, render, screen } from '@testing-library/react' +import { beforeEach, describe, expect, it, vi } from 'vitest' + +import PostImportThumbnailPreview from '@/components/posts/import/PostImportThumbnailPreview' + +describe ('PostImportThumbnailPreview', () => { + beforeEach (() => { + vi.clearAllMocks () + globalThis.URL.createObjectURL = vi.fn (() => 'blob:preview') + globalThis.URL.revokeObjectURL = vi.fn () + }) + + it ('renders the remote URL directly without a backend proxy', () => { + render ( + ) + + expect (screen.getByRole ('img')).toHaveAttribute ( + 'src', + 'https://example.com/thumbnail.jpg') + expect (screen.getByRole ('img')).toHaveAttribute ( + 'referrerpolicy', + 'no-referrer') + }) + + it ('shows the empty frame after the remote image fails', () => { + const { container } = render ( + ) + + fireEvent.error (screen.getByRole ('img')) + + expect (screen.queryByRole ('img')).toBeNull () + expect (container.querySelector ('div.rounded.border.bg-muted')).not.toBeNull () + expect (container.textContent).toBe ('') + }) + + it ('uses and revokes an object URL only when the remote URL is blank', () => { + const file = new File (['image'], 'thumbnail.png', { type: 'image/png' }) + const { rerender, unmount } = render ( + ) + + expect (screen.getByRole ('img')).toHaveAttribute ('src', 'blob:preview') + + rerender ( + ) + expect (screen.getByRole ('img')).toHaveAttribute ( + 'src', + 'https://example.com/remote.jpg') + + unmount () + expect (globalThis.URL.revokeObjectURL).toHaveBeenCalledWith ('blob:preview') + }) +}) diff --git a/frontend/src/components/posts/import/PostImportThumbnailPreview.tsx b/frontend/src/components/posts/import/PostImportThumbnailPreview.tsx new file mode 100644 index 0000000..6afad5f --- /dev/null +++ b/frontend/src/components/posts/import/PostImportThumbnailPreview.tsx @@ -0,0 +1,25 @@ +import PostThumbnailPreview from '@/components/posts/PostThumbnailPreview' + +import type { FC } from 'react' + +type Props = { + url: string + file?: File + alt?: string + className?: string } + + +const PostImportThumbnailPreview: FC = ( + { url, + file, + alt = 'サムネール', + className = 'h-16 w-16' }, +) => ( + ) + +export default PostImportThumbnailPreview diff --git a/frontend/src/components/posts/import/postImportRowStatus.test.ts b/frontend/src/components/posts/import/postImportRowStatus.test.ts new file mode 100644 index 0000000..1f61191 --- /dev/null +++ b/frontend/src/components/posts/import/postImportRowStatus.test.ts @@ -0,0 +1,29 @@ +import { describe, expect, it } from 'vitest' + +import { displayPostImportStatus } from '@/components/posts/import/postImportRowStatus' +import { buildPostImportRow } from '@/test/postImportFactories' + +describe ('displayPostImportStatus', () => { + it ('shows only ready, warning, and skipped states', () => { + expect (displayPostImportStatus (buildPostImportRow ())).toBe ('ready') + expect (displayPostImportStatus (buildPostImportRow ({ + status: 'warning', + fieldWarnings: { title: ['warning'] } }))).toBe ('warning') + expect (displayPostImportStatus (buildPostImportRow ({ + skipReason: 'existing', + existingPostId: 2 }))).toBe ('skipped') + expect (displayPostImportStatus (buildPostImportRow ({ + skipReason: 'manual' }))).toBe ('skipped') + }) + + it ('distinguishes validation, failure, and created states', () => { + expect (displayPostImportStatus (buildPostImportRow ({ + status: 'error', + validationErrors: { title: ['invalid'] } }))).toBe ('error') + expect (displayPostImportStatus (buildPostImportRow ({ + importStatus: 'failed' }))).toBe ('failed') + expect (displayPostImportStatus (buildPostImportRow ({ + importStatus: 'created', + createdPostId: 3 }))).toBe ('created') + }) +}) diff --git a/frontend/src/components/posts/import/postImportRowStatus.ts b/frontend/src/components/posts/import/postImportRowStatus.ts new file mode 100644 index 0000000..03b4eb9 --- /dev/null +++ b/frontend/src/components/posts/import/postImportRowStatus.ts @@ -0,0 +1,34 @@ +import type { PostImportRow } from '@/lib/postImportTypes' + +export type PostImportDisplayStatus = + 'ready' + | 'error' + | 'skipped' + | 'warning' + | 'created' + | 'failed' + +export type PostImportBadgeValue = PostImportDisplayStatus + +const hasWarnings = (row: PostImportRow): boolean => + Object.values (row.fieldWarnings ?? { }).some (messages => messages.length > 0) + || row.baseWarnings.length > 0 + +export const displayPostImportStatus = ( + row: PostImportRow, +): PostImportDisplayStatus | null => + row.status === 'pending' + ? null + : (row.importStatus === 'failed') + ? 'failed' + : (row.skipReason != null || row.importStatus === 'skipped') + ? 'skipped' + : (row.importStatus === 'created') + ? 'created' + : (row.status === 'error') + ? 'error' + : (Object.values (row.validationErrors ?? { }).some (messages => messages.length > 0)) + ? 'error' + : ((hasWarnings (row) || row.status === 'warning') + ? 'warning' + : 'ready') diff --git a/frontend/src/components/ui/dialog.tsx b/frontend/src/components/ui/dialog.tsx index a6e9a8e..e3c154c 100644 --- a/frontend/src/components/ui/dialog.tsx +++ b/frontend/src/components/ui/dialog.tsx @@ -1,10 +1,10 @@ -"use client" +'use client' -import * as React from "react" -import * as DialogPrimitive from "@radix-ui/react-dialog" -import { X } from "lucide-react" +import * as DialogPrimitive from '@radix-ui/react-dialog' +import { X } from 'lucide-react' +import * as React from 'react' -import { cn } from "@/lib/utils" +import { cn } from '@/lib/utils' const Dialog = DialogPrimitive.Root @@ -15,111 +15,109 @@ const DialogPortal = DialogPrimitive.Portal const DialogClose = DialogPrimitive.Close const DialogOverlay = React.forwardRef< - React.ElementRef, - React.ComponentPropsWithoutRef ->(({ className, ...props }, ref) => ( - )) + React.ElementRef, + React.ComponentPropsWithoutRef +>( + ({ className, ...props }, ref) => ( + )) DialogOverlay.displayName = DialogPrimitive.Overlay.displayName const DialogContent = React.forwardRef< - React.ElementRef, - React.ComponentPropsWithoutRef ->(({ className, children, ...props }, ref) => ( - - - - {children} + React.ElementRef, + React.ComponentPropsWithoutRef +>( + ({ className, children, ...props }, ref) => ( + + + + {children} - - - 閉ぢる - - - )) + + + 閉ぢる + + + )) DialogContent.displayName = DialogPrimitive.Content.displayName -const DialogHeader = ({ - className, - ...props -}: React.HTMLAttributes) => ( -
) -DialogHeader.displayName = "DialogHeader" +const DialogHeader = ( + { className, ...props }: React.HTMLAttributes, +) => ( +
) +DialogHeader.displayName = 'DialogHeader' -const DialogFooter = ({ - className, - ...props -}: React.HTMLAttributes) => ( -
) -DialogFooter.displayName = "DialogFooter" +const DialogFooter = ( + { className, ...props }: React.HTMLAttributes, +) => ( +
) +DialogFooter.displayName = 'DialogFooter' const DialogTitle = React.forwardRef< - React.ElementRef, - React.ComponentPropsWithoutRef ->(({ className, ...props }, ref) => ( - )) + React.ElementRef, + React.ComponentPropsWithoutRef +>( + ({ className, ...props }, ref) => ( + )) DialogTitle.displayName = DialogPrimitive.Title.displayName const DialogDescription = React.forwardRef< - React.ElementRef, - React.ComponentPropsWithoutRef ->(({ className, ...props }, ref) => ( - )) + React.ElementRef, + React.ComponentPropsWithoutRef +>( + ({ className, ...props }, ref) => ( + )) DialogDescription.displayName = DialogPrimitive.Description.displayName export { - Dialog, - DialogPortal, - DialogOverlay, - DialogClose, - DialogTrigger, - DialogContent, - DialogHeader, - DialogFooter, - DialogTitle, - DialogDescription, + Dialog, + DialogPortal, + DialogOverlay, + DialogClose, + DialogTrigger, + DialogContent, + DialogHeader, + DialogFooter, + DialogTitle, + DialogDescription, } diff --git a/frontend/src/lib/api.ts b/frontend/src/lib/api.ts index 654c51f..7f9148d 100644 --- a/frontend/src/lib/api.ts +++ b/frontend/src/lib/api.ts @@ -8,6 +8,7 @@ import type { AxiosError, AxiosRequestConfig } from 'axios' type Opt = { params?: AxiosRequestConfig['params'] headers?: Record + signal?: AbortSignal responseType?: 'blob' } const client = axios.create ({ baseURL: API_BASE_URL }) diff --git a/frontend/src/lib/dialogues/useDialogue.ts b/frontend/src/lib/dialogues/useDialogue.ts new file mode 100644 index 0000000..579c824 --- /dev/null +++ b/frontend/src/lib/dialogues/useDialogue.ts @@ -0,0 +1,72 @@ +import { createContext, useContext } from 'react' + +import type { ReactNode } from 'react' + +type DialogueVariant = 'default' | 'danger' + +type ConfirmOptions = { title: string + description?: ReactNode + confirmText?: string + cancelText?: string + variant?: DialogueVariant } + +type AlertOptions = { title: string + description?: ReactNode + okText?: string } + +type Choice = { value: T + label: string + variant?: DialogueVariant } + +type ChoiceOptions = { title: string + description?: ReactNode + choices: Choice[] + cancelText?: string } + +type DialogueFormAction = { + label: string + placement?: 'start' | 'end' + variant?: DialogueVariant + disabled?: boolean + onSelect: () => Promise | boolean | void } + +type DialogueFormControls = { + close: () => void + setActions: (actions: DialogueFormAction[]) => void + confirm: (options: ConfirmOptions) => Promise } + +type DialogueFormOptions = { title: string + description?: ReactNode + body: (controls: DialogueFormControls) => ReactNode + cancelText?: string + size?: 'default' | 'large' } + +type DialogueAPI = + { confirm: (options: ConfirmOptions) => Promise + alert: (options: AlertOptions) => Promise + choice: (options: ChoiceOptions) => Promise + form: (options: DialogueFormOptions) => Promise } + +const DialogueContext = createContext (null) + +const useDialogue = () => { + const dialogue = useContext (DialogueContext) + + if (dialogue == null) + throw new Error ('useDialogue must be used inside DialogueProvider') + + return dialogue +} + +export { DialogueContext, useDialogue } +export default useDialogue +export type { + AlertOptions, + Choice, + ChoiceOptions, + ConfirmOptions, + DialogueAPI, + DialogueFormAction, + DialogueFormControls, + DialogueFormOptions, + DialogueVariant } diff --git a/frontend/src/lib/postImportRows.test.ts b/frontend/src/lib/postImportRows.test.ts new file mode 100644 index 0000000..f239f0f --- /dev/null +++ b/frontend/src/lib/postImportRows.test.ts @@ -0,0 +1,292 @@ +import { describe, expect, it } from 'vitest' + +import { creatableImportRows, + buildNextEditedRow, + canEditResultRow, + canEditReviewRow, + canRetryResultRow, + hasExactSourceRows, + initialisePreviewRows, + mergeImportResults, + mergeValidatedImportRow, + mergeValidatedImportRows, + processableImportRows, + replaceImportRow, + resultRepairMode, + resultRowMessages, + resultSummaryCounts, + retryImportRow, + reviewSummaryCounts } from '@/lib/postImportRows' +import { buildPostImportRow } from '@/test/postImportFactories' + +describe ('post import row state', () => { + it ('separates processable existing rows from creatable rows', () => { + const ready = buildPostImportRow ({ sourceRow: 1 }) + const existing = buildPostImportRow ({ + sourceRow: 2, + skipReason: 'existing', + existingPostId: 20 }) + const manual = buildPostImportRow ({ + sourceRow: 3, + skipReason: 'manual' }) + const invalid = buildPostImportRow ({ + sourceRow: 4, + status: 'error', + validationErrors: { url: ['invalid'] } }) + const created = buildPostImportRow ({ + sourceRow: 5, + importStatus: 'created', + createdPostId: 40 }) + const rows = [ready, existing, manual, invalid, created] + + expect (processableImportRows (rows)).toEqual ([ready]) + expect (creatableImportRows (rows)).toEqual ([ready]) + expect (reviewSummaryCounts (rows)).toEqual ({ + creatable: 1, + manualSkipped: 1, + existingSkipped: 1, + pendingOrError: 1 }) + }) + + it ('preserves terminal rows while merging validation results', () => { + const created = buildPostImportRow ({ + sourceRow: 1, + importStatus: 'created', + createdPostId: 10, + attributes: { title: 'created title' } }) + const pending = buildPostImportRow ({ + sourceRow: 2, + fieldWarnings: { title: ['old warning'] }, + provenance: { title: 'manual' }, + attributes: { title: 'manual title' } }) + const validated = [ + buildPostImportRow ({ sourceRow: 1, attributes: { title: 'changed' } }), + buildPostImportRow ({ + sourceRow: 2, + fieldWarnings: { title: ['fetch warning'], tags: ['tag warning'] }, + provenance: { title: 'manual' }, + attributes: { title: 'manual title' } })] + + const result = mergeValidatedImportRows ([created, pending], validated) + + expect (result[0]).toBe (created) + expect (result[1]?.fieldWarnings).toEqual ({ tags: ['tag warning'] }) + }) + + it ('updates the reset snapshot only after metadata URL changes', () => { + const current = buildPostImportRow ({ + attributes: { title: 'manual title' }, + metadataUrl: 'https://example.com/old' }) + const validated = buildPostImportRow ({ + attributes: { title: 'new metadata title' }, + metadataUrl: 'https://example.com/new', + fieldWarnings: { title: ['warning'] }, + baseWarnings: ['base warning'] }) + + const result = mergeValidatedImportRows ([current], [validated])[0] + + expect (result?.resetSnapshot).toMatchObject ({ + attributes: { title: 'new metadata title' }, + fieldWarnings: { title: ['warning'] }, + baseWarnings: ['base warning'], + metadataUrl: 'https://example.com/new' }) + }) + + it ('merges result states and clears incompatible post identifiers', () => { + const created = mergeImportResults ([buildPostImportRow ({ + skipReason: 'existing', + existingPostId: 2, + recoverable: true, + importStatus: 'pending' })], [{ + sourceRow: 1, + status: 'created', + post: { id: 3 } }])[0] + const skipped = mergeImportResults ([buildPostImportRow ({ + createdPostId: 3, + importStatus: 'created', + recoverable: true })], [{ + sourceRow: 1, + status: 'skipped', + existingPostId: 4 }])[0] + const failed = mergeImportResults ([buildPostImportRow ({ + skipReason: 'existing', + existingPostId: 4 })], [{ + sourceRow: 1, + status: 'failed', + recoverable: true, + errors: { base: ['failure'] } }])[0] + + expect (created).toMatchObject ({ + importStatus: 'created', + createdPostId: 3, + existingPostId: undefined, + recoverable: undefined, + skipReason: undefined }) + expect (skipped).toMatchObject ({ + importStatus: 'skipped', + existingPostId: 4, + createdPostId: undefined, + recoverable: undefined, + skipReason: 'existing' }) + expect (failed).toMatchObject ({ + importStatus: 'failed', + recoverable: true, + createdPostId: undefined, + existingPostId: undefined, + skipReason: undefined, + importErrors: { base: ['failure'] } }) + }) + + it ('retries only the selected failed row and counts results exclusively', () => { + const rows = [ + buildPostImportRow ({ sourceRow: 1, importStatus: 'created', createdPostId: 1 }), + buildPostImportRow ({ sourceRow: 2, importStatus: 'skipped', + existingPostId: 2, skipReason: 'existing' }), + buildPostImportRow ({ sourceRow: 3, importStatus: 'failed', + recoverable: true, importErrors: { base: ['failed'] } }), + buildPostImportRow ({ sourceRow: 4, importStatus: 'pending', + recoverable: true, validationErrors: { base: ['failed'] } })] + + expect (resultSummaryCounts (rows)).toEqual ({ created: 1, skipped: 1, failed: 2 }) + expect (retryImportRow (rows, 3)[2]).toMatchObject ({ + importStatus: 'pending', + importErrors: undefined }) + expect (retryImportRow (rows, 4)[3]).toBe (rows[3]) + }) + + it ('deduplicates messages and keeps repair mode only for repairable rows', () => { + const repairable = buildPostImportRow ({ + sourceRow: 1, + importStatus: 'pending', + validationErrors: { title: ['invalid'], base: ['duplicate'] }, + importErrors: { base: ['duplicate'], url: ['network'] } }) + const complete = buildPostImportRow ({ + sourceRow: 2, + importStatus: 'created', + createdPostId: 2 }) + + expect (resultRowMessages (repairable)).toEqual (['invalid', 'duplicate', 'network']) + expect (resultRepairMode ([repairable, complete])).toBe ('failed') + expect (resultRepairMode ([complete])).toBe ('all') + }) + + it ('classifies editable and retryable rows by terminal and recoverable state', () => { + const ready = buildPostImportRow () + const skipped = buildPostImportRow ({ + importStatus: 'skipped', + skipReason: 'existing', + existingPostId: 2 }) + const hardFailed = buildPostImportRow ({ + importStatus: 'failed', + importErrors: { base: ['failed'] } }) + const pendingInvalid = buildPostImportRow ({ + importStatus: 'pending', + recoverable: true, + validationErrors: { title: ['invalid'] } }) + const pendingValid = buildPostImportRow ({ + importStatus: 'pending', + recoverable: true }) + + expect (canEditReviewRow (ready)).toBe (true) + expect (canEditReviewRow (skipped)).toBe (false) + expect (canEditReviewRow (hardFailed)).toBe (false) + expect (canEditResultRow (pendingInvalid)).toBe (true) + expect (canRetryResultRow (pendingInvalid)).toBe (false) + expect (canRetryResultRow (pendingValid)).toBe (true) + }) + + it ('detects missing, duplicate, and extra source rows exactly', () => { + expect (hasExactSourceRows ([1, 2], [{ sourceRow: 1 }, { sourceRow: 2 }])).toBe (true) + expect (hasExactSourceRows ([1, 2], [{ sourceRow: 1 }])).toBe (false) + expect (hasExactSourceRows ([1, 2], [{ sourceRow: 1 }, { sourceRow: 1 }])).toBe (false) + expect (hasExactSourceRows ([1, 2], [{ sourceRow: 1 }, { sourceRow: 3 }])).toBe (false) + }) + + it ('merges only the validated source row and preserves other row edits', () => { + const edited = buildPostImportRow ({ + sourceRow: 1, + attributes: { title: 'edited row' }, + provenance: { title: 'manual' } }) + const other = buildPostImportRow ({ + sourceRow: 2, + attributes: { title: 'keep me' }, + provenance: { title: 'manual' } }) + const validated = buildPostImportRow ({ + sourceRow: 1, + attributes: { title: 'validated row' } }) + + const result = mergeValidatedImportRow ([edited, other], validated) + + expect (result[0]?.attributes.title).toBe ('validated row') + expect (result[1]?.attributes.title).toBe ('keep me') + }) + + it ('replaces only the targeted source row and preserves the others', () => { + const original = buildPostImportRow ({ + sourceRow: 1, + importStatus: 'failed', + recoverable: true, + importErrors: { base: ['failed'] } }) + const other = buildPostImportRow ({ + sourceRow: 2, + attributes: { title: 'keep edited row' } }) + const restored = buildPostImportRow ({ + sourceRow: 1, + importStatus: 'failed', + recoverable: true, + importErrors: { base: ['failed'] } }) + + const result = replaceImportRow ([original, other], restored) + + expect (result[0]).toEqual (restored) + expect (result[1]?.attributes.title).toBe ('keep edited row') + }) + + it ('copies reset snapshot values instead of sharing mutable records', () => { + const row = buildPostImportRow ({ fieldWarnings: { title: ['warning'] } }) + const initialised = initialisePreviewRows ([row])[0] + expect (initialised).toBeDefined () + if (initialised == null) + return + + initialised.attributes.title = 'changed' + initialised.fieldWarnings.title?.push ('another') + + expect (initialised.resetSnapshot.attributes.title).toBe ('') + expect (initialised.resetSnapshot.fieldWarnings.title).toEqual (['warning']) + }) + + it ('builds the next edited row with shared repair semantics', () => { + const row = buildPostImportRow ({ + url: 'https://example.com/original', + importStatus: 'failed', + recoverable: true, + importErrors: { base: ['failed'] }, + attributes: { title: 'old title', tags: 'old-tag', duration: '2' }, + provenance: { title: 'automatic', tags: 'automatic', url: 'manual' }, + tagSources: { automatic: 'old-tag', manual: '' } }) + + const nextRow = buildNextEditedRow ( + row, + { + url: 'https://example.com/edited', + title: 'edited title', + thumbnailBase: '', + originalCreatedFrom: '', + originalCreatedBefore: '', + duration: '2', + tags: 'edited-tag', + parentPostIds: '' }, + true) + + expect (nextRow.importStatus).toBe ('pending') + expect (nextRow.importErrors).toBeUndefined () + expect (nextRow.url).toBe ('https://example.com/edited') + expect (nextRow.attributes.title).toBe ('edited title') + expect (nextRow.attributes.duration).toBe ('2') + expect (nextRow.attributes.tags).toBe ('edited-tag') + expect (nextRow.provenance.title).toBe ('manual') + expect (nextRow.provenance.url).toBe ('manual') + expect (nextRow.tagSources?.manual).toBe ('edited-tag') + }) +}) diff --git a/frontend/src/lib/postImportRows.ts b/frontend/src/lib/postImportRows.ts new file mode 100644 index 0000000..366a767 --- /dev/null +++ b/frontend/src/lib/postImportRows.ts @@ -0,0 +1,421 @@ +import type { PostImportEditableDraft, + PostImportResultRow, + PostImportRow } from '@/lib/postImportTypes' + +const THUMBNAIL_MISSING_WARNING = 'サムネールなし' + +export const hasThumbnailBaseValue = (value: unknown): boolean => + typeof value === 'string' && value.trim () !== '' + +export const hasVideoTag = (value: unknown): boolean => + typeof value === 'string' + && value.split (/\s+/).includes ('動画') + +export const isExistingSkipRow = (row: PostImportRow): boolean => + row.skipReason === 'existing' + + +export const isManualSkipRow = (row: PostImportRow): boolean => + row.skipReason === 'manual' + + +const hasSkipReason = (row: PostImportRow): boolean => + row.skipReason != null + +export const compactMessageRecord = ( + messages: Record, +): Record => + Object.fromEntries ( + Object.entries (messages).filter (([, values]) => values.length > 0)) + + +export const hasErrorMessages = ( + messages: Record, +): boolean => + Object.values (messages).some (values => values.length > 0) + + +const hasValidationErrors = (row: PostImportRow): boolean => + hasErrorMessages (row.validationErrors ?? { }) + +const isRecoverableRow = (row: PostImportRow): boolean => + row.recoverable === true + +const isRepairableImportStatus = (row: PostImportRow): boolean => + isRecoverableRow (row) + && (row.importStatus === 'failed' || row.importStatus === 'pending') + + +export const isNonRecoverableFailedRow = (row: PostImportRow): boolean => + row.importStatus === 'failed' && row.recoverable !== true + + +export const isTerminalRow = (row: PostImportRow): boolean => + row.importStatus === 'created' + || row.importStatus === 'skipped' + || isNonRecoverableFailedRow (row) + + +export const isCompletedReviewRow = (row: PostImportRow): boolean => + row.importStatus === 'created' + || row.importStatus === 'skipped' + || hasSkipReason (row) + + +export const validatableImportRows = (rows: PostImportRow[]): PostImportRow[] => + rows.filter (row => !(hasSkipReason (row)) && !(isTerminalRow (row))) + +const buildResetSnapshot = (row: PostImportRow) => ({ + url: row.url, + attributes: { ...row.attributes }, + displayTags: row.displayTags?.map (tag => ({ + name: tag.name, + category: tag.category, + sectionLiterals: tag.sectionLiterals == null ? undefined : [...tag.sectionLiterals] })) ?? [], + provenance: { ...row.provenance }, + tagSources: { + automatic: row.tagSources?.automatic ?? '', + manual: row.tagSources?.manual ?? '' }, + fieldWarnings: Object.fromEntries ( + Object.entries (row.fieldWarnings).map (([key, values]) => [key, [...values]])), + baseWarnings: [...row.baseWarnings], + metadataUrl: row.metadataUrl }) + +const deduped = (values: string[]): string[] => + [...new Set (values)] + + +const thumbnailWarnings = (row: PostImportRow): string[] => { + const current = row.fieldWarnings.thumbnailBase ?? [] + const others = current.filter (message => message !== THUMBNAIL_MISSING_WARNING) + return hasThumbnailBaseValue (row.attributes.thumbnailBase) || row.thumbnailFile != null + ? others + : deduped ([...others, THUMBNAIL_MISSING_WARNING]) +} + + +export const applyThumbnailWarning = (row: PostImportRow): PostImportRow => ({ + ...row, + fieldWarnings: compactMessageRecord ({ + ...row.fieldWarnings, + thumbnailBase: thumbnailWarnings (row) }) }) + + +export const applyThumbnailWarnings = (rows: PostImportRow[]): PostImportRow[] => + rows.map (row => applyThumbnailWarning (row)) + + +export const processableImportRows = (rows: PostImportRow[]): PostImportRow[] => + validatableImportRows (rows).filter (row => { + if (row.status === 'pending') + return false + if (row.importStatus === 'created') + return false + if (row.importStatus === 'skipped') + return false + if (row.importStatus === 'failed') + return false + if (hasValidationErrors (row)) + return false + return row.importStatus == null || row.importStatus === 'pending' + }) + +export const creatableImportRows = (rows: PostImportRow[]): PostImportRow[] => + processableImportRows (rows).filter (row => !(hasSkipReason (row))) + + +export const reviewSummaryCounts = (rows: PostImportRow[]) => ({ + creatable: rows.filter (row => creatableImportRows ([row]).length > 0).length, + manualSkipped: rows.filter (row => isManualSkipRow (row)).length, + existingSkipped: rows.filter (row => isExistingSkipRow (row)).length, + pendingOrError: rows.filter (row => + !(isCompletedReviewRow (row)) + && creatableImportRows ([row]).length === 0).length }) + + +export const resultSummaryCounts = (rows: PostImportRow[]) => + rows.reduce ( + (counts, row) => { + if (row.importStatus === 'created') + { + ++counts.created + return counts + } + if (row.importStatus === 'skipped') + { + ++counts.skipped + return counts + } + if (row.importStatus === 'failed' + || (row.recoverable === true && row.importStatus === 'pending')) + ++counts.failed + return counts + }, + { created: 0, skipped: 0, failed: 0 }) + + +export const resultRepairMode = ( + rows: PostImportRow[], +): 'all' | 'failed' => + rows.some (row => hasValidationErrors (row) || isRepairableImportStatus (row)) + ? 'failed' + : 'all' + + +export const canEditReviewRow = (row: PostImportRow): boolean => + !(hasSkipReason (row) + || row.importStatus === 'created' + || row.importStatus === 'skipped' + || (row.importStatus === 'failed' && row.recoverable !== true)) + + +export const canEditResultRow = (row: PostImportRow): boolean => + row.skipReason == null + && row.recoverable === true + && (row.importStatus === 'failed' + || (row.importStatus === 'pending' && hasValidationErrors (row))) + + +export const canRetryResultRow = (row: PostImportRow): boolean => + row.skipReason == null + && row.recoverable === true + && (row.importStatus === 'failed' + || (row.importStatus === 'pending' && !(hasValidationErrors (row)))) + + +export const resultRowMessages = (row: PostImportRow): string[] => + [...new Set ([ + ...Object.values (row.validationErrors ?? { }).flat (), + ...Object.values (row.importErrors ?? { }).flat ()])] + + +export const resultRowWarnings = (row: PostImportRow): string[] => + [...new Set ([ + ...Object.values (row.fieldWarnings ?? { }).flat (), + ...row.baseWarnings])] + + +const isManualChange = ( + current: unknown, + next: string, +): boolean => + next !== String (current ?? '') + + +export const buildNextEditedRow = ( + editingRow: PostImportRow, + draft: PostImportEditableDraft, + urlChanged: boolean, +): PostImportRow => { + const nextProvenance = { ...editingRow.provenance } + const nextAttributes = { ...editingRow.attributes } + const nextTagSources = { + automatic: editingRow.tagSources?.automatic ?? '', + manual: editingRow.tagSources?.manual ?? '' } + const draftFields = [ + ['title', draft.title], + ['thumbnailBase', draft.thumbnailBase], + ['originalCreatedFrom', draft.originalCreatedFrom], + ['originalCreatedBefore', draft.originalCreatedBefore], + ['duration', draft.duration], + ['parentPostIds', draft.parentPostIds]] as const + draftFields.forEach (([field, value]) => { + nextAttributes[field] = value + nextProvenance[field] = + isManualChange (editingRow.attributes[field], value) + ? 'manual' + : (editingRow.provenance[field] ?? 'automatic') + }) + nextAttributes.tags = draft.tags + if (isManualChange (editingRow.attributes.tags, draft.tags)) + { + nextProvenance.tags = 'manual' + nextTagSources.manual = draft.tags + } + else + { + nextProvenance.tags = editingRow.provenance.tags ?? 'automatic' + nextTagSources.manual = editingRow.tagSources?.manual ?? '' + } + + return { + ...editingRow, + url: draft.url, + attributes: nextAttributes, + displayTags: + editingRow.displayTags?.map (tag => ({ + name: tag.name, + category: tag.category, + sectionLiterals: tag.sectionLiterals == null ? undefined : [...tag.sectionLiterals] })), + thumbnailFile: draft.thumbnailFile, + provenance: { + ...nextProvenance, + url: urlChanged ? 'manual' : (editingRow.provenance.url ?? 'manual') }, + tagSources: nextTagSources, + importStatus: editingRow.importStatus === 'created' ? 'created' : 'pending', + importErrors: undefined } +} + + +export const hasExactSourceRows = ( + expected: number[], + actual: Array<{ sourceRow: number }>, +): boolean => { + if (expected.length !== actual.length) + return false + + const expectedSorted = [...expected].sort ((a, b) => a - b) + const actualSorted = actual.map (row => row.sourceRow).sort ((a, b) => a - b) + return expectedSorted.every ((value, index) => value === actualSorted[index]) +} + + +export const replaceImportRow = ( + rows: PostImportRow[], + nextRow: PostImportRow, +): PostImportRow[] => + rows.map (row => row.sourceRow === nextRow.sourceRow ? nextRow : row) + + +export const mergeValidatedImportRow = ( + rows: PostImportRow[], + validated: PostImportRow, +): PostImportRow[] => { + const current = rows.find (row => row.sourceRow === validated.sourceRow) + if (current == null) + return rows + + const [merged] = mergeValidatedImportRows ([current], [validated]) + return merged == null ? rows : replaceImportRow (rows, merged) +} + + +export const mergeValidatedImportRows = ( + current: PostImportRow[], + validated: PostImportRow[], +): PostImportRow[] => { + const validatedMap = new Map (validated.map (row => [row.sourceRow, row])) + return current.map (previous => { + if ( + previous.importStatus === 'created' + || previous.importStatus === 'skipped' + || previous.importStatus === 'failed') + return previous + + const row = validatedMap.get (previous.sourceRow) + if (row == null) + return previous + + const fieldWarnings = + hasErrorMessages (row.fieldWarnings) || row.metadataUrl !== previous.metadataUrl + ? { ...row.fieldWarnings } + : { ...previous.fieldWarnings } + for (const [field, origin] of Object.entries (row.provenance)) + { + if (origin === 'manual') + delete fieldWarnings[field] + } + + return { + ...previous, + url: row.url, + attributes: row.attributes, + displayTags: + row.displayTags?.map (tag => ({ + name: tag.name, + category: tag.category, + sectionLiterals: + tag.sectionLiterals == null ? undefined : [...tag.sectionLiterals] })), + provenance: row.provenance, + tagSources: row.tagSources, + skipReason: row.skipReason, + existingPostId: row.existingPostId, + existingPost: row.existingPost, + fieldWarnings, + baseWarnings: + row.baseWarnings.length > 0 || row.metadataUrl !== previous.metadataUrl + ? row.baseWarnings + : previous.baseWarnings, + validationErrors: row.validationErrors, + status: row.status, + metadataUrl: row.metadataUrl, + resetSnapshot: + row.metadataUrl !== previous.metadataUrl + ? buildResetSnapshot (row) + : previous.resetSnapshot } + }) +} + + +export const mergeImportResults = ( + rows: PostImportRow[], + results: PostImportResultRow[], +): PostImportRow[] => { + const resultMap = new Map (results.map (row => [row.sourceRow, row])) + return rows.map (row => { + const result = resultMap.get (row.sourceRow) + if (result == null) + return row + + switch (result.status) + { + case 'created': + return { + ...row, + importStatus: 'created', + recoverable: undefined, + skipReason: undefined, + createdPostId: result.post.id, + existingPostId: undefined, + existingPost: undefined, + fieldWarnings: compactMessageRecord ( + result.fieldWarnings ?? row.fieldWarnings), + baseWarnings: result.baseWarnings ?? row.baseWarnings, + importErrors: compactMessageRecord (result.errors ?? { }) } + case 'skipped': + return { + ...row, + importStatus: 'skipped', + recoverable: undefined, + skipReason: 'existing', + createdPostId: undefined, + existingPostId: result.existingPostId, + existingPost: result.existingPost ?? row.existingPost, + fieldWarnings: compactMessageRecord ( + result.fieldWarnings ?? row.fieldWarnings), + baseWarnings: result.baseWarnings ?? row.baseWarnings, + importErrors: compactMessageRecord (result.errors ?? { }) } + case 'failed': + return { + ...row, + importStatus: 'failed', + recoverable: result.recoverable === true ? true : undefined, + skipReason: undefined, + createdPostId: undefined, + existingPostId: undefined, + existingPost: undefined, + fieldWarnings: compactMessageRecord ( + result.fieldWarnings ?? row.fieldWarnings), + baseWarnings: result.baseWarnings ?? row.baseWarnings, + importErrors: compactMessageRecord (result.errors ?? { }) } + } + }) +} + + +export const retryImportRow = ( + rows: PostImportRow[], + sourceRow: number, +): PostImportRow[] => + rows.map (row => + row.sourceRow === sourceRow + && row.importStatus === 'failed' + && row.recoverable === true + ? { ...row, importStatus: 'pending', importErrors: undefined } + : row) + +export const initialisePreviewRows = (rows: PostImportRow[]): PostImportRow[] => + applyThumbnailWarnings ( + rows.map (row => ({ + ...row, + resetSnapshot: buildResetSnapshot (row) }))) diff --git a/frontend/src/lib/postImportSourceValidation.test.ts b/frontend/src/lib/postImportSourceValidation.test.ts new file mode 100644 index 0000000..91f0484 --- /dev/null +++ b/frontend/src/lib/postImportSourceValidation.test.ts @@ -0,0 +1,52 @@ +import { describe, expect, it } from 'vitest' + +import { + countImportSourceLines, + validateImportSource, +} from '@/lib/postImportSourceValidation' + +describe ('post import source validation', () => { + it ('counts trimmed non-empty CRLF and LF rows', () => { + expect (countImportSourceLines (' one \r\n\r\n two\n')).toBe (2) + }) + + it ('reports invalid protocols and malformed URLs with original line numbers', () => { + const issues = validateImportSource ( + '\nftp://example.com/file\nhttps://exa mple.com/path') + + expect (issues).toEqual ([ + { + sourceRow: 2, + message: 'HTTP または HTTPS の URL ではありません．', + url: 'ftp://example.com/file' }, + { + sourceRow: 3, + message: 'URL の形式が不正です．', + url: 'https://exa mple.com/path' }]) + }) + + it ('detects duplicates after frontend URL normalisation', () => { + const issues = validateImportSource ( + 'https://EXAMPLE.com/path/\nhttps://example.com/path') + + expect (issues).toEqual ([{ + sourceRow: 2, + message: '1 行目と同じ URL です．', + url: 'https://example.com/path' }]) + }) + + it ('rejects oversized URLs and rows beyond the maximum count', () => { + const oversized = `https://example.com/${ 'a'.repeat (20 * 1024) }` + const tooMany = Array.from ( + { length: 101 }, + (_, index) => `https://example.com/${ index }`).join ('\n') + + expect (validateImportSource (oversized)[0]).toMatchObject ({ + sourceRow: 1, + message: 'URL が長すぎます．' }) + expect (validateImportSource (tooMany).at (-1)).toEqual ({ + sourceRow: 101, + message: '取込件数は 100 件までです．', + url: 'https://example.com/100' }) + }) +}) diff --git a/frontend/src/lib/postImportSourceValidation.ts b/frontend/src/lib/postImportSourceValidation.ts new file mode 100644 index 0000000..cc37ee7 --- /dev/null +++ b/frontend/src/lib/postImportSourceValidation.ts @@ -0,0 +1,121 @@ +import type { PostImportSourceIssue } from '@/lib/postImportTypes' + +const MAX_ROWS = 100 +const MAX_URL_BYTES = 20 * 1024 + + +const truncateUrl = (value: string): string => + value.length > 120 ? `${ value.slice (0, 117) }…` : value + + +const bytesize = (value: string): number => + new TextEncoder ().encode (value).length + + +const parseImportUrl = (value: string): URL | null => { + const trimmed = value.trim () + if (!(trimmed)) + return null + + try + { + return new URL (trimmed) + } + catch + { + return null + } +} + + +const normaliseImportUrl = (url: URL): string => { + url.hostname = url.hostname.toLowerCase () + if (url.pathname.endsWith ('/')) + url.pathname = url.pathname.replace (/\/+$/, '') + return url.toString () +} + + +export const extractImportSourceUrls = (source: string): string[] => + source + .split (/\r\n|\n|\r/) + .map (line => line.trim ()) + .filter (line => line !== '') + + +export const countImportSourceLines = (source: string): number => + extractImportSourceUrls (source).length + + +export const validateImportSource = ( + source: string, +): PostImportSourceIssue[] => { + const lines = source.split (/\r\n|\n|\r/) + const issues: PostImportSourceIssue[] = [] + const seen = new Map () + let count = 0 + + lines.forEach ((rawLine, index) => { + const value = rawLine.trim () + if (!(value)) + return + + ++count + const sourceRow = index + 1 + const displayUrl = truncateUrl (value) + if (count > MAX_ROWS) + { + issues.push ({ + sourceRow, + message: `取込件数は ${ MAX_ROWS } 件までです．`, + url: displayUrl }) + return + } + if (bytesize (value) > MAX_URL_BYTES) + { + issues.push ({ + sourceRow, + message: 'URL が長すぎます．', + url: displayUrl }) + return + } + const parsed = parseImportUrl (value) + if (parsed == null) + { + issues.push ({ + sourceRow, + message: 'URL の形式が不正です．', + url: displayUrl }) + return + } + if (!(parsed.protocol === 'http:' || parsed.protocol === 'https:')) + { + issues.push ({ + sourceRow, + message: 'HTTP または HTTPS の URL ではありません．', + url: displayUrl }) + return + } + if (!(parsed.host)) + { + issues.push ({ + sourceRow, + message: 'URL の形式が不正です．', + url: displayUrl }) + return + } + const normalised = normaliseImportUrl (parsed) + const duplicateRow = seen.get (normalised) + if (duplicateRow != null) + { + issues.push ({ + sourceRow, + message: `${ duplicateRow } 行目と同じ URL です．`, + url: displayUrl }) + return + } + seen.set (normalised, sourceRow) + }) + + return issues +} diff --git a/frontend/src/lib/postImportStorage.test.ts b/frontend/src/lib/postImportStorage.test.ts new file mode 100644 index 0000000..832971a --- /dev/null +++ b/frontend/src/lib/postImportStorage.test.ts @@ -0,0 +1,40 @@ +import { beforeEach, describe, expect, it, vi } from 'vitest' + +import { + clearPostImportSourceDraft, + loadPostImportSourceDraft, + savePostImportSourceDraft, +} from '@/lib/postImportStorage' + +describe ('post import source draft storage', () => { + beforeEach (() => { + sessionStorage.clear () + vi.restoreAllMocks () + }) + + it ('round-trips and clears the URL list source draft', () => { + expect (savePostImportSourceDraft ('https://example.com')).toBe (true) + expect (loadPostImportSourceDraft ()).toEqual ({ + source: 'https://example.com' }) + + clearPostImportSourceDraft () + + expect (loadPostImportSourceDraft ()).toEqual ({ source: '' }) + }) + + it ('ignores malformed stored drafts', () => { + sessionStorage.setItem ('post-import-source-draft', '{') + + expect (loadPostImportSourceDraft ()).toEqual ({ source: '' }) + }) + + it ('reports storage access failures without throwing', () => { + const onError = vi.fn () + vi.spyOn (Storage.prototype, 'setItem').mockImplementation (() => { + throw new DOMException ('quota') + }) + + expect (savePostImportSourceDraft ('source', onError)).toBe (false) + expect (onError).toHaveBeenCalledWith ('ブラウザへ保存できませんでした．') + }) +}) diff --git a/frontend/src/lib/postImportStorage.ts b/frontend/src/lib/postImportStorage.ts new file mode 100644 index 0000000..ec29cad --- /dev/null +++ b/frontend/src/lib/postImportStorage.ts @@ -0,0 +1,94 @@ +import type { StorageErrorHandler } from '@/lib/postImportTypes' + +const SOURCE_DRAFT_KEY = 'post-import-source-draft' + + +const readStorage = ( + key: string, + onError?: StorageErrorHandler, +): string | null => { + if (typeof window === 'undefined') + return null + + try + { + return sessionStorage.getItem (key) + } + catch + { + onError?.('保存済みデータを読み込めませんでした．') + return null + } +} + + +const writeStorage = ( + key: string, + value: string, + onError?: StorageErrorHandler, +): boolean => { + if (typeof window === 'undefined') + return false + + try + { + sessionStorage.setItem (key, value) + return true + } + catch + { + onError?.('ブラウザへ保存できませんでした．') + return false + } +} + + +const removeStorage = ( + key: string, + onError?: StorageErrorHandler, +) => { + if (typeof window === 'undefined') + return + + try + { + sessionStorage.removeItem (key) + } + catch + { + onError?.('保存済みデータを削除できませんでした．') + } +} + + +export const loadPostImportSourceDraft = ( + onError?: StorageErrorHandler, +): { source: string } => { + const raw = readStorage (SOURCE_DRAFT_KEY, onError) + if (raw == null) + return { source: '' } + + try + { + const value = JSON.parse (raw) as { source?: string } + return { source: typeof value.source === 'string' ? value.source : '' } + } + catch + { + return { source: '' } + } +} + + +export const savePostImportSourceDraft = ( + source: string, + onError?: StorageErrorHandler, +): boolean => + writeStorage (SOURCE_DRAFT_KEY, JSON.stringify ({ source }), onError) + + +export const clearPostImportSourceDraft = ( + onError?: StorageErrorHandler, +) => { + removeStorage (SOURCE_DRAFT_KEY, onError) +} diff --git a/frontend/src/lib/postImportTypes.ts b/frontend/src/lib/postImportTypes.ts new file mode 100644 index 0000000..f709cb8 --- /dev/null +++ b/frontend/src/lib/postImportTypes.ts @@ -0,0 +1,98 @@ +import type { Category } from '@/types' + +export type PostImportOrigin = 'automatic' | 'manual' +export type PostImportRepairMode = 'all' | 'failed' +export type PostImportStatus = + 'pending' + | 'created' + | 'skipped' + | 'failed' +export type PostImportSkipReason = 'existing' | 'manual' +export type PostImportResultStatus = 'created' | 'skipped' | 'failed' +export type PostImportAttributeValue = string | number + +export type PostImportDisplayTag = { + name: string + category: Category + sectionLiterals?: string[] } + +export type PostImportResetSnapshot = { + url: string + attributes: Record + displayTags: PostImportDisplayTag[] + provenance: Record + tagSources: Record + fieldWarnings: Record + baseWarnings: string[] + metadataUrl?: string } + +export type PostImportExistingPost = { + id: number + title: string + url: string + thumbnail?: string | null + thumbnailBase?: string | null } + +export type PostImportRow = { + sourceRow: number + url: string + attributes: Record + fieldWarnings: Record + baseWarnings: string[] + validationErrors: Record + importErrors?: Record + provenance: Record + tagSources?: Record + status: 'pending' | 'ready' | 'warning' | 'error' + skipReason?: PostImportSkipReason + existingPostId?: number + existingPost?: PostImportExistingPost + metadataUrl?: string + displayTags?: PostImportDisplayTag[] + resetSnapshot: PostImportResetSnapshot + createdPostId?: number + importStatus?: PostImportStatus + recoverable?: boolean + thumbnailFile?: File } + +export type PostImportResultRow = + | { + sourceRow: number + status: 'created' + post: { id: number } + fieldWarnings?: Record + baseWarnings?: string[] + errors?: Record } + | { + sourceRow: number + status: 'skipped' + existingPostId: number + existingPost?: PostImportExistingPost + fieldWarnings?: Record + baseWarnings?: string[] + errors?: Record } + | { + sourceRow: number + status: 'failed' + fieldWarnings?: Record + baseWarnings?: string[] + errors?: Record + recoverable?: boolean } + +export type PostImportSourceIssue = { + sourceRow: number + message: string + url: string } + +export type PostImportEditableDraft = { + url: string + title: string + thumbnailBase: string + originalCreatedFrom: string + originalCreatedBefore: string + duration: string + tags: string + parentPostIds: string + thumbnailFile?: File } + +export type StorageErrorHandler = (message: string) => void diff --git a/frontend/src/lib/postNewQueryState.test.ts b/frontend/src/lib/postNewQueryState.test.ts new file mode 100644 index 0000000..720dc73 --- /dev/null +++ b/frontend/src/lib/postNewQueryState.test.ts @@ -0,0 +1,43 @@ +import { describe, expect, it } from 'vitest' + +import { + buildPostNewReviewPath, + hasPostNewReviewState, + isPostNewReviewPathWithinLimit, + parsePostNewReviewUrls, + postNewReviewPathByteLength, +} from '@/lib/postNewQueryState' + +describe ('post new review URL state', () => { + it ('encodes each URL separately and preserves literal plus signs', () => { + const urls = [ + 'https://example.com/one+a', + 'https://example.com/two?value=b+c'] + + const path = buildPostNewReviewPath (urls) + + expect (path).toBe ( + '/posts/new?urls=https%3A%2F%2Fexample.com%2Fone%2Ba' + + '+https%3A%2F%2Fexample.com%2Ftwo%3Fvalue%3Db%2Bc') + expect (parsePostNewReviewUrls (path.slice ('/posts/new'.length))).toEqual (urls) + }) + + it ('uses only the raw urls parameter as review state', () => { + expect (hasPostNewReviewState ('?session_id=old&meta=old')).toBe (false) + expect (hasPostNewReviewState ('?unknown=value&urls=')).toBe (true) + expect (parsePostNewReviewUrls ('?unknown=value&urls=one+two&meta=old')) + .toEqual (['one', 'two']) + }) + + it ('allows at most a 6 143 byte request target', () => { + const baseUrl = 'https://example.com/' + const baseLength = postNewReviewPathByteLength ([baseUrl]) + const allowed = `${ baseUrl }${ 'a'.repeat (6_143 - baseLength) }` + const denied = `${ allowed }a` + + expect (postNewReviewPathByteLength ([allowed])).toBe (6_143) + expect (isPostNewReviewPathWithinLimit ([allowed])).toBe (true) + expect (postNewReviewPathByteLength ([denied])).toBe (6_144) + expect (isPostNewReviewPathWithinLimit ([denied])).toBe (false) + }) +}) diff --git a/frontend/src/lib/postNewQueryState.ts b/frontend/src/lib/postNewQueryState.ts new file mode 100644 index 0000000..86bcb4e --- /dev/null +++ b/frontend/src/lib/postNewQueryState.ts @@ -0,0 +1,58 @@ +const POST_NEW_REVIEW_PATH_PREFIX = '/posts/new?urls=' +const MAX_POST_NEW_REVIEW_TARGET_BYTES = 6_144 + +const textEncoder = new TextEncoder () + + +const rawUrlsParam = (search: string): string | null => { + const query = search.startsWith ('?') ? search.slice (1) : search + if (query === '') + return null + + for (const segment of query.split ('&')) + { + if (segment === 'urls') + return '' + if (segment.startsWith ('urls=')) + return segment.slice ('urls='.length) + } + + return null +} + + +export const buildPostNewReviewPath = (urls: string[]): string => + `${ POST_NEW_REVIEW_PATH_PREFIX }${ urls.map (url => encodeURIComponent (url)).join ('+') }` + + +export const postNewReviewPathByteLength = (urls: string[]): number => + textEncoder.encode (buildPostNewReviewPath (urls)).byteLength + + +export const isPostNewReviewPathWithinLimit = (urls: string[]): boolean => + postNewReviewPathByteLength (urls) < MAX_POST_NEW_REVIEW_TARGET_BYTES + + +export const parsePostNewReviewUrls = (search: string): string[] => { + const raw = rawUrlsParam (search) + if (raw == null) + return [] + + return raw + .split ('+') + .filter (segment => segment !== '') + .map (segment => { + try + { + return decodeURIComponent (segment) + } + catch + { + return segment + } + }) +} + + +export const hasPostNewReviewState = (search: string): boolean => + rawUrlsParam (search) != null diff --git a/frontend/src/lib/useKeyboardShortcuts.tsx b/frontend/src/lib/useKeyboardShortcuts.tsx index cc650bd..d6e955f 100644 --- a/frontend/src/lib/useKeyboardShortcuts.tsx +++ b/frontend/src/lib/useKeyboardShortcuts.tsx @@ -25,7 +25,6 @@ import { getEffectiveKeyBindings, setClientKeyboardSettings, } from '@/lib/settings' -import { useUnsavedChangesGuard } from '@/lib/useUnsavedChangesGuard' import type { KeyBinding, @@ -97,7 +96,6 @@ const focusSearchTarget = (): void => { export const KeyboardShortcutsProvider = ({ children }: PropsWithChildren) => { const location = useLocation () const navigate = useNavigate () - const { confirmDiscardNavigation } = useUnsavedChangesGuard () const [keyboardSettings, setKeyboardSettingsState] = useState (() => getClientKeyboardSettings ()) @@ -149,11 +147,8 @@ export const KeyboardShortcutsProvider = ({ children }: PropsWithChildren) => { }, []) const guardedNavigate = useCallback ((path: string) => { - confirmDiscardNavigation ().then (confirmed => { - if (confirmed) - navigate (path) - }) - }, [confirmDiscardNavigation, navigate]) + navigate (path) + }, [navigate]) const builtinHandlers = useMemo ( () => ({ diff --git a/frontend/src/lib/useUnsavedChangesGuard.test.tsx b/frontend/src/lib/useUnsavedChangesGuard.test.tsx new file mode 100644 index 0000000..8c945ba --- /dev/null +++ b/frontend/src/lib/useUnsavedChangesGuard.test.tsx @@ -0,0 +1,77 @@ +import { fireEvent, screen, waitFor, within } from '@testing-library/react' +import { useState } from 'react' +import { useLocation, useNavigate } from 'react-router-dom' +import { describe, expect, it, vi } from 'vitest' + +import { useUnsavedChangesGuard } from '@/lib/useUnsavedChangesGuard' +import { renderWithProviders } from '@/test/render' + +const GuardHarness = () => { + const [dirty, setDirty] = useState (true) + const location = useLocation () + const navigate = useNavigate () + const discard = vi.fn (() => setDirty (false)) + const { allowNextNavigation } = useUnsavedChangesGuard ({ + dirty, + onDiscard: discard }) + + return ( + <> + {location.pathname}{location.search} + + + ) +} + +describe ('useUnsavedChangesGuard', () => { + it ('blocks route changes and resets a cancelled transition', async () => { + renderWithProviders (, { route: '/current' }) + + fireEvent.click (screen.getByRole ('button', { name: 'move' })) + + expect (within (await screen.findByRole ('dialog')).getByText ( + '変更が破棄してページ移動しますか?')).toBeInTheDocument () + expect (screen.getByLabelText ('location')).toHaveTextContent ('/current') + + fireEvent.click (screen.getByRole ('button', { name: '取消' })) + + await waitFor (() => { + expect (screen.queryByRole ('dialog')).not.toBeInTheDocument () + }) + expect (screen.getByLabelText ('location')).toHaveTextContent ('/current') + }) + + it ('proceeds with the blocked transition after discard is confirmed', async () => { + renderWithProviders (, { route: '/current' }) + + fireEvent.click (screen.getByRole ('button', { name: 'move' })) + fireEvent.click (await screen.findByRole ('button', { + name: '変更を破棄して移動' })) + + await waitFor (() => { + expect (screen.getByLabelText ('location')).toHaveTextContent ('/next?tab=one') + }) + }) + + it ('allows only the next navigation without leaving a bypass token', async () => { + renderWithProviders (, { route: '/current' }) + + fireEvent.click (screen.getByRole ('button', { name: 'allowed move' })) + await waitFor (() => { + expect (screen.getByLabelText ('location')).toHaveTextContent ('/allowed') + }) + + fireEvent.click (screen.getByRole ('button', { name: 'move' })) + + expect (within (await screen.findByRole ('dialog')).getByText ( + '変更が破棄してページ移動しますか?')).toBeInTheDocument () + expect (screen.getByLabelText ('location')).toHaveTextContent ('/allowed') + }) +}) diff --git a/frontend/src/lib/useUnsavedChangesGuard.tsx b/frontend/src/lib/useUnsavedChangesGuard.tsx index c007812..868e538 100644 --- a/frontend/src/lib/useUnsavedChangesGuard.tsx +++ b/frontend/src/lib/useUnsavedChangesGuard.tsx @@ -1,6 +1,15 @@ -import { createContext, useCallback, useContext, useMemo, useState } from 'react' +import { + createContext, + useCallback, + useContext, + useEffect, + useMemo, + useRef, + useState, +} from 'react' +import { useBlocker } from 'react-router-dom' -import { useDialogue } from '@/components/dialogues/DialogueProvider' +import { useDialogue } from '@/lib/dialogues/useDialogue' import type { FC, PropsWithChildren } from 'react' @@ -8,12 +17,16 @@ type UnsavedChangesSource = { dirty: boolean discard: () => void | Promise } +type UseUnsavedChangesGuardOptions = { + dirty: boolean + onDiscard?: () => void | Promise } + type UnsavedChangesGuardContextValue = { - hasUnsavedChanges: boolean - registerUnsavedChangesSource: ( - source: UnsavedChangesSource | null, - ) => void - confirmDiscardNavigation: () => Promise } + hasUnsavedChanges: boolean + registerUnsavedChangesSource: ( + source: UnsavedChangesSource, + ) => () => void + allowNextNavigation: () => void } const UnsavedChangesGuardContext = createContext (null) @@ -21,49 +34,171 @@ const UnsavedChangesGuardContext = export const UnsavedChangesGuardProvider: FC = ({ children }) => { const dialogue = useDialogue () - const [source, setSource] = useState (null) + const sourcesRef = useRef (new Map ()) + const bypassNextNavigationRef = useRef (null) + const [revision, setRevision] = useState (0) + const dialogueOpenRef = useRef (false) + const handlingBlockedTransitionRef = useRef (false) const registerUnsavedChangesSource = useCallback (( - nextSource: UnsavedChangesSource | null, - ) => { - setSource (nextSource) + source: UnsavedChangesSource, + ): (() => void) => { + const sourceId = Symbol ('unsaved-changes-source') + sourcesRef.current.set (sourceId, source) + setRevision (current => current + 1) + + return () => { + if (!(sourcesRef.current.delete (sourceId))) + return + + setRevision (current => current + 1) + } }, []) - const confirmDiscardNavigation = useCallback (async (): Promise => { - if (!(source?.dirty)) + const allowNextNavigation = useCallback (() => { + const token = Symbol ('allowed-navigation') + bypassNextNavigationRef.current = token + + queueMicrotask (() => { + if (bypassNextNavigationRef.current === token) + bypassNextNavigationRef.current = null + }) + }, []) + + const sources = useMemo ( + () => [...sourcesRef.current.values ()], + [revision], + ) + const dirtySources = useMemo ( + () => sources.filter (source => source.dirty), + [sources], + ) + const hasUnsavedChanges = dirtySources.length > 0 + const hasUnsavedChangesRef = useRef (hasUnsavedChanges) + hasUnsavedChangesRef.current = hasUnsavedChanges + const shouldBlock = useCallback (() => { + if (bypassNextNavigationRef.current != null) + { + bypassNextNavigationRef.current = null + return false + } + + return hasUnsavedChangesRef.current + }, []) + const blocker = useBlocker (shouldBlock) + + const confirmDiscardChanges = useCallback (async (): Promise => { + if (!(hasUnsavedChanges)) return true - const confirmed = await dialogue.confirm ({ - title: '未保存の変更があります', - description: 'このまま移動すると、保存していない変更は失われます。', - cancelText: 'このページに残る', - confirmText: '変更を破棄して移動', - variant: 'danger' }) - if (!(confirmed)) + if (dialogueOpenRef.current) return false - await source.discard () - return true - }, [dialogue, source]) + dialogueOpenRef.current = true + + try + { + const confirmed = await dialogue.confirm ({ + title: '変更が破棄してページ移動しますか?', + confirmText: '変更を破棄して移動', + variant: 'danger' }) + if (!(confirmed)) + return false + + return true + } + finally + { + dialogueOpenRef.current = false + } + }, [dialogue, hasUnsavedChanges]) + + useEffect (() => { + if (blocker.state !== 'blocked' || handlingBlockedTransitionRef.current) + return + + handlingBlockedTransitionRef.current = true + void (async () => { + try + { + const confirmed = await confirmDiscardChanges () + if (!(confirmed)) + { + blocker.reset () + return + } + + let discardResults: Array> + try + { + discardResults = dirtySources.map (source => source.discard ()) + } + catch + { + blocker.reset () + return + } + + blocker.proceed () + await Promise.allSettled (discardResults) + } + finally + { + handlingBlockedTransitionRef.current = false + } + }) () + }, [blocker, confirmDiscardChanges, dirtySources]) + + useEffect (() => { + if (!(hasUnsavedChanges)) + return + + const handleBeforeUnload = (event: BeforeUnloadEvent) => { + event.preventDefault () + event.returnValue = '' + } + + window.addEventListener ('beforeunload', handleBeforeUnload) + return () => { + window.removeEventListener ('beforeunload', handleBeforeUnload) + } + }, [hasUnsavedChanges]) const value = useMemo (() => ({ - hasUnsavedChanges: source?.dirty === true, + hasUnsavedChanges, registerUnsavedChangesSource, - confirmDiscardNavigation, - }), [confirmDiscardNavigation, registerUnsavedChangesSource, source?.dirty]) + allowNextNavigation, + }), [ + allowNextNavigation, + hasUnsavedChanges, + registerUnsavedChangesSource, + ]) return ( - {children} + {children} ) } -export const useUnsavedChangesGuard = (): UnsavedChangesGuardContextValue => { +export const useUnsavedChangesGuard = ( + options?: UseUnsavedChangesGuardOptions, +): UnsavedChangesGuardContextValue => { const context = useContext (UnsavedChangesGuardContext) if (context == null) throw new Error ('UnsavedChangesGuardProvider が必要です．') + const { registerUnsavedChangesSource } = context + + useEffect (() => { + if (options == null) + return + + return registerUnsavedChangesSource ({ + dirty: options.dirty, + discard: options.onDiscard ?? (() => undefined) }) + }, [options?.dirty, options?.onDiscard, registerUnsavedChangesSource]) + return context } diff --git a/frontend/src/pages/materials/MaterialListPage.tsx b/frontend/src/pages/materials/MaterialListPage.tsx index 9629733..7a5b718 100644 --- a/frontend/src/pages/materials/MaterialListPage.tsx +++ b/frontend/src/pages/materials/MaterialListPage.tsx @@ -13,7 +13,7 @@ import MainArea from '@/components/layout/MainArea' import { SITE_TITLE } from '@/config' import { fetchMaterials, parseMaterialFilter } from '@/lib/materials' import { materialsKeys } from '@/lib/queryKeys' -import { dateString, inputClass } from '@/lib/utils' +import { cn, dateString, inputClass } from '@/lib/utils' import type { FC, FormEvent } from 'react' @@ -113,10 +113,11 @@ const clearedTagSelectionPath = ( const MaterialThumb: FC<{ material: Material }> = ({ material }) => (
+ className={cn ( + 'flex aspect-square h-[180px] w-[180px] items-center justify-center', + 'overflow-hidden rounded-lg border border-stone-200 bg-white text-center', + 'text-stone-900 shadow-sm dark:border-stone-700 dark:bg-stone-900', + 'dark:text-stone-100')}> {material.thumbnail ? : ( @@ -482,7 +483,7 @@ const MaterialListPage: FC = () => { className={inputClass (invalid)}> - + )} @@ -531,30 +532,32 @@ const MaterialListPage: FC = () => {
- -
diff --git a/frontend/src/pages/posts/PostImportReviewPage.test.tsx b/frontend/src/pages/posts/PostImportReviewPage.test.tsx new file mode 100644 index 0000000..9baec54 --- /dev/null +++ b/frontend/src/pages/posts/PostImportReviewPage.test.tsx @@ -0,0 +1,157 @@ +import { fireEvent, screen, waitFor } from '@testing-library/react' +import { useLocation } from 'react-router-dom' +import { beforeEach, describe, expect, it, vi } from 'vitest' + +import PostImportReviewPage from '@/pages/posts/PostImportReviewPage' +import { buildUser } from '@/test/factories' +import { renderWithProviders } from '@/test/render' + +import type { ReactNode } from 'react' + +const api = vi.hoisted (() => ({ + apiGet: vi.fn (), + apiPost: vi.fn (), + isApiError: vi.fn (() => false) })) + +vi.mock ('@/lib/api', () => api) +vi.mock ('framer-motion', () => ({ + AnimatePresence: ({ children }: { children?: ReactNode }) => <>{children}, + motion: { + div: ({ children }: { children?: ReactNode }) =>
{children}
, + main: ({ children }: { children?: ReactNode }) =>
{children}
} })) + +const metadata = (url: string, title: string) => ({ + url, + title, + thumbnailBase: 'https://example.com/thumbnail.jpg', + originalCreatedFrom: '', + originalCreatedBefore: '', + duration: '', + tags: '', + parentPostIds: [], + fieldWarnings: { }, + baseWarnings: [], + displayTags: [] }) + + +const LocationProbe = () => { + const location = useLocation () + return {location.pathname}{location.search} +} + + +const renderReviewPage = (urls: string[]) => { + const search = urls.map (url => encodeURIComponent (url)).join ('+') + return renderWithProviders ( + <> + + + , + { route: `/posts/new?urls=${ search }` }) +} + +describe ('PostImportReviewPage', () => { + beforeEach (() => { + vi.clearAllMocks () + api.isApiError.mockReturnValue (false) + }) + + it ('fetches metadata with at most four concurrent requests', async () => { + const resolvers: Array<(value: ReturnType) => void> = [] + api.apiGet.mockImplementation ((_path, options) => + new Promise (resolve => { + resolvers.push (resolve) + const url = String (options.params.url) + void url + })) + const urls = Array.from ( + { length: 6 }, + (_, index) => `https://example.com/${ index + 1 }`) + + renderReviewPage (urls) + + await waitFor (() => expect (api.apiGet).toHaveBeenCalledTimes (4)) + + resolvers[0]?.(metadata (urls[0]!, 'first')) + + expect (await screen.findAllByText ('first')).toHaveLength (2) + await waitFor (() => expect (api.apiGet).toHaveBeenCalledTimes (5)) + expect (screen.getByRole ('button', { name: '追加' })).toBeDisabled () + + for (let index = 1; index < resolvers.length; ++index) + resolvers[index]?.(metadata (urls[index]!, `post ${ index + 1 }`)) + }) + + it ('keeps successful rows when another metadata request fails', async () => { + api.apiGet + .mockResolvedValueOnce (metadata ('https://example.com/one', 'first')) + .mockRejectedValueOnce (new TypeError ('network')) + + renderReviewPage (['https://example.com/one', 'https://example.com/two']) + + expect (await screen.findAllByText ('first')).toHaveLength (2) + await waitFor (() => { + expect (screen.getAllByText ('登録不可')).toHaveLength (2) + }) + expect (screen.getAllByRole ('button', { name: '編輯' })[0]).toBeEnabled () + }) + + it ('shows existing posts with Active Storage thumbnail precedence', async () => { + api.apiGet.mockResolvedValue ({ + ...metadata ('https://example.com/post', ''), + existingPostId: 24, + existingPost: { + id: 24, + title: 'existing post', + url: 'https://example.com/post', + thumbnail: 'https://example.com/storage.jpg', + thumbnailBase: 'https://example.com/base.jpg' } }) + + renderReviewPage (['https://example.com/post']) + + const disclosure = await screen.findByRole ('button', { + name: '既存投稿による自動スキップ 1件' }) + expect (screen.getByRole ('button', { name: '追加' })).toBeDisabled () + + fireEvent.click (disclosure) + + expect ((await screen.findAllByRole ('img', { name: 'サムネール' }))[0]) + .toHaveAttribute ('src', 'https://example.com/storage.jpg') + }) + + it ('keeps a recoverable bulk failure on the review screen', async () => { + api.apiGet.mockResolvedValue ( + metadata ('https://example.com/post', 'new post')) + api.apiPost.mockResolvedValue ({ + results: [{ + status: 'failed', + recoverable: true, + errors: { title: ['invalid title'] } }] }) + + renderReviewPage (['https://example.com/post']) + + fireEvent.click (await screen.findByRole ('button', { name: '追加' })) + + expect (await screen.findAllByText ('登録失敗')).toHaveLength (2) + expect (screen.getAllByText ('invalid title')).toHaveLength (2) + expect (screen.getAllByRole ('button', { name: '編輯' })[0]).toBeEnabled () + expect (screen.getAllByRole ('button', { name: '再試行' })[0]).toBeEnabled () + expect (screen.getByLabelText ('current-location')).toHaveTextContent ('/posts/new?') + }) + + it ('navigates to /posts after every submitted row is created', async () => { + api.apiGet.mockResolvedValue ( + metadata ('https://example.com/post', 'new post')) + api.apiPost.mockResolvedValue ({ + results: [{ status: 'created', post: { id: 42 } }] }) + + renderReviewPage (['https://example.com/post']) + + fireEvent.click (await screen.findByRole ('button', { name: '追加' })) + + await waitFor (() => { + expect (screen.getByLabelText ('current-location')).toHaveTextContent ('/posts') + }) + expect (api.apiPost).toHaveBeenCalledWith ('/posts/bulk', expect.any (FormData)) + }) +}) diff --git a/frontend/src/pages/posts/PostImportReviewPage.tsx b/frontend/src/pages/posts/PostImportReviewPage.tsx new file mode 100644 index 0000000..9dcc181 --- /dev/null +++ b/frontend/src/pages/posts/PostImportReviewPage.tsx @@ -0,0 +1,1189 @@ +import { AnimatePresence, motion } from 'framer-motion' +import { useCallback, useEffect, useMemo, useRef, useState } from 'react' +import { Helmet } from 'react-helmet-async' +import { ChevronRight } from 'lucide-react' +import { useLocation, useNavigate } from 'react-router-dom' + +import PageTitle from '@/components/common/PageTitle' +import MainArea from '@/components/layout/MainArea' +import PrefetchLink from '@/components/PrefetchLink' +import PostThumbnailPreview from '@/components/posts/PostThumbnailPreview' +import PostImportRowForm from '@/components/posts/import/PostImportRowForm' +import PostImportRowSummary from '@/components/posts/import/PostImportRowSummary' +import { Button } from '@/components/ui/button' +import { toast } from '@/components/ui/use-toast' +import { SITE_TITLE } from '@/config' +import { apiGet, apiPost, isApiError } from '@/lib/api' +import useDialogue from '@/lib/dialogues/useDialogue' +import { parsePostNewReviewUrls } from '@/lib/postNewQueryState' +import { validateImportSource } from '@/lib/postImportSourceValidation' +import { + applyThumbnailWarning, + buildNextEditedRow, + canEditReviewRow, + canRetryResultRow, + compactMessageRecord, + hasErrorMessages, + hasThumbnailBaseValue, + isCompletedReviewRow, + isExistingSkipRow, + isManualSkipRow, + isNonRecoverableFailedRow, + mergeImportResults, + processableImportRows, + replaceImportRow, + resultRepairMode, + resultRowMessages, + reviewSummaryCounts, + retryImportRow, +} from '@/lib/postImportRows' +import { clearPostImportSourceDraft } from '@/lib/postImportStorage' +import { useClientBehaviourSettings } from '@/lib/useClientBehaviourSettings' +import { useUnsavedChangesGuard } from '@/lib/useUnsavedChangesGuard' +import { cn } from '@/lib/utils' +import { canEditContent } from '@/lib/users' +import Forbidden from '@/pages/Forbidden' + +import type { FC } from 'react' + +import type { + PostImportDisplayTag, + PostImportEditableDraft, + PostImportResetSnapshot, + PostImportResultRow, + PostImportRow, +} from '@/lib/postImportTypes' +import type { User } from '@/types' + +type Props = { user: User | null } + +type PostMetadataResponse = { + url: string + title?: string + thumbnailBase?: string + originalCreatedFrom?: string + originalCreatedBefore?: string + parentPostIds?: string + duration?: string + videoMs?: number + tags?: string + displayTags?: PostImportDisplayTag[] + fieldWarnings?: Record + baseWarnings?: string[] + validationErrors?: Record + existingPostId?: number + existingPost?: { + id: number + title: string + url: string + thumbnail?: string | null + thumbnailBase?: string | null } | null } + +type BulkApiRow = { + status: 'created' | 'skipped' | 'failed' + post?: { id: number } + existingPostId?: number + existingPost?: { + id: number + title: string + url: string + thumbnail?: string | null + thumbnailBase?: string | null } | null + fieldWarnings?: Record + baseWarnings?: string[] + errors?: Record + baseErrors?: string[] + recoverable?: boolean } + +type ExistingSkippedRowsProps = { + id: string + rows: PostImportRow[] } + +const DUPLICATE_URL_MESSAGE = 'URL が重複しています．' +const BULK_PROCESSING_FAILED_MESSAGE = '登録中にエラーが発生しました．' + +const emptyAttributes = () => ({ + title: '', + thumbnailBase: '', + originalCreatedFrom: '', + originalCreatedBefore: '', + duration: '', + tags: '', + parentPostIds: '' }) + + +const emptyProvenance = () => ({ + url: 'manual', + title: 'automatic', + thumbnailBase: 'automatic', + originalCreatedFrom: 'automatic', + originalCreatedBefore: 'automatic', + duration: 'automatic', + tags: 'automatic', + parentPostIds: 'automatic' }) as const + + +const emptyTagSources = () => ({ + automatic: '', + manual: '' }) + + +const cloneDisplayTags = ( + tags: PostImportDisplayTag[] | undefined, +): PostImportDisplayTag[] => + tags?.map (tag => ({ + name: tag.name, + category: tag.category, + sectionLiterals: + tag.sectionLiterals == null + ? undefined + : [...tag.sectionLiterals] })) ?? [] + + +const cloneMessageRecord = ( + record: Record, +): Record => + Object.fromEntries ( + Object.entries (record).map (([key, messages]) => [key, [...messages]])) + + +const buildEmptyResetSnapshot = (url: string): PostImportResetSnapshot => ({ + url, + attributes: emptyAttributes (), + displayTags: [], + provenance: emptyProvenance (), + tagSources: emptyTagSources (), + fieldWarnings: { }, + baseWarnings: [] }) + + +const buildInitialRows = (urls: string[]): PostImportRow[] => { + const source = urls.join ('\n') + const issues = validateImportSource (source) + const issuesByRow = issues.reduce> ((result, issue) => { + result[issue.sourceRow] = [...(result[issue.sourceRow] ?? []), issue.message] + return result + }, { }) + + return applyDuplicateUrlErrors ( + urls.map ((url, index) => { + const sourceRow = index + 1 + const rowIssues = issuesByRow[sourceRow] ?? [] + const validationErrors: Record = + rowIssues.length > 0 + ? { url: [...rowIssues] } + : { } + return { + sourceRow, + url, + attributes: emptyAttributes (), + fieldWarnings: { }, + baseWarnings: [], + validationErrors, + provenance: emptyProvenance (), + tagSources: emptyTagSources (), + status: rowIssues.length > 0 ? 'error' : 'pending', + displayTags: [], + resetSnapshot: buildEmptyResetSnapshot (url) } + })) +} + + +const rowMetadataPending = (row: PostImportRow): boolean => + row.status === 'pending' + + +const rowOperationBusy = ( + row: PostImportRow, + submitting: boolean, + busyRowIds: Set, +): boolean => + submitting + || busyRowIds.has (row.sourceRow) + || rowMetadataPending (row) + + +const rowSkipBusy = ( + row: PostImportRow, + submitting: boolean, + busyRowIds: Set, +): boolean => + submitting + || busyRowIds.has (row.sourceRow) + || (rowMetadataPending (row) && !(isManualSkipRow (row))) + + +const shouldFetchMetadata = (row: PostImportRow): boolean => + row.status === 'pending' + && row.skipReason !== 'manual' + && row.importStatus !== 'created' + && row.importStatus !== 'skipped' + && !(isNonRecoverableFailedRow (row)) + && row.metadataUrl == null + + +const shouldHydrateExistingPost = (row: PostImportRow): boolean => + row.skipReason === 'existing' + && row.existingPostId != null + && row.existingPost == null + && row.importStatus !== 'created' + && row.importStatus !== 'skipped' + + +const shouldFetchReviewRow = (row: PostImportRow): boolean => + shouldFetchMetadata (row) || shouldHydrateExistingPost (row) + + +const applyDuplicateUrlErrors = ( + rows: PostImportRow[], +): PostImportRow[] => { + const counts = rows.reduce> ((result, row) => { + if (row.url !== '' + && !(isManualSkipRow (row)) + && row.importStatus !== 'created' + && row.importStatus !== 'skipped') + result[row.url] = (result[row.url] ?? 0) + 1 + return result + }, { }) + + return rows.map (row => { + const urlErrors = (row.validationErrors.url ?? []).filter ( + message => message !== DUPLICATE_URL_MESSAGE) + if ((counts[row.url] ?? 0) < 2) + { + const nextValidationErrors = compactMessageRecord ({ + ...row.validationErrors, + url: urlErrors }) + const hasErrors = hasErrorMessages (nextValidationErrors) + return urlErrors.length === (row.validationErrors.url ?? []).length + ? row + : { + ...row, + validationErrors: nextValidationErrors, + skipReason: + row.skipReason === 'manual' + ? 'manual' + : row.existingPostId != null + ? 'existing' + : undefined, + status: hasErrors ? 'error' : row.status } + } + + return { + ...row, + skipReason: row.skipReason === 'manual' ? 'manual' : undefined, + validationErrors: compactMessageRecord ({ + ...row.validationErrors, + url: [...new Set ([...urlErrors, DUPLICATE_URL_MESSAGE])] }), + status: 'error' } + }) +} + + +const buildDryRunFormData = (row: PostImportRow): FormData => { + const formData = new FormData () + formData.append ('url', row.url) + formData.append ('title', String (row.attributes.title ?? '')) + formData.append ('thumbnail_base', String (row.attributes.thumbnailBase ?? '')) + formData.append ('tags', String (row.attributes.tags ?? '')) + formData.append ('parent_post_ids', String (row.attributes.parentPostIds ?? '')) + formData.append ( + 'original_created_from', + String (row.attributes.originalCreatedFrom ?? '')) + formData.append ( + 'original_created_before', + String (row.attributes.originalCreatedBefore ?? '')) + formData.append ('duration', String (row.attributes.duration ?? '')) + if (!(hasThumbnailBaseValue (row.attributes.thumbnailBase)) && row.thumbnailFile != null) + formData.append ('thumbnail', row.thumbnailFile) + return formData +} + + +const mergeDryRunRow = ( + currentRow: PostImportRow, + result: PostMetadataResponse, +): PostImportRow => { + const fieldWarnings = compactMessageRecord (result.fieldWarnings ?? { }) + const hasWarnings = + Object.values (fieldWarnings).some (messages => messages.length > 0) + || (result.baseWarnings?.length ?? 0) > 0 + + return applyThumbnailWarning ({ + ...currentRow, + url: result.url, + attributes: { + ...currentRow.attributes, + title: result.title ?? '', + thumbnailBase: result.thumbnailBase ?? '', + originalCreatedFrom: result.originalCreatedFrom ?? '', + originalCreatedBefore: result.originalCreatedBefore ?? '', + duration: result.duration ?? '', + tags: result.tags ?? '', + parentPostIds: String ( + result.parentPostIds ?? currentRow.attributes.parentPostIds ?? '') }, + displayTags: cloneDisplayTags (result.displayTags), + fieldWarnings, + baseWarnings: result.baseWarnings ?? [], + validationErrors: { }, + importErrors: undefined, + status: hasWarnings ? 'warning' : 'ready', + skipReason: + result.existingPostId != null || result.existingPost?.id != null + ? 'existing' + : undefined, + existingPostId: result.existingPostId ?? result.existingPost?.id, + existingPost: result.existingPost ?? undefined, + importStatus: + currentRow.importStatus === 'created' + ? 'created' + : 'pending', + recoverable: undefined }) +} + + +const buildPreviewResetSnapshot = ( + preview: PostMetadataResponse, +): PostImportRow['resetSnapshot'] => ({ + url: preview.url, + attributes: { + title: preview.title ?? '', + thumbnailBase: preview.thumbnailBase ?? '', + originalCreatedFrom: preview.originalCreatedFrom ?? '', + originalCreatedBefore: preview.originalCreatedBefore ?? '', + duration: preview.duration ?? '', + tags: preview.tags ?? '', + parentPostIds: String (preview.parentPostIds ?? '') }, + displayTags: cloneDisplayTags (preview.displayTags), + provenance: emptyProvenance (), + tagSources: { + automatic: preview.tags ?? '', + manual: '' }, + fieldWarnings: preview.fieldWarnings ?? { }, + baseWarnings: preview.baseWarnings ?? [], + metadataUrl: preview.url }) + + +const indexedBulkResults = ( + rows: PostImportRow[], + results: BulkApiRow[], +): PostImportResultRow[] => + rows.map ((row, index) => { + const result = results[index] + const errors = + result?.baseErrors?.length + ? { + ...(result.errors ?? { }), + base: [...new Set ([...(result.errors?.base ?? []), ...result.baseErrors])] } + : result?.errors + if (result?.status === 'created' && result.post != null) + { + return { + sourceRow: row.sourceRow, + status: 'created', + post: result.post, + fieldWarnings: result.fieldWarnings, + baseWarnings: result.baseWarnings, + errors } + } + if (result?.status === 'skipped') + { + const existingPostId = + result.existingPostId + ?? result.existingPost?.id + ?? row.existingPostId + if (existingPostId == null) + { + return { + sourceRow: row.sourceRow, + status: 'failed', + fieldWarnings: result.fieldWarnings, + baseWarnings: result.baseWarnings, + errors: { + ...(errors ?? { }), + base: [...new Set ([...(errors?.base ?? []), + BULK_PROCESSING_FAILED_MESSAGE])] }, + recoverable: false } + } + return { + sourceRow: row.sourceRow, + status: 'skipped', + existingPostId, + existingPost: result.existingPost ?? row.existingPost, + fieldWarnings: result.fieldWarnings, + baseWarnings: result.baseWarnings, + errors } + } + return { + sourceRow: row.sourceRow, + status: 'failed', + fieldWarnings: result?.fieldWarnings, + baseWarnings: result?.baseWarnings, + errors, + recoverable: result?.recoverable } + }) + + +const mergePreviewRow = ( + currentRow: PostImportRow, + preview: PostMetadataResponse, +): PostImportRow => { + const fieldWarnings = compactMessageRecord (preview.fieldWarnings ?? { }) + const baseWarnings = preview.baseWarnings ?? [] + const validationErrors = compactMessageRecord (preview.validationErrors ?? { }) + const metadataChanged = currentRow.metadataUrl !== preview.url + const hasWarnings = + Object.values (fieldWarnings).some (messages => messages.length > 0) + || baseWarnings.length > 0 + const hasErrors = + Object.values (validationErrors).some (messages => messages.length > 0) + const nextRow: PostImportRow = { + ...currentRow, + attributes: { ...currentRow.attributes }, + provenance: { ...currentRow.provenance }, + displayTags: cloneDisplayTags (currentRow.displayTags), + tagSources: { + automatic: currentRow.tagSources?.automatic ?? '', + manual: currentRow.tagSources?.manual ?? '' }, + url: preview.url, + fieldWarnings, + baseWarnings, + validationErrors, + existingPostId: preview.existingPostId ?? preview.existingPost?.id, + existingPost: preview.existingPost ?? undefined, + skipReason: + preview.existingPostId != null || preview.existingPost?.id != null + ? 'existing' + : currentRow.skipReason === 'manual' + ? 'manual' + : undefined, + metadataUrl: preview.url, + resetSnapshot: + metadataChanged + ? buildPreviewResetSnapshot (preview) + : currentRow.resetSnapshot, + status: + hasErrors + ? 'error' + : hasWarnings + ? 'warning' + : 'ready' } + + if (currentRow.provenance.title !== 'manual') + nextRow.attributes.title = preview.title ?? '' + if (currentRow.provenance.thumbnailBase !== 'manual') + nextRow.attributes.thumbnailBase = preview.thumbnailBase ?? '' + if (currentRow.provenance.originalCreatedFrom !== 'manual') + nextRow.attributes.originalCreatedFrom = preview.originalCreatedFrom ?? '' + if (currentRow.provenance.originalCreatedBefore !== 'manual') + nextRow.attributes.originalCreatedBefore = preview.originalCreatedBefore ?? '' + if (currentRow.provenance.tags !== 'manual') + { + nextRow.attributes.tags = preview.tags ?? '' + nextRow.displayTags = cloneDisplayTags (preview.displayTags) + } + nextRow.tagSources!.automatic = preview.tags ?? '' + if (currentRow.provenance.parentPostIds !== 'manual') + nextRow.attributes.parentPostIds = String (preview.parentPostIds ?? '') + if (currentRow.provenance.duration !== 'manual') + nextRow.attributes.duration = preview.duration ?? '' + + return applyThumbnailWarning (nextRow) +} + + +const mergeExistingSkippedRow = ( + currentRow: PostImportRow, + preview: PostMetadataResponse, +): PostImportRow => + preview.existingPostId != null || preview.existingPost?.id != null + ? { + ...currentRow, + url: preview.url, + existingPostId: preview.existingPostId ?? preview.existingPost?.id, + existingPost: preview.existingPost ?? currentRow.existingPost, + skipReason: 'existing', + metadataUrl: preview.url } + : currentRow + + +const buildBulkFormData = (rows: PostImportRow[]): FormData => { + const formData = new FormData () + formData.append ( + 'posts', + JSON.stringify ( + rows.map (row => ({ + url: row.url, + title: row.attributes.title ?? '', + thumbnail_base: row.attributes.thumbnailBase ?? '', + tags: row.attributes.tags ?? '', + parent_post_ids: row.attributes.parentPostIds ?? '', + original_created_from: row.attributes.originalCreatedFrom ?? '', + original_created_before: row.attributes.originalCreatedBefore ?? '', + duration: row.attributes.duration ?? '' })))) + rows.forEach ((row, index) => { + if (!(hasThumbnailBaseValue (row.attributes.thumbnailBase)) && row.thumbnailFile != null) + formData.append (`thumbnails[${ index }]`, row.thumbnailFile) + }) + return formData +} + + +const ExistingSkippedRows: FC = ({ id, rows }) => { + return ( +
+
+ {rows.map (row => { + const post = row.existingPost + if (post == null) + { + return ( +
+ +
+
+ {row.url} +
+
+
) + } + return ( + + +
+
+ {post.title} +
+
+ {post.url} +
+
+ ) + })} +
+
) +} + + +const editableRowDirty = (row: PostImportRow): boolean => + row.skipReason === 'manual' + || row.url !== row.resetSnapshot.url + || String (row.attributes.title ?? '') + !== String (row.resetSnapshot.attributes.title ?? '') + || String (row.attributes.thumbnailBase ?? '') + !== String (row.resetSnapshot.attributes.thumbnailBase ?? '') + || String (row.attributes.originalCreatedFrom ?? '') + !== String (row.resetSnapshot.attributes.originalCreatedFrom ?? '') + || String (row.attributes.originalCreatedBefore ?? '') + !== String (row.resetSnapshot.attributes.originalCreatedBefore ?? '') + || String (row.attributes.duration ?? '') + !== String (row.resetSnapshot.attributes.duration ?? '') + || String (row.attributes.tags ?? '') + !== String (row.resetSnapshot.attributes.tags ?? '') + || String (row.attributes.parentPostIds ?? '') + !== String (row.resetSnapshot.attributes.parentPostIds ?? '') + || row.thumbnailFile != null + + +const PostImportReviewPage: FC = ({ user }) => { + const editable = canEditContent (user) + const dialogue = useDialogue () + const location = useLocation () + const navigate = useNavigate () + const behaviourSettings = useClientBehaviourSettings () + const animationMode = behaviourSettings.animation ?? 'normal' + const existingRowsTransition = + animationMode === 'off' + ? { duration: 0 } + : animationMode === 'reduced' + ? { duration: .08, ease: 'linear' as const } + : { duration: .2, ease: 'easeOut' as const } + + const [rows, setRows] = useState (null) + const [metadataLoading, setMetadataLoading] = useState (false) + const [submitting, setSubmitting] = useState (false) + const [busyRowIds, setBusyRowIds] = useState> (new Set ()) + const [editingRow, setEditingRow] = useState (null) + const [showExistingRows, setShowExistingRows] = useState (false) + const rowsRef = useRef ([]) + const metadataSequenceRef = useRef (0) + const discardChanges = useCallback (() => undefined, []) + + const dirty = useMemo ( + () => rows?.some (row => editableRowDirty (row)) ?? false, + [rows], + ) + const { allowNextNavigation } = useUnsavedChangesGuard ({ + dirty, + onDiscard: discardChanges, + }) + + const commitRows = useCallback ((nextRows: PostImportRow[]) => { + rowsRef.current = nextRows + setRows (nextRows) + return nextRows + }, []) + + const updateRows = useCallback (( + updater: (currentRows: PostImportRow[]) => PostImportRow[], + ) => { + const nextRows = applyDuplicateUrlErrors (updater (rowsRef.current)) + return commitRows (nextRows) + }, [commitRows]) + + const currentRowBySource = useCallback (( + sourceRow: number, + ): PostImportRow | null => + rowsRef.current.find (row => row.sourceRow === sourceRow) ?? null, + []) + + const finishImport = useCallback (() => { + allowNextNavigation () + clearPostImportSourceDraft (message => + toast ({ title: '入力内容を削除できませんでした', description: message })) + navigate ('/posts') + }, [allowNextNavigation, navigate]) + + useEffect (() => { + const urls = parsePostNewReviewUrls (location.search) + const initialRows = buildInitialRows (urls) + commitRows (initialRows) + + let active = true + const sequence = ++metadataSequenceRef.current + const controllers = new Set () + let nextIndex = 0 + const targetSourceRows = initialRows + .filter (row => shouldFetchReviewRow (row)) + .map (row => row.sourceRow) + + setMetadataLoading (targetSourceRows.length > 0) + + const mergeCurrentRow = ( + sourceRow: number, + updater: (row: PostImportRow) => PostImportRow, + ) => { + updateRows (currentRows => + currentRows.map (row => + row.sourceRow === sourceRow + ? updater (row) + : row)) + } + + const worker = async () => { + while (active) + { + const sourceRow = targetSourceRows[nextIndex] + ++nextIndex + if (sourceRow == null) + return + + const currentRow = currentRowBySource (sourceRow) + if (currentRow == null || !(shouldFetchReviewRow (currentRow))) + continue + + const controller = new AbortController () + controllers.add (controller) + + try + { + const preview = await apiGet ('/posts/metadata', { + params: { url: currentRow.url }, + signal: controller.signal }) + if (!(active) || metadataSequenceRef.current !== sequence) + return + + mergeCurrentRow (sourceRow, row => + row.importStatus === 'created' + || row.importStatus === 'skipped' + || isNonRecoverableFailedRow (row) + || row.skipReason === 'manual' + ? row + : (shouldFetchMetadata (row) + ? mergePreviewRow (row, preview) + : mergeExistingSkippedRow (row, preview))) + } + catch (requestError) + { + if (controller.signal.aborted) + return + if (!(active) || metadataSequenceRef.current !== sequence) + return + + if (!(isApiError<{ + errors?: Record + baseErrors?: string[] + }> (requestError))) + { + mergeCurrentRow (sourceRow, row => ({ + ...row, + status: 'error' })) + continue + } + + if (requestError.response?.status === 422) + { + const rowErrors = compactMessageRecord ({ + ...(requestError.response.data.errors ?? { }), + ...(requestError.response.data.baseErrors?.length + ? { + base: requestError.response.data.baseErrors } + : { }) }) + mergeCurrentRow (sourceRow, row => ({ + ...row, + validationErrors: rowErrors, + status: 'error' })) + } + else + { + mergeCurrentRow (sourceRow, row => ({ + ...row, + status: 'error' })) + } + } + finally + { + controllers.delete (controller) + } + } + } + + void Promise.all ( + Array.from ( + { length: Math.min (4, targetSourceRows.length) }, + () => worker (), + )).finally (() => { + if (active && metadataSequenceRef.current === sequence) + setMetadataLoading (false) + }) + + return () => { + active = false + controllers.forEach (controller => controller.abort ()) + } + }, [commitRows, currentRowBySource, location.search, updateRows]) + + useEffect (() => { + if (editingRow == null || resultRepairMode (rowsRef.current) !== 'failed') + return + + const element = document.getElementById (`post-import-row-${ editingRow.sourceRow }`) + element?.scrollIntoView ({ block: 'center', behavior: 'smooth' }) + }, [editingRow, rows]) + + const reviewRowsState = rows ?? [] + const counts = useMemo ( + () => reviewSummaryCounts (reviewRowsState), + [reviewRowsState], + ) + const sortedRows = + resultRepairMode (reviewRowsState) === 'failed' + ? [...reviewRowsState].sort ((a, b) => { + const aRepair = + a.recoverable === true + && (a.importStatus === 'failed' + || (a.importStatus === 'pending' + && hasErrorMessages (a.validationErrors))) + ? 0 + : 1 + const bRepair = + b.recoverable === true + && (b.importStatus === 'failed' + || (b.importStatus === 'pending' + && hasErrorMessages (b.validationErrors))) + ? 0 + : 1 + return aRepair - bRepair || a.sourceRow - b.sourceRow + }) + : reviewRowsState + const existingRows = sortedRows.filter (row => isExistingSkipRow (row)) + const reviewRows = sortedRows.filter (row => !(isExistingSkipRow (row))) + const processingRows = processableImportRows (reviewRowsState) + const canSubmit = processingRows.length > 0 + + const toggleManualSkip = async (sourceRow: number, checked: boolean) => { + const currentRow = currentRowBySource (sourceRow) + if (currentRow == null) + return + if (rowSkipBusy (currentRow, submitting, busyRowIds)) + return + + updateRows (currentRows => + currentRows.map (row => { + if (row.sourceRow !== sourceRow) + return row + if (isExistingSkipRow (row) || row.importStatus === 'created') + return row + return { + ...row, + skipReason: + checked + ? 'manual' + : row.existingPostId != null + ? 'existing' + : undefined } + })) + } + + const saveDraft = async ( + row: PostImportRow, + { draft, resetRequested }: { + draft: PostImportEditableDraft + resetRequested: boolean }, + ): Promise<{ saved: boolean + row: PostImportRow | null }> => { + const currentRow = currentRowBySource (row.sourceRow) + if (currentRow == null) + return { saved: false, row: null } + if (!(canEditReviewRow (currentRow))) + return { saved: false, row: null } + + const baseRow = + resetRequested + ? { + ...currentRow, + url: currentRow.resetSnapshot.url, + attributes: { ...currentRow.resetSnapshot.attributes }, + displayTags: cloneDisplayTags (currentRow.resetSnapshot.displayTags), + provenance: { ...currentRow.resetSnapshot.provenance }, + tagSources: { ...currentRow.resetSnapshot.tagSources }, + fieldWarnings: cloneMessageRecord (currentRow.resetSnapshot.fieldWarnings), + baseWarnings: [...currentRow.resetSnapshot.baseWarnings], + metadataUrl: currentRow.resetSnapshot.metadataUrl, + thumbnailFile: undefined } + : currentRow + const urlChanged = draft.url !== baseRow.url + const nextRow = buildNextEditedRow (baseRow, draft, urlChanged) + let candidateRow = nextRow + + try + { + candidateRow = + urlChanged + ? mergePreviewRow ( + nextRow, + await apiGet ('/posts/metadata', { + params: { url: nextRow.url } })) + : nextRow + const dryRun = await apiPost ( + '/posts?dry=1', + buildDryRunFormData (candidateRow)) + const mergedRow = mergeDryRunRow (candidateRow, dryRun) + const mergedRows = updateRows (currentRows => + replaceImportRow (currentRows, mergedRow)) + const mergedTarget = mergedRows.find ( + current => current.sourceRow === baseRow.sourceRow) + if (mergedTarget == null) + return { saved: false, row: null } + if (hasErrorMessages (mergedTarget.validationErrors)) + return { saved: false, row: mergedTarget } + return { saved: true, row: null } + } + catch (requestError) + { + if (isApiError<{ + errors?: Record + baseErrors?: string[] + }> (requestError) + && requestError.response?.status === 422) + { + const errorRow = { + ...candidateRow, + validationErrors: compactMessageRecord ({ + ...(requestError.response.data.errors ?? { }), + ...(requestError.response.data.baseErrors?.length + ? { + base: requestError.response.data.baseErrors } + : { }) }), + baseWarnings: [], + fieldWarnings: { }, + status: 'error' as const } + return { saved: false, row: applyThumbnailWarning (errorRow) } + } + + toast ({ title: '行の再検証に失敗しました' }) + return { saved: false, row: null } + } + } + + const openEditingDialogue = async (row: PostImportRow) => { + const saveRowDraft = ( + { draft, resetRequested }: { + draft: PostImportEditableDraft + resetRequested: boolean }, + ) => + saveDraft (row, { draft, resetRequested }) + + await dialogue.form ({ + title: '投稿を編輯', + cancelText: '取消', + size: 'large', + body: controls => ( + ) }) + } + + const editRow = async (row: PostImportRow) => { + const currentRow = currentRowBySource (row.sourceRow) + if (currentRow == null) + return + if (!(canEditReviewRow (currentRow))) + return + if (rowOperationBusy (currentRow, submitting, busyRowIds)) + return + + setEditingRow (currentRow) + try + { + await openEditingDialogue (currentRow) + } + finally + { + setEditingRow (current => + current?.sourceRow === currentRow.sourceRow + ? null + : current) + } + } + + const retry = async (sourceRow: number) => { + const originalRow = currentRowBySource (sourceRow) + if (originalRow == null) + return + if (rowOperationBusy (originalRow, submitting, busyRowIds)) + return + + setBusyRowIds (current => new Set ([...current, sourceRow])) + try + { + updateRows (currentRows => retryImportRow (currentRows, sourceRow)) + const target = currentRowBySource (sourceRow) + if (target == null) + return + + const result = await apiPost<{ results: BulkApiRow[] }>( + '/posts/bulk', + buildBulkFormData ([target])) + if (result.results.length !== 1) + { + updateRows (currentRows => replaceImportRow (currentRows, originalRow)) + toast ({ title: '登録結果が不完全でした' }) + return + } + + const indexedResults = indexedBulkResults ([target], result.results) + const nextRows = applyDuplicateUrlErrors ( + mergeImportResults (rowsRef.current, indexedResults)) + .map (row => applyThumbnailWarning (row)) + commitRows (nextRows) + if (nextRows.every (row => isCompletedReviewRow (row))) + finishImport () + } + catch + { + updateRows (currentRows => replaceImportRow (currentRows, originalRow)) + toast ({ title: '再試行に失敗しました' }) + } + finally + { + setBusyRowIds (current => { + const next = new Set (current) + next.delete (sourceRow) + return next + }) + } + } + + const submit = async () => { + if (metadataLoading || submitting || busyRowIds.size > 0) + return + + const latestProcessingRows = processableImportRows (rowsRef.current) + if (latestProcessingRows.length === 0) + return + + setSubmitting (true) + try + { + const result = await apiPost<{ results: BulkApiRow[] }>( + '/posts/bulk', + buildBulkFormData (latestProcessingRows)) + if (result.results.length !== latestProcessingRows.length) + { + toast ({ title: '登録結果が不完全でした' }) + return + } + + const indexedResults = indexedBulkResults (latestProcessingRows, result.results) + const nextRows = applyDuplicateUrlErrors ( + mergeImportResults (rowsRef.current, indexedResults)) + .map (row => applyThumbnailWarning (row)) + commitRows (nextRows) + if (nextRows.every (row => isCompletedReviewRow (row))) + finishImport () + } + catch + { + toast ({ title: '登録に失敗しました' }) + } + finally + { + setSubmitting (false) + } + } + + if (!(editable)) + return + + if (rows == null) + return null + + return ( + <> + + {`追加内容確認 | ${ SITE_TITLE }`} + + + +
+ 追加内容確認 + + {existingRows.length > 0 && ( +
+ + + {showExistingRows && ( + + + )} + +
)} + +
+ {reviewRows.map ((row, index) => ( +
+ editRow (row)} + onToggleSkip={checked => toggleManualSkip (row.sourceRow, checked)} + onRetry={canRetryResultRow (row) ? () => retry (row.sourceRow) : undefined} + rowMessages={resultRowMessages (row)}/> +
))} +
+
+ + + 0} + canSubmit={canSubmit} + creatableCount={counts.creatable} + manualSkippedCount={counts.manualSkipped} + existingSkippedCount={counts.existingSkipped} + pendingOrErrorCount={counts.pendingOrError} + onBack={() => navigate ('/posts/new')} + onSubmit={() => submit ()}/> + ) +} + + +const PostImportFooter = ( + { metadataLoading, + submitting, + busyRowsPresent, + canSubmit, + creatableCount, + manualSkippedCount, + existingSkippedCount, + pendingOrErrorCount, + onBack, + onSubmit }: { metadataLoading: boolean + submitting: boolean + busyRowsPresent: boolean + canSubmit: boolean + creatableCount: number + manualSkippedCount: number + existingSkippedCount: number + pendingOrErrorCount: number + onBack: () => void + onSubmit: () => void }, +) => ( +
+
+
+ 作成対象 {creatableCount}件 + 手動スキップ {manualSkippedCount}件 + 既存投稿による自動スキップ {existingSkippedCount}件 + 未処理 {pendingOrErrorCount}件 +
+
+ + +
+
+
) + +export default PostImportReviewPage diff --git a/frontend/src/pages/posts/PostImportSourcePage.test.tsx b/frontend/src/pages/posts/PostImportSourcePage.test.tsx new file mode 100644 index 0000000..6e28d94 --- /dev/null +++ b/frontend/src/pages/posts/PostImportSourcePage.test.tsx @@ -0,0 +1,67 @@ +import { fireEvent, screen } from '@testing-library/react' +import { beforeEach, describe, expect, it, vi } from 'vitest' + +import PostImportSourcePage from '@/pages/posts/PostImportSourcePage' +import { buildUser } from '@/test/factories' +import { renderWithProviders } from '@/test/render' + +const router = vi.hoisted (() => ({ navigate: vi.fn () })) + +vi.mock ('react-router-dom', async importOriginal => ({ + ...await importOriginal (), + useNavigate: () => router.navigate })) + +describe ('PostImportSourcePage', () => { + beforeEach (() => { + sessionStorage.clear () + vi.clearAllMocks () + }) + + it ('validates an empty source only after Next is pressed', () => { + renderWithProviders () + + expect (screen.queryByText ('URL を入力してください．')).not.toBeInTheDocument () + + fireEvent.click (screen.getByRole ('button', { name: '次へ' })) + + expect (screen.getByText ('URL を入力してください．')).toBeInTheDocument () + expect (router.navigate).not.toHaveBeenCalled () + }) + + it ('reports frontend URL issues with original line numbers', () => { + renderWithProviders () + const input = screen.getByRole ('textbox', { name: '' }) + + fireEvent.change (input, { + target: { value: '\nftp://example.com/file\nhttps://example.com/valid' } }) + fireEvent.click (screen.getByRole ('button', { name: '次へ' })) + + expect (screen.getByText (/2 行目: HTTP または HTTPS/)).toBeInTheDocument () + expect (screen.getByText ('ftp://example.com/file')).toBeInTheDocument () + expect (input).toHaveAttribute ('aria-invalid', 'true') + expect (router.navigate).not.toHaveBeenCalled () + }) + + it ('navigates with individually encoded URLs without calling an API', () => { + renderWithProviders () + + fireEvent.change (screen.getByRole ('textbox', { name: '' }), { + target: { + value: 'https://example.com/one+a\nhttps://example.com/two?value=b+c' } }) + fireEvent.click (screen.getByRole ('button', { name: '次へ' })) + + expect (router.navigate).toHaveBeenCalledWith ( + '/posts/new?urls=https%3A%2F%2Fexample.com%2Fone%2Ba' + + '+https%3A%2F%2Fexample.com%2Ftwo%3Fvalue%3Db%2Bc') + }) + + it ('disables Next when the encoded request target reaches 4096 bytes', () => { + renderWithProviders () + const input = screen.getByRole ('textbox', { name: '' }) + + fireEvent.change (input, { + target: { value: `https://example.com/${ 'a'.repeat (6_200) }` } }) + + expect (screen.getByRole ('button', { name: '次へ' })).toBeDisabled () + }) +}) diff --git a/frontend/src/pages/posts/PostImportSourcePage.tsx b/frontend/src/pages/posts/PostImportSourcePage.tsx new file mode 100644 index 0000000..fc65b69 --- /dev/null +++ b/frontend/src/pages/posts/PostImportSourcePage.tsx @@ -0,0 +1,181 @@ +import { useEffect, useRef, useState } from 'react' +import { Helmet } from 'react-helmet-async' +import { useNavigate } from 'react-router-dom' + +import FieldError from '@/components/common/FieldError' +import Form from '@/components/common/Form' +import FormField from '@/components/common/FormField' +import PageTitle from '@/components/common/PageTitle' +import TextArea from '@/components/common/TextArea' +import MainArea from '@/components/layout/MainArea' +import { Button } from '@/components/ui/button' +import { toast } from '@/components/ui/use-toast' +import { SITE_TITLE } from '@/config' +import { + buildPostNewReviewPath, + isPostNewReviewPathWithinLimit, +} from '@/lib/postNewQueryState' +import { + countImportSourceLines, + extractImportSourceUrls, + validateImportSource, +} from '@/lib/postImportSourceValidation' +import { + loadPostImportSourceDraft, + savePostImportSourceDraft, +} from '@/lib/postImportStorage' +import { canEditContent } from '@/lib/users' +import Forbidden from '@/pages/Forbidden' + +import type { FC } from 'react' + +import type { User } from '@/types' + +type Props = { user: User | null } + +const MAX_ROWS = 100 +const SOURCE_ERROR_ID = 'post-import-source-error' +const SOURCE_ISSUES_ID = 'post-import-source-issues' + + +const PostImportSourcePage: FC = ({ user }) => { + const editable = canEditContent (user) + const navigate = useNavigate () + const [source, setSource] = useState ('') + const [loading, setLoading] = useState (false) + const [sourceIssues, setSourceIssues] = useState> ([]) + const [sourceError, setSourceError] = useState (null) + const saveTimer = useRef (null) + const editedRef = useRef (false) + + const lineCount = countImportSourceLines (source) + const sourceUrls = extractImportSourceUrls (source) + const withinPathLimit = isPostNewReviewPathWithinLimit (sourceUrls) + const messages = sourceError != null ? [sourceError] : [] + const sourceDescribedBy = + [sourceError != null ? SOURCE_ERROR_ID : null, + sourceIssues.length > 0 ? SOURCE_ISSUES_ID : null] + .filter (value => value != null) + .join (' ') + + useEffect (() => { + const draft = loadPostImportSourceDraft (message => + toast ({ title: '保存済み入力を復元できませんでした', description: message })) + + if (!(editedRef.current)) + { + setSource (current => + current === '' + ? draft.source + : current) + } + }, []) + + useEffect (() => { + if (saveTimer.current != null) + window.clearTimeout (saveTimer.current) + + saveTimer.current = window.setTimeout (() => { + savePostImportSourceDraft (source, message => + toast ({ title: '入力内容を保存できませんでした', description: message })) + }, 300) + + return () => { + if (saveTimer.current != null) + window.clearTimeout (saveTimer.current) + } + }, [source]) + + const preview = async () => { + if (lineCount === 0) + { + setSourceIssues ([]) + setSourceError ('URL を入力してください．') + return + } + + const issues = validateImportSource (source) + if (issues.length > 0) + { + setSourceIssues (issues) + setSourceError (null) + return + } + if (!(withinPathLimit)) + return + + setLoading (true) + setSourceIssues ([]) + setSourceError (null) + try + { + navigate (buildPostNewReviewPath (sourceUrls)) + } + finally + { + setLoading (false) + } + } + + if (!(editable)) + return + + return ( + + + {`広場に投稿を追加 | ${ SITE_TITLE }`} + + + + 広場に投稿を追加 + + {() => ( +