XinJe_PLC_Skill/references/known-ingestion-issues.md

Known Ingestion Issues

Use this reference before searching Xinje’s website, downloading official files, or extracting Chinese PDF text. It records repeated problems and the working fixes.

PowerShell And UTF-8

• PowerShell may display UTF-8 output as mojibake even when the file is correct.
• Prefer running bundled Python with -X utf8.
• When reading Markdown generated from PDFs, use Get-Content -Encoding UTF8.
• If command output looks garbled, verify with Select-String or Python before assuming the source is corrupt.

Xinje Download Center API

• Download-center page: https://www.xinje.com/web/downloadCenter/index.
• File list endpoint: /web/downloadCenter/file.
• Use headers:
  • User-Agent: Mozilla/5.0
  • X-Requested-With: XMLHttpRequest
• Category tab parameters are easy to misread. The page’s changePage(page, seriesId, oneId, twoId) call uses oneId for top-level file categories:
  • oneId=16: 产品手册
  • oneId=19: 彩页/样本
  • oneId=30: 软件工具
  • oneId=31: 产品图纸
  • oneId=33: EPLAN
  • oneId=34: 样例程序
  • oneId=49: 认证证书
• Do not assume twoId=16 means 产品手册; that returns empty results for the top-level manual tab.

Download URL Problems

• Direct https://cdn.xinje.com/... URLs may return 403 unless requested with a browser-like user agent and the download-center referer.
• /web/file-download/down/{id} can return a signed OSS URL.
• In some environments, signed OSS URLs may fail with 400 InvalidArgument and the message that both Signature and Authorization are present. When that happens, try the direct catalog CDN URL without query-string signature, with the download-center referer.
• Always record file ID, title, version, update date, and catalog URL before downloading.
• In the Codex sandbox, direct network sockets may fail with WinError 10013. Re-run the same download command with approval/escalation rather than changing the URL logic.
• For batch downloads, prefer scripts/download_catalog_ids.py; it uses the catalog URL and browser-like headers and creates filenames with version/update-date suffixes.

PDF Extraction

• Use pdfplumber for page-range text extraction and pypdf for quick page count / keyword scans.
• Do not trust extracted layout for tables without checking pages visually or cross-checking nearby text.
• For large manuals, first extract the front-matter table of contents and create a coverage map.
• For Chinese PDFs, extracted text can be correct while terminal display is wrong; save as UTF-8 Markdown and inspect with UTF-8 mode.
• pdfplumber can be slow across many large PDFs. For broad inventory, use scripts/probe_pdf_manuals.py with a small --pages value; for deep extraction, target a specific PDF and page range.

Skill Packaging

• Keep full catalogs in assets/catalogs/ instead of loading large tables in SKILL.md.
• Keep original manuals in assets/manuals/ only when they are public and useful as source-of-truth fallback.
• Do not add a new PDF to assets/manuals/ silently. Update coverage-map.md, source-index.md, and at least one task-oriented reference or full-manual overview in the same version.