Skip to content

anubis: new, 1.20.0 #11561

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 1 commit into from
Jul 5, 2025
Merged

anubis: new, 1.20.0 #11561

merged 1 commit into from
Jul 5, 2025

Conversation

darkyzhou
Copy link
Contributor

@darkyzhou darkyzhou commented Jul 1, 2025 •
edited by aosc-buildit-bot
Loading

Topic Description

  • anubis: new, 1.20.0

Package(s) Affected

  • anubis: 1.20.0

Security Update?

No

Build Order

#buildit anubis

Test Build(s) Done

Primary Architectures

  • AMD64 amd64
  • AArch64 arm64
  • LoongArch 64-bit loongarch64

Secondary Architectures

  • Loongson 3 loongson3
  • PowerPC 64-bit (Little Endian) ppc64el
  • RISC-V 64-bit riscv64

@darkyzhou darkyzhou added the new-package New AOSC OS package label Jul 1, 2025
@darkyzhou darkyzhou marked this pull request as ready for review July 2, 2025 06:22
Cyanoxygen
Cyanoxygen reviewed Jul 2, 2025
View reviewed changes
MingcongBai
MingcongBai requested changes Jul 3, 2025
View reviewed changes
@darkyzhou
Copy link
Contributor Author

darkyzhou commented Jul 3, 2025 •
edited
Loading

Works on my LoongArch VM, you can see it by visiting https://hydra.nix4loong.cn

(Please ignore the hostname "nixos", the VM actually runs AOSC)

root@nixos ~# systemctl status [email protected] --no-pager -l
● [email protected] - "Anubis HTTP defense proxy (instance main)"
     Loaded: loaded (/usr/lib/systemd/system/[email protected]; enabled; preset: disabled)
     Active: active (running) since Wed 2025-07-02 00:56:53 CST; 1 day 9h ago
 Invocation: edebbeb822bc435fa8f771f9cae32a92
   Main PID: 285625 (anubis)
      Tasks: 9 (limit: 4487)
     Memory: 9M (peak: 21.5M)
        CPU: 13.851s
     CGroup: /system.slice/system-anubis.slice/[email protected]
             └─285625 /usr/bin/anubis
7月 02 00:56:53 nixos anubis[285625]: {"time":"2025-07-02T00:56:53.739624417+08:00","level":"WARN","source":{"function":"github.com/TecharoHQ/anubis/lib/policy.ParseConfig","file":"/var/cache/acbs/build/acbs.8fynuwnb/anubis/lib/policy/policy.go","line":121},"msg":"You have specified a Thoth specific check but you have no Thoth client configured. Please read https://anubis.techaro.lol/docs/admin/thoth for more information","check":"asn","settings":{"match":[13335,136907,45102]}}
7月 02 00:56:53 nixos anubis[285625]: {"time":"2025-07-02T00:56:53.745374451+08:00","level":"WARN","source":{"function":"main.main","file":"/var/cache/acbs/build/acbs.8fynuwnb/anubis/cmd/anubis/main.go","line":327},"msg":"generating random key, Anubis will have strange behavior when multiple instances are behind the same load balancer target, for more information: see https://anubis.techaro.lol/docs/admin/installation#key-generation"}
7月 02 00:56:53 nixos anubis[285625]: {"time":"2025-07-02T00:56:53.745427831+08:00","level":"WARN","source":{"function":"main.main","file":"/var/cache/acbs/build/acbs.8fynuwnb/anubis/cmd/anubis/main.go","line":341},"msg":"REDIRECT_DOMAINS is not set, Anubis will only redirect to the same domain a request is coming from, see https://anubis.techaro.lol/docs/admin/configuration/redirect-domains"}
7月 02 00:56:53 nixos anubis[285625]: {"time":"2025-07-02T00:56:53.746535797+08:00","level":"INFO","source":{"function":"main.main","file":"/var/cache/acbs/build/acbs.8fynuwnb/anubis/cmd/anubis/main.go","line":392},"msg":"listening","url":"http://localhost:8923","difficulty":4,"serveRobotsTXT":false,"target":"http://192.168.3.3:3000","version":"devel","use-remote-address":false,"debug-benchmark-js":false,"og-passthrough":false,"og-expiry-time":86400000000000,"base-prefix":"","cookie-expiration-time":604800000000000,"rule-error-ids":{"ai-catchall":"75bb751cc7c631a5","ai-clients":"b3728715388cb593","ai-crawlers-search":"b0faade6eb1303e1","ai-crawlers-training":"26301432c34cf028","headless-chrome":"4d1dbaddfcc0f385","headless-chromium":"290aba64f1527af9","lightpanda":"44d2de56a972dc6d","us-artificial-intelligence-scraper":"f9d6578d179b858b"}}
7月 02 02:42:49 nixos anubis[285625]: {"time":"2025-07-02T02:42:49.162056284+08:00","level":"INFO","source":{"function":"github.com/TecharoHQ/anubis/lib.(*Server).checkRules","file":"/var/cache/acbs/build/acbs.8fynuwnb/anubis/lib/anubis.go","line":220},"msg":"explicit deny","user_agent":"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/125.0.6422.60 Safari/537.36","accept_language":"","priority":"u=0, i","x-forwarded-for":"","x-real-ip":"192.168.2.10","check_result":{"name":"bot/headless-chrome","rule":"DENY","weight":0}}
7月 02 02:42:58 nixos anubis[285625]: {"time":"2025-07-02T02:42:58.583721341+08:00","level":"INFO","source":{"function":"github.com/TecharoHQ/anubis/lib.(*Server).checkRules","file":"/var/cache/acbs/build/acbs.8fynuwnb/anubis/lib/anubis.go","line":220},"msg":"explicit deny","user_agent":"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/125.0.6422.60 Safari/537.36","accept_language":"","priority":"u=0, i","x-forwarded-for":"","x-real-ip":"192.168.2.10","check_result":{"name":"bot/headless-chrome","rule":"DENY","weight":0}}
7月 02 02:50:20 nixos anubis[285625]: {"time":"2025-07-02T02:50:20.8506473+08:00","level":"INFO","source":{"function":"github.com/TecharoHQ/anubis/lib.(*Server).checkRules","file":"/var/cache/acbs/build/acbs.8fynuwnb/anubis/lib/anubis.go","line":220},"msg":"explicit deny","user_agent":"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/125.0.6422.60 Safari/537.36","accept_language":"","priority":"u=0, i","x-forwarded-for":"","x-real-ip":"192.168.2.10","check_result":{"name":"bot/headless-chrome","rule":"DENY","weight":0}}
7月 03 07:12:34 nixos anubis[285625]: {"time":"2025-07-03T07:12:34.743966992+08:00","level":"INFO","source":{"function":"github.com/TecharoHQ/anubis/lib.(*Server).checkRules","file":"/var/cache/acbs/build/acbs.8fynuwnb/anubis/lib/anubis.go","line":220},"msg":"explicit deny","user_agent":"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/125.0.6422.60 Safari/537.36","accept_language":"","priority":"u=0, i","x-forwarded-for":"","x-real-ip":"192.168.2.10","check_result":{"name":"bot/headless-chrome","rule":"DENY","weight":0}}
7月 03 07:19:02 nixos anubis[285625]: {"time":"2025-07-03T07:19:02.092271874+08:00","level":"INFO","source":{"function":"github.com/TecharoHQ/anubis/lib.(*Server).checkRules","file":"/var/cache/acbs/build/acbs.8fynuwnb/anubis/lib/anubis.go","line":220},"msg":"explicit deny","user_agent":"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/125.0.6422.60 Safari/537.36","accept_language":"","priority":"u=0, i","x-forwarded-for":"","x-real-ip":"192.168.2.10","check_result":{"name":"bot/headless-chrome","rule":"DENY","weight":0}}
7月 03 07:19:05 nixos anubis[285625]: {"time":"2025-07-03T07:19:05.285813668+08:00","level":"INFO","source":{"function":"github.com/TecharoHQ/anubis/lib.(*Server).checkRules","file":"/var/cache/acbs/build/acbs.8fynuwnb/anubis/lib/anubis.go","line":220},"msg":"explicit deny","user_agent":"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/125.0.6422.60 Safari/537.36","accept_language":"","priority":"u=0, i","x-forwarded-for":"","x-real-ip":"192.168.2.10","check_result":{"name":"bot/headless-chrome","rule":"DENY","weight":0}}

root@nixos ~# robots2policy 
Usage of robots2policy:
robots2policy [options] -input <robots.txt>
  -action string
    	default action for disallowed paths: ALLOW, DENY, CHALLENGE, WEIGH (default "CHALLENGE")
  -crawl-delay-weight int
    	if > 0, add weight adjustment for crawl-delay (difficulty adjustment)
  -deny-user-agents string
    	action for specifically blocked user agents: DENY, CHALLENGE (default "DENY")
  -format string
    	output format: yaml or json (default "yaml")
  -help
    	show help
  -input string
    	path to robots.txt file (use - for stdin)
  -name string
    	name for the generated policy (default "robots-txt-policy")
  -output string
    	output file path (use - for stdout, defaults to stdout)
Examples:
  # Convert local robots.txt file
  robots2policy -input robots.txt -output policy.yaml
  # Convert from URL
  robots2policy -input https://example.com/robots.txt -format json
  # Read from stdin, write to stdout
  curl https://example.com/robots.txt | robots2policy -input -

@aosc-buildit-bot
Copy link
Contributor

Dickens-topic report:

anubis introduced at 1.20.0~pre20250701T171106Z on amd64, arm64, loongarch64, loongson3, ppc64el, riscv64:

125 added, 0 removed, size +58.5 MB 
--- a
+++ b
@@ -0,0 +1,125 @@
+drwxr-xr-x     ./
+drwxr-xr-x     ./etc/
+drwxr-xr-x     ./etc/anubis/
+-rw-r--r--     ./etc/anubis/default.env
+drwxr-xr-x     ./usr/
+drwxr-xr-x     ./usr/bin/
+-rwxr-xr-x     ./usr/bin/anubis
+-rwxr-xr-x     ./usr/bin/robots2policy
+drwxr-xr-x     ./usr/lib/
+drwxr-xr-x     ./usr/lib/systemd/
+drwxr-xr-x     ./usr/lib/systemd/system/
+-rw-r--r--     ./usr/lib/systemd/system/[email protected]
+drwxr-xr-x     ./usr/share/
+drwxr-xr-x     ./usr/share/doc/
+drwxr-xr-x     ./usr/share/doc/anubis/
+-rw-r--r--     ./usr/share/doc/anubis/LICENSE
+-rw-r--r--     ./usr/share/doc/anubis/README.md
+drwxr-xr-x     ./usr/share/doc/anubis/data/
+drwxr-xr-x     ./usr/share/doc/anubis/data/apps/
+-rw-r--r--     ./usr/share/doc/anubis/data/apps/allow-api-routes.yaml
+-rw-r--r--     ./usr/share/doc/anubis/data/apps/bookstack-saml.yaml
+-rw-r--r--     ./usr/share/doc/anubis/data/apps/gitea-rss-feeds.yaml
+-rw-r--r--     ./usr/share/doc/anubis/data/apps/qualys-ssl-labs.yml
+-rw-r--r--     ./usr/share/doc/anubis/data/apps/searx-checker.yml
+-rw-r--r--     ./usr/share/doc/anubis/data/botPolicies.json
+-rw-r--r--     ./usr/share/doc/anubis/data/botPolicies.yaml
+drwxr-xr-x     ./usr/share/doc/anubis/data/bots/
+-rw-r--r--     ./usr/share/doc/anubis/data/bots/_deny-pathological.yaml
+-rw-r--r--     ./usr/share/doc/anubis/data/bots/aggressive-brazilian-scrapers.yaml
+-rw-r--r--     ./usr/share/doc/anubis/data/bots/ai-catchall.yaml
+-rw-r--r--     ./usr/share/doc/anubis/data/bots/ai-robots-txt.yaml
+-rw-r--r--     ./usr/share/doc/anubis/data/bots/cloudflare-workers.yaml
+-rw-r--r--     ./usr/share/doc/anubis/data/bots/headless-browsers.yaml
+drwxr-xr-x     ./usr/share/doc/anubis/data/bots/irc-bots/
+-rw-r--r--     ./usr/share/doc/anubis/data/bots/irc-bots/archlinux-phrik.yaml
+-rw-r--r--     ./usr/share/doc/anubis/data/bots/irc-bots/gentoo-chat.yaml
+-rw-r--r--     ./usr/share/doc/anubis/data/bots/us-ai-scraper.yaml
+drwxr-xr-x     ./usr/share/doc/anubis/data/clients/
+-rw-r--r--     ./usr/share/doc/anubis/data/clients/ai.yaml
+-rw-r--r--     ./usr/share/doc/anubis/data/clients/git.yaml
+-rw-r--r--     ./usr/share/doc/anubis/data/clients/go-get.yaml
+-rw-r--r--     ./usr/share/doc/anubis/data/clients/mistral-mistralai-user.yaml
+-rw-r--r--     ./usr/share/doc/anubis/data/clients/openai-chatgpt-user.yaml
+drwxr-xr-x     ./usr/share/doc/anubis/data/clients/small-internet-browsers/
+-rw-r--r--     ./usr/share/doc/anubis/data/clients/small-internet-browsers/_permissive.yaml
+-rw-r--r--     ./usr/share/doc/anubis/data/clients/small-internet-browsers/netsurf.yaml
+-rw-r--r--     ./usr/share/doc/anubis/data/clients/small-internet-browsers/palemoon.yaml
+-rw-r--r--     ./usr/share/doc/anubis/data/clients/x-firefox-ai.yaml
+drwxr-xr-x     ./usr/share/doc/anubis/data/common/
+-rw-r--r--     ./usr/share/doc/anubis/data/common/allow-api-like.yaml
+-rw-r--r--     ./usr/share/doc/anubis/data/common/allow-private-addresses.yaml
+-rw-r--r--     ./usr/share/doc/anubis/data/common/json-api.yaml
+-rw-r--r--     ./usr/share/doc/anubis/data/common/keep-internet-working.yaml
+-rw-r--r--     ./usr/share/doc/anubis/data/common/rfc-violations.yaml
+drwxr-xr-x     ./usr/share/doc/anubis/data/crawlers/
+-rw-r--r--     ./usr/share/doc/anubis/data/crawlers/_allow-good.yaml
+-rw-r--r--     ./usr/share/doc/anubis/data/crawlers/ai-search.yaml
+-rw-r--r--     ./usr/share/doc/anubis/data/crawlers/ai-training.yaml
+-rw-r--r--     ./usr/share/doc/anubis/data/crawlers/applebot.yaml
+-rw-r--r--     ./usr/share/doc/anubis/data/crawlers/bingbot.yaml
+-rw-r--r--     ./usr/share/doc/anubis/data/crawlers/duckduckbot.yaml
+-rw-r--r--     ./usr/share/doc/anubis/data/crawlers/googlebot.yaml
+-rw-r--r--     ./usr/share/doc/anubis/data/crawlers/internet-archive.yaml
+-rw-r--r--     ./usr/share/doc/anubis/data/crawlers/kagibot.yaml
+-rw-r--r--     ./usr/share/doc/anubis/data/crawlers/marginalia.yaml
+-rw-r--r--     ./usr/share/doc/anubis/data/crawlers/mojeekbot.yaml
+-rw-r--r--     ./usr/share/doc/anubis/data/crawlers/openai-gptbot.yaml
+-rw-r--r--     ./usr/share/doc/anubis/data/crawlers/openai-searchbot.yaml
+-rw-r--r--     ./usr/share/doc/anubis/data/crawlers/qwantbot.yaml
+drwxr-xr-x     ./usr/share/doc/anubis/data/meta/
+-rw-r--r--     ./usr/share/doc/anubis/data/meta/README.md
+-rw-r--r--     ./usr/share/doc/anubis/data/meta/ai-block-aggressive.yaml
+-rw-r--r--     ./usr/share/doc/anubis/data/meta/ai-block-moderate.yaml
+-rw-r--r--     ./usr/share/doc/anubis/data/meta/ai-block-permissive.yaml
+drwxr-xr-x     ./usr/share/doc/anubis/docs/
+-rw-r--r--     ./usr/share/doc/anubis/docs/CHANGELOG.md
+drwxr-xr-x     ./usr/share/doc/anubis/docs/admin/
+-rw-r--r--     ./usr/share/doc/anubis/docs/admin/algorithm-selection.mdx
+-rw-r--r--     ./usr/share/doc/anubis/docs/admin/caveats-gitea-forgejo.mdx
+drwxr-xr-x     ./usr/share/doc/anubis/docs/admin/configuration/
+drwxr-xr-x     ./usr/share/doc/anubis/docs/admin/configuration/challenges/
+-rw-r--r--     ./usr/share/doc/anubis/docs/admin/configuration/challenges/metarefresh.mdx
+-rw-r--r--     ./usr/share/doc/anubis/docs/admin/configuration/challenges/proof-of-work.mdx
+-rw-r--r--     ./usr/share/doc/anubis/docs/admin/configuration/custom-status-codes.mdx
+-rw-r--r--     ./usr/share/doc/anubis/docs/admin/configuration/expressions.mdx
+-rw-r--r--     ./usr/share/doc/anubis/docs/admin/configuration/import.mdx
+-rw-r--r--     ./usr/share/doc/anubis/docs/admin/configuration/impressum.mdx
+-rw-r--r--     ./usr/share/doc/anubis/docs/admin/configuration/open-graph.mdx
+-rw-r--r--     ./usr/share/doc/anubis/docs/admin/configuration/redirect-domains.mdx
+-rw-r--r--     ./usr/share/doc/anubis/docs/admin/configuration/subrequest-auth.mdx
+-rw-r--r--     ./usr/share/doc/anubis/docs/admin/configuration/thresholds.mdx
+-rw-r--r--     ./usr/share/doc/anubis/docs/admin/default-allow-behavior.mdx
+drwxr-xr-x     ./usr/share/doc/anubis/docs/admin/environments/
+-rw-r--r--     ./usr/share/doc/anubis/docs/admin/environments/apache.mdx
+-rw-r--r--     ./usr/share/doc/anubis/docs/admin/environments/caddy.mdx
+-rw-r--r--     ./usr/share/doc/anubis/docs/admin/environments/docker-compose.mdx
+-rw-r--r--     ./usr/share/doc/anubis/docs/admin/environments/kubernetes.mdx
+-rw-r--r--     ./usr/share/doc/anubis/docs/admin/environments/nginx.mdx
+-rw-r--r--     ./usr/share/doc/anubis/docs/admin/environments/traefik.mdx
+drwxr-xr-x     ./usr/share/doc/anubis/docs/admin/frameworks/
+-rw-r--r--     ./usr/share/doc/anubis/docs/admin/frameworks/htmx.mdx
+-rw-r--r--     ./usr/share/doc/anubis/docs/admin/frameworks/wordpress.mdx
+-rw-r--r--     ./usr/share/doc/anubis/docs/admin/installation.mdx
+-rw-r--r--     ./usr/share/doc/anubis/docs/admin/native-install.mdx
+-rw-r--r--     ./usr/share/doc/anubis/docs/admin/policies.mdx
+-rw-r--r--     ./usr/share/doc/anubis/docs/admin/robots2policy.mdx
+-rw-r--r--     ./usr/share/doc/anubis/docs/admin/thoth.mdx
+drwxr-xr-x     ./usr/share/doc/anubis/docs/design/
+-rw-r--r--     ./usr/share/doc/anubis/docs/design/how-anubis-works.mdx
+-rw-r--r--     ./usr/share/doc/anubis/docs/design/why-proof-of-work.mdx
+drwxr-xr-x     ./usr/share/doc/anubis/docs/developer/
+-rw-r--r--     ./usr/share/doc/anubis/docs/developer/building-anubis.md
+-rw-r--r--     ./usr/share/doc/anubis/docs/developer/code-quality.md
+-rw-r--r--     ./usr/share/doc/anubis/docs/developer/local-dev.md
+-rw-r--r--     ./usr/share/doc/anubis/docs/developer/signed-commits.md
+-rw-r--r--     ./usr/share/doc/anubis/docs/funding.md
+-rw-r--r--     ./usr/share/doc/anubis/docs/index.mdx
+drwxr-xr-x     ./usr/share/doc/anubis/docs/user/
+-rw-r--r--     ./usr/share/doc/anubis/docs/user/frequently-asked-questions.mdx
+-rw-r--r--     ./usr/share/doc/anubis/docs/user/known-broken-extensions.md
+-rw-r--r--     ./usr/share/doc/anubis/docs/user/known-instances.md
+-rw-r--r--     ./usr/share/doc/anubis/docs/user/why-see-challenge.md
+drwxr-xr-x     ./usr/share/licenses/
+drwxr-xr-x     ./usr/share/licenses/anubis/
+-rw-r--r--     ./usr/share/licenses/anubis/LICENSE

@MingcongBai MingcongBai merged commit cc5fcf8 into stable Jul 5, 2025
@MingcongBai MingcongBai deleted the anubis-new branch July 5, 2025 02:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@Cyanoxygen Cyanoxygen Cyanoxygen left review comments

@MingcongBai MingcongBai MingcongBai approved these changes

Assignees
No one assigned
Labels
new-package New AOSC OS package
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@darkyzhou @aosc-buildit-bot @MingcongBai @Cyanoxygen