https://offpunk.net ← that’s the original project that this might get upstreamed to in some parts
Note: since publication I have found numerous bugs in my implementation. Rewrites upcoming. I just can’t work silently without at least a blog post every day.
Update: now please apply the subsequent patches before trying out anything. The code here below was bad.
Here is the new post: Amending my Offpunk redirection implementation
What does my large patch bring? (a post-publish added summary)
So far, Offpunk only allows you to redirect from one netloc (usually host or host:port) to another. At that, it won’t match if the origin netloc had an explicit port number specified. I wanted a redirections engine that would allow me to redirect to a different port, on a different host, under a path and with some suffix — like .json.
I also wanted to eliminate the hardcoded redirects and domain blocks and put them into a user’s rc file. But that required adding them to user’s rc. I didn’t want to alter a user’s existing offpunkrc, so I came up with the idea to create a pre-rc for the user (later, only such that would already have their rc, as new users can just get the actual rc pre-filled and only get a pre-rc created if new defaults come from upstream).
A side-glance into what are special-website commands now
I still want to later also move all kinds of defaults from the options dictionary, especially the default commands, to user rc… As a <figure>, what follows is a fragment of the options dictionary and all the do_ methods for the commands search, websearch, wikipedia, xkcd and gus.
"wikipedia": "gemini://gemi.dev/cgi-bin/wp.cgi/view/%s?%s", "search": "gemini://kennedy.gemi.dev/search?%s", "websearch": "https://wiby.me/?q=%s", "accept_bad_ssl_certificates": False, "default_protocol": "gemini",
Initial confusion and renaming the big class
(a separate commit, not attached)
First thing that initially made it hard for me to find my way around in this code was that the main command-loop of the textual user interface was in a class named GeminiClient, which made it seem like I stumbled upon just the gemini part of the implementation. So i renamed it to TouringShell in my worktree.
But now on with the big ugly commit. Unfortunately, it has been 16 hours since I last sat down to that and I can no longer write a coherent blog post tonight, so here is my commit message:
Redirections can be detailed now. Things like pre-rc autocreation also
offblocklist.py had a hardcoded list of redirections that required
'none'-ing each entry as per current Offpunk version to disable.
It is now replaced with a new mechanism of creating an offpunkprerc file
that contains them all as redirect rules. Except now we don't match
suffix domains too, because those rules are split into example.com and
into *.example.com, with example.com also adding as www.example.com
unless -!www switch is applied like "redirect example.com -!www ...".
The redirects now also allow a -silent switch that, since they are now
commands reporting the result, makes only the final count printed
when rc queue is done.
They now also use fnmatch glob library to match on the original domains
since previously it was just checking for "*" prefix to suffix check.
It is now also possible to match on either the domain for any explicit
port as well as the domain particularly with no explicit port —
— the -noport toggle allows for the latter, the former is default.
There are new options like -^?___ that remove the ___ query parameter,
-/___ for pattern test on path, -?___ ___ for pattern test on
values of parameters, -#___ for pattern test on URL fragment,
and an 'unless' keyword that inverts those pattern tests in the command.
A new templating engine is introduced, allowing to write a special kind
of destination URL string where host, scheme, netloc, path, query, and
fragment, can all be templated in with curly braces Python templating.
Intentionally, host is not the same as netloc and also if explicit port
is not specified, the :{port} (or any other character in place of colon)
gets removed.
Both config files are also loaded into opnk and opnk is extended with
an --rc-redirects option that makes it also apply the redirects.
---
I put in here a very opinionated approach that, when good defaults are preconfigured by distribution/authors into config files,
we should neither
- just not mention new additions to the user on updates
- ask them to add stuff themselves by hand right when they get the memo
- ask them to merge files during routine upgrade
- hardcode stuff because it’s better than not put it anywhere just to avoid asking people to change their configs
and that we can actually do a cool innovative thing by:
- Creating a preconfiguration file for those who had their config for a while
- Make a flag for the config …
Or, as I say in the help for –pre-config flag in the subsequent another commit “Preconfig creation opt-out” that improves on the idea (introduces the aforementioned flag):
This one gets created if it doesn’t exist. Unless your config starts with ‘set dont_create_preconfig’. If you have this file, usually it is expected to contain upstream defaults. Future releases may include new rc defaults. If you move all the current preconfigured defaults to your main rc and delete prerc, a future release could create a new one for you and ask you again to add a subsequent ‘set dont_create_preconfig 1’ to your main rc. New users get the most recent opt-out by default as their rc contain newest defaults.
Let’s look through a bunch of stuff. From the first file, a redirections_stuff.py that I put in a new directory I called implementation_parts just not to make a mess, we can see how matching the origin domains for the redirect command now has fnmatch glob pattern matching and special logic to not have explicit port in URL for example accidentally circumventing our simple blocks for corporate scum hosts:
+netloc_port_regex = re.compile(r'^.+(?P<colonport>:\d{1,5})$')
+ for key, rule in redirects.keys():
+ if not rule.noport:
+ netloc_port_match = netloc_port_regex.match(netloc)
+ if netloc_port_match and not netloc_port_regex.match(key):
+ colon_port = netloc_port_match["colonport"]
+ netloc = netloc[:-len(colon_port)]
+ if fnmatch.fnmatch(netloc, key):
Let’s look at the logic for the applicability decider for the additional narrowing flags in a later stage:
+def process_redirection_rule(parsed, rule):
+ applicable = True
+ for (true, glob) in rule.pathpattern:
+ applicable |= true == fnmatch.fnmatch(parsed.path, glob)
+ if not applicable: break
+ for (true, glob) in rule.fragpattern:
+ applicable |= true == fnmatch.fnmatch(parsed.path, glob)
+ if not applicable: break
+ truequeryglob_hitboard = {nameglob: {
+ valueglob for (true, valueglob) in valueglobs if true
+ } for nameglob, valueglobs in rule.parampatterns.items()}
+ query = urllib.parse_qs(parsed.query)
+ for actual_param_name, values in query.items():
+ for nameglob, (true, valueglob) in rule.parampatterns.items():
+ if not fnmatch.fnmatch(actual_param_name, nameglob): continue
+ result = any(fnmatch.fnmatch(value, valueglob) for value in values)
+ applicable |= true == result
+ if not applicable: break
+ if true and result:
+ truequeryglob_hitboard[nameglob].discard(valueglob)
+ if not applicable: break
+
+ for purge_glob in rule.purge_query:
+ if fnmatch.fnmatch(actual_param_name, purge_glob):
+ query.pop(purge_glob)
+ applicable &= not any(True for valueglobs_hitboard
+ in truequeryglob_hitboard
+ if valueglobs_hitboard)
+ if applicable:
Here,
- parsed is the result object of urlparse.
- The true in those tuples is whether the particular matcher was before or after a possible unless keyword.
Below we have the capable-but-unwieldy templating system and the original basic one, also very much expanded:
+schema_regex = re.compile(r'^[a-zA-Z][a-zA-Z0-9+.-]*?(?=://)')
+port_templating = re.compile(r'.[{]port[}]')
. . .
+ if applicable:
+ querystring = urllib.urlencode(query, doseq=True)
+ if rule.destination and '{' in str(rule.destination):
+ templating = dict(
+ host = parsed.hostname,
+ net = parsed.netloc,
+ path = parsed.path.removeprefix('/'),
+ scheme = parsed.scheme,
+ query = querystring,
+ fragment = parsed.fragment,
+ )
+ port = parsed.port
+ destination = rule.destination
+ if port:
+ templating.update(port=port)
+ else:
+ destination = port_templating.sub('', destination)
+
+ url = destination.format(templating)
+ parsed = urllib.parse.urlparse(url, scheme=parsed.scheme)
+ else:
+ r = dict(query=querystring)
+ if rule.destination:
+ destination_has_schema = schema_regex.match(rule.destination)
+ if not destination_has_schema:
+ r.update(netloc=rule.destination)
+ else:
+ schema = destination_has_schema[0]
+ r.update(schema=schema)
+ with_slashslash = len(schema) + 3
+ if len(rule.destination) > with_slashslash:
+ r.update(netloc=rule.destination[with_slashslash:])
+ parsed = parsed._replace(**r)
+ url = urllib.parse.urlunparse(parsed)
+ return parsed, url
I will probably skip detailing how the rules are parsed and stuff. Here, have what I wrote into the help-not-help for redirections (I should have looked into the help, but I put it in the result of listing redirections after calling the redirect command by itself:
To block subdomains, prefix with *.: “redirect *.origine.com BLOCK”
Rules duplicate to www. subdomain too by default, unless you add -!www
If a port is not specified, it is not matched. Unless -noport
Add -silent switch anywhere after the origin domain to add silently
After domain pattern, even if it’s just ‘*’, you can add
-^?name to remove params ?name= from the URL,
-/path/*.jpg to apply the rule only to jpg files in a path
-?param value* to apply the rule only if ?param=valuesomething
-#fragment to apply the rule only if #fragment in URL
‘unless’ to invert the meaning of subsequent flagsIf a curly brace will be found in your redirect destination, templating
will be applied to it. That allows you to template in:
– the ‘{net}’ (netloc, that is host or host:port or more),
– the ‘{host} (just the hostname without the port even if specified)
– the ‘{port} (any ‘_{port}’ will be removed from template,
together with *one single preceding character*, usually a ‘:’, if no port found)
the ‘{path}’, the ‘{scheme}’, the ‘{query}’ and the ‘{fragment}’.
For example, a way to redirect to gemini on same subdomain is to:
redirect *.example.com gemini://{host}/{path}?{query}#{fragment}For one domain the basic syntax also now allows that you use
redirect example.com gemini://example.comThis should have went to ‘help redirect’ instead haha
So why do I want this all? Well, I think just using the 3rd party front-ends isn’t something we should do if we have a properly extensible browser ourselves, without struggling with stuff like CORS and CSRF. So we can just first have the redirections engine redirect us to, for example, some unofficlal Reddit json, and then we can match the url of such cached json and know that these are reddit jsons that we open with a particular handler, per their actual url pattern. Later I intended to move forward towards seeing whether the handlers concept is developed enough for such matching and opening, and extending if necessary.
Going back to those pre-config files, here is how the managing of two rc files was initially like, in the earlier commit of these two:
+def init_both_configs(argparse_args, + skip_go=False, interactive=True, verbose=True): + yield from init_config(argparse_args.pre_config_file, + skip_go, interactive, verbose, + create_if_not_exists=interactive) + yield from init_config(argparse_args.config_file, + skip_go, interactive, verbose)
Now it the logic has the whole opt-out clause behavior:
def init_both_configs(argparse_args, skip_go=False, interactive=True, verbose=True): existed = os.path.exists(argparse_args.config_file) config = init_config(argparse_args.config_file, skip_go, interactive, verbose, create_if_not_exists=(interactive and not existed), optout_from_preconfig=True) first_from_config = next(config, None) user_excluded_from_preconfig = (first_from_config and first_from_config.strip() == "set dont_create_preconfig") if not user_excluded_from_preconfig: print(first_from_config.strip()) yield from init_config(argparse_args.pre_config_file, skip_go, interactive, verbose, create_if_not_exists=(interactive and not user_excluded_from_preconfig )) if first_from_config: if not user_excluded_from_preconfig: yield first_from_config yield from config
The optout_from_preconfig=True makes it write first the “set dont_create_preconfig” on top if it’s creating the file. Later it skips yielding it because that is not a real Offpunk option and it would probably result in a “What?” line from Offpunk.
Anyway, below are the two patch files in entirety. But I still have to write some unit tests and refine at least the strings.
If you by chance think this reads good or when my subsequent amends will at some point read good, please say something.
From 40f892a03de89fa173aebb8c9ab9700234df90a4 Mon Sep 17 00:00:00 2001
Date: Sat, 24 Jan 2026 20:28:05 +0100
Subject: [PATCH 1/2] Redirections can be detailed now. Things like pre-rc
autocreation also
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
offblocklist.py had a hardcoded list of redirections that required
'none'-ing each entry as per current Offpunk version to disable.
It is now replaced with a new mechanism of creating an offpunkprerc file
that contains them all as redirect rules. Except now we don't match
suffix domains too, because those rules are split into example.com and
into *.example.com, with example.com also adding as www.example.com
unless -!www switch is applied like "redirect example.com -!www ...".
The redirects now also allow a -silent switch that, since they are now
commands reporting the result, makes only the final count printed
when rc queue is done.
They now also use fnmatch glob library to match on the original domains
since previously it was just checking for "*" prefix to suffix check.
It is now also possible to match on either the domain for any explicit
port as well as the domain particularly with no explicit port —
— the -noport toggle allows for the latter, the former is default.
There are new options like -^?___ that remove the ___ query parameter,
-/___ for pattern test on path, -?___ ___ for pattern test on
values of parameters, -#___ for pattern test on URL fragment,
and an 'unless' keyword that inverts those pattern tests in the command.
A new templating engine is introduced, allowing to write a special kind
of destination URL string where host, scheme, netloc, path, query, and
fragment, can all be templated in with curly braces Python templating.
Intentionally, host is not the same as netloc and also if explicit port
is not specified, the :{port} (or any other character in place of colon)
gets removed.
Both config files are also loaded into opnk and opnk is extended with
an --rc-redirects option that makes it also apply the redirects.
---
create_preconfig.py | 42 ++++
implementation_parts/__init__.py | 0
implementation_parts/redirections_stuff.py | 264 +++++++++++++++++++++
offblocklist.py | 34 ---
offpunk.py | 91 ++-----
offutils.py | 51 +++-
opnk.py | 38 ++-
7 files changed, 394 insertions(+), 126 deletions(-)
create mode 100644 create_preconfig.py
create mode 100644 implementation_parts/__init__.py
create mode 100644 implementation_parts/redirections_stuff.py
delete mode 100644 offblocklist.py
diff --git a/create_preconfig.py b/create_preconfig.py
new file mode 100644
index 0000000..8af6643
--- /dev/null
+++ b/create_preconfig.py
@@ -0,0 +1,42 @@
+import os
+
+default_preconfig_content = """
+redirect *.reddit.com teddit.net -silent
+redirect reddit.com teddit.net -silent
+redirect *.medium.com scribe.rip -silent
+redirect medium.com scribe.rip -silent
+redirect * -^?utm_source -silent
+
+"""
+
+blocked = {
+ "twitter.com",
+ "x.com",
+ # "youtube.com",
+ # "youtu.be",
+ "facebook.com",
+ "facebook.net",
+ "fbcdn.net",
+ "linkedin.com",
+ "*licdn.com",
+ "*admanager.google.com",
+ "*google-health-ads.blogspot.com",
+ "*firebase.google.com",
+ "*google-webfonts-helper.herokuapp.com",
+ "*tiktok.com" ,
+ "*doubleclick.net",
+ "*google-analytics.com" ,
+ "*ads.yahoo.com",
+ "*advertising.amazon.com",
+ "*advertising.theguardian.com",
+ "*advertise.newrepublic.com",
+}
+
+def yield_preconfig_content():
+ yield from default_preconfig_content.splitlines(keepends=True)
+ yield from (f"redirect {item} -silent BLOCK" + os.linesep for item in blocked)
+
+def preconfig_creation(filepath,
+ preconfig_lines=yield_preconfig_content()):
+ with open(filepath, "x", encoding="utf-8") as fp:
+ fp.writelines(preconfig_lines)
\ No newline at end of file
diff --git a/implementation_parts/__init__.py b/implementation_parts/__init__.py
new file mode 100644
index 0000000..e69de29
diff --git a/implementation_parts/redirections_stuff.py b/implementation_parts/redirections_stuff.py
new file mode 100644
index 0000000..bd9bd6e
--- /dev/null
+++ b/implementation_parts/redirections_stuff.py
@@ -0,0 +1,264 @@
+import fnmatch
+import re
+import urllib.parse
+from collections import defaultdict
+from types import SimpleNamespace
+
+from offutils import _
+
+netloc_port_regex = re.compile(r'^.+(?P<colonport>:\d{1,5})$')
+
+def redirections_engine(redirects, url):
+ blocked = None
+ parsed = urllib.parse.urlparse(url)
+ netloc = parsed.netloc
+ # we block/redirect even subdomains
+ for key, rule in redirects.items():
+ if not rule.noport_toggle:
+ netloc_port_match = netloc_port_regex.match(netloc)
+ if netloc_port_match and not netloc_port_regex.match(key):
+ colon_port = netloc_port_match["colonport"]
+ netloc = netloc[:-len(colon_port)]
+ if fnmatch.fnmatch(netloc, key):
+ if rule == "blocked":
+ blocked = ""
+ blocked += _("Blocked URL: ") + url + "\n"
+ blocked += _("This website has been blocked with the following rule:\n")
+ blocked += key + "\n"
+ blocked += _("Use the following redirect command to unblock it:\n")
+ blocked += "redirect %s NONE" % key
+ break
+ else:
+ parsed, url = process_redirection_rule(parsed, rule)
+ return blocked, url
+
+schema_regex = re.compile(r'^[a-zA-Z][a-zA-Z0-9+.-]*?(?=://)')
+
+port_templating = re.compile(r'.[{]port[}]')
+
+def process_redirection_rule(parsed, rule):
+ applicable = True
+ for (true, glob) in rule.pathpattern:
+ applicable |= true == fnmatch.fnmatch(parsed.path, glob)
+ if not applicable: break
+ for (true, glob) in rule.fragpattern:
+ applicable |= true == fnmatch.fnmatch(parsed.path, glob)
+ if not applicable: break
+ truequeryglob_hitboard = {nameglob: {
+ valueglob for (true, valueglob) in valueglobs if true
+ } for nameglob, valueglobs in rule.parampatterns.items()}
+ query = urllib.parse.parse_qs(parsed.query)
+ for actual_param_name, values in query.items():
+ for nameglob, (true, valueglob) in rule.parampatterns.items():
+ if not fnmatch.fnmatch(actual_param_name, nameglob): continue
+ result = any(fnmatch.fnmatch(value, valueglob) for value in values)
+ applicable |= true == result
+ if not applicable: break
+ if true and result:
+ truequeryglob_hitboard[nameglob].discard(valueglob)
+ if not applicable: break
+
+ for purge_glob in rule.purge_query:
+ if fnmatch.fnmatch(actual_param_name, purge_glob):
+ query.pop(purge_glob)
+ applicable &= not any(True for valueglobs_hitboard
+ in truequeryglob_hitboard
+ if valueglobs_hitboard)
+ if applicable:
+ querystring = urllib.parse.urlencode(query, doseq=True)
+ if rule.destination and '{' in str(rule.destination):
+ templating = dict(
+ host = parsed.hostname,
+ net = parsed.netloc,
+ path = parsed.path.removeprefix('/'),
+ scheme = parsed.scheme,
+ query = querystring,
+ fragment = parsed.fragment,
+ )
+ port = parsed.port
+ destination = rule.destination
+ if port:
+ templating.update(port=port)
+ else:
+ destination = port_templating.sub('', destination)
+
+ url = destination.format(templating)
+ parsed = urllib.parse.urlparse(url, scheme=parsed.scheme)
+ else:
+ r = dict(query=querystring)
+ if rule.destination:
+ destination_has_schema = schema_regex.match(rule.destination)
+ if not destination_has_schema:
+ r.update(netloc=rule.destination)
+ else:
+ schema = destination_has_schema[0]
+ r.update(schema=schema)
+ with_slashslash = len(schema) + 3
+ if len(rule.destination) > with_slashslash:
+ r.update(netloc=rule.destination[with_slashslash:])
+ parsed = parsed._replace(**r)
+ url = urllib.parse.urlunparse(parsed)
+ return parsed, url
+
+
+def redirection_parse_dest_and_conditions(line):
+ www_toggle = False
+ silent_toggle = False
+ noport_toggle = False
+ purge_query = set()
+ pathpattern = []
+ fragpattern = []
+ parampatterns = defaultdict(list)
+ parampattern_being_added = None
+ unless = False
+ destination = None
+ for item in line.split():
+ message_in_such_scenario = None
+ if parampattern_being_added:
+ message_in_such_scenario \
+ = f"""
+Query parameter pattern for {parampattern_being_added} was supposed to be here,
+but '{item}' found!
+"""
+ match item:
+ case '-silent':
+ assert not parampattern_being_added, message_in_such_scenario
+ silent_toggle = True
+ case '-!www':
+ assert not parampattern_being_added, message_in_such_scenario
+ assert not www_toggle, f"Two {item} toggles?"
+ www_toggle = True
+ case '-noport':
+ assert not parampattern_being_added, message_in_such_scenario
+ assert not www_toggle, f"Two {item} toggles?"
+ noport_toggle = True
+ case 'unless':
+ assert not parampattern_being_added, message_in_such_scenario
+ unless = not unless
+ case item if item.startswith("-^?"):
+ assert not parampattern_being_added, message_in_such_scenario
+ x = item.removeprefix("-^?")
+ assert not x in purge_query, f"Two identical {item} param purging options?"
+ purge_query.add(x)
+ case item if item.startswith("-/"):
+ assert not parampattern_being_added, message_in_such_scenario
+ x = item[1:]
+ assert not (not unless, x) in pathpattern, \
+ f"Two identical {item} path pattern options!"
+ assert not (unless, x) in pathpattern, \
+ f"Two identical {item} path pattern options but opposing each other!"
+ pathpattern.append((not unless, x))
+ case item if item.startswith("-#"):
+ assert not parampattern_being_added, message_in_such_scenario
+ x = item[1:]
+ assert not (not unless, x) in pathpattern, \
+ f"Two identical {item} fragment pattern options!"
+ assert not (unless, x) in pathpattern, \
+ f"Two identical {item} fragment pattern options but opposing each other!"
+ fragpattern.append((not unless, x))
+ case item if item.startswith("-?"):
+ assert not parampattern_being_added, message_in_such_scenario
+ parampattern_being_added = item.removeprefix("-?")
+ case _:
+ if parampattern_being_added:
+ our = parampatterns[parampattern_being_added]
+ assert not (not unless, item) in our, \
+ f"Two identical {item} patterns for param {parampattern_being_added}!"
+ assert not (unless, item) in our, \
+ f"Two opposing {item} patterns for param {parampattern_being_added}!"
+ parampatterns[parampattern_being_added].append((not unless, item))
+ else:
+ assert not destination, \
+ f"Can't have destination {item}, it's already {destination}!"
+ destination = item
+ return SimpleNamespace(line=" ".join(item for item in line.split(" ") if item != "-silent"),
+ www_toggle=www_toggle,
+ silent_toggle=silent_toggle,
+ noport_toggle=noport_toggle,
+ purge_query=purge_query,
+ pathpattern=pathpattern,
+ fragpattern=fragpattern,
+ parampatterns=parampatterns,
+ destination=destination)
+
+
+def redirects_cli(redirects, line, silent_redirects_counter):
+ if len(line.split()) == 1:
+ if line in redirects:
+ successprint = _("%s is redirected to %s") % (line, redirects[line])
+ else:
+ successprint = _("Please add a destination to redirect %s") % line
+ elif len(line.split()) >= 2:
+ while True:
+ orig, dest = line.split(" ", 1)
+ match orig:
+ case "-silent", "-!www":
+ line = f"{dest} {orig}"
+ case _:
+ break
+ if dest.lower() == "none":
+ if orig in redirects:
+ redirects.pop(orig)
+ successprint = _("Redirection for %s has been removed") % orig
+ else:
+ successprint = _("%s was not redirected. Nothing has changed.") % orig
+ elif dest.lower() == "block":
+ redirects[orig] = "blocked"
+ successprint = _("%s will now be blocked") % orig
+ else:
+ terms = redirection_parse_dest_and_conditions(dest)
+ redirects[orig] = terms
+ repeated_message = lambda vals: _("%s will now be redirected to %s") % vals
+ successprint = repeated_message((orig, dest))
+ if not terms.www_toggle and not orig.startswith("*"):
+ orig = 'www.' + orig
+ redirects[orig] = terms
+ successprint = "\n".join((successprint,
+ _("Because of no -!www toggle, also ") + repeated_message((orig, dest))))
+ if terms.silent_toggle:
+ successprint = ""
+ silent_redirects_counter += 1
+
+ else:
+ successprint = _("Current redirections:\n")
+ successprint += "--------------------\n"
+ for r in redirects:
+ value = redirects[r]
+ value = "blocked" if value == "blocked" else value.line
+ successprint += "%s\t->\t%s\n" % (r, value)
+ successprint += _('\nTo add new, use "redirect origine.com destination.org"')
+ successprint += _('\nTo remove a redirect, use "redirect origine.com NONE"')
+ successprint += (
+ _('\nTo completely block a website, use "redirect origine.com BLOCK"')
+ )
+ successprint += _('\nTo block subdomains, prefix with *.: "redirect *.origine.com BLOCK"')
+ successprint += _('\nRules duplicate to www. subdomain too by default, unless you add -!www')
+ successprint += _('\nIf a port is not specified, it is not matched. Unless -noport')
+ successprint += _('\nAdd -silent switch anywhere after the origin domain to add silently')
+ successprint += _(
+ """\n
+ After domain pattern, even if it's just '*', you can add
+ -^?name to remove params ?name= from the URL,
+ -/path/*.jpg to apply the rule only to jpg files in a path
+ -?param value* to apply the rule only if ?param=valuesomething
+ -#fragment to apply the rule only if #fragment in URL
+ 'unless' to invert the meaning of subsequent flags
+ """)
+ successprint += _(
+ """\n
+ If a curly brace will be found in your redirect destination, templating
+ will be applied to it. That allows you to template in:
+ - the '{net}' (netloc, that is host or host:port or more),
+ - the '{host} (just the hostname without the port even if specified)
+ - the '{port} (any '_{port}' will be removed from template,
+ together with *one single preceding character*, usually a ':', if no port found)
+ the '{path}', the '{scheme}', the '{query}' and the '{fragment}'.
+ For example, a way to redirect to gemini on same subdomain is to:
+ redirect *.example.com gemini://{host}/{path}?{query}#{fragment}
+
+ For one domain the basic syntax also now allows that you use
+ redirect example.com gemini://example.com
+ """
+ )
+ successprint += _("\n\nThis should have went to 'help redirect' instead haha")
+ return successprint, silent_redirects_counter
diff --git a/offblocklist.py b/offblocklist.py
deleted file mode 100644
index 58d8ce8..0000000
--- a/offblocklist.py
+++ /dev/null
@@ -1,34 +0,0 @@
-# The following are the default redirections from Offpunk
-# Those are by default because they should make sens with offpunk
-
-redirects = {
- "*reddit.com" : "teddit.net",
- "*medium.com" : "scribe.rip",
- }
-
-
-#following are blocked URLs. Visiting them with offpunk doesn’t make sense.
-#Blocking them will save a lot of bandwith
-
-blocked = {
- "twitter.com",
- "x.com",
-# "youtube.com",
-# "youtu.be",
- "facebook.com",
- "facebook.net",
- "fbcdn.net",
- "linkedin.com",
- "*licdn.com",
- "*admanager.google.com",
- "*google-health-ads.blogspot.com",
- "*firebase.google.com",
- "*google-webfonts-helper.herokuapp.com",
- "*tiktok.com" ,
- "*doubleclick.net",
- "*google-analytics.com" ,
- "*ads.yahoo.com",
- "*advertising.amazon.com",
- "*advertising.theguardian.com",
- "*advertise.newrepublic.com",
-}
diff --git a/offpunk.py b/offpunk.py
index eeadc42..1a3f002 100755
--- a/offpunk.py
+++ b/offpunk.py
@@ -19,7 +19,6 @@ import gettext
import ansicat
import netcache
-import offblocklist
import offthemes
import opnk
from offutils import (
@@ -30,12 +29,14 @@ from offutils import (
term_width,
unmode_url,
xdg,
- init_config,
+ add_config_file_options,
+ init_both_configs,
send_email,
_HAS_XDGOPEN,
_LOCALE_DIR,
find_root,
)
+from implementation_parts.redirections_stuff import redirections_engine, redirects_cli
gettext.bindtextdomain('offpunk', _LOCALE_DIR)
gettext.textdomain('offpunk')
@@ -207,9 +208,8 @@ class GeminiClient(cmd.Cmd):
"prompt_close": "> ",
}
self.set_prompt("ON")
- self.redirects = offblocklist.redirects
- for i in offblocklist.blocked:
- self.redirects[i] = "blocked"
+ self.redirects = {}
+ self.silent_redirects_counter = 0
term_width(new_width=self.options["width"])
self.log = {
"start_time": time.time(),
@@ -346,29 +346,11 @@ class GeminiClient(cmd.Cmd):
)
return
# Code to translate URLs to better frontends (think twitter.com -> nitter)
- parsed = urllib.parse.urlparse(url)
- netloc = parsed.netloc
- if netloc.startswith("www."):
- netloc = netloc[4:]
- # we block/redirect even subdomains
- for key in self.redirects.keys():
- match = key == netloc
- if key.startswith("*"):
- match = netloc.endswith(key[1:])
- if match:
- if self.redirects[key] == "blocked":
- text = ""
- text += _("Blocked URL: ")+url + "\n"
- text += _("This website has been blocked with the following rule:\n")
- text += key + "\n"
- text += _("Use the following redirect command to unblock it:\n")
- text += "redirect %s NONE" %key
- if handle and not self.sync_only:
- print(text)
- return
- else:
- parsed = parsed._replace(netloc=self.redirects[key])
- url = urllib.parse.urlunparse(parsed)
+ blocked, url = redirections_engine(self.redirects, url)
+ if blocked:
+ if handle and not self.sync_only:
+ print(blocked)
+ return
params = {}
params["timeout"] = self.options["short_timeout"]
if limit_size:
@@ -417,7 +399,7 @@ class GeminiClient(cmd.Cmd):
self._update_history(modedurl)
else:
# we are asked not to handle or in sync_only mode
- if self.support_http or parsed.scheme not in ["http", "https"]:
+ if self.support_http or not url.startswith('http'):
netcache.fetch(url, **params)
@needs_gi
@@ -489,37 +471,9 @@ class GeminiClient(cmd.Cmd):
# Settings
def do_redirect(self, line):
"""Display and manage the list of redirected URLs. This features is mostly useful to use privacy-friendly frontends for popular websites."""
- if len(line.split()) == 1:
- if line in self.redirects:
- print(_("%s is redirected to %s") % (line, self.redirects[line]))
- else:
- print(_("Please add a destination to redirect %s") % line)
- elif len(line.split()) >= 2:
- orig, dest = line.split(" ", 1)
- if dest.lower() == "none":
- if orig in self.redirects:
- self.redirects.pop(orig)
- print(_("Redirection for %s has been removed") % orig)
- else:
- print(_("%s was not redirected. Nothing has changed.") % orig)
- elif dest.lower() == "block":
- self.redirects[orig] = "blocked"
- print(_("%s will now be blocked") % orig)
- else:
- self.redirects[orig] = dest
- print(_("%s will now be redirected to %s") % (orig, dest))
- else:
- toprint = _("Current redirections:\n")
- toprint += "--------------------\n"
- for r in self.redirects:
- toprint += "%s\t->\t%s\n" % (r, self.redirects[r])
- toprint += _('\nTo add new, use "redirect origine.com destination.org"')
- toprint += _('\nTo remove a redirect, use "redirect origine.com NONE"')
- toprint += (
- _('\nTo completely block a website, use "redirect origine.com BLOCK"')
- )
- toprint += _('\nTo block also subdomains, prefix with *: "redirect *origine.com BLOCK"')
- print(toprint)
+ successprint, self.silent_redirects_counter = \
+ redirects_cli(self.redirects, line, self.silent_redirects_counter)
+ if successprint: print(successprint)
def do_set(self, line):
"""View or set various options."""
@@ -2294,11 +2248,7 @@ def main():
nargs="*",
help=_("Launch this command after startup"),
)
- parser.add_argument(
- "--config-file",
- metavar="FILE",
- help=_("use this particular config file instead of default"),
- )
+ add_config_file_options(parser)
parser.add_argument(
"--sync",
action="store_true",
@@ -2413,22 +2363,25 @@ def main():
depth = int(args.depth)
else:
depth = 1
- torun_queue += init_config(rcfile=args.config_file, interactive=False)
+ torun_queue += init_both_configs(args, interactive=False)
for line in torun_queue:
# This doesn’t seem to run on sync. Why?
gc.onecmd(line)
gc.call_sync(refresh_time=refresh_time, depth=depth, lists=args.url)
else:
# We are in the normal mode. First process config file
- torun_queue += init_config(rcfile=args.config_file,interactive=True)
+ torun_queue += init_both_configs(args, interactive=True)
print(_("Welcome to Offpunk!"))
#TRANSLATORS keep 'help', it's a literal command
print(_("Type `help` to get the list of available command."))
for line in torun_queue:
gc.onecmd(line)
+ if 0 != gc.silent_redirects_counter:
+ print(_("%d silent redirect rules applied.") % gc.silent_redirects_counter)
+ gc.silent_redirects_counter = 0
if args.command:
- for cmd in args.command:
- gc.onecmd(cmd)
+ for cmd_from_args in args.command:
+ gc.onecmd(cmd_from_args)
while True:
try:
gc.cmdloop()
diff --git a/offutils.py b/offutils.py
index 336c18d..2ba32a6 100644
--- a/offutils.py
+++ b/offutils.py
@@ -1,5 +1,4 @@
#!/bin/python
-
# This file contains some utilities common to offpunk, ansicat and netcache.
# Currently, there are the following utilities:
#
@@ -15,6 +14,7 @@ import urllib.parse
import gettext
import cert_migration
+import create_preconfig
import netcache
import netcache_migration
@@ -147,18 +147,43 @@ def xdg(folder="cache"):
print(_("No XDG folder for %s. Check your code.") % folder)
return None
+def add_config_file_options(parser):
+ parser.add_argument(
+ "--config-file",
+ metavar="FILE",
+ default=os.path.join(xdg("config"), "offpunkrc"),
+ help="".join((_("use this particular config file instead of default"),
+ "(%(default)s)"))
+ )
+ parser.add_argument(
+ "--pre-config-file",
+ metavar="FILE",
+ default=os.path.join(xdg("config"), "offpunkprerc"),
+ help=" ".join((
+ _("use this particular config file instead of default"),
+ "(%(default)s)",
+ _("— but this is a prior one executed additionally beforehand."),
+ _("This one gets created if it doesn't exist."),
+ ))
+ )
+
+def init_both_configs(argparse_args,
+ skip_go=False, interactive=True, verbose=True):
+ yield from init_config(argparse_args.pre_config_file,
+ skip_go, interactive, verbose,
+ create_if_not_exists=interactive)
+ yield from init_config(argparse_args.config_file,
+ skip_go, interactive, verbose)
#Return a list of the commands that must be run
#if skip_go = True, any command changing the url will be ignored (go, tour)
#if not interactive, only redirects and handlers are considered
-def init_config(rcfile=None,skip_go=False,interactive=True,verbose=True):
- cmds = []
- if not rcfile:
- rcfile = os.path.join(xdg("config"), "offpunkrc")
+def init_config(rcfile=None,
+ skip_go=False,interactive=True,verbose=True,create_if_not_exists=False):
if os.path.exists(rcfile):
if verbose:
print(_("Using config %s") % rcfile)
- with open(rcfile,"r") as fp:
+ with open(rcfile,"r",encoding="utf-8") as fp:
for line in fp:
line = line.strip()
#Is this a command to go to an url ?
@@ -166,15 +191,19 @@ def init_config(rcfile=None,skip_go=False,interactive=True,verbose=True):
#Is this a command necessary, even when non-interactive ?
is_necessary = any(line.startswith(x) for x in ("redirect","handler","set"))
if is_necessary:
- cmds.append(line)
+ yield line
elif interactive:
if skip_go and is_go:
if verbose:
print(_("Skipping startup command \"%s\" due to provided URL")%line)
continue
else:
- cmds.append(line)
- return cmds
+ yield line
+ elif create_if_not_exists:
+ print(_("Initializing %s with preconfigured rc defaults") % rcfile)
+ create_preconfig.preconfig_creation(rcfile)
+ yield from init_config(rcfile, skip_go, interactive, verbose)
+
# An IPV6 URL should be put between []
# We try to detect them has location with more than 2 ":"
@@ -252,7 +281,7 @@ def unmode_url(url):
#This function gives the root of an URL
# expect if the url contains /user/ or ~username/
#in that case, it considers it as a muli-user servers
-# it returns the root URL
+# it returns the root URL
# except if "return_value=name" then it return a name for that root
# which is hostname by default or username if applicable
# if absolute is set, it doesn’t care about users
@@ -421,4 +450,4 @@ def looks_like_base64(src, baseurl):
imgurl = None
else:
imgurl = urllib.parse.urljoin(baseurl, imgname)
- return imgurl, imgdata
+ return imgurl, imgdata
\ No newline at end of file
diff --git a/opnk.py b/opnk.py
index 5d315d1..0b4d2d4 100755
--- a/opnk.py
+++ b/opnk.py
@@ -16,17 +16,18 @@ import ansicat
import netcache
import offutils
from offutils import (
- GREPCMD,
- is_local,
- mode_url,
- run,
- term_width,
- unmode_url,
- init_config,
- send_email,
- _HAS_XDGOPEN,
- _LOCALE_DIR,
- )
+ GREPCMD,
+ is_local,
+ mode_url,
+ run,
+ term_width,
+ unmode_url,
+ send_email,
+ _HAS_XDGOPEN,
+ _LOCALE_DIR,
+ add_config_file_options,
+)
+from implementation_parts.redirections_stuff import redirections_engine, redirects_cli
gettext.bindtextdomain('offpunk', _LOCALE_DIR)
gettext.textdomain('offpunk')
@@ -111,6 +112,7 @@ class opencache:
self.mime_handlers = {}
self.last_mode = {}
self.last_width = term_width(absolute=True)
+ self.redirects = {}
def _get_handler_cmd(self, mimetype,file_extension=None):
# Now look for a handler for this mimetype
@@ -220,6 +222,10 @@ class opencache:
if not offutils.is_local(inpath):
if mode:
kwargs["images_mode"] = mode
+ blocked, inpath = redirections_engine(self.redirects, inpath)
+ if blocked:
+ print(blocked)
+ return False, inpath
cachepath, inpath = netcache.fetch(inpath, **kwargs)
if not cachepath:
return False, inpath
@@ -391,14 +397,22 @@ def main():
help=_("maximum age, in second, of the cached version before \
redownloading a new version"),
)
+ parser.add_argument(
+ "--rc-redirects",
+ action='store_true',
+ help=_("whether additionally respect the 'redirect' command"),
+ )
+ add_config_file_options(parser)
args = parser.parse_args()
cache = opencache()
#we read the startup config and we only care about the "handler" command
- cmds = init_config(skip_go=True,interactive=False,verbose=False)
+ cmds = offutils.init_both_configs(args, skip_go=True, interactive=False, verbose=False)
for cmd in cmds:
splitted = cmd.split(maxsplit=2)
if len(splitted) >= 3 and splitted[0] == "handler":
cache.set_handler(splitted[1],splitted[2])
+ elif args.rc_redirects and len(splitted) >= 2 and splitted[0] == "redirect":
+ redirects_cli(cache.redirects, " ".join(splitted[1:]), 0)
# if the second argument is an integer, we associate it with the previous url
# to use as a link_id
if len(args.content) == 2 and args.content[1].isdigit():
--
2.52.0
From 2a3fdb7f736f0c76bf3cd2468d18412379678192 Mon Sep 17 00:00:00 2001
Date: Sun, 25 Jan 2026 01:45:58 +0100
Subject: [PATCH 2/2] Preconfig creation opt-out
---
create_preconfig.py | 5 ++++-
offutils.py | 29 ++++++++++++++++++++++++-----
2 files changed, 28 insertions(+), 6 deletions(-)
diff --git a/create_preconfig.py b/create_preconfig.py
index 8af6643..3b18222 100644
--- a/create_preconfig.py
+++ b/create_preconfig.py
@@ -37,6 +37,9 @@ def yield_preconfig_content():
yield from (f"redirect {item} -silent BLOCK" + os.linesep for item in blocked)
def preconfig_creation(filepath,
- preconfig_lines=yield_preconfig_content()):
+ preconfig_lines=yield_preconfig_content(),
+ optout_from_preconfig=False):
with open(filepath, "x", encoding="utf-8") as fp:
+ if optout_from_preconfig:
+ fp.write("set dont_create_preconfig" + os.linesep)
fp.writelines(preconfig_lines)
\ No newline at end of file
diff --git a/offutils.py b/offutils.py
index 2ba32a6..fc3851c 100644
--- a/offutils.py
+++ b/offutils.py
@@ -164,22 +164,41 @@ def add_config_file_options(parser):
"(%(default)s)",
_("— but this is a prior one executed additionally beforehand."),
_("This one gets created if it doesn't exist."),
+ _("Unless your config starts with 'set dont_create_preconfig'."),
+ _("If you have this file, usually it is expected to contain upstream defaults."),
+ _("Future releases may include new rc defaults."),
+ _("If you move all the current preconfigured defaults to your main rc and delete prerc,"),
+ _("a future release could create a new one for you and ask you again to add a subsequent"),
+ 'set dont_create_preconfig 1', _("to your main rc."),
+ _("New users get the most recent opt-out by default as their rc contain newest defaults.")
))
)
def init_both_configs(argparse_args,
skip_go=False, interactive=True, verbose=True):
+ existed = os.path.exists(argparse_args.config_file)
+ config = init_config(argparse_args.config_file,
+ skip_go, interactive, verbose,
+ create_if_not_exists=(interactive and not existed),
+ optout_from_preconfig=True)
+ first_from_config = next(config, None)
+ user_excluded_from_preconfig = (first_from_config and
+ first_from_config.strip() == "set dont_create_preconfig")
+ if not user_excluded_from_preconfig: print(first_from_config.strip())
yield from init_config(argparse_args.pre_config_file,
skip_go, interactive, verbose,
- create_if_not_exists=interactive)
- yield from init_config(argparse_args.config_file,
- skip_go, interactive, verbose)
+ create_if_not_exists=(interactive and not user_excluded_from_preconfig))
+ if first_from_config:
+ if not user_excluded_from_preconfig: yield first_from_config
+ yield from config
#Return a list of the commands that must be run
#if skip_go = True, any command changing the url will be ignored (go, tour)
#if not interactive, only redirects and handlers are considered
def init_config(rcfile=None,
- skip_go=False,interactive=True,verbose=True,create_if_not_exists=False):
+ skip_go=False,interactive=True,verbose=True,
+ create_if_not_exists=False,
+ optout_from_preconfig=False):
if os.path.exists(rcfile):
if verbose:
print(_("Using config %s") % rcfile)
@@ -201,7 +220,7 @@ def init_config(rcfile=None,
yield line
elif create_if_not_exists:
print(_("Initializing %s with preconfigured rc defaults") % rcfile)
- create_preconfig.preconfig_creation(rcfile)
+ create_preconfig.preconfig_creation(rcfile, optout_from_preconfig=optout_from_preconfig)
yield from init_config(rcfile, skip_go, interactive, verbose)
--
2.52.0
Comments on “Experimentally expanding Offpunk browser Part 1 (nightly)”