As it turned out the next afternoon, my code was buggy.
This is a sequel to the post Experimentally expanding Offpunk browser Part 1 (nightly) from yesterday.
The blocked domains were saving as redirection with destination URL interpreted as //BLOCK/, deleting redirections was broken because of the same CLI parsing issue, and it was still only possible to have one rule for one domain glob, no matter how specific through the criterions.
The only reason why -silent was working for them was because of this.
I am now hashing the entire criterion into the dict, and for that I had to add frozendict dependency to the project because good to write words sometimes, because I just turned that into a tuple.
More bugs found: especially that I didn’t really apply the specificity of the rules to BLOCK rules, and they were always acting on the netloc regardless. Now lemme just drop you the patches right below. First is a tiny patch for more predictable behavior of fragment templating, then the big one.
Update: I posted another bugfix post to this draft implementation of mine, see Bugfix for list URI for my Offpunk redirections implementation draft
Note: this post publishes the patches, but doesn’t diff any code nicely. These patch files are not exactly made for directly reading except for the commit message of the longer one, which I will maybe paste here too:
Big amends to my redirects implementation
Fixed BLOCK/NONE redirs saving as destination. Fixed -silent for them.
From my previous commit, BLOCK and NONE redirections were saving with incorrect “BLOCK” or “NONE” destination treated as a URL. Fixed.
No flags (-silent, -!www, -noport) were enacted for them. Fixed.
There was a limit of one rule per origin netloc — now the whole criterion is hashed in the dict.
The original line under which the rule was created is no longer preserved — it is now recreated from the actual objects.
The internal representation of block is now uppercase BLOCK — that is due to my repeated confusion when it was otherwise.
The redirect command with one argument will now print all the rules for that origin netloc — a consequence of new hashing.
Rearranged the function applying redirection rules — until now the BLOCK rules weren’t actually respecting the precise toggles, just the netloc.
Also the fragment rules were accidentally matching the path rules.
Also the silent redirect counter will again not count the www double.
Also there is a bunch of type hints now that help a lot.
From 86afe6bdca6d9352d282e4005084106d8c31932c Mon Sep 17 00:00:00 2001
Date: Sun, 25 Jan 2026 19:55:45 +0100
Subject: [PATCH 1/2] Redirections templating: omit the # for the {fragment}.
The fragment is no longer passed to the templating with the #.
---
implementation_parts/redirections_stuff.py | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/implementation_parts/redirections_stuff.py b/implementation_parts/redirections_stuff.py
index bd9bd6e..8e2021a 100644
--- a/implementation_parts/redirections_stuff.py
+++ b/implementation_parts/redirections_stuff.py
@@ -150,7 +150,7 @@ but '{item}' found!
pathpattern.append((not unless, x))
case item if item.startswith("-#"):
assert not parampattern_being_added, message_in_such_scenario
- x = item[1:]
+ x = item[2:]
assert not (not unless, x) in pathpattern, \
f"Two identical {item} fragment pattern options!"
assert not (unless, x) in pathpattern, \
--
2.52.0
From 501f6f5ac8c847f70678b31c722569b3d2bd45ef Mon Sep 17 00:00:00 2001
Date: Sun, 25 Jan 2026 19:54:16 +0100
Subject: [PATCH 2/2] Big amends to my redirects implementation
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
Fixed BLOCK/NONE redirs saving as destination. Fixed -silent for them.
From my previous commit, BLOCK and NONE redirections were saving with
incorrect "BLOCK" or "NONE" destination treated as a URL. Fixed.
No flags (-silent, -!www, -noport) were enacted for them. Fixed.
There was a limit of one rule per origin netloc — now the whole
criterion is hashed in the dict.
The original line under which the rule was created is no longer
preserved — it is now recreated from the actual objects.
The internal representation of block is now uppercase BLOCK
— that is due to my repeated confusion when it was otherwise.
The redirect command with one argument will now print all the rules
for that origin netloc — a consequence of new hashing.
Rearranged the function applying redirection rules — until now the
BLOCK rules weren't actually respecting the precise toggles, just
the netloc.
Also the fragment rules were accidentally matching the path rules.
Also the silent redirect counter will again not count the www double.
Also there is a bunch of type hints now that help a lot.
---
implementation_parts/redirection_classes.py | 70 +++++
implementation_parts/redirections_stuff.py | 329 ++++++++++++--------
2 files changed, 276 insertions(+), 123 deletions(-)
create mode 100644 implementation_parts/redirection_classes.py
diff --git a/implementation_parts/redirection_classes.py b/implementation_parts/redirection_classes.py
new file mode 100644
index 0000000..5cba31d
--- /dev/null
+++ b/implementation_parts/redirection_classes.py
@@ -0,0 +1,70 @@
+from typing import NamedTuple, Literal
+
+class RedirectionRewrite(NamedTuple):
+ locator: str
+ purge_query: any
+
+ def recover_line(self) -> str:
+ result = []
+ if self.locator:
+ result.append(self.locator)
+ result.extend(f"-^?{q}" for q in self.purge_query)
+ return " ".join(result)
+
+RedirectionDestination = Literal["BLOCK"] | RedirectionRewrite
+
+TruthinessGlobPair = tuple[bool, str]
+FrozenTruthinessGlobPairs = tuple[TruthinessGlobPair, ...]
+FrozenParamGlobDictEntry = tuple[str, FrozenTruthinessGlobPairs]
+
+class RedirectionOrigin(NamedTuple):
+ netloc: str
+ noport_toggle: bool
+ pathpattern: FrozenTruthinessGlobPairs
+ fragpattern: FrozenTruthinessGlobPairs
+ parampatterns: tuple[FrozenParamGlobDictEntry, ...]
+
+ def recover_line(self):
+ result = [self.netloc]
+ if self.noport_toggle:
+ result.append("-noport")
+
+ def before_after_unless(should):
+ yield from (f"-{path}" for true, path in self.pathpattern if true == should)
+ yield from (f"-#{fragment}" for true, fragment in self.fragpattern if true == should)
+ for param_glob, per_param_patterns in self.parampatterns:
+ for name, values in per_param_patterns:
+ for (true, value) in values:
+ if true == should:
+ yield from (f"-?{name}", value)
+
+ result.extend(before_after_unless(True))
+ result.append("unless")
+ result.extend(before_after_unless(False))
+ if result[-1] == "unless": result.pop()
+ return " ".join(result)
+
+class RedirectionTerms(NamedTuple):
+ www_toggle: bool
+ silent_toggle: bool
+ noport_toggle: bool
+ purge_query: set[str]
+ pathpattern: list[tuple[bool, str]]
+ fragpattern: list[tuple[bool, str]]
+ parampatterns: dict[str, list[tuple[bool, str]]]
+ destination: str | None
+
+ def origin(self, netloc: str) -> RedirectionOrigin:
+ return RedirectionOrigin(netloc=netloc,
+ noport_toggle=self.noport_toggle,
+ pathpattern=tuple(self.pathpattern),
+ fragpattern=tuple(self.fragpattern),
+ parampatterns=tuple(
+ (k, tuple(v)) for k, v in self.parampatterns.items()),
+ )
+
+ def dest(self) -> RedirectionDestination:
+ if self.destination == "BLOCK":
+ return "BLOCK"
+ return RedirectionRewrite(locator=self.destination,
+ purge_query=self.purge_query)
\ No newline at end of file
diff --git a/implementation_parts/redirections_stuff.py b/implementation_parts/redirections_stuff.py
index 8e2021a..9a75c93 100644
--- a/implementation_parts/redirections_stuff.py
+++ b/implementation_parts/redirections_stuff.py
@@ -1,103 +1,143 @@
-import fnmatch
+from fnmatch import fnmatch
+from os import linesep
import re
import urllib.parse
+from urllib.parse import ParseResult
from collections import defaultdict
-from types import SimpleNamespace
+from implementation_parts.redirection_classes import RedirectionOrigin, RedirectionRewrite, RedirectionTerms, \
+ FrozenParamGlobDictEntry, RedirectionDestination
from offutils import _
netloc_port_regex = re.compile(r'^.+(?P<colonport>:\d{1,5})$')
-def redirections_engine(redirects, url):
+def remove_port_if_criterion_port_inexplicit(criterion, netloc):
+ if not criterion.noport_toggle and not netloc_port_regex.match(criterion.netloc):
+ netloc_port_match = netloc_port_regex.match(netloc)
+ if netloc_port_match:
+ colon_port = netloc_port_match["colonport"]
+ netloc = netloc[:-len(colon_port)]
+ return netloc
+
+def redirections_engine(redirects: dict[RedirectionOrigin, RedirectionDestination],
+ url: str):
blocked = None
- parsed = urllib.parse.urlparse(url)
+ parsed: ParseResult = urllib.parse.urlparse(url)
netloc = parsed.netloc
- # we block/redirect even subdomains
- for key, rule in redirects.items():
- if not rule.noport_toggle:
- netloc_port_match = netloc_port_regex.match(netloc)
- if netloc_port_match and not netloc_port_regex.match(key):
- colon_port = netloc_port_match["colonport"]
- netloc = netloc[:-len(colon_port)]
- if fnmatch.fnmatch(netloc, key):
- if rule == "blocked":
- blocked = ""
- blocked += _("Blocked URL: ") + url + "\n"
- blocked += _("This website has been blocked with the following rule:\n")
- blocked += key + "\n"
- blocked += _("Use the following redirect command to unblock it:\n")
- blocked += "redirect %s NONE" % key
- break
- else:
- parsed, url = process_redirection_rule(parsed, rule)
+ for criterion, rewrite in redirects.items():
+ applicable = fnmatch(
+ remove_port_if_criterion_port_inexplicit(criterion, netloc),
+ criterion.netloc)
+ if not applicable: continue
+ further_applicable, query = process_redirection_applicability(rewrite, criterion, parsed)
+ # query parameter purge is now enacted in `query` variable
+ applicable &= further_applicable
+ if not applicable: continue
+ blocked = is_redirection_blocking(rewrite, criterion, url)
+ if blocked: break
+ rewrite: RedirectionRewrite
+ parsed, url = process_redirection_destination(rewrite, parsed, query)
return blocked, url
schema_regex = re.compile(r'^[a-zA-Z][a-zA-Z0-9+.-]*?(?=://)')
port_templating = re.compile(r'.[{]port[}]')
-def process_redirection_rule(parsed, rule):
+def process_redirection_applicability(dest: RedirectionDestination,
+ orig: RedirectionOrigin,
+ parsed: ParseResult):
+ """
+ This method will not check the applicability on netloc.
+ This method only checks the -[/?#],
+ as well as purges the query parameters as per -^? specifiers.
+ :param dest: just for the purge of query parameters
+ :param orig: the criterions to consider, except netloc
+ :param parsed: the already parsed url
+ :return:
+ """
applicable = True
- for (true, glob) in rule.pathpattern:
- applicable |= true == fnmatch.fnmatch(parsed.path, glob)
+ for (true, glob) in orig.pathpattern:
+ applicable |= true == fnmatch(parsed.path, glob)
if not applicable: break
- for (true, glob) in rule.fragpattern:
- applicable |= true == fnmatch.fnmatch(parsed.path, glob)
+ for (true, glob) in orig.fragpattern:
+ applicable |= true == fnmatch(parsed.fragment, glob)
if not applicable: break
truequeryglob_hitboard = {nameglob: {
valueglob for (true, valueglob) in valueglobs if true
- } for nameglob, valueglobs in rule.parampatterns.items()}
+ } for nameglob, valueglobs in orig.parampatterns}
query = urllib.parse.parse_qs(parsed.query)
for actual_param_name, values in query.items():
- for nameglob, (true, valueglob) in rule.parampatterns.items():
- if not fnmatch.fnmatch(actual_param_name, nameglob): continue
- result = any(fnmatch.fnmatch(value, valueglob) for value in values)
+ for item in orig.parampatterns:
+ nameglob: str
+ globdictentry: FrozenParamGlobDictEntry
+ nameglob, globdictentry = item
+ true: bool
+ valueglob: str
+ true, valueglob = globdictentry
+ if not fnmatch(actual_param_name, nameglob): continue
+ result = any(fnmatch(value, valueglob) for value in values)
applicable |= true == result
if not applicable: break
if true and result:
truequeryglob_hitboard[nameglob].discard(valueglob)
if not applicable: break
- for purge_glob in rule.purge_query:
- if fnmatch.fnmatch(actual_param_name, purge_glob):
- query.pop(purge_glob)
+ if isinstance(dest, RedirectionRewrite):
+ for purge_glob in dest.purge_query:
+ if fnmatch(actual_param_name, purge_glob):
+ query.pop(purge_glob)
applicable &= not any(True for valueglobs_hitboard
in truequeryglob_hitboard
if valueglobs_hitboard)
- if applicable:
- querystring = urllib.parse.urlencode(query, doseq=True)
- if rule.destination and '{' in str(rule.destination):
- templating = dict(
- host = parsed.hostname,
- net = parsed.netloc,
- path = parsed.path.removeprefix('/'),
- scheme = parsed.scheme,
- query = querystring,
- fragment = parsed.fragment,
- )
- port = parsed.port
- destination = rule.destination
- if port:
- templating.update(port=port)
- else:
- destination = port_templating.sub('', destination)
+ return applicable, query
- url = destination.format(templating)
- parsed = urllib.parse.urlparse(url, scheme=parsed.scheme)
+def is_redirection_blocking(dest: RedirectionDestination,
+ criterion: RedirectionOrigin,
+ url: str):
+ blocked = None
+ if dest == "BLOCK":
+ blocked = ""
+ blocked += _("Blocked URL: ") + url + "\n"
+ blocked += _("This website has been blocked with the following rule:\n")
+ blocked += criterion.recover_line() + "\tBLOCK\n"
+ blocked += _("Use the following redirect command to unblock it:\n")
+ blocked += "redirect %s NONE" % criterion.recover_line()
+ return blocked
+
+def process_redirection_destination(dest, parsed, query):
+ querystring = urllib.parse.urlencode(query, doseq=True)
+ if dest.locator and '{' in str(dest.locator):
+ templating = dict(
+ host=parsed.hostname,
+ net=parsed.netloc,
+ path=parsed.path.removeprefix('/'),
+ scheme=parsed.scheme,
+ query=querystring,
+ fragment=parsed.fragment,
+ )
+ port = parsed.port
+ destination = dest.locator
+ if port:
+ templating.update(port=str(port))
else:
- r = dict(query=querystring)
- if rule.destination:
- destination_has_schema = schema_regex.match(rule.destination)
- if not destination_has_schema:
- r.update(netloc=rule.destination)
- else:
- schema = destination_has_schema[0]
- r.update(schema=schema)
- with_slashslash = len(schema) + 3
- if len(rule.destination) > with_slashslash:
- r.update(netloc=rule.destination[with_slashslash:])
- parsed = parsed._replace(**r)
- url = urllib.parse.urlunparse(parsed)
+ destination = port_templating.sub('', destination)
+
+ url = destination.format(templating)
+ parsed = urllib.parse.urlparse(url, scheme=parsed.scheme)
+ else:
+ r = dict(query=querystring)
+ if dest.locator:
+ destination_has_schema = schema_regex.match(dest.locator)
+ if not destination_has_schema:
+ r.update(netloc=dest.locator)
+ else:
+ schema = destination_has_schema[0]
+ r.update(schema=schema)
+ with_slashslash = len(schema) + 3
+ if len(dest.locator) > with_slashslash:
+ r.update(netloc=dest.locator[with_slashslash:])
+ parsed = parsed._replace(**r)
+ url = urllib.parse.urlunparse(parsed)
return parsed, url
@@ -109,9 +149,9 @@ def redirection_parse_dest_and_conditions(line):
pathpattern = []
fragpattern = []
parampatterns = defaultdict(list)
- parampattern_being_added = None
+ parampattern_being_added : (str | None) = None
unless = False
- destination = None
+ destination : (str | None) = None
for item in line.split():
message_in_such_scenario = None
if parampattern_being_added:
@@ -171,8 +211,7 @@ but '{item}' found!
assert not destination, \
f"Can't have destination {item}, it's already {destination}!"
destination = item
- return SimpleNamespace(line=" ".join(item for item in line.split(" ") if item != "-silent"),
- www_toggle=www_toggle,
+ return RedirectionTerms(www_toggle=www_toggle,
silent_toggle=silent_toggle,
noport_toggle=noport_toggle,
purge_query=purge_query,
@@ -182,60 +221,104 @@ but '{item}' found!
destination=destination)
-def redirects_cli(redirects, line, silent_redirects_counter):
+def redirects_add(redirects, orig_netloc: str, terms: RedirectionTerms):
+ qualifier = terms.origin(orig_netloc)
+ destination = terms.dest()
+ qualifier_line = qualifier.recover_line()
+ if isinstance(destination, str):
+ match destination.upper():
+ case 'NONE':
+ if qualifier in redirects:
+ redirects.pop(qualifier)
+ return _("Redirection for %s has been removed") % qualifier_line
+ else:
+ return _("%s was not redirected. Nothing has changed.") % qualifier_line
+ case 'BLOCK':
+ redirects[qualifier] = "BLOCK"
+ return _("%s will now be blocked") % qualifier_line
+ case _:
+ assert False, f"Unexpected string leaked through implementation: '{destination}'."
+ else:
+ redirects[qualifier] = destination
+ return _("%s will now be redirected to %s") % (
+ qualifier_line,
+ destination.recover_line()
+ )
+
+def redirects_for_netloc(redirects: dict[RedirectionOrigin, RedirectionDestination],
+ origin: str):
+ for orig, dest in redirects.items():
+ if orig.netloc == origin:
+ yield orig, dest
+
+def redirects_cli(redirects: dict[RedirectionOrigin, RedirectionDestination],
+ line: str, silent_redirects_counter: int):
if len(line.split()) == 1:
- if line in redirects:
- successprint = _("%s is redirected to %s") % (line, redirects[line])
- else:
- successprint = _("Please add a destination to redirect %s") % line
+ successprint = print_redirections_for_netloc_glob(line, redirects)
elif len(line.split()) >= 2:
- while True:
- orig, dest = line.split(" ", 1)
- match orig:
- case "-silent", "-!www":
- line = f"{dest} {orig}"
- case _:
- break
- if dest.lower() == "none":
- if orig in redirects:
- redirects.pop(orig)
- successprint = _("Redirection for %s has been removed") % orig
- else:
- successprint = _("%s was not redirected. Nothing has changed.") % orig
- elif dest.lower() == "block":
- redirects[orig] = "blocked"
- successprint = _("%s will now be blocked") % orig
+ silent_redirects_counter, successprint = \
+ redirections_adding(line, redirects, silent_redirects_counter)
+ else:
+ successprint = print_current_redirections(redirects)
+ return successprint, silent_redirects_counter
+
+
+def redirections_adding(line, redirects, silent_redirects_counter):
+ common_flags_set = {"-silent", "-!www"}
+ while True:
+ orig: str
+ dest: str
+ orig, dest = line.split(" ", 1)
+ if orig in common_flags_set:
+ line = f"{dest} {orig}" # Put the flag at the end
else:
- terms = redirection_parse_dest_and_conditions(dest)
- redirects[orig] = terms
- repeated_message = lambda vals: _("%s will now be redirected to %s") % vals
- successprint = repeated_message((orig, dest))
- if not terms.www_toggle and not orig.startswith("*"):
- orig = 'www.' + orig
- redirects[orig] = terms
- successprint = "\n".join((successprint,
- _("Because of no -!www toggle, also ") + repeated_message((orig, dest))))
- if terms.silent_toggle:
- successprint = ""
- silent_redirects_counter += 1
+ break
+ terms = redirection_parse_dest_and_conditions(dest)
+ successprint = redirects_add(redirects, orig, terms)
+ if not terms.www_toggle and not orig.startswith("*") \
+ and not orig.startswith("www."):
+ orig = 'www.' + orig
+ successprint = "\n".join((
+ successprint,
+ _("Because of no -!www toggle, also ") + \
+ redirects_add(redirects, orig, terms)))
+ if terms.silent_toggle:
+ successprint = ""
+ if terms.silent_toggle:
+ successprint = ""
+ silent_redirects_counter += 1
+ return silent_redirects_counter, successprint
+
+def print_redirections_for_netloc_glob(line, redirects):
+ if line in redirects:
+ successprint = linesep.join((
+ _("%s is redirected to %s") % tuple(x.recover_line() for x in tup) \
+ for tup in redirects_for_netloc(redirects, line)
+ ))
else:
- successprint = _("Current redirections:\n")
- successprint += "--------------------\n"
- for r in redirects:
- value = redirects[r]
- value = "blocked" if value == "blocked" else value.line
- successprint += "%s\t->\t%s\n" % (r, value)
- successprint += _('\nTo add new, use "redirect origine.com destination.org"')
- successprint += _('\nTo remove a redirect, use "redirect origine.com NONE"')
- successprint += (
- _('\nTo completely block a website, use "redirect origine.com BLOCK"')
- )
- successprint += _('\nTo block subdomains, prefix with *.: "redirect *.origine.com BLOCK"')
- successprint += _('\nRules duplicate to www. subdomain too by default, unless you add -!www')
- successprint += _('\nIf a port is not specified, it is not matched. Unless -noport')
- successprint += _('\nAdd -silent switch anywhere after the origin domain to add silently')
- successprint += _(
+ successprint = _("Please add a destination to redirect %s") % line
+ return successprint
+
+
+def print_current_redirections(redirects):
+ successprint = _("Current redirections:\n")
+ successprint += "--------------------\n"
+ orig: RedirectionOrigin
+ dest: RedirectionDestination
+ for orig, dest in redirects.items():
+ value = "blocked" if dest == "BLOCK" else dest.recover_line()
+ successprint += "%s\t->\t%s\n" % (orig.recover_line(), value)
+ successprint += _('\nTo add new, use "redirect origine.com destination.org"')
+ successprint += _('\nTo remove a redirect, use "redirect origine.com NONE"')
+ successprint += (
+ _('\nTo completely block a website, use "redirect origine.com BLOCK"')
+ )
+ successprint += _('\nTo block subdomains, prefix with *.: "redirect *.origine.com BLOCK"')
+ successprint += _('\nRules duplicate to www. subdomain too by default, unless you add -!www')
+ successprint += _('\nIf a port is not specified, it is not matched. Unless -noport')
+ successprint += _('\nAdd -silent switch anywhere after the origin domain to add silently')
+ successprint += _(
"""\n
After domain pattern, even if it's just '*', you can add
-^?name to remove params ?name= from the URL,
@@ -244,7 +327,7 @@ def redirects_cli(redirects, line, silent_redirects_counter):
-#fragment to apply the rule only if #fragment in URL
'unless' to invert the meaning of subsequent flags
""")
- successprint += _(
+ successprint += _(
"""\n
If a curly brace will be found in your redirect destination, templating
will be applied to it. That allows you to template in:
@@ -259,6 +342,6 @@ def redirects_cli(redirects, line, silent_redirects_counter):
For one domain the basic syntax also now allows that you use
redirect example.com gemini://example.com
"""
- )
- successprint += _("\n\nThis should have went to 'help redirect' instead haha")
- return successprint, silent_redirects_counter
+ )
+ successprint += _("\n\nThis should have went to 'help redirect' instead haha")
+ return successprint
--
2.52.0
Comments on “Amending my Offpunk redirection implementation”