HTTP uses GMT [1,2] rather than UTC as timezone for timestamps. However,
the Last-Modified header used UTC which confused at least wget.
Before, UTC was used:
$ wget --no-check-certificate -S --spider https://cb_pages_tests.localhost.mock.directory:4430/images/827679288a.jpg
...
Last-Modified: Sun, 11 Sep 2022 08:37:42 UTC
...
Last-modified header invalid -- time-stamp ignored.
...
After, GMT is used:
$ wget --no-check-certificate -S --spider https://cb_pages_tests.localhost.mock.directory:4430/images/827679288a.jpg
...
Last-Modified: Sun, 11 Sep 2022 08:37:42 GMT
...
(no last-modified-header-invalid warning)
[1]: https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Last-Modified
[2]: https://www.rfc-editor.org/rfc/rfc9110#name-date-time-formatsFixes#364
---
Whatt I noticed is that the If-Modified-Since header isn't accepted (neither with GMT nor with UTC):
```
$ wget --header "If-Modified-Since: Sun, 11 Sep 2022 08:37:42 GMT" --no-check-certificate -S --spider https://cb_pages_tests.localhost.mock.directory:4430/images/827679288a.jpg
Spider mode enabled. Check if remote file exists.
--2024-07-15 23:31:41-- https://cb_pages_tests.localhost.mock.directory:4430/images/827679288a.jpg
Resolving cb_pages_tests.localhost.mock.directory (cb_pages_tests.localhost.mock.directory)... 127.0.0.1
Connecting to cb_pages_tests.localhost.mock.directory (cb_pages_tests.localhost.mock.directory)|127.0.0.1|:4430... connected.
WARNING: The certificate of ‘cb_pages_tests.localhost.mock.directory’ is not trusted.
WARNING: The certificate of ‘cb_pages_tests.localhost.mock.directory’ doesn't have a known issuer.
HTTP request sent, awaiting response...
HTTP/1.1 200 OK
Allow: GET, HEAD, OPTIONS
Cache-Control: public, max-age=600
Content-Length: 124635
Content-Type: image/jpeg
Etag: "073af1960852e2a4ef446202c7974768b9881814"
Last-Modified: Sun, 11 Sep 2022 08:37:42 GMT
Referrer-Policy: strict-origin-when-cross-origin
Server: pages-server
Strict-Transport-Security: max-age=63072000; includeSubdomains; preload
Date: Mon, 15 Jul 2024 21:31:42 GMT
Length: 124635 (122K) [image/jpeg]
Remote file exists
```
I would have expected a 304 (Not Modified) rather than a 200 (OK). I assume this is simply not supported and on production 304 is returned by a caching proxy in front of pages-server.
Reviewed-on: https://codeberg.org/Codeberg/pages-server/pulls/365
Reviewed-by: crapStone <codeberg@crapstone.dev>
Co-authored-by: Peter Gerber <peter@arbitrary.ch>
Co-committed-by: Peter Gerber <peter@arbitrary.ch>
Remove leading slashes from captured portions of paths when
redirecting using splats. This makes a directive like
"/articles/* /posts/:splat 302" behave as described in
FEATURES.md, i.e. "/articles/foo" now redirects to
"/posts/foo" rather than to "/posts//foo". Fixes#269.
This also changes the behavior of a redirect like
"/articles/* /posts:splat 302". "/articles/foo" will now
redirect to "/postsfoo" rather than to "/posts/foo".
This change also fixes an issue where paths like
"/articles123" would be incorrectly matched by the above
patterns.
Reviewed-on: https://codeberg.org/Codeberg/pages-server/pulls/308
Reviewed-by: crapStone <codeberg@crapstone.dev>
Co-authored-by: Daniel Erat <dan@erat.org>
Co-committed-by: Daniel Erat <dan@erat.org>
Move repetitive code from Options.matchRedirects into a new
Redirect.rewriteURL method and add a new test file.
No functional changes are intended; this is in preparation
for a later change to address #269.
Reviewed-on: https://codeberg.org/Codeberg/pages-server/pulls/304
Reviewed-by: crapStone <codeberg@crapstone.dev>
Co-authored-by: Daniel Erat <dan@erat.org>
Co-committed-by: Daniel Erat <dan@erat.org>
Hello 👋
since it affected my deployment of the pages server I started to look into the problem of the blank pages and think I found a solution for it:
1. There is no check if the file response is empty, neither in cache retrieval nor in writing of a cache. Also the provided method for checking for empty responses had a bug.
2. I identified the redirect response to be the issue here. There is a cache write with the full cache key (e. g. rawContent/user/repo|branch|route/index.html) happening in the handling of the redirect response. But the written body here is empty. In the triggered request from the redirect response the server then finds a cache item to the key and serves the empty body. A quick fix is the check for empty file responses mentioned in 1.
3. The decision to redirect the user comes quite far down in the upstream function. Before that happens a lot of stuff that may not be important since after the redirect response comes a new request anyway. Also, I suspect that this causes the caching problem because there is a request to the forge server and its error handling with some recursions happening before. I propose to move two of the redirects before "Preparing"
4. The recursion in the upstream function makes it difficult to understand what is actually happening. I added some more logging to have an easier time with that.
5. I changed the default behaviour to append a trailing slash to the path to true. In my tested scenarios it happened anyway. This way there is no recursion happening before the redirect.
I am not developing in go frequently and rarely contribute to open source -> so feedback of all kind is appreciated
closes#164
Reviewed-on: https://codeberg.org/Codeberg/pages-server/pulls/292
Reviewed-by: 6543 <6543@obermui.de>
Reviewed-by: crapStone <codeberg@crapstone.dev>
Co-authored-by: Hoernschen <julian.hoernschemeyer@mailbox.org>
Co-committed-by: Hoernschen <julian.hoernschemeyer@mailbox.org>
- Currently if the canonical domain validations fails(either for
legitimate reasons or for bug reasons like the request to Gitea/Forgejo
failing) it will use main domain certificate, which in the case for
custom domains will warrant a security error as the certificate isn't
issued to the custom domain.
- This patch handles this situation more gracefully and instead only
disallow obtaining a certificate if the domain validation fails, so in
the case that a certificate still exists it can still be used even if
the canonical domain validation fails. There's a small side effect,
legitimate users that remove domains from `.domain` will still be able
to use the removed domain(as long as the DNS records exists) as long as
the certificate currently hold by pages-server isn't expired.
- Given the increased usage in custom domains that are resulting in
errors, I think it ways more than the side effect.
- In order to future-proof against future slowdowns of instances, add a retry mechanism to the domain validation function, such that it's more likely to succeed even if the instance is not responding.
- Refactor the code a bit and add some comments.
Co-authored-by: Gusted <postmaster@gusted.xyz>
Co-authored-by: 6543 <6543@obermui.de>
Reviewed-on: https://codeberg.org/Codeberg/pages-server/pulls/160
Reviewed-by: 6543 <6543@obermui.de>
Co-authored-by: Gusted <gusted@noreply.codeberg.org>
Co-committed-by: Gusted <gusted@noreply.codeberg.org>
- Currently any error generated by requesting the `.domains` file of a repository would be logged under the info log level, which isn't the correct log level when we exclude the not found error.
- Use warn log level if the error isn't the not found error.
Co-authored-by: Gusted <postmaster@gusted.xyz>
Reviewed-on: https://codeberg.org/Codeberg/pages-server/pulls/162
Reviewed-by: Otto <otto@codeberg.org>
we have big functions that handle all stuff ... we should split this into smaler chuncks so we could test them seperate and make clear cuts in what happens where
Reviewed-on: https://codeberg.org/Codeberg/pages-server/pulls/135
- Logs are currently indicating that it's returning `nil` in valid
scenarios, therefor this patch adds extra logging in this code to
better understand what it is doing in this function.
Co-authored-by: Gusted <williamzijl7@hotmail.com>
Reviewed-on: https://codeberg.org/Codeberg/pages-server/pulls/130
Reviewed-by: 6543 <6543@obermui.de>
Co-authored-by: Gusted <gusted@noreply.codeberg.org>
Co-committed-by: Gusted <gusted@noreply.codeberg.org>
- Actually log useful information at their respective log level.
- Add logs in hot-paths to be able to deep-dive and debug specific requests (see server/handler.go)
- Add more information to existing fields(e.g. the host that the user is visiting, this was noted by @fnetX).
Co-authored-by: Gusted <williamzijl7@hotmail.com>
Reviewed-on: https://codeberg.org/Codeberg/pages-server/pulls/116
Reviewed-by: 6543 <6543@noreply.codeberg.org>
Co-authored-by: Gusted <gusted@noreply.codeberg.org>
Co-committed-by: Gusted <gusted@noreply.codeberg.org>
solves #56.
- The expected filename is `404.html`, like GitHub Pages
- Each repo/branch can have one `404.html` file at it's root
- If a repo does not have a `pages` branch, the 404.html file from the `pages` repository is used
- You get status code 404 (unless you request /404.html which returns 200)
- The error page is cached
---
close#56
Reviewed-on: https://codeberg.org/Codeberg/pages-server/pulls/81
Reviewed-by: 6543 <6543@noreply.codeberg.org>
Co-authored-by: crystal <crystal@noreply.codeberg.org>
Co-committed-by: crystal <crystal@noreply.codeberg.org>
move forward with refactoring:
- initial implementation of a smal "gitea client for fasthttp"
- move constant into const.go
Co-authored-by: 6543 <6543@obermui.de>
Reviewed-on: https://codeberg.org/Codeberg/pages-server/pulls/34
Reviewed-by: Otto Richter <otto@codeberg.org>
