Take redirections into account in UrlChecker

[danicheg]

This PR aims to increase the likelihood of URLs (such as those related to release notes) being propagated into PR descriptions. In #3583, I noticed that we omit URLs in the description of the final PR when facing redirections on GitHub, GitLab, etc. While it may seem that the current PR supersedes the mentioned one, I’m confident it does not — and here’s why.

❯ curl -I http://github.com/lettuce-io/lettuce-core/releases/tag/6.5.3.RELEASE
HTTP/1.1 301 Moved Permanently
Content-Length: 0
Location: https://github.com/lettuce-io/lettuce-core/releases/tag/6.5.3.RELEASE

❯ curl -I https://github.com/lettuce-io/lettuce-core/releases/tag/6.5.3.RELEASE
HTTP/2 301
server: GitHub.com
location: https://github.com/redis/lettuce/releases/tag/6.5.3.RELEASE

In the first request, we simply received a tweak to the URL’s Scheme (http->https). In the second request, however, we encountered an actual redirection to the existing resource. FWIW, an attentive reader may notice that these two redirections are handled by different middleware (HTTP/1.1 vs HTTP/2).

As always, I'm open to any concerns or suggestions to help drive this to merge.

[codecov]

Codecov Report

❌ Patch coverage is 91.22807% with 5 lines in your changes missing coverage. Please review.
✅ Project coverage is 89.82%. Comparing base (1e8e1ea) to head (56b8ad9).
⚠️ Report is 614 commits behind head on main.

Files with missing lines	Patch %	Lines
.../org/scalasteward/core/nurture/UpdateInfoUrl.scala	0.00%	4 Missing ⚠️
...calasteward/core/nurture/UpdateInfoUrlFinder.scala	96.87%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #3586      +/-   ##
==========================================
+ Coverage   89.80%   89.82%   +0.01%     
==========================================
  Files         174      174              
  Lines        5031     5090      +59     
  Branches      445      445              
==========================================
+ Hits         4518     4572      +54     
- Misses        513      518       +5     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[fthomas]

While it may seem that the current PR supersedes the mentioned one, I’m confident it does not — and here’s why.

How about allowing two recursive calls instead of just one? It should work for the lettuce case then, right?

[danicheg]

How about allowing two recursive calls instead of just one? It should work for the lettuce case then, right?

I think so. But when it comes to recursion, it may promptly become three calls, and so on. IMHO, it is beneficial to sort out these scheme-related things when preparing metadata from Coursier. This might be helpful in other places besides UpdateInfoUrlFinder.

[fthomas]

But when it comes to recursion, it may promptly become three calls, and so on.

Three calls would be okay for me, too. The benefit of allowing two calls is that it "fixes" the lettuce case without modifying the scheme.

IMHO, it is beneficial to sort out these scheme-related things when preparing metadata from Coursier. This might be helpful in other places besides UpdateInfoUrlFinder.

I don't see how it would be beneficial. From my perspective merging this PR with two recursive calls would be better than merging this PR as-is and #3583 because it results in less LOC that needs to be maintained.

[danicheg]

@fthomas ah, unfortunately, simply doing two recursive calls doesn't help:

❯ curl -I http://github.com/lettuce-io/lettuce-core/tag/6.5.4.RELEASE
HTTP/1.1 301 Moved Permanently
Location: https://github.com/lettuce-io/lettuce-core/tag/6.5.4.RELEASE

❯ curl -I https://github.com/lettuce-io/lettuce-core/tag/6.5.4.RELEASE
HTTP/2 404

So, if you want to keep all the logic in the org.scalasteward.core.util.UrlChecker boundaries, we should do that schema replacement there. But I'm unsure if that abstraction would still be UrlChecker. Alright, we can bypass this by first obtaining the stable URL of repoUrl and then checking the URL for the release notes. But again, this makes it more complicated.

[danicheg]

@fthomas Alright, I applied your suggestion. Now we can abandon #3583. I'm unsure if the LOC appeared lower than it initially was though.

[danicheg]

@fthomas What do you think about this initiative? I really want to improve the state of missing links to release notes for a number of dependencies I rely on across various projects. I'm open to completely reworking the PR in whatever way you and other maintainers would find suitable.