What's New Pussycat?
Charly's Column – urlwatch
Experienced system administrators attach great importance to always being up to date when it comes to information technology. Urlwatch is a command-line tool that presents the latest news from websites that do not offer RSS feeds by email.
Some years ago, I reported on the Miniflux RSS feed aggregator [1] in this column, and I still use it. Miniflux is lean, fast, and easy to use. The media have been predicting the death of RSS feeds for what feels like an eternity, but it still has not happened. However, some websites simply don't offer feeds. I have to do something different here. What I need is a tool that alerts me when a particular website changes.
That's far more complicated than it sounds at first. Using a web service for this purpose (there are countless numbers of them) is out of the question for reasons of data economy. However, the biggest problem is something else. Things on websites that I don't find relevant are constantly changing. For example, a daily newspaper that I regularly read displays job ads that change with every reload. If I was notified every time, it would drive me crazy. There has to be a better way – and there is: urlwatch.
Written in Python 3, urlwatch is included in most popular distributions. If you want to know exactly if and which version of urlwatch is available in your favorite distro, you can find out on Repology.org [2]; DIY enthusiasts can use GitHub [3]. For example, I instructed urlwatch to keep an eye on the online news page for Linux Magazine (Listing 1) – pointlessly, because it actually still offers RSS.
Listing 1
Monitoring Websites
01 $ urlwatch --add url=http://www.linux-magazine.com/Online/News,name=LinMag
In my home directory below .config/urlwatch/
, urlwatch has now created a configuration file named urls.yaml
(Listing 2). So far, so good – but I want to be sure that I am only notified when a new news ticker post is published. I want urlwatch to ignore any other changes to the website.
Listing 2
urls.yaml
01 kind: url 02 name: LinMag 03 url: http://www.linux-magazine.com/Online/News
To do this, I briefly immersed myself in the website's HTML source code and discovered that every news ticker entry is introduced by a div
tag that contains the specification "td_module_9 td_module_wrap"
as its class
specification. This prompted me to append the following line to the configuration in urls.yaml
:
filter: element-by-id:td_module_9
Now urlwatch only alerts me when a new ticker entry appears on the website. There is a rich selection of alerting types; I opted for plain old email. All I have to do is enter the appropriate mail server, as well as the sender and recipient addresses, below .config/urlwatch/
in the urlwatch.yaml
configuration file's report
section. And, lo and behold, the electronic mailman was soon ringing the bell (Figure 1).
Infos
- "The Sys Admin's Daily Grind: Miniflux" by Charly Kühnast, Linux Magazine, issue 164, July 2014, http://www.linux-magazine.com/Issues/2014/164/Charly-s-Column-Miniflux/(language)/eng-US
- urlwatch packages: https://repology.org/project/urlwatch/versions
- urlwatch on GitHub: https://github.com/thp/urlwatch
Buy this article as PDF
(incl. VAT)
Buy Linux Magazine
Subscribe to our Linux Newsletters
Find Linux and Open Source Jobs
Subscribe to our ADMIN Newsletters
Support Our Work
Linux Magazine content is made possible with support from readers like you. Please consider contributing when you’ve found an article to be beneficial.
News
-
Plasma 6.3 Ready for Public Beta Testing
Plasma 6.3 will ship with KDE Gear 24.12.1 and KDE Frameworks 6.10, along with some new and exciting features.
-
Budgie 10.10 Scheduled for Q1 2025 with a Surprising Desktop Update
If Budgie is your desktop environment of choice, 2025 is going to be a great year for you.
-
Firefox 134 Offers Improvements for Linux Version
Fans of Linux and Firefox rejoice, as there's a new version available that includes some handy updates.
-
Serpent OS Arrives with a New Alpha Release
After months of silence, Ikey Doherty has released a new alpha for his Serpent OS.
-
HashiCorp Cofounder Unveils Ghostty, a Linux Terminal App
Ghostty is a new Linux terminal app that's fast, feature-rich, and offers a platform-native GUI while remaining cross-platform.
-
Fedora Asahi Remix 41 Available for Apple Silicon
If you have an Apple Silicon Mac and you're hoping to install Fedora, you're in luck because the latest release supports the M1 and M2 chips.
-
Systemd Fixes Bug While Facing New Challenger in GNU Shepherd
The systemd developers have fixed a really nasty bug amid the release of the new GNU Shepherd init system.
-
AlmaLinux 10.0 Beta Released
The AlmaLinux OS Foundation has announced the availability of AlmaLinux 10.0 Beta ("Purple Lion") for all supported devices with significant changes.
-
Gnome 47.2 Now Available
Gnome 47.2 is now available for general use but don't expect much in the way of newness, as this is all about improvements and bug fixes.
-
Latest Cinnamon Desktop Releases with a Bold New Look
Just in time for the holidays, the developer of the Cinnamon desktop has shipped a new release to help spice up your eggnog with new features and a new look.