Posted
about 2 years
ago
Leonardo and I are thrilled to announce the first CRAN release of dtts. The dtts package builds on top of both our nanotime package and the well-loved and widely-used data.table package by Matt, Arun, Jan, and numerous collaborators.
In a very
... [More]
rough nutshell, you can think of dtts as combining both these potent ingredients to produce something not-entirely-unlike the venerable xts package by our friends Jeff and Josh—but using highest-precision nanosecond increments rather than not-quite-microseconds or dates.
The package is still somewhat rare and bare: it is mostly “just” alignment operators. But because of the power and genius of data.table not all that much more is needed because data.table gets us fifteen years of highly refined, tuned and tested code for data slicing, dicing, and aggregation. To which we now humbly add nanosecond-resolution indexing and alignment.
The package had been simmering for some time, and does of course take advantage of (a lot of) earlier work by Leonardo on his ztsdb project. We look forward to user feedback and suggestions at the GitHub repo.
If you like this or other open-source work I do, you can now sponsor me at GitHub.
This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.
[Less]
|
Posted
about 2 years
ago
Debian Reunion Hamburg 2022 from May 23 to 30
As last year there will be a Debian Reunion Hamburg 2022 event taking place at the same location as previous years, from May 23rd until the 30th.
This is just a preliminary announcement to get the
... [More]
word out, that this event will happen, so you can ponder attending. The wiki page has more information and some fine folks have even already registered!
A few things still need to be sorted out, eg a call for papers and a call for sponsors. If you want to help with that or have questions about the event, please reach out via #debconf-hamburg on irc.oftc.net or via the debconf-hamburg mailinglist.
I'm very much looking forward to meet some of you again soon and getting to know some others for the first time! Yay. It's been a long time... [Less]
|
Posted
about 2 years
ago
fs.com s5850 and s8050 series type switches have a secret mode which
lets you enter a regular shell from the switch cli, like so:
hostname # start shell
Password:
The command and password are not documented by the manufacturer,
i wondered wether
... [More]
if its possible to extract that password from
the firmware. After all: its my device, and i want to have access
to all the features!
Download the latest firmware image for those switch types and let binwalk do
its magic:
$ wget https://img-en.fs.com/file/user_manual/s5850-and-s8050-series-switches-fsos-v7-2-5r-software.zip
binwalk FSOS-S5850-v7.2.5.r.bin -e
This will extract an regular cpio archive, including the switch root FS:
$ file 2344D4
2344D4: ASCII cpio archive (SVR4 with no CRC)
$ cpio --no-absolute-filenames -idv < 2344D4
The extracted files include the passwd file with hashes:
cat etc/passwd
root:$1$ZrdxfwMZ$1xAj.S6emtA7gWD7iwmmm/:0:0:root:/root:/bin/sh
nms:$1$nUbsGtA7$5omXOHPNK.ZzNd5KeekUq/:0:0:root:/ftp:/bin/sh
Let john do its job:
$ wget https://github.com/brannondorsey/naive-hashcat/releases/download/data/rockyou.txt
$ sudo john etc/passwd --wordlist=rockyou.txt
(nms)
(root)
2g 0:00:04:03 100% 0.008220g/s 58931p/s 58935c/s 58935C/s nancy..!*!hahaha!*!
Thats it (wont reveal the password here, but well: its an easy one ;))
Now have fun poking around on your switches firmware:
hostname # start shell
Password:
[root@hostname /mnt/flash]$ ps axw
PID TTY STAT TIME COMMAND
1 ? Ss 0:29 init
2 ? S 0:06 [kthreadd]
[..]
[root@hostname /mnt/flash]$ uname -a
Linux hostname 2.6.35-svn37723 #1 Thu Aug 22 20:43:19 CST 2019 ppc unknow
even tho the good things wont work, but i guess its time to update the firmware
anyways:
[root@hostname /mnt/flash]$ tcpdump -pni vlan250
tcpdump: can't get TPACKET_V3 header len on packet socket: Invalid argument [Less]
|
Posted
about 2 years
ago
Sony RX100-III, relegated to a webcam
Sometimes I have remote meetings with Google Meet. Unlike the other
video-conferencing services that I use (Bluejeans, Zoom), my video
was stretched out of proportion under Google Meet with Firefox.
I
... [More]
haven't found out why this was happening, but I did figure out a
work-around.
Thanks to Daniel Silverstone, Rob Kendrick, Gregor Herrmann and Ben Allen for
pointing me in the right direction!
Hardware
The lovely Sony RX-100 mk3 that I bought in 2015 has
spent most of its life languishing unused. During the Pandemic, once
I was working from home all the time, I decided to press-gang it into
service as a better-quality webcam. Newer models of this camera —
the mark 4 onwards — have support for a USB mode called "PC Remote",
which effectively makes them into webcams. Unfortunately my mark 3
does not support this, but it does have HDMI out, so I picked up a
cheap "HDMI to USB Video Capture Card" from eBay.
Video modes
Before: wrong aspect ratio
This device offers a selection of different video modes over a webcam
interface. I used qv4l2 to
explore the different modes. It became clear that the camera was
outputting a signal at 16:9, but the modes on offer from the dongle
were for a range of different aspect ratios. The picture for these
other ratios was not letter or pillar-boxed, but stretched to fit.
I also noticed that the modes which had the correct aspect ratio were
at very low framerates: 1920x1080@5fps, 1360x768@8fps, 1280x720@10fps.
It felt to me that I would look unnatural at such a low framerate. The
most promising mode was close to the right ratio, 720x480 and 30 fps.
Software
After: corrected aspect ratio
My initial solution is to use the v4l2loopback
kernel module, which provides a virtual loop-back webcam interface. I
can write video data to it from one process, and read it back from
another. Loading it as follows:
modprobe v4l2loopback exclusive_caps=1
The option exclusive_caps configures the module into a mode where it
initially presents a write-only interface, but once a process has opened
a file handle, it then switches to read-only for subsequent processes.
Assuming there are no other camera devices connected at the time of
loading the module, it will create /dev/video0.1
I experimented briefly with OBS Studio, the
very versatile and feature-full streaming tool, which confirmed that I
could use filters on the source video to fix the aspect ratio, and emit
the result to the virtual device. I don't otherwise use OBS, though, so
I achieve the same result using ffmpeg:
fmpeg -s 720x480 -i /dev/video1 -r 30 -f v4l2 -vcodec rawvideo \
-pix_fmt yuyv422 -s 720x405 /dev/video0
The source options are to select the source video mode I want. The
codec and pixel formats are to match what is being emitted (I determined
that using ffprobe on the camera device). The resizing is triggered by
supplying a different size to the -s parameter. I think that is equivalent
to explicitly selecting a "scale" filter, and there might be other filters
that could be used instead (to add pillar boxes for example).
This worked just as well. In Google Meet, I select the Virtual Camera,
and Google Meet is presented with only one video mode, in the correct aspect
ratio, and no configurable options for it, so it can't misbehave.
Future
I'm planning to automate the loading (and unloading) of the module and starting
the ffmpeg process in response to the real camera device being plugged or
unplugged, using systemd events and services. (I don't leave the camera plugged
in all the time due to some bad USB behaviour I've experienced if I do so.)
If I get that working, I will write a follow-up.
you can request a specific device name/number with another module option.↩
[Less]
|
Posted
about 2 years
ago
A new major release of RcppRedis arrived on CRAN today. RcppRedis is one of several packages connecting R to the fabulous Redis in-memory datastructure store (and much more). RcppRedis does not pretend to be feature complete, but it may do some
... [More]
things faster than the other interfaces, and also offers an optional coupling with MessagePack binary (de)serialization via RcppMsgPack. The package has carried production loads for several years now.
This release integrates support for pub/sub, a popular messaging pattern in which one or more clients can subscribe to one or more ‘channels’. Whenever a client instances publishes, the Redis server immediately updates all clients listening on the given channel. This pattern is fairly common to transmit data to listeners. A there is a bit more to explain about this, I also added a brand-new vignette describing pub/sub with RcppRedis, along with another introductory vignette about Redis itself. We blogged about this exciting new feature and its particular use for market monitoring in R4 #36 recently too.
The pub/sub feature was available in package rredis by Bryan Lewis and has now been ported over by Bryan in a truly elegant yet compact implementation. We placed the code for the pub/sub examples, both for a single symbol (SP 500) as well as for a set of (futures) symbols, into a new examples/ subdirectory.
Other changes in this release are the removal of the build-dependency on Boost (or, rather, my BH package), an update to the included hiredis library (used if no system-wide version is found), and an updated to the UCRT build for R. That last one is a bit of a sore spot and nobody at CRAN deemed it necessary to tell me they were waiting for me to make this change; communication with the CRAN team can still be “challenging” (and I am being generous here). Anyway, the package is now on CRAN so all is well now, at long last.
The detailed changes list follows.
Changes in version 0.2.0 (2022-03-08)
Two simple C++11 features remove needs for BH and lexical_cast() (Dirk in #45 addressing #44).
Redis pub/sub is now supported (Dirk in #43, Bryan in #46).
Bryan Lewis is now a coauthor.
Added pub/sub examples for single and multiple Globex symbols.
The included hiredis sources have been updated to release 1.0.2.
Two vignettes have been added to introduce Redis and to described a live market-monitoring application included in directory pubsub/.
The UCRT build was updated per a suggestion by Tomas.
Courtesy of CRANberries, there is also a diffstat report for this release. More information is on the RcppRedis page.
If you like this or other open-source work I do, you can now sponsor me at GitHub.
This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.
[Less]
|
Posted
about 2 years
ago
After upgrading my MythTV machine to Debian
Bullseye and MythTV
31, my Streamzap remote
control stopped working correctly: the up and down buttons were
working, but the OK button wasn't.
Here's the complete solution that made it work with the
... [More]
built-in kernel
support (i.e. without LIRC).
Button re-mapping
Since some of the buttons were working, but not others, I figured that the
buttons were probably not mapped to the right keys.
Inspired by these old v4l-utils-based
instructions,
I made my own custom keymap by by copying the original keymap:
cp /lib/udev/rc_keymaps/streamzap.toml /etc/rc_keymaps/
and then modifying it to adapt it to what MythTV needs. This is what I ended
up with:
?protocols
name = "streamzap"
protocol = "rc-5-sz"
[protocols.scancodes]
0x28c0 = "KEY_0"
0x28c1 = "KEY_1"
0x28c2 = "KEY_2"
0x28c3 = "KEY_3"
0x28c4 = "KEY_4"
0x28c5 = "KEY_5"
0x28c6 = "KEY_6"
0x28c7 = "KEY_7"
0x28c8 = "KEY_8"
0x28c9 = "KEY_9"
0x28ca = "KEY_ESC"
0x28cb = "KEY_MUTE"
0x28cc = "KEY_UP"
0x28cd = "KEY_RIGHTBRACE"
0x28ce = "KEY_DOWN"
0x28cf = "KEY_LEFTBRACE"
0x28d0 = "KEY_UP"
0x28d1 = "KEY_LEFT"
0x28d2 = "KEY_ENTER"
0x28d3 = "KEY_RIGHT"
0x28d4 = "KEY_DOWN"
0x28d5 = "KEY_M"
0x28d6 = "KEY_ESC"
0x28d7 = "KEY_L"
0x28d8 = "KEY_P"
0x28d9 = "KEY_ESC"
0x28da = "KEY_BACK"
0x28db = "KEY_FORWARD"
0x28dc = "KEY_R"
0x28dd = "KEY_PAGEUP"
0x28de = "KEY_PAGEDOWN"
0x28e0 = "KEY_D"
0x28e1 = "KEY_I"
0x28e2 = "KEY_END"
0x28e3 = "KEY_A"
Note that the keycodes can be found in the kernel source
code.
With my own keymap in place at /etc/rc_keymaps/streamzap.toml, I changed
/etc/rc_maps.cfg to have the kernel driver automatically use it:
--- a/rc_maps.cfg
+++ b/rc_maps.cfg
@@ -126,7 +126,7 @@
* rc-real-audio-220-32-keys real_audio_220_32_keys.toml
* rc-reddo reddo.toml
* rc-snapstream-firefly snapstream_firefly.toml
-* rc-streamzap streamzap.toml
+* rc-streamzap /etc/rc_keymaps/streamzap.toml
* rc-su3000 su3000.toml
* rc-tango tango.toml
* rc-tanix-tx3mini tanix_tx3mini.toml
Button repeat delay
To adjust the delay before button presses are repeated, I followed these
old out-of-date
instructions
on the MythTV wiki and put the following in
/etc/udev/rules.d/streamzap.rules:
ACTION=="add", ATTRS{idVendor}=="0e9c", ATTRS{idProduct}=="0000", RUN+="/usr/bin/ir-keytable -s rc0 -D 1000 -P 250"
Note that the -d option has been replaced with -s in the latest version
of ir-keytable.
To check that the Streamzap is indeed detected as rc0 on your system, use
this command:
$ ir-keytable
Found /sys/class/rc/rc0/ with:
Name: Streamzap PC Remote Infrared Receiver (0e9c:0000)
Driver: streamzap
Default keymap: rc-streamzap
...
Make sure you don't pass the -c to ir-keytable or else it will clear the
keymap set via /etc/rc_maps.cfg, removing all of the button mappings. [Less]
|
Posted
about 2 years
ago
Wait! Just like yesterday when I was accepted as an Outreachy intern and the first half of the internship is finished😲. How time flies when you are having a good time🎃
As part of the requirements for the final application during the contribution
... [More]
period for the Outreachy internship, I needed to provide a timeline to achieve our goal on my outreachy task which is transitioning of dependencies in node16 and webpack5. Having consulted my mentors who implied that the packages depending on webpack and nodejs combined are so numerous that its impossible to finish all within a space of three months but we have steps to guide us through the entire process to achieve most of our goals which are ➡
Find a list of packages(failing rebuild and testing with autopkgtest, of reverse dependencies of webpack and nodejs) to fix.
See if new upstream versions are available that support nodejs16 and webpack5 respectively.
See if the new upstream version works and doesn't fail while rebuilding or testing with autopkgtest.
Report bug 🐞 in Debian if any fails to rebuild or test with autopkgtest.
Forward bugs upstream if needed.
Fix packages and forward patches.
As of this writing(though a little late🕔) we have successfully rebuilt all reverse dependencies of webpack5 and split them equally each for I and my co-intern for all Javascript modules as ruby💎 packages also depend on webpack which is a total of 44 packages. Filed a bug report on Debian bug tracking system for failing packages, also the original maintainer or uploader of the package to the Debian archive mostly Debian developers also get a mail in references to the package bug 🐞report. Sometimes the uploader who also receives the bug report decides to help out to fix the package and forward the patch upstream if need be. We have also filed an issue to upstream repo mostly via github👆 where some respond and create PR to solve those errors and others are plain aversive to the whole idea. PR from the upstream developer is cherry-picked and a patch is created by us to incorporate the code into our own working repository. some package upstream maintainer rejects such issues or doesn't respond, we take it upon ourselves to fix the package. The total number of packages that are successfully updated and ready to be merged is 10 packages while 12 packages remain on my own end to be updated.
One of the most challenging packages to update so far was prop-type as its runs its large test suite using jest of a lower version 19.0.2 compared to that of Debian OS which is version 27.5.1 updating and migrating its API's and methods to use the Debian updated version is so challenging after several googling, testing out the solution from StackOverflow, trials, and errors, reading documentations we eventually made progress with the help of my mentor, co-intern and the whole community member. It's so crazy that when I got it working I said to myself. phew😅😌 it's not rocket science why can I figure it out sooner than expected🤷♀️
I initially proposed that I would be halfway done with the project by now, I guess the reason am not able to achieve some of our goals which are finishing up with the packages for webpack and moving to transition some of the nodejs packages at all is DEBUGGING. Yes DEBUGGING! where you never can predict what the solution is. is the problem coming from Debian? or dependencies of the package you are working on, upstream bug, or dependencies of dependencies of the package you are working on, so many questions to answer. You can't easily find a solution to a bug as it takes time to try out so many guesses more of an educated guess, or even try out all the solutions from stack overflow and still no viable progress. Obviously, you cannot really know about something to set up a plan for unless you get right into it.
One way of doing this, if I have to start again is the truly understand how the javascript package work under the hood, how its handles different interaction between packages, some of its dos and don't of transpiling, bundling, testing, e.t.c
I guess my unrealistic goals need to be modified because some drawback that was not envisaged popped up and I underestimated the complexity of the tasks, which will be reducing the number of packages to update in transitioning of nodejs from what I planned😢
My major focus for the second half of the internship is to fix bugs and errors I discover, file bug reports for future bugs to seek help from co-maintainer or developers, file issues upstream and close those whose bugs are already resolved for the remaining 12 packages, and ultimately successfully uploading all reverse dependencies. Also diving into transitioning of nodejs16.
Thanks for stopping by🙏 [Less]
|
Posted
about 2 years
ago
Leonardo and I are pleased to another update to our nanotime package bringing it to version 0.3.6 which landed on CRAN earlier today.
nanotime relies on the RcppCCTZ package (as well as the RcppDate package for additional C++ operations) and
... [More]
offers efficient high(er) resolution time parsing and formatting up to nanosecond resolution, and the bit64 package for the actual integer64 arithmetic. Initially implemented using the S3 system, it has benefitted greatly from a rigorous refactoring by Leonardo who not only rejigged nanotime internals in S4 but also added new S4 types for periods, intervals and durations.
This release corrects subsetting with %in% operator, integrates it better fit in the mixed S3/S4 setup, fixes a negative period parse, and updates class comparisons to rely on inherits(). The NEWS snippet has the full more details.
Changes in version 0.3.6 (2022-03-06)
Fix incorrect subsetting with operator %in% (Leonardo in #100 fixing #99).
Fix incorrect parsing for negative nanoperiod (Leonardo in #100 fixing #96).
Test for class via inherits() (Dirk).
Thanks to my CRANberries there is also a diff to the previous version. More details and examples are at the nanotime page; code, issue tickets etc at the GitHub repository – and all documentation is provided at the nanotime documentation site.
If you like this or other open-source work I do, you can now sponsor me at GitHub.
This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.
[Less]
|
Posted
about 2 years
ago
FTP master
This month I accepted 484 and rejected 73 packages. The overall number of packages that got accepted was 495.
The overall number of rejected packages was 76, which is about 15% of the uploads to NEW. While most of the maintainers do a
... [More]
great job when creating their debian/copyright, others are a bit lax. Unfortunately those people seem to be more enthusiastic when fighting for changes in NEW processing or even removing NEW.
One argument in discussions about NEW is that the copyright verification of packages can be done by the community after accepting the packages in the archive.
Last month I did not get any hint that such checks have been done by anybody. As the past already showed several times, this community based checks simply do not exist.
So in the end poorly maintained copyright information will rot in the archive and I am not sure that this really corresponds with the Debian Social Contract.
Debian LTS
This was my ninety-second month that I did some work for the Debian LTS initiative, started by Raphael Hertzog at Freexian.
This month my all in all workload has been 40h. During that time I did LTS and normal security uploads of:
[DLA 2928-1] htmldoc security update for three CVEs
[#1004049] buster-pu: zziplib debdiff was approved and package uploaded
[#1004050] bullseye-pu: zziplib debdiff was approved and package uploaded
[#1004055] buster-pu: debdiff was approved and package uploaded
[#1006493] bullseye-pu: htmldoc/1.9.11-4+deb11u2
[#1006494] buster-pu: htmldoc/1.9.3-1+deb10u3
[#1006550] buster-pu: tiff/4.1.0+git191117-2~deb10u4
[#1006551] bullseye-pu: tiff/4.2.0-1+deb11u1
Unfortunately salsa went down at the end of the month, so several planned uploads did not happen and have to be delayed to March.
I also continued to work on security support for golang packages. Further I worked on packages in NEW on security-master and injected missing sources. Last but not least I did some days of frontdesk duties.
Debian ELTS
This month was the forty-fourth ELTS month.
During my allocated time I uploaded:
ELA-567-1 for apache2
ELA-567-2 for apache2
ELA-568-1 for ksh
ELA-569-1 for tiff
ELA-570-1 for htmldoc
Further I worked on cyrus-sasl but did not do an upload yet.
Last but not least I did some days of frontdesk duties.
Debian Printing
As announced last month I uploaded a new version of cups.
Altogether I uploaded new upstream versions or improved packaging of:
… cups
… cups-filters
Debian Astro
This month I uploaded new upstream versions or improved packaging of:
… openvlbi
Other stuff
This month I uploaded new upstream versions or improved packaging of:
… duktape
… meep
… bitshuffle
… several golang packages
[Less]
|
Posted
about 2 years
ago
Welcome to the February 2022 report from the Reproducible Builds project. In these reports, we try to round-up the important things we and others have been up to over the past month. As ever, if you are interested in contributing to the project
... [More]
, please visit our Contribute page on our website.
Jiawen Xiong, Yong Shi, Boyuan Chen, Filipe R. Cogo and Zhen Ming Jiang have published a new paper titled Towards Build Verifiability for Java-based Systems (PDF). The abstract of the paper contains the following:
Various efforts towards build verifiability have been made to C/C++-based systems, yet the techniques for Java-based systems are not systematic and are often specific to a particular build tool (eg. Maven). In this study, we present a systematic approach towards build verifiability on Java-based systems.
GitBOM is a flexible scheme to track the source code used to generate build artifacts via Git-like unique identifiers. Although the project has been active for a while, the community around GitBOM has now started running weekly community meetings.
The paper Chris Lamb and Stefano Zacchiroli is now available in the March/April 2022 issue of IEEE Software. Titled Reproducible Builds: Increasing the Integrity of Software Supply Chains (PDF), the abstract of the paper contains the following:
We first define the problem, and then provide insight into the challenges of making real-world software build in a “reproducible” manner-this is, when every build generates bit-for-bit identical results. Through the experience of the Reproducible Builds project making the Debian Linux distribution reproducible, we also describe the affinity between reproducibility and quality assurance (QA).
In openSUSE, Bernhard M. Wiedemann posted his monthly reproducible builds status report.
On our mailing list this month, Thomas Schmitt started a thread around the SOURCE_DATE_EPOCH specification related to formats that cannot help embedding potentially timezone-specific timestamp. (Full thread index.)
The Yocto Project is pleased to report that it’s core metadata (OpenEmbedded-Core) is now reproducible for all recipes (100% coverage) after issues with newer languages such as Golang were resolved. This was announced in their recent Year in Review publication. It is of particular interest for security updates so that systems can have specific components updated but reducing the risk of other unintended changes and making the sections of the system changing very clear for audit.
The project is now also making heavy use of “equivalence” of build output to determine whether further items in builds need to be rebuilt or whether cached previously built items can be used. As mentioned in the article above, there are now public servers sharing this equivalence information. Reproducibility is key in making this possible and effective to reduce build times/costs/resource usage.
diffoscope
diffoscope is our in-depth and content-aware diff utility. Not only can it locate and diagnose reproducibility issues, it can provide human-readable diffs from many kinds of binary formats. This month, Chris Lamb prepared and uploaded versions 203, 204, 205 and 206 to Debian unstable, as well as made the following changes to the code itself:
Bug fixes:
Fix a file(1)-related regression where Debian .changes files that contained non-ASCII text were not identified as such, therefore resulting in seemingly arbitrary packages not actually comparing the nested files themselves. The non-ASCII parts were typically in the Maintainer or in the changelog text. […][…]
Fix a regression when comparing directories against non-directories. […][…]
If we fail to scan using binwalk, return False from BinwalkFile.recognizes. […]
If we fail to import binwalk, don’t report that we are missing the Python rpm module! […]
Testsuite improvements:
Add a test for recent file(1) issue regarding .changes files. […]
Use our assert_diff utility where we can within the test_directory.py set of tests. […]
Don’t run our binwalk-related tests as root or fakeroot. The latest version of binwalk has some new security protection against this. […]
Codebase improvements:
Drop the _PATH suffix from module-level globals that are not paths. […]
Tidy some control flow in Difference._reverse_self. […]
Don’t print a warning to the console regarding NT_GNU_BUILD_ID changes. […]
In addition, Mattia Rizzolo updated the Debian packaging to ensure that diffoscope and diffoscope-minimal packages have the same version. […]
Debian-related updates
Vagrant Cascadian wrote to the debian-devel mailing list after noticing that the binutils source package contained unreproducible logs in one of its binary packages. Vagrant expanded the discussion to one about all kinds of build metadata in packages and outlines a number of potential solutions that support reproducible builds and arbitrary metadata.
Vagrant also started a discussion on debian-devel after identifying a large number of packages that embed build paths via RPATH when building with CMake, including a list of packages (grouped by Debian maintainer) affected by this issue. Maintainers were requested to check whether their package still builds correctly when passing the -DCMAKE_BUILD_RPATH_USE_ORIGIN=ON directive.
On our mailing list this month, kpcyrd announced the release of rebuilderd-debian-buildinfo-crawler a tool to parse the Packages.xz Debian package index file, attempts to discover the right .buildinfo file from buildinfos.debian.net and outputs it in a format that can be understood by rebuilderd. The tool, which is available on GitHub, solves a problem regarding correlating Debian version numbers with their builds.
bauen1 provided two patches for debian-cd, the software used to make Debian installer images. This involved passing --invariant and -i deb00001 to mkfs.msdos(8) and avoided embedding timestamps into the gzipped Packages and Translations files. After some discussion, the patches in question were merged and will be included in debian-cd version 3.1.36.
Roland Clobus wrote another in-depth status update about status of ‘live’ Debian images, summarising the current situation that “all major desktops build reproducibly with bullseye, bookworm and sid”.
The python3.10 package was uploaded to Debian by doko, fixing an issue where [.pyc files were not reproducible because the elements in frozenset data structures were not ordered reproducibly. This meant that to creating a bit-for-bit reproducible Debian chroot which included .pyc files was not reproducible. As of writing, the only remaining unreproducible parts of a standard chroot is man-db, but Guillem Jover has a patch for update-alternatives which will likely be part of the next release of dpkg.
Elsewhere in Debian, 139 reviews of Debian packages were added, 29 were updated and 17 were removed this month adding to our knowledge about identified issues. A large number of issue types have been updated too, including the addition of captures_kernel_variant, erlang_escript_file, captures_build_path_in_r_rdb_rds_databases, captures_build_path_in_vo_files_generated_by_coq and build_path_in_vo_files_generated_by_coq.
Website updates
There were quite a few changes to the Reproducible Builds website and documentation this month as well, including:
Chris Lamb:
Considerably rework the Who is involved? page. […][…]
Move the contributors.sh Bash/shell script into a Python script. […][…][…]
Daniel Shahaf:
Try a different Markdown footnote content syntax to work around a rendering issue. […][…][…]
Holger Levsen:
Make a huge number of changes to the Who is involved? page, including pre-populating a large number of contributors who cannot be identified from the metadata of the website itself. […][…][…][…][…]
Improve linking to sponsors in sidebar navigation. […]
drop sponsors paragraph as the navigation is clearer now. […]
Add Mullvad VPN as a bronze-level sponsor . […][…]
Vagrant Cascadian:
Remove a stray parenthesis from the Who is involved? page. […]
Upstream patches
The Reproducible Builds project attempts to fix as many currently-unreproducible packages as possible. February’s patches included the following:
Bernhard M. Wiedemann:
btop (sort-related issue)
complexity (date)
giac (update the version with upstreamed date patch)
htcondor (use CMake timestamp)
libint (readdir system call related)
libnet (date-related issue)
librime-lua (sort filesystem ordering)
linux_logo (sort-related issue)
micro-editor (date-related issue)
openvas-smb (date-related issue)
ovmf (sort-related issue)
paperjam (date-related issue)
python-PyQRCode (date-related issue)
quimb (single-CPU build failure)
radare2 (Meson date/time-related issue)
radare2 (Rework SOURCE_DATE_EPOCH usage to be portable)
siproxd (date, with Sebastian Kemper + follow-up
xonsh (Address Space Layout Randomisation-related issue)
xsnow (date & tar(1)-related issue)
zip (toolchain issue related to filesystem ordering)
Chris Lamb:
#1005029 filed against ltsp (forwarded upstream).
#1005197 filed against pcmemtest.
#1005825 filed against hatchling.
#1005826 filed against mpl-sphinx-theme (forwarded upstream)
#1005827 filed against gap-hapcryst.
#1005901 filed against tree-puzzle.
#1005954 filed against jcabi-aspects.
#1005955 filed against paper-icon-theme.
Roland Clobus:
#1006358 filed against libxmlb.
Vagrant Cascadian:
#1005408 filed against wcwidth.
#1005420 filed against xir.
#1005421 filed against xir.
#1005726 filed against ruby-github-markup.
#1005727 filed against ruby-tioga.
#1005792 filed against btop.
#1005793 filed against libadwaita-1.
#1005794 filed against snibbetracker.
#1006252 filed against cctbx.
#1006254 filed against mdnsd.
#1006256 filed against gmerlin.
#1006302 filed against beav.
#1006385 filed against krita.
#1006407 filed against qt6-base.
#1006455 filed against onevpl-intel-gpu.
#1006471 filed against ruby3.0.
#1006473 filed against nix.
#1006474 filed against foma.
#1006476 filed against ruby3.0.
Testing framework
The Reproducible Builds project runs a significant testing framework at tests.reproducible-builds.org, to check packages and other artifacts for reproducibility. This month, the following changes were made:
Daniel Golle:
Update the OpenWrt configuration to not depend on the host LLVM, adding lines to the .config seed to build LLVM for eBPF from source. […]
Preserve more OpenWrt-related build artifacts. […]
Holger Levsen:
Temporary use a different Git tree when building OpenWrt as our tests had been broken since September 2020. This was reverted after the patch in question was accepted by Paul Spooren into the canonical openwrt.git repository the next day.
Various improvements to debugging OpenWrt reproducibility. […][…][…][…][…]
Ignore useradd warnings when building packages. […]
Update the script to powercycle armhf architecture nodes to add a hint to where nodes named virt-*. […]
Update the node health check to also fix failed logrotate and man-db services. […]
Mattia Rizzolo:
Update the website job after contributors.sh script was rewritten in Python. […]
Make sure to set the DIFFOSCOPE environment variable when available. […]
Vagrant Cascadian:
Various updates to the diffoscope timeouts. […][…][…]
Node maintenance was also performed by Holger Levsen […] and Vagrant Cascadian […].
Finally…
If you are interested in contributing to the Reproducible Builds project, please visit our Contribute page on our website. However, you can get in touch with us via:
IRC: #reproducible-builds on irc.oftc.net.
Twitter: @ReproBuilds
Mailing list: [email protected]
[Less]
|