Most packages these days are signed using GPG so that the user can be sure that they are from a trusted source. We should probably do the same and publish the fingerprint on our home page.
The PackageKit Error in Fedora 9 looks like this: "Malicious software can damage your computer and cause other harm. Are you *sure* you want to install this package?" Looks bad for ThinLinc, another reason to sign our packages.
A big problem with this is how should the users trust the key? They'll have to download it the same way they downloaded the packages. So the level of trust should be the same. However if the key keeps getting reused then only the first download needs to take extra steps to verify the key. After that the users can keep verifying upgrades as they know the key is trusted. This requires us to keep using the same key for a long time though, which is unlike what distributions do who generally replace the keys for every version.
We are missing gnupg in cenbuild, so this would need to be packaged if we want to be able to sign rpms as a part of the build process. Just running ./autogen.sh and then ./configure for gnupg 2.4.4 results in the following output: > configure: > *** > *** You need libgpg-error to build this program. > ** This library is for example available at > *** https://gnupg.org/ftp/gcrypt/gpgrt > *** (at least version 1.46 is required.) > *** > configure: > *** > *** You need libgcrypt to build this program. > ** This library is for example available at > *** https://gnupg.org/ftp/gcrypt/libgcrypt/ > *** (at least version 1.9.1 (API 1) is required.) > *** > configure: > *** > *** You need libassuan to build this program. > *** This library is for example available at > *** https://gnupg.org/ftp/gcrypt/libassuan/ > *** (at least version 2.5.0 (API 2) is required). > *** > configure: > *** > *** You need libksba to build this program. > *** This library is for example available at > *** https://gnupg.org/ftp/gcrypt/libksba/ > *** (at least version 1.6.3 using API 1 is required). > *** > configure: > *** > *** It is now required to build with support for the > *** New Portable Threads Library (nPth). Please install this > *** library first. The library is for example available at > *** https://gnupg.org/ftp/gcrypt/npth/ > *** (at least version 1.2 (API 1) is required). > *** > configure: error: > *** > *** Required libraries not found. Please consult the above messages > *** and install them before running configure again. > *** There are a few things that we need to have in place before we can sign rpms.
We have now added GnuPG to cenbuild. It is used by e.g. rpmsign when signing rpm packages. See %_gpg_sign_cmd in /usr/lib/rpm/macros. Some of the features in GnuPG have been disabled in the cenbuild package, especially those relating to starting daemons/services. Most notably, we have disabled building the keyboxd daemon component of GnuPG, hence keyboxd is not a part of the cenbuild package. We noticed that it would start when running e.g. "gpg --list-keys", and then remained running in the background. We don't want such things running indefinitely in the build environment.
A GnuPG config folder, generated outside of cenbuild, will likely contain a file called common.conf, containing the line "use-keyboxd". If this folder is to be used with GnuPG inside cenbuild, the file has to be either deleted or moved such that GnuPG in cenbuild uses it's old method of handling keys. See https://github.com/gpg/gnupg/blob/40227e42/README#L128
The keys we create for signing our rpms can be set to expire. We have therefore looked at the behavior surrounding the usage of packages that have been signed with an expired key, and we found the following: * An rpm can not be signed with a key that has expired, it says "gpg: signing failed: Unusable secret key". * A signed rpm can be installed even after the signing key has expired. - running "rpm --checksig <rpm package>" will still say that the signature is valid after expiration. - running "rpm -i <rpm package>" installs the package without any issues. * An rpm can be signed with an expired key if the system clock is set back before the expiration date. In conclusion, the expiration date does not add any security. Having an expiration date on our key could instill a false sense of security.
(In reply to Pierre Ossman from comment #4) > A big problem with this is how should the users trust the key? They'll have > to download it the same way they downloaded the packages. So the level of > trust should be the same. > > > However if the key keeps getting reused then only the first download needs > to take extra steps to verify the key. After that the users can keep > verifying upgrades as they know the key is trusted. > > This requires us to keep using the same key for a long time though, which is > unlike what distributions do who generally replace the keys for every > version. We looked at the rpm repository keys for different providers: * Google Chrome has a primary key without expiration date, and subkeys rolling on three-year intervals with 1.5 year overlap. * Microsoft VSCode has only a primary key without expiration date. * GitLab had a primary key with 6 years expiration date, and a subkey also with 6 years expiration date. There doesn't seem to be any de facto standard for how these keys are set up. When looking at the different ways of configuring our keys we have to take a few things into account. For starters, do we want only a primary key, or do we want subkeys? Further, do we want to set an expiration date or not? Arguments for using subkeys: With subkeys, it would be possible to have different keys for different purposes. It would also, in theory, be possible to have the keys in different locations. Subkeys can not, by themselves, be updated with a new expiration date. This means subkeys can be handled with less care. If a subkey is leaked, the impact is less severe. Arguments for only a primary key: We don't need keys with different purposes, only one key for signing. The idea is that you want separation between the primary key and the subkeys. As part of that, we would want different passphrases. By default, the entire set of keys (primary key and its subkeys) share the same passphrase. Without using non-standard workarounds, we can't separate the two. Hence, we don't gain any security by having subkeys. Arguments for setting an expiry date: An expiration date shows that keys are rotated and are not used indefinitely. This is a clear signal to admins that they need to keep track of future key updates. It would also be a useful reminder for us (the developers) that keys need updating, since our rpm builds would fail. Arguments against setting expiration date: Expiration dates do not provide any actual security benefits. Firstly, there's no central authority check for rpm packages e.g. some online-based verification. Secondly, the system clock can be changed to work around the expiration date -- see comment #16. Third, the key expiration date may be extended, which can be confusing for admins and misused by someone who got hold of our leaked key. Having an expiration date set would require us to also implement a routine for key renewal. This may still be beneficial for us, but it is outside the scope of this bug. In conclusion, we see no apparent benefits to having subkeys and/or expiration dates for our use case. Both features seem to be useful in a context where all parties are able to synchronize trust. As explained above, this is not the case when signing rpms. Thus, we will move forward with only one primary key without expiration date. We also aim to implement a simple make file check that acts as a soft reminder to create new keys at a set interval.
The tool we want to use to sign rpms (rpmsign) expects the passphrase to be input manually via an interactive ncurses prompt called pinentry. This prompt does not allow us to provide a passphrase via a file or stdin. This is problematic for us since we want to automate things. However, rpmsign uses gpg below the surface, which does give us more control. This can be seen in the rpm macro called "%__gpg_sign_cmd". We can modify parts of the gpg-call using the option "--define" for rpmsign. As can be seen in the below link, we should be able to use the gpg option "--passphrase-file" [1]. The "--define" option for rpmsign sets macros. The only macro that is suitable for the "--passphrase-file" option to gpg is called "_gpg_sign_cmd_extra_args". Sadly, "_gpg_sign_cmd_extra_args" is only available in newer versions of RPM. We currently have version 4.9 in cenbuild, we need to upgrade to a newer version for this to work. We have verified on a Fedora 39 machine with version 4.19 that modifying this macro works like intended. [1] https://www.gnupg.org/documentation/manuals/gnupg24/gpg.1.html
It should be noted that another method exists for modifying the gpg options. You can overwrite the "%__gpg_sign_cmd" entirely. This method would not require an upgraded rpm in cenbuild. The first option, which we have seen works, is to have a ~/.rpmmacros file where you can redefine gpg_sign_cmd to your liking. The problem is that the path of this rpmmacros file cannot be changed. This makes things difficult since we want all build related files to be contained within our source tree. The second option is to use rpmsign's "--define" parameter. This method has not been fully explored. The parsing of the macro is not fully straight forward. That means that we would likely have to translate internal macros in gpg_sign_cmd. However, we might not want to replace the entire "%__gpg_sign_cmd" since we know that it could be updated as part of a new rpm version. We want to keep our changes to rpm as small as possible. Due to the drawbacks of replacing gpg_sign_cmd, we will start by investigating an upgrade of rpm as described in comment #18.
Turns out that we are missing Lua as a requirement for rpm in cenbuild. I looked at the lua source, and it is quite minimal, and can be built with any ANSI C compiler. Sadly, it is not using Autoconf or CMake which means that we need to do a more manual build configuration, especially if we want to be able to include it in non-linux archs in cenbuild. Lets look at only building for linux arch for now since these are the only ones that require the rpm package.
The rpm package's configure script uses pkg-config to find lua, and hence we need to make sure that the lua package provides a lua.pc-file in the proper location for it to work. It is possible to generate the contents of lua.pc with a recipe in the lua make file. But the contents of it is insufficient for (modern versions of?) pkg-config. The make file only provides: * Version * prefix * libdir * includedir While we also need it to provide [1]: * Name * Description * Libs * Cflags [1] see e.g. /usr/lib64/pkgconfig/lua.pc locally. This means that we might as well create our own lua.pc at build time, since most of the info we need is available at that time.
I also found that the default prefix path /usr/local/ is hardcoded at one place in the source code, even if the Makefile suggests that it is OK to set it to something else while building. Just to be sure, I added a patch that sets that path to /usr/ in the source code, which happens to be the prefix we set when we build packages, independent of arch for native packages. This is maybe not the most robust way of handling it, and we will likely have to revisit this issue in the future if we need to build lua for our target archs.
Work on upgrading rpm and its dependencies was moved over to bug 5435.
We are now signing the client and server rpms automatically as a part of the build procedure. Not that we are not doing any kind of codesigning on the payload, there is only a header signature added to the packages at the moment.
We are now signing the client and server rpms automatically as a part of the build procedure. Note that we are not doing any kind of code signing on the payload, there is only a header signature added to the packages at the moment.
I also made it such that the build will continue even if the call to rpmsign did not return 0 as exit status. We want to still be able to build our software, the signature is not essential for us in that way. Instead, we have make sure that the rpms are signed via autotests.
I added autotests to test_rpms.py that do the following: Check that we provide exactly one gpg-pubkey ============================================ All gpg-pubkeys-files found in the zip-bundle are provided to test_rpms.py, but we verify that there is only one resulting public key in the database after all files have been imported. Check that all rpms have a signature ==================================== This is done for both rpm(v4) and rpmv3 signatures. Check that all signatures use RSA/SHA256 ======================================== This seems to be the standard on modern rpms (also on FIPS enabled systems). Check that all the public key matches signatures on all rpms ============================================================ We use the command rpm --checksig <rpm-name> to verify this.
We are now signing the client and server rpms automatically as a part of the build procedure.
In comment 35 it was stated that only the header was signed while the payload was not. This is not true, the signature is applied to the packages using the rpmsign command, which signs the payload. Due to some confusing formatting when comparing output from "rpm --checksig --verbose <package name>" between our packages and some arbitrary Fedora-package, I mistakenly assumed that our packages were missing a payload signature which could be seen on the Fedora packages. Comparison of our server-rpm and Fedora's which-package: Kernel-headers rpm: > kernel-headers-6.7.3-200.fc39.x86_64.rpm: > Header V4 RSA/SHA256 Signature, key ID 18b8e74c: OK > Header SHA256 digest: OK > Header SHA1 digest: OK > Payload SHA256 digest: OK > V4 RSA/SHA256 Signature, key ID 18b8e74c: OK > MD5 digest: OK ThinLinc server rpm: > thinlinc-server-4.16.0post-3532.x86_64.rpm: > Header V4 RSA/SHA256 Signature, key ID 5e02d204: OK > Header SHA256 digest: OK > Header SHA1 digest: OK > Payload SHA256 digest: OK > MD5 digest: OK Turns out that the missing field represent signatures compatible with rpm < 4.14 (rpmv3). Rpm v3 signatures can be added to packages to improve backward compatibility.
We are now adding rpmv3 signatures to our packages as well.
We added a convenience function in the makefile to generate a new gpg-key. We also updated the wiki and README with instructions for what steps need to be done when a key has to be updated. Tested by generating a new key and could see that it was included in the server bundle. With that, this bug should be ready for testing.
Tested server build 3547 and client build 3444. ✅ We should provide signed rpm packages. ✅ We should sign packages automatically during the build procedure. Our RPM packages are signed automatically at build time. We tested installing the server and client RPMs on RHEL 7 and RHEL 9, and Deb packages on Ubuntu 16.04 and Ubuntu 22.04. Installation was successful in all instances, even without trusting our public key. ✅ The validity of the signature should be affirmed in the build procedure. There are autotests in place that run after successful builds that check for a valid signature. Looked through the commits (until r40831) as well as the documentation and they look good. Note that building RPMs locally does not sign the package. This is by design as only our build server signs RPMs currently.
Tested building with the password file in the correct location and could see that the RPM was signed during the building process. Removing the file simply skipped signing the RPM and printed a helpful message. Using a password file with the incorrect password prints an error message during build, but does not halt the building process and produces an unsigned RPM.
Regarding the convenience function in the makefile for generating a GPG-key, we noticed we had not been consistent in what phrase we use to identify it over the different make targets. One of the phrases was also very general, so we made it a bit more specific to only match the key in question, if we happen to add more keys in the future.
Tested the check and generate-gpg-key make targets while following the README instructions, and they worked perfectly fine. Also tested signing a ThinLinc rpm with the newly created key and it worked as intended. This was not done the automatic way in a server/client build process, but rather simply by calling the rpmsign command, as constructed in the makefiles, directly on an existing rpm file. This part of the bug can be considered to be resolved.
Everything seems to be in order, and the acceptance criteria are met as detailed in comment #64. Closing.
