By Samuel Moelius
This time last year, we wrote about the more than 190 Trail of Bits-authored pull requests that were merged into non-Trail of Bits repositories in 2021. In 2022, we continued that trend by having more than 400 pull requests merged into non-Trail of Bits repositories!
Why is this significant? While we take great pride in the tools that we develop, we recognize that we benefit from tools maintained outside of Trail of Bits. When one of those tools doesn’t work as we expect, we try to fix it. When a tool doesn’t fill the need we think it was meant to, we try to improve it. In short, we try to give back to the community that gives so much to us.
Here are a few highlights from the list of PRs at the end of this blog post:
- Clippy is a collection of over 550 lints to catch common mistakes and improve Rust code. We added the crate_in_macro_def and unnecessary_find_map lints, and contributed improvements and bugfixes to lints such as empty_line_after_outer_attribute, expect_used/unwrap_used, extra_unused_lifetimes, needless_borrow, needless_lifetimes, unnecessary_to_owned, and unnecessary_filter_map.
- HEVM is an implementation of the Ethereum virtual machine with symbolic execution capabilities. Our contributions to HEVM included simplifying its use of the SMT solver, improving its performance, fixing a memory leak, and adding tests.
- Envoy is a high-performance open source edge and service proxy that makes the network transparent to applications. We implemented the initial version of the Unified Header Validation (UHV) component within Envoy for validating all request and response headers for HTTP/1 and HTTP/2. We took the existing header validation logic, consolidated it into the UHV component, performed an assessment to determine where the logic was not fully RFC compliant, and then fixed or implemented any gaps to ensure that the default configuration strictly adheres to the RFC standards. The new component provides a single entry point for all HTTP request and response validation that makes it a much easier code base to maintain, audit, extend, customize, and fix any newly discovered attack vectors.
- pyca/cryptography is a package that provides cryptographic recipes and primitives to Python developers. We improved its support for Certificate Transparency and made numerous usability improvements.
- Vcpkg is a C/C++ package manager for Windows, Linux, and MacOS. We fixed a bug in Vcpkg itself and made improvements to packages such as flatbuffers, grpc, gtest, ixwebsocket. libcpplocate, llvm, mbedtls, tcb-span, and z3.
- Warehouse is the software that powers PyPI, the official package index for the Python programming language. We made numerous feature improvements and bugfixes, including support for expiring API tokens, support for credential-free package uploads with OIDC, a refactor of core permissions internals, enhancements to PyPI’s vulnerability feed, and improvements to user-facing error messages.
The projects named below represent software of the highest quality. Software of this caliber doesn’t come from just merging PRs and publishing new releases; it comes from careful planning, prioritizing features, familiarity with related projects, and an understanding of the role that a project plays within the larger software ecosystem. We thank these projects’ maintainers both for the work the public sees and for innumerable hours spent on work the public doesn’t see.
We wish you a happy, safe, and similarly productive 2023!
Some of Trail of Bits’s 2022 Open-Source Contributions
Cryptography
Tech Infrastructure
- abodelot/jquery.json-viewer
- aio-libs/aiohttp
- curl/curl
- curl/curl-fuzzer
- di/bump
- di/pip-api
- GaloisInc/daedalus
- googleapis/google-auth-library-python
- Homebrew/homebrew-core
- solc-select 0.2.1 (new formula) #107977
- crytic-compile 0.2.3 (new formula) #108010
- slither-analyzer 0.8.3 (new formula) #108016
- echidna 2.0.2 (new formula) #108045
- echidna 2.0.3 #110107
- crytic-compile 0.2.4 #112418
- slither-analyzer 0.9.0 #112423
- solc-select 1.0.1 #112793
- slither-analyzer 0.9.1 #114631
- echidna 2.0.4 #116508
- iovisor/ubpf
- killercup/cargo-edit
- llvm/llvm-project
- microsoft/ebpf-for-windows
- microsoft/vcpkg
- [libcpplocate] New port #23173
- [tcb-span] Add new port #23393
- [llvm] Fix LLVM install for ‘utils’ feature #23399
- [ixwebsocket] Update to v11.3.3 #23548
- [gtest] Remove -Werror #23780
- [mbedtls] Update to latest 2.x LTS version #23787
- [flatbuffers] Update to 2.0.6 #24208
- [z3] Update to 4.8.15 #24209
- [z3] Update to 4.8.16 #24407
- [grpc] Fix path quoting #24948
- [mbedtls] Update to v2.28.1 #25894
- [z3] Update to v4.9.1 #25911
- [z3] Update to v4.10.2 #25954
- [grpc] Fix protobuf protoc executable variable #26199
- [vcpkg] Fix cross compiling macOS #26240
- [z3] Update to 4.11.0 #26429
- NixOS/nixpkgs
- haskellPackages.clash-prelude: fix build by disabling tests #178868
- semgrep: 0.106.0 -> 0.108.0 #185771
- echidna: 1.7.3 -> 2.0.2 #190144
- echidna: 2.0.2 -> 2.0.3 #190775
- libff: dynamic by default #190786
- xed: 12.0.1 -> 2022.08.11 #191045
- echidna: 2.0.3 -> 2.0.4 #202542
- haskellPackages: remove unnecessary overrides from ghc-9.2.x #202604
- haskellPackages: configuration cleanup #202615
- haskellPackages: configuration common cleanup #203180
- haskellPackages: unbreak selected packages #203327
- haskellPackages: unbreak selected packages #203489
- nodejs/node
- Add script for vulnerability checking of Node.js dependencies #43362
- Add support for using API key to vuln checking script #43909
- Update `undici` CPE in vulnerability checking script #44128
- Add automation for updating `base64` dependency #45300
- Add automation for updating `acorn` dependency #45357
- Add automation for updating `libuv` dependency #45362
- tools: add missing step in update-base64.sh script #45509
- Remove dependency vulnerability checker #45675
- osquery/osquery
- Fix check for PIE support #7234
- Improve Pidfile handling #7304
- Prevent the audit event system from using too much memory #7329
- Fix globToRegex truncating UTF16 characters #7430
- Do not run clang-tidy on third party libraries #7432
- Change the JSON of the results from an event scheduled query to an array #7434
- Add new metrics and improve description of existing ones in osquery_schedule #7438
- Fix a crash when Yara uses its own strutils functions #7439
- bpf: Improve socket event handling #7446
- Update cppcheck to version 2.6.3 and skip analysis for third party code #7455
- Fix submodule cache for macOS CI runner #7456
- Add third party libraries target #7467
- Add BOOST_USE_ASAN define when enabling Asan #7469
- Enable fuzzing and Asan on Windows, enable Asan on macOS #7470
- Fix user_time and system_time unit in processes table on M1 #7473
- Fix watchdog not killing unhealthy worker/extension fast enough #7474
- Fix some warnings about unrecognized special characters #7478
- yum_sources: Include the mirrorlist URL in the table output #7479
- Fix third party libraries flags leaking to osquery targets #7480
- Replace WmiRequest constructor with static factory method #7489
- Change cpu_info test to ensure *at least* one socket is present #7490
- Improve scheduled query denylisting and scheduler shutdown #7492
- Fix the test_http_server.py –persist option #7497
- bpf: Disable the BPF publisher in case of error #7500
- Mark wall_time column in osquery_schedule as hidden #7501
- Add a mechanism to reduce memory retained on Linux #7502
- Fix crash due to interaction between distributed and config plugin #7504
- libs: Update OpenSSL from version 1.1.1l to 1.1.1n #7506
- Implement a performant cache for users and groups on Windows #7516
- Remove libelfin and elf parsing tables #7524
- Update librpm to 4.17.0 #7529
- Eliminate removal of nonblocking flag for “special” files #7530
- Fixes to unblock the CI #7533
- Drop shortcut_files table #7547
- libs: Update zlib from v1.2.11 to v1.2.12 #7548
- libs: Update libdpkg from version v1.19.0.5 to v1.21.7 #7549
- Prevent ebpfpub linking against the system zlib #7557
- Restore some release checks #7558
- Add an option to specify a path to the openssl archive #7559
- Prevent CLI_FLAGs to be set via config #7561
- Change where the macOS Info.plist is generated #7566
- Fix DebPackages.test_sanity test #7569
- certificates table: Add Linux support #7570
- CHANGELOG 5.2.3 #7571
- Fix release tests for Linux aarch64 #7572
- Use additional instead of index for admindir in deb_packages #7573
- CHANGELOG 5.3.0 #7575
- Explicitly set context for the tables reading utmpx databases #7578
- certificates: Refactor the OpenSSL utilities #7581
- Warn about setting CLI_FLAGs in the config #7583
- Remove the test_daemon_sighup test #7584
- Restore macOS `kernel_panics` table on modern macOS #7585
- Replace `OS X` with `macOS` in table specs #7587
- Fix MBCS support on Windows #7593
- Remove CLI flags settings from osquery.example.conf #7595
- Correct the section where the users and groups service flags are described #7596
- Fix shared_resources accessing uninitialized variables #7600
- Remove redundant string conversion #7603
- Update the “new release” issue template #7607
- Fix a UUID typo in the `disk_encryption` table #7608
- Add an option to build with the leak sanitizer #7609
- Fix SchedulerTests.test_scheduler_drift_accumulation flakyness #7613
- Fix multiple Yara leaks #7615
- Initialize users and groups services on all tests that need them #7620
- Do not catch table or registry exceptions when running tests #7621
- Remove unnecessary string copy #7625
- Fix system-info support for Unicode characters on Windows #7626
- libs: Update sqlite to version 3.38.5 #7628
- libs: Update OpenSSL to version 1.1.1o #7629
- `wmi_bios_info`: Include Win32_BIOS attributes for all systems #7631
- Port `memory_devices` table to Windows #7633
- Improve config parsing and osqueryfuzz-config performance #7635
- Implement a split and trim function using std::string_view #7636
- deb_packages: Do not display arch info in the package name #7638
- Fix thrift server shutting down when dropping privileges #7639
- Update `shared_resources` table to add type names, fix type/maximum_allowed handling #7645
- Fix AWS certificate verification failing on all services #7652
- time: Fix the Windows local_timezone column value #7656
- Port platform_info table to M1 Macs #7660
- Remove the lldp_neighbors table #7664
- ci: Update osquery-packaging commit to the latest one #7667
- cmake: Add an option to enable or disable using ccache #7671
- cmake: Prevent defining some Linux only targets on other platforms #7672
- libs: Update OpenSSL to version 1.1.1q #7674
- Add documentation about 3rd-party dependency security #7684
- tpm_info: Refactor, ensure boolean values are always up to date #7686
- Port the `secureboot` table to macOS #7692
- Fix a crash when parsing ATC config with no columns #7693
- ci: Update and temporarily disable the macOS Catalina test job #7700
- test: Fix Mdfind.test_sanity flakyness #7701
- Fix bug in GetHomeDirectories filesystem function #7705
- Update minimum macOS support from 10.12 to 10.14 #7707
- Add `firmware_type` column to `platform_info` table on Windows. #7710
- Fix `GetMemorySize` for Windows `memory_devices` table #7711
- Correct macOS version support check for unified_log.mm #7713
- Improvements to osquery AWS logic #7714
- Temporarily disable memory_devices integration test #7717
- Add validation integration test for memory_devices #7722
- Increase mdfind query timeout to 30 seconds #7725
- platform_info: Add `firmware_type` to macOS #7727
- libs: Update libxml2 to v2.9.14 #7729
- libs: Update sqlite to version 3.39.2 #7736
- mdfind: Reduce table overhead and support quick interruption #7738
- test: Fix platform-info.test_sanity on Windows #7742
- `secureboot`: Acquire the necessary process privileges on Windows #7743
- ci: Migrate jobs from ubuntu-18.04 to ubuntu-20.04 #7745
- Fix a leak and improve users and groups APIs on Windows #7755
- Fix `process_file_events` subscriber being incorrectly initialized #7759
- docs: Correct the description on how to configure and use Yara signature urls #7769
- build: Remove unused find_packages modules and submodule #7771
- misc: Delete temporary CTest files #7782
- ci: Add a job and helper scripts to periodically scan for CVEs #7787
- `processes`: Stabilize the `start_time` column value on macOS and Linux #7788
- ci: Update how we set github workflow step outputs #7791
- Fix deadlock when logging happens during a database reset #7798
- Fix handling of some errors during an AWS HTTP request #7811
- ci: Fix python version when installing modules and testing on macos #7813
- processes: Fix the procfs memory unit kB, which is 1024 bytes not 1000 #7818
- Do not access the AWS SDK request content type if missing #7834
- ci: Update some actions to remove deprecation warnings #7864
- cve: Ignore zstd CVE-2021-24031 #7865
- docs: Update the list of pages #7866
- libs: update Thrift to 0.17 #7868
- cve: Ignore libcryptsetup cves #7871
- cve: Ignore libdpkg CVE-2022-1664 #7872
- cve: Ignore libgcrypt cves #7873
- libs: Update zlib to 1.2.13 #7874
- libs: Update libarchive to 3.6.2 #7877
- Docs: mention the recent adoption of automatic CVE scanning #7878
- cmake: Remove forced static libraries search for osquery-toolchain #7881
- libs: Update libxml2 to 2.10.3 #7882
- git: Ignore compile_commands.json and pyrightconfig.json #7885
- ci: Automatically cancel old PR jobs #7887
- test: Fix flaky test_daemon_sigint #7888
- test: Add an option to run only selected python testcases #7890
- CHANGELOG 5.7.0 #7894
- ci: Improve error message when a library is missing from the manifest #7899
- pallets/werkzeug
- pypi/warehouse
- python-version: bump to 3.8.9 #10626
- docs/getting-started: formatting fixes, add macOS troubleshooting #10627
- Interfaces and services for JWK management #10628
- docs/application: update the project structure #10634
- Models, routes and views for creating OIDC publishers #10753
- Add `ExpiryCaveat` #11122
- warehouse, tests: pick DB changes from #11122 #11157
- Refactor: Migrate to 2.0-style security policies #11218
- OIDC: More claims for GitHub’s provider #11239
- GitHub OIDC: validate `job_workflow_ref` #11263
- OIDC macaroon minting for GitHub #11272
- Revert #11313 #11315
- macaroons/security_policy: avoid exceptions during user lookup #11322
- tests, warehouse: avoid potential recursion in authenticated_userid #11333
- warehouse, tests: check that session’s user still exists #11341
- warehouse, tests: require a matched route for session auth #11351
- Add a caveat for project IDs #11857
- API: Add “summary” field to vulnerability reports #11858
- Better upload errors when using API tokens #11885
- warehouse, tests: remove the journal view #11962
- vulnerabilities: expose withdrawn state on vulnerabilities #12443
- warehouse: add initial pending OIDC provider models #12572
- python/cpython
- pypa/pip
- rust-lang/rust
- rust-lang/rustc-dev-guide
- rust-lang/rustfix
- sarugaku/resolvelib
- snipe/snipe-it
- vehemont/nvdlib
- vityafx/serde-aux
- zulip/zulip
Software testing tools
- AFLplusplus/AFLplusplus
- AFLplusplus/LibAFL
- assert-rs/trycmd
- cgaebel/pipe
- GaloisInc/FAW
- garyttierney/intellij-ghidra
- google/gofuzz
- google/oss-fuzz
- googleprojectzero/weggli
- gstrauss/mcdb
- Homegear/Homegear
- jkrh/kvms
- kometchtech/docker-build
- lief-project/LIEF
- Manishearth/compiletest-rs
- microsoft/binskim
- ned14/quickcpplib
- NationalSecurityAgency/ghidra
- pwndbg/pwndbg
- update unicorn to 2.0.0 #1034
- colorful tip of the day #1046
- Fix aarch64 regs display #1054
- Fix context args crash on missing instruction #1055
- Remove shell commands registration #1064
- Improve search –next speed and add –trunc-out flag #1066
- Revert “Remove shell commands registration” #1073
- small refactor of vmmap module #1078
- Fix coredump debugging #1079
- Revert “Refactor heap code” #1084
- fix vis_heap_chunk test on CI? #1086
- Fix heap test binaries build #1087
- Remove QuietSloppyParsedCommand once and for all #1091
- tests.sh: add [filter] and –pdb #1092
- black all da code #1103
- fix #1098: dX cmds trunc out on x86 binaries #1104
- vmmap: use pwndbg.info.auxv instead of gdb.execute #1107
- ArgparsedCommand: fix `help cmd` and `cmd –help` behavior #1108
- improve start and entry commands description #1109
- fix errno command #1112
- fix #1111 errno command edge case #1126
- fix distance command #1146
- fix qemu vmmap showing coredump mappings #1148
- Improve vmmap on coredump files #1149
- add patch command #1150
- Fix #1153 nextproginstr command #1158
- Show arch and emulation status on disasm banner #1160
- Fix #1165: set context-clear-screen on resetting scrollback #1166
- silence heap_bugs.c build warnings #1172
- Enhance heap with for static-linked binaries & remove typeinfo bloat #1173
- search command: remove unused string optional arg #1180
- Fix disable_colors formatting & test ctx disasm showing fds #1186
- fix #1190: telescope -r with addr as count #1198
- tips: add set show-flags on tip #1200
- add show-flags and show-compact-regs to ctx regs banner #1201
- remove defcon.py #1203
- bugreport command: use code listings #1204
- Bump gdb pt dump #1205
- Delete .sublime-settings #1206
- Update README with GDB build steps #1220
- fix #1221: ipi command multi-line inputs #1222
- events.py: remove unused Pause class #1223
- Fix #1197: dont display ctx on reg/mem changes #1239
- allow setting gdblib.regs.= #1267
- Fix #1256: fixes next cmds hangs on segfaults #1268
- Fix #1189: fixes patch command’s arch=… value #1269
- Pwndbg configuration: do not set history expansion #1292
- Fix parameter default values #1307
- Fix invalid zig path in tests makefile & suppress compilation warning #1308
- Increase CI timeout to 20 minutes #1309
- Fix setting empty ctx sections #1310
- lint.sh: lint only pwndbg files #1312
- fix lint #1356
- tests.sh: del joblog if –keep not passed #1360
- Fix lexer for coloring negative numbers in asm #1367
- Merged #1351 PR: Run tests in docker images #1370
- Remove instr operands padding in enhance #1372
- tips.py: add tip about Pwndbg’s signal handling #1373
- Fix tests reporting in parallel execution #1379
- tests zig cc: silence unused vars warnings #1382
- Fix debian10 ci #1383
- fix test_loads_binary_with_core_without_crashing on debian10 #1389
- tests reference-binary.c: dont rely on connect to 1.1.1.1 #1390
- Fix vmmap coredump test #1391
- version.py: fix build_id after recent refactors #1393
- fix #1188: incorrect 32-bit syscall display on x64 #1407
- abi.py: don’t recreate ABI dicts #1408
- Fix #1399: cymbol command on old GDB #1409
- tests.sh: fix –pdb (set SERIAL when –pdb is set) #1410
- Fix archlinux ci tests #1411
- returntocorp/semgrep-rules
- rust-fuzz/book
- rust-lang/rust-clippy
- Change `unnecessary_to_owned` `into_iter` suggestions to `MaybeIncorrect` #8201
- Format `if_chain` invocations in clippy_utils #8370
- Fix some `unnecessary_filter_map` false positives #8479
- Add `unnecessary_find_map` lint #8489
- Fix `unncessary_to_owned` false positive #8509
- Add `crate_in_macro_def` lint #8576
- Extend `extra_unused_lifetimes` to handle impl lifetimes #8737
- Address `unnecessary_to_owned` false positive #8794
- Optionally allow `expect` and `unwrap` in tests #8802
- Improve “unknown field” error messages #8823
- Check `.fixed` paths’ existence in `run_ui` #8844
- Add test for #8855 #8857
- Fix `empty_line_after_outer_attribute` false positive #8892
- Fix `extra_unused_lifetimes` false positive #9037
- Enhance `needless_borrow` to consider trait implementations #9136
- Add `ui_cargo_toml_metadata` test #9216
- Fix `to_string_in_format_args` false positive #9259
- Further enhance `needless_borrow`, mildly refactor `redundant_clone` #9386
- Upgrade `compiletest-rs` dependency #9523
- Expand internal lint `unnecessary_def_path` #9566
- Fix bug introduced by #9386 #9635
- Fix `needless_borrow` false positive #9674
- Add `lintcheck` to packages linted by `dogfood` test #9691
- Improve `possible_borrower` #9701
- Fix `needless_borrow` false positive #9710 #9711
- Improve `needless_lifetimes` #9743
- Update CONTRIBUTING.md with changelog guidance #9753
- Address issues 9739 and 9782 #9791
- Fix #9771 (`unnecessary_to_owned` false positive) #9796
- Fix typo in `expect_used` and `unwrap_used` warning messages #9863
- Move `line_span` to source.rs #9873
- Use `walk_generic_arg` #9930
- Fix 10021 #10027
- S2E/s2e-env
- S2E/s2e
- unicorn-engine/unicorn
- Z3Prover/z3
Blockchain software
- coral-xyz/anchor
- ethereum/hevm
- Cleanup flake.nix, fix nix-build on macOS #34
- Cleanup GHC2021 standard extensions #42
- Code simplifications in EVM.SMT #43
- Performance: BA.unpack -> BA.convert #45
- Make Ethereum tests pass #56
- Move nixpkgs to unstable and clean up #72
- Change gas to Word64 to improve performance #73
- Remove sbv from cabal dependencies #88
- metaplex-foundation/metaplex
- OpenZeppelin/openzeppelin-contracts
- primitivefinance/rmm-core
- solana-labs/rbpf
- ton-blockchain/ton