FreeBSD Project "ideas" List

Over the years, the FreeBSD Project has built up a list of ideas for implementation work that seems like it might be a good idea, but no hands are available to do it. If you would like to contribute to the FreeBSD Project, you might peruse this list to get a sense of the kinds of work available to do. Obviously, contributions are not limited to this list!

Please do contact us before starting on it though -- sometimes items remain on the list after they are completed, and sometimes they are just ideas, rather than a recipe for success. Searching our mailing list archives may turn up the discussions leading to ideas being put on the list. Frequently the initial goal would be to simply investigate the idea, rather than produce code. Many project ideas list contacts, who it is worth sending an e-mail. Otherwise (and perhaps as well), send e-mail to our hackers@ mailing list.

For Google Summer of Code students:

For the list of Google Summer of Code ideas, visit: SummerOfCodeIdeas

Please note: ideas on this list are generally considered unsuitable for Summer of Code projects for many possible reasons (too large, too small, no available mentor, etc) but there may still be ideas here that could form the basis of a good GSoC task.

Contents

  1. FreeBSD Project "ideas" List
  2. Wireless Projects
  3. Embedded Projects
    1. Reduced FreeBSD kernel size for embedded
    2. Make creating a bus easier
    3. Variable hints
    4. ARM cleanup
  4. File System Projects
    1. Improve the performance of dump/restore
    2. Filesystem decompression layer
    3. Bring back the RAIDframe port
    4. OpenBFS
    5. Porting HFS+
  5. Kernel Projects
    1. Document all sysctls
    2. Document the sound subsystem
    3. Kernel fuzzing suite
    4. DTrace
    5. DWARF2 call frame information (GSoC 2011)
    6. Implement support for kernel Address Sanitizer
    7. Suspend to disk
    8. Sync FreeBSD i386 boot code with DragonFly
    9. Solaris Doors IPC Implementation
    10. Userspace mount() implementation
  6. Virtualization Projects
    1. bhyve gdb-stub/dcons integration
    2. VirtualBox shared folder support for FreeBSD guests
  7. Networking Projects
    1. SCPS, Space Communication Protocol Standards
  8. Porting Projects
    1. Port FreeBSD to new platforms
    2. Port FreeBSD on Tablet device
  9. Testing and Continuous Integration projects
    1. POSIX compliance testing framework
    2. Travis Continuous Integration Support for FreeBSD
  10. Userland / Installation Tools Projects
    1. Switch procstat from subcommand flags to verbs
    2. BSD-licensed ELF Tools
    3. BSD-licensed Text-Processing Tools
    4. NDMP data server
    5. Port prebind from OpenBSD
    6. Proxy auto-config file support for libfetch
    7. PXE Installer
    8. Improve cron(8) and atrun(8)
    9. libpw
    10. resurrect memory leak detector libmprof
    11. Safe crash dumps
    12. Import syslogd improvements from NetBSD
    13. Add support for usbdump file-format to wireshark and vusb-analyzer
    14. Support for setting base system build options via dialog(1)
    15. RAID and disk monitoring suite
    16. Cross-building FreeBSD from Linux and/or Mac OSX
  11. Global Projects (may touch everything)
    1. EPUB Support in Documentation Build Infrastructure
    2. PerfVisor (PERFormance adVISOR)
  12. Other Projects

Wireless Projects

These are now on WifiIdeasPage.

Embedded Projects

Reduced FreeBSD kernel size for embedded

Technical Contact: imp@

Description

The FreeBSD kernel has been optimized over the years for a server or workstation environment. Memory is plentiful in these environments, so little attention was given to the size of the kernel. There's a number items in the kernel that can be made optional without reducing affecting the functionality needed in an embedded environment. These include things like not compiling in strings into the kernel, less agressively inlining code, making some non-optional features optional and investigating compile time flags. This task requires identifying potentially optional kernel content and building the infrastructure to make that content optional.

Requirements

Make creating a bus easier

Technical Contact: imp@

Description

There's about a dozen busses in the tree now that manage resources and activate children. They are far too hard to create. We need to abstract out the basics for these buses and provide a way to allow these buses to be a subclass of this new base class.

Requirements

Variable hints

Technical Contact: imp@

Description

Often times in the embedded world, you know what kind of built-in devices are on a SoC (System on a Chip) only because you know the specific model of that SoC. It is desirable to have a mechanism that code on these machines can use to load one of several sets of hints, which can then be used to populate the bus.

Requirements

* Good C programming skills

ARM cleanup

Technical Contact: imp@

Description

Adding a new board to the arm code is a lot harder than it needs to be. A lot of benefit could be had by creating tables for memory ranges, etc, and having more generic initialization code. Much of this can also be Machine Independent (MI).

Requirements

File System Projects

Improve the performance of dump/restore

Description

A performance evaluation of the split cache (as is) and an unified cache (like e.g. NetBSD) would be interesting. More details in this mail to the hackers mailing list. Additional improvements are welcome too.

Requirements

Filesystem decompression layer

Solaris 10 and newer provide dcfs; a read-only decompression stacking filesystem layer for UFS. Files are initially compressed by a userland fiocompress utility. The filesystem layer is very simple, it is implemented in single file that permits transparent decompression without the end user knowing if the file is compressed or not. While the implementation is really simple it is very useful by making possible to install in systems with little memory or for quick compression of files in typical read-only directories like /usr/bin. While the Solaris fiocompress implementation uses zlib, it would be easy to use more modern algorithms like lz4 or snappy.

References: File-System Development with Stackable Layers

Bring back the RAIDframe port

https://people.freebsd.org/~scottl/rf/

Description

NetBSD has a software RAID framework that is very useful for prototyping or otherwise implementing RAID solutions. Around FreeBSD 5 there was an heroic effort to port it, hovever, the intrfaces to devices and disks was changed and the port never caught up so the code was removed from FreeBSD in Revision 127066.

Requirements

OpenBFS

Practical Filesystem Design Book

Description

Before working at Apple, Dominic Giampaolo wrote a 64 bit journalling filesystem for BeOS. The objectives were to have a modern filesystem capable of streaming for multimedia applications. It also supports indexing and Extended Attributes for desktop use. Haiku has implemented a version of the filesystem based on the documentation which is used as their base filesystem and is available under an MIT license. Porting it, or rather using the existing implementation as the basis for a new one, would make an interesting case for an alternative filesystem.

Requirements

Porting HFS+

HFS Wiki

Description

The Hierarchical File System was developed by Apple Inc. for use in MacOS. With the Release of MacOS X it received many new features, and the source code was made available as part of XNU. An initial FreeBSD 5.3 HFS port was made and although it was subsequently abandoned and support for locking has to be added, it would be excellent reference material for an updated port. A port would also be a good reference for bringing other interesting filesystems from Apple's Darwin.

Requirements

Kernel Projects

Document all sysctls

Technical Contact: mat@, brd@, eadler@

Description

The sysctl(8) utility retrieves kernel states and allows processes with appropriate privilege to change kernel states. On request it is able to display description lines which document the kernel state. Unfortunately not every sysctl is documented. This task is possible to share with other volunteers. mat has done some development in Perforce, in the mat_sysctl_cleanup branch.

Requirements

Document the sound subsystem

Technical Contact: netchild@

Description

Requirements

Kernel fuzzing suite

Description

FreeBSD's memguard(9), and the compiler stack protection offer a good framework to detect memory leaks and buffer overflows in the kernel and the complete OS is frequently checked with static analysis tools, but we lack kernel specific fuzzer testing tools to aid in such detection. Originally the linux Trinity fuzzer was the main example of such tool, Dmitry Vyukov's syzkaller is somewhat more promising, and as of lately there is also the TriForceAFL used for OpenBSD.

A native tool would be good but perhaps just running the Trinity tool under the linux emulator, along with memguard(9), would reveal general bugs in the kernel.

Reference: syzkaller for freebsd posting

DTrace

Technical Contact: rwatson@

Homepage: Perforce repository, DTrace for FreeBSD

DTrace is a dynamic tracing facility designed by Sun Microsystems and released in Solaris 10. They have since released the major part of Solaris under the banner of OpenSolaris and the Common Development and Distribution License (CDDL) 1.0.

Requirements

DWARF2 call frame information (GSoC 2011)

The DWARF Debugging Standard

Description

A debug kernel is not able to show stack traces with cross exceptions anymore. This is because we do not emit any dwarf2 call frame information for any assembler code, since gdb switched to the dwarf2 format. A volunteer should annotate every assembler file [*.[sS]] with dwarf2 call frame information. This was started as a GSoC but needs more work.

Requirements

Implement support for kernel Address Sanitizer

The kernel Address Sanitizer KAsan is a dynamic memory error detector developed by Google based on the similar userland tool. It provides a fast and comprehensive solution for finding use-after-free and out-of-bounds bugs. It requires instrumentation in the kernel.

Requirements

Suspend to disk

Implement a suspend/resume from disk mechanism. Possibly use the dump functions to dump pages to disk, then use ACPI to put the system in S4 or power-off. Resume would require changes to the loader to load the memory image directly and then begin executing again.

Requirements

Sync FreeBSD i386 boot code with DragonFly

Technical Contact: jhb@

DragonFly invested a lot of time to clean up and document it. Additionally they fixed some bugs. Interesting files in the DragonFly CVS are sys/boot/i386/bootasm.h, sys/boot/i386/bootasmdef.c, sys/boot/boot0/*, sys/boot/boot2/*, sys/boot/i386/btx/*, sys/boot/i386/cdboot/*, sys/boot/i386/libi386/amd64_tramp.S, sys/boot/i386/libi386/biosdisk.c and sys/boot/i386/loader/main.c. An interested volunteer has to compare and evaluate both implementations and port interesting/good parts.

Requirements

Solaris Doors IPC Implementation

Fast Sockets, An IPC Library to Boost Application Performance

An Implementation of the Solaris Doors API for Linux

Doors provide a mechanism for processes to issue remote procedure calls to functions in other processes running on the same system. The door APIs were developed by Sun Microsystems as a core part of the Spring operating system and were officially available in Solaris 2.6. They are extensively used in Illumos.

In addition to the Solaris/Illumos port, which is well documented, there is also an outdated implementation for linux that can serve for comparison. The project would consist in understanding how the existing code works and designing a completely new, but compatible, implementation for FreeBSD.

Requirements

- Interest in Inter-process Communications

- Ability to understand code and the existing implementations in Illumos and linux.

- Capacity to run your own tests and benchmarks.

Userspace mount() implementation

Technical Contact: brooks@

The mount() system call has a badly designed interface, is obsolete, and has largely been replaced by the nmount() system call. It should be straightforward to implement a user space wrapper that parses the passed structure and constructs an nmount() call. This should allow mount() to be removed on new architectures and space constrained systems.

Requirements

Virtualization Projects

bhyve gdb-stub/dcons integration

FreeBSD's 'bhyve' Hypervisor has a feature where it allows the kernel debugger to communicate with the outside world via a socket. Unfortunately it is quite slow, being polled one byte at a time. In addition it still uses the kernel on the VM to do all the work. Some avenues of improvement could include using the existing dcons memory buffer driver so that a whole buffer might be transferred at a time, or to make the current driver much faster. One might also intercept some of the commands (memory read for example) and perform them directly in the hypervisor so that cooperation of the virtualized system is not required to examine memory. It is even possible that by manipulating the processor flags appropriately, one could single step or 'break' the guest without its cooperation at all.

Requirements

VirtualBox shared folder support for FreeBSD guests

Technical Contact: lwhsu@, gonzo@, decke@, vbox@

Description

Oracle VirtualBox does unofficially support FreeBSD as host and guest operating system. VirtualBox shared folder support allows to access folders of your host from within the guest system. This is similar how you would use network shares in Windows networks except that shared folders do not need require networking, only the VirtualBox Guest Additions.

This task was part of GSoC 2013 where the main focus was on porting the code to FreeBSD. The result was almost working read only support but more bugs need to be fixed and read write support needs to be added and tested.

Requirements

Networking Projects

SCPS, Space Communication Protocol Standards

SCPS is a protocol suite designed to allow communication over challenging environments. Originally developed jointly by NASA and DoD's USSPACECOM, these protocols are used for commercial, educational, and military environments. A student project in this area would involve implementing various network protocols according to specification (SCPS File Protocol, similar to FTP; SCPS-Transport Protocol, based on TCP; and others.)

Note that European Space Agency has now an ESA Summer of Code in Space program and while FreeBSD is not a mentoring organization, interested students could motivate such a process.

References: Consultative Committee for Space Data Systems

Requirements

Porting Projects

Port FreeBSD to new platforms

Porting to new platforms is a good way to learn the internals of FreeBSD and serves to check the general portability of the base system. While there are important efforts to maintain an external toolchain working it would be ideal to start working on newer platforms that are already supported by the toolchain in base and where the hardware support is either easy to find or available through emulation.

References:

Requirements

Port FreeBSD on Tablet device

Porting FreeBSD to arm, intel tablets and next mobile device.

Requirements

Testing and Continuous Integration projects

POSIX compliance testing framework

Description

Standards compliance has always been one of the main objectives for the FreeBSD project. In the past we had some efforts to follow regular testing procedures but we haven't followed up the efforts with proper and sustainable infrastructure.

The Open Group has made some testsuites freely available. In particular there used to be a FreeBSD port of the TET testing suite, but this was removed due to a lack of maintenance and other issues. The lsb-vsx Linux testsuite should also be cleaned up so that together we are able to do regular testing on both linux-emulation and FreeBSD-native support.

Any compliance issue should be reported in Bugzilla and an attempt to contact the respective group should be made to draw a plan towards compliance.

Travis Continuous Integration Support for FreeBSD

Technical Contact: rodrigc@,

Description

Travis Continuous Integration is a very popular Continuous Integration system used by projects hosted on GitHub. If a GitHub project has a .travis.yml config file in the root directory, the Travis system will build and test the project if new code is committed to the GitHub project.

Currently, Travis only supports Linux and MacOS X. The Travis project closed issue 1818 which was a request to add FreeBSD support to Travis, due to lack of resources. However, having this support would be very useful, and allow FreeBSD to test many third party projects on GitHub.

Requirements

Userland / Installation Tools Projects

Switch procstat from subcommand flags to verbs

Technical Contact: brooks@

Description

The procstat command has a number of flags which are in practice subcommands. The single letter namespace means that new flags often have poor mnemonics and locally added commands often end up colliding with newly added values. Procstat should be modified to take command verbs in addition to existing flags. Argument handling should be table driven to make it easy to add arguments with minimal merge conflicts.

Requirements

BSD-licensed ELF Tools

Technical Contact: jkoshy@, kaiw@

Create BSD-licensed versions of ELF processing tools (e.g., ld, dbx, as and others) using the ELF(3) and GELF(3) API set. Identify overlapping functions in those tools and create a library out of the common functions. Identify parts which can be generated by tools (e.g., machine code parser generators) to support our Tier-1 and Tier-2 architectures.

References:

Requirements

BSD-licensed Text-Processing Tools

Part of Summer of Code 2010, Part of Summer of Code 2008

Technical Contact: gabor@

grep: It has been committed to the base system and available as an alternative of GNU grep. The compatibility is good but the performance is quite behind GNU grep, which prevents us from using it as a default. There are also some problems of regular expressions involved. It is under active development by gabor@.

diff/diff3/sdiff: Many command-line options are supported but some features are still missing. Maybe the three programs can be integrated into a single binary, this should be evaluated. A thorough performance benchmark should also be done. See SummerOfCode2012/JesseHagewood for last status.

mdocml: Some groff features are very hard to implement but they aren't strictly needed to render our man pages. Yet some manuals do not compile with mdocml. Investigate the reasons and create a migration plan.

Requirements

NDMP data server

URL: The NDMP Initiative

The NDMP initiative was launched to create an open standard protocol for network-based backup for network-attached storage. Major commercial storage systems come with a compliant service. This allows major commercial backup systems to backup such NAS devices. Including a NDMP disk server into FreeBSD would allow to play nice out of the box (modulo some configuring) regarding backups in a corporate environment.

Requirements

Port prebind from OpenBSD

The OpenBSD prebind is a secure implementation of prelinking that is compatible with address space randomization. Prelinking allows to speed up application startup when a lot of libraries are involved. This should show a noticeable effect with e.g. GNOME/KDE.

Requirements

Proxy auto-config file support for libfetch

A proxy auto-config (PAC) file contains a JavaScript function "FindProxyForURL(url, host)" that determines which HTTP or SOCKS proxy, if any, to use to access a given URL. In most application the file may be specified manually or discovered using the Web Proxy Autodiscovery Protocol. Support for PAC files in libfetch would make fetch more versitle.

Supporting PAC files nominally requires a fairly complete JavaScript implementation. Google's V8 JavaScript engine is BSD Licensed, however it compiles code to native machine code so platform support is an issue. However, the parser etc may provide a good starting point, and other engines may also exist and should be evaluated. A minimalist implementation of the language with commonly used constructs such as if/else, string comparison, and functions would be sufficient in many cases.

References:

Requirements

PXE Installer

It would be great to have a bundled PXE installer. This would allow one to boot an install server from a FreeSBIE live CD-ROM on one box, set the BIOS on subsequent boxes to PXE boot, and then have the rest happen by magic. This would be very helpful for installing cluster nodes, etc.

m@ is working on a bundled PXE installer as part of his BSDInstaller project within the Google Summer of Code 2006. The PXE Installer is working but some non-PXE related issues have to be solved before it can enter the tree.

Requirements

Improve cron(8) and atrun(8)

Currently, cron(8) and atrun(8) are outdated in their implementation. Here are some directions for improvement:

Requirements

libpw

Technical Contact:

Create a library to be able to manage users/groups easily, it also should have a pam/nss-like plugin framework for different account system.

libutil has pw_* and gr_* undocumented functions which allow user/group manipulation

writing pw_*() and gr_*() manpages

known users for that library: pkgng, pw(8)

Requirements

resurrect memory leak detector libmprof

Technical Contact: julian@

There used to be, many years ago, a port called mprof, which would do one thing, but do it very well: Find memory leaks. Unfortunately it has bitrotted and no longer works. It reads symbols from the executable, and correlates it with a detailed memory allocation trace to produce a very useful memory allocation leak descrition.

Requirements

Safe crash dumps

Technical Contact: gavin@

Crash dumps are important for collecting debugging information but are also disabled by default because they can consume much space in /var if the user doesn't pay attention to them.

What we miss is a safe way to enable crash dumps by default without having to worry about them filling up /var.

Requirements

Import syslogd improvements from NetBSD

Technical Contacts: emaste@, markj@

Note - there is work in progress on this project, please contact Ed or Mark for details.

NetBSD's syslogd has a number of improvements from a former Google Summer of Code project available for porting:

The changes are in NetBSD's repository in src/usr.sbin/syslogd, available for viewing on their cvsweb:

Requirements

Add support for usbdump file-format to wireshark and vusb-analyzer

Technical Contact: hselasky@

Support for the usbdump file-format has now been added to wireshark:

https://wiki.freebsd.org/SummerOfCode2017/usbdump-wireshark

Support for setting base system build options via dialog(1)

Technical Contact: ??

Our ports support setting build options via dialog(1) for ages. Recently, with the pkgng invention, it was made possible also to pack port's build options into binary package - later, a functionality to track dependencies based on the build options may be added.

Our base system build infrastructure now prefers consistent /etc/src.conf options in favor to older ad-hoc /etc/make.conf options. Currently, there are only binary ones, thus possible to map to dialog(1) checkboxes. The idea is to be able to do cd /usr/src && make config to see familiar dialog(1) interface - just as in any port in /usr/ports (it should be complemented with usual make showconfig and make rmconfig, of course).

This idea, amongst direct simplification of average user's life, is valuable in regard to possible future packaging of the FreeBSD base system to one or a few pkgng packages. Such packaging is desired in some environments of large production server farms for easier managing/upgrading servers, see https://github.com/z0nt/pkg for such a project. In the future, however, this could be useful for tracking pkgng packages dependencies based on world's build options (e.g. a port requires a world built with WITH_IDEA or not built WITHOUT_PF, etc.).

@bapt: note that it is already possible to store the informations about the options used to build base using the actually framework and without the need of the dialog(1) interface.

Requirements

RAID and disk monitoring suite

Technical Contact:

There have been several organizations that have independently developed RAID and disk failure monitoring tools. These should be gathered together into a unified group of daemons and monitoring scripts to provide a consistent view of disk status.

http://svnweb.freebsd.org/base/user/sbruno/ard/ http://svnweb.freebsd.org/base/user/sbruno/mfid/ http://svnweb.freebsd.org/base/user/sbruno/mptd/

Requirements

Cross-building FreeBSD from Linux and/or Mac OSX

Technical Contact: brooks@

FreeBSD's build system is self-contained, but only really designed to be run on FreeBSD as it makes assumptions about the host platform and available tools. Being able to build a fully-functional FreeBSD userland and kernel from either Linux or Mac OSX without having to use FreeBSD inside a virtual machine would allow more people to make use of and build products out of FreeBSD easier.

The NetBSD approach of bootstrapping via build.sh is one way to go. Another is to make autoconf versions of the key build tools and ideally add them to debian/macports/homebrew/etc so users can install the set of things they need and build from there. The bmake program is typically already available (perhaps in a somewhat old form so it may be viable to use in the bootstrap process).

On Mac OSX an additional complication exists in that the default filesystem is case-insensitive. At least to begin with, creating a case-sensitive filesystem to do this work on is recommended. Some work has already been done here, and it may be desirable to build upon that.

Requirements

Global Projects (may touch everything)

EPUB Support in Documentation Build Infrastructure

Suggested Summer of Code project idea

Enhance the FreeBSD Documentation Project build infrastructure to generate EPUB format output suitable for eBook readers from such as iPads and Kindles.

Requirements

PerfVisor (PERFormance adVISOR)

Coordination: netchild@

The goal of this project is to get a tool which can analyze a system for performance bottlenecks and maybe even give some hints what to do next.

This is not a GSoC project. Maybe small parts of it can be done during a GSoC.

Prerequisites you should know/read to understand the project steps:

Project phases

Initial phase

Improvement phase

Visualization phase

Cloud phase (everything is cloudy nowadays...)

Datawarehouse phase (buzzword bingo!)

Discussion

The first item in the initial phase is a big item. It may be better to see the complete initial phase as an iterative or divide and conquer step. First determine some high level resources and process all steps of the initial phase on them. Then repeat the initial phase again by braking up the high level resources in more detailed items (e.g. first "all CPU in system" as one resource, second "each CPU in system" as a resrouce, third breaking up each CPU via hardware performance counters).

GSoC info

Each phase is a big project of its own. Do not expect to be able to do it during some weekends or during a GSoC. If you think you can, you did not get the full scope, think again. If you still insinst after rethinking, be our guest, but you better prepare a very good outline what you want to do when, how, at which level of detail (e.g. come with a list of resources for the first item in the initial phase), and also include what you will NOT do/cover/handle but could be related (e.g. CPU internal performance counters if you want to handle the CPU performance side of this). You already need to know FreeBSD (you use it already on your server or desktop since several months) and you shall not be afraid to ask questions or discuss on the mailinglists. This is not a project for 2h per day, if you are not motivated to spend time on this, you better chose a different topic.

This can be made "big enough" to be a project suitable for finals at your university (depending on the requirements of your university, with or without extending it over the end of the GSoC).

Other Projects

If you are interested in working on a project not explicitly mentioned above, you may want to contact one of the potential Technical Contacts below:

Additionally, there are a lot of interesting mailing lists that can be used when searching information about specific subjects.

For porting ideas go to WantedPorts and for Beginner tasks check out JuniorJobs

IdeasPage (last edited 2018-01-24 15:16:57 by PedroGiffuni)