• Status
• 2005

2005 Nov

* Make Network MPSAFE

• Make all network interrupt service routines MPSAFE.

* Update libedit * Update libedit from NetBSD * Import NetBSD's ftp client * Make tsleep/wakeup MPSAFE

• Make most of the tsleep/wakeup core code MP safe. On an SMP box, this code is actually more efficient then the old code.

* Update em(4) driver

* Sync with Intel's em(4) driver version 3.2.15

• Add support for 82571EB, 82572EI and 82573E adapters
• Nuke DBG_STATS related code and struct adapter's fields
• Flow control water marks are now calculated from hardware's register value instead of using hardwired values(FC_DEFAULT_{LOW, HIGH}_THRESH)
• Flow control pause time is bumped to 512us (old value is 131us) (*)
• In em_detach(), call bus_teardown_intr() if adapter.int_handler_tag != NULL
• Update Intel's LICENSE and README
• Update README to reflect recent changes to device polling
• Minor style changes

* Allow 'options SMP' *WITHOUT* 'options APIC_IO'

• Allow 'options SMP' *WITHOUT* 'options APIC_IO'. That is, an ability to produce an SMP-capable kernel that uses the PIC/ICU instead of the IO APICs for interrupt routing. SMP boxes with broken BIOSes (namely my Shuttle XPC SN95G5) could very well have serious interrupt routing problems when operating in IO APIC mode. One solution is to not use the IO APICs. That is, to run only the Local APICs for the SMP management.

* Start removal of FreeBSD package system

• Move the freebsd package system from /usr/sbin to /usr/freebsd_pkg/sbin and the manual pages to /usr/freebsd_pkg/man.
• Adjust the upgrade target to remove the old pkg_* binaries and manual pages from their original locations.
• Adjust the nrelease build to access the FreeBSD packaging tools via /usr/freebsd_pkg/sbin.

2005 Oct

* Update em(4) driver

* Sync with the FreeBSD em(4) driver

• Add support for Intel PRO/1000 GT Desktop Adapter
• Add support for Intel 82546GB dual port PCIE adapter
• Add fix for 82547 which corrects an issue with Jumbo frames larger than 10k
• Add fix for vlan tagged frames not being properly bridged
• Correct TBI workaround
• Correct LED operation issues
• Do not count RNBC (internal buffer full) towards the RX error count, since it's not really an error
• Correct a workaround that should only be applied to one adapter. Workaround was causing device hangs when incorrectly applied to other adapters (*)
• Run em_local_timer() once per second instead of running it once per 2 seconds. This makes gathering of stats more precise

Userland 1:1 threading changes

• Move thread-local members from struct proc into new struct lwp.
• Add a LIST_HEAD(lwp) p_lwps to struct proc. This links a proc with its lwps.
• Add a td_lwp member to struct thread which links a thread to its lwp, if it exists. This won't replace td_proc completely to save indirections.
• For now embed one struct lwp into struct proc and set up preprocessor linkage so that semantics don't change for the rest of the kernel. Once all consumers are converted to take a struct lwp instead of a struct proc, this will go away.
• Convert functions to use struct lwp instead struct proc.
• Move profiling and intervall timer data to struct proc.
• Switch the userland scheduler to use lwps instead of procs.

2005 Sep

* Enable wide character support in ncurses

* Port the rue(4) driver from FreeBSD

• This driver supports: RealTek RTL8150 based USB ethernet devices

  • Melco LUA-KTX
  • GREEN HOUSE GH-USB100B
  • Billionton ThumbLAN USBKR2-100B

* Update to groff 1.19.2 * Added axe(4) driver

• Provides support for USB Ethernet adapters based on the ASIX Electronics AX88172 USB 2.0 chipset.

* Continued Jscan development (part 6 of ?)

• Rework and expand the algorithms in JSCAN, part 6/?. Preliminary support for UNDO (-u) on mirrors. We now have the ability to roll a mirror forwards or backwards by a specific number of raw journaling records (-c), or to roll it backwards completely. However, only basic remove and write UNDOs currently work, UNDOs for other transaction types and UNDO's that start mid-transaction are untested.

* Continued Jscan development (part 5 of ?)

• Rework and expand the algorithms in JSCAN, part 5/?.
• Revamp the scanning code, cleaning up the API considerably and allowing forward or backwards reads relative to a supplied record, which makes scanning a lot more intuitive.
• Implement the -c option to specify the number of raw records to process. This is primarily intended for running a mirror forwards or backwards by a certain number of records.
• Implement mid-transaction recovery. Mid-transaction recovery is needed when a batch operation has stopped at a raw record that is in the middle of a meta-transaction (a meta-transaction can consist of many raw records). Stoppage can occur for many reasons, such as the batch being run while the system is still writing out the records related to a meta-transaction, or due to journal restarts, unexpected program termination, or command-line limited record counts with -c.
  • This feature has been tested a little with -m -c 1 to iterate a mirror forwards in time one raw record at a time across a large WRITE transaction that covers many raw records. It is a pre-requisite for being able to iterate mirrors forwards and backwards in time.

Continued Jscan development (part 4 of ?)

* Rework and expand the algorithms in JSCAN, part 4/?.

• Fix a bug in the handling of PAD records. PAD records are special cased records in that the transaction id field may not be valid. This occurs because a PAD record must be able to fill up the alignment space, which is only 16 bytes. The journal_rawrecend structure winds up overlapping the transid of the journal_rawrecbeg structure in such cases. We must skip such records.

• Fix a bug in jfile->jf_pos tracking when doing forward scans. Since we supporting being able to scan forwards or reverse from any point, we have to properly track the file seek offset for forward scans.

Tested with /usr/obj recorded (-r) and mirrored (-m) through a buildworld. A diff -r after the buildworld completed showed a perfect copy! The buildworld with both forward and reverse (undo) journaling enabled generated 24 50MB journal files (1.2 GB of journal data). Continued Jscan development (part 3 of ?)

Rework and expand the algorithms in JSCAN, part 3/?.

• Have the debug dump print out raw record transaction id's.
• Fix a number of bugs in the prefix set code (the files were being strtoul'd in base 10 rather then base 16).
• Make multiple commands on the command line work (e.g. -w and -m together).
• Fix a few jseek() bugs.
• Add '-f' which operates like tail -f and allows a batch operation on e.g. a prefix file set to poll for more data.

• Still TODO: ACK protocol, raw output (-o/-O) protocol, temporary mode for record files (-W), and many other things. Continued Jscan development (part 2 of ?)

Rework and expand the algorithms in JSCAN, part 2/?.

• Implement session tracking so multiple output commands can be independantly tracked. However, currently only one output command may be specified (e.g. -w or -m, not both, yet)
• The session code now updates transaction id files, giving us a general restart capability on the userland side.
• Prefix set record mode (-w) should now work, including journal restarts.
• Running a mirror (-m) on an active prefix set should now operate properly in batch mode.

• Many things are still not working yet.

* Update OpenSSH to 4.2

* Continued Jscan development (part 1 of 2)

• Implement a new infrastructure in the userland jscan program to create a framework for supporting the following:
  • Pipe-throughs (without storing data in a buffering file).
  • File-based buffering
  • Streamed replication of journaled data.
  • Sequenced output files limited to a specific size (to match against backup media)
  • Transaction id tracking to support journal restarts and resynchronization. This is needed to make off-site and networked backups reliable.
  • Decoupled journal operations. e.g. (once finished) we will be able to run a jscan in realtime to record journal data, and another one in batch to take that data and use it to maintain a mirror or to send it to another machine in a restartable and resynchronizable manner.
  • Infrastructure to be able to scan a journal forwards or backwards by an arbitrary number of raw records and maintain a mirror in both the forwards (redo) and backwards (undo) direction based on that ability.
  • Infrastructure that will aid in the preferred backup methodology of maintaining a real time mirror and UNDO/REDO journal on a 100% full backup partition such that one is able to free space on the partition simply by removing the oldest journaling file.
  • Infrastructure that will make it easier to solve the overlapping meta-transaction problem.
  NOTE: this is just part 1/2. This code and the new options are NON WORKING at the moment.

* K&R to ANSI C

• Started working on making the code to be ANSI C compliant.

* Update to groff 1.19.1

2005 Aug

* Machine-Independent Spinlock Implementation

• It has the advantages of
  • being written in C except for the most low-level atomic swap primitive, which is universally supported on current processor architectures
  • having a very small inlined memory footprint for spin_lock(), with the slow-path defered to a subroutine call
  • only requiring a bus-locked operation for lock acquisition, and not requiring a bus-locked operation for lock release
  • doing a non-bus-locked check first in the spin loop to reduce bus contention
  • doing exponential backoff in the uncommon contested case, which Sun has found to reduce bus contention by a factor of 5 or more

* Update to TexInfo 4.8

* USB bugfixes

• Allow 32 chars in the saved vendor string. Fixes kern/29760. Also, add more vendor ID's and identification strings. Add suspend/resume support.

* Filesystem journaling

• Implement UNDO records, including writing out file data that is about to be overwritten and the mtime prior to a write, rename, or remove. UNDO records allow us to create a reversable journal, where we can wind a mirror forwards or backwards simply by scanning the journal forwards or backwards.

* Update the TWE 3ware/AMCC driver

• bringing in all the fixes made by Vinod Kashyap and Paul Saab in FreeBSD-5/STABLE, and retaining those by David Rhodus and Matthew Dillon.

* Remove PC98 Support

* More Broadcom Support

• Added more Broadcom ID's for 5705K, 5714C, 5721, 5750, 5750M, 5751M, 5789 chipsets.

* Remove old ACPI code

• Get rid of 4.x-derived ACPI code.

* VESA support under VMWare

• Allow the VESA code to handle devices that don't claim to be VGA devices. This fixes VESA support when running under VMWare.

2005 Jul

* Update Sendmail to 8.13.4

• Bring DragonFly's sendmail infrastructure up to date, including updating to version 8.13.4 which includes native DragonFlyBSD support.

* Added shlock(1)

• A program for the safe creation of lock files from shell scripts.

* IPI messaging code bugfix

* Fix a critical bug in the IPI messaging code, effecting SMP systems.

* Switch to zlib 1.2.3

* Switch to zlib 1.2.3 and remove the fix for the inflate bug since it's already included.

* USB bugfixes

* Fix numerous extremely serious bugs in OHCI's iso-synchronous code.

* Journaling Restart Support

• Add journaling restart support, required to produce a robust journaling environment. If a journal is writing to one stream and the stream breaks or dies or otherwise fails, this feature gives us the ability to restart the journaling stream on a new descriptor without losing any data. The journaling restart code does a shutdown() of the old descriptor, waits for both directions to cease operation, installs a new stream descriptor, and resets the FIFO index to the last acknowledged offset.

* Import OpenSSH-4.1p1 * Import OpenPAM Figwort and switch to OpenPAM. * Updated OpenSSL to 0.9.8 * Bug fix in zlib

• Fix possible buffer overflow in zlib (FreeBSD-SA-05:16.zlib).

* Update to file-4.14 * Continued work on jscan

• Major continuing work on jscan, the userland backend for the journaling system -- bug fixes, support for unknown nesting record sizes, use a 64 bit integer to calculate the completed transaction size, and implement (partial) support for mirroring.

* Update mailwrapper

• Merge various changes from NetBSD, OpenBSD and FreeBSD. Make error messages that originate from mailwrapper more understandable.

*Citrus sync and bug fixes

• Syncing up with NetBSD's Citrus, add some functions to Citrus, and fix bugs in Citrus library.

* Add Broadcom Support

• Add support for Broadcom BCM4401-B0 chipset.

2005 Jun

* Scheduler changes

• An infrastructure for multiple schedulers in the kernel has been added. This is the prerequirement for runtime selectable schedulers. At the same time a number of bugs in the 4.4BSD derived scheduler have been fixed, effectively reverting it almost back to the 4.4BSD version.

* Improved file descriptor allocation

• The algorithm to determine the next free file descriptor of a process has been replaced with a shiny new implementation inspired by Solaris. The old algorithm needed a linear search which can be very slow for processes with a high number of open files, the new algorithm scales in logarithmic times.

* Improved Kernel Tracing infrastructure

• The ktr(4) facility has been revamped to be more extensible and easier to use. It now supports an unlimited number of facilities and much easier writing of the interesting information.

* MBUF cache

• The MBUF allocator has been changed to use a generic object cache as backend. This allows utilizing pre-initialized data structures and CPU-local allocations.

* SPL removal

• The old interrupt deferral code based on spl*() has been removed. It was used in the BSD kernel to selectively mask interrupts of a specific class without having to physically disable them. On current CPUs like the Pentium 4, masking the interrupts in hardware is very expensive compared to running the trap gate and returning directly when the interrupt can't currently be handled. Instead of the SPL model, critical sections are now being used and will be replaced with more fine-grained protection mechanisms in the longer run.

* GCC 4.0 added

• Since GCC 2.95.4 has been retired a while ago, GCC 4.0.0 was chosen to replace it as the alternative system compiler.

* Support for multiple time counters

• The time counting infrastructure in the kernel has been extended to allow changing the primary counter at boot time. Available counters are ACPI counter (default) and i8254 counters used before.

2005 Apr

* ELF ABI tagging

• The kernel ELF loader supports the .note.ABI-tag, which specifies the OS kernel and version a program was developed for. This makes the old tagging with brandelf obsolete and allows running Linux binaries without any changes.

* libc_r fixes

• A number of bugs in libc_r have been fixed. The kqueue override was incomplete which could be observed e.g. in Firefox when the contacted DNS server timed out.

* GCC 3.4 update

• The in-tree GCC has been updated to the 3.4.4 release version.

* Multiple boot profile support

• The support to choose between different configurations on boot time has been merged from NetBSD. This is very handy for notebook users to switch between a roaming profile and a stationary configuration. A profile consists of various configuration files like /etc/rc.conf or /etc/inetd.conf, which are stored in subdirectories and replace the normal version based on the chosen profile.

* Add support for TCP Appropiate Byte Counting

• Support for RFC 3465 has been added, which improves the TCP congestion window handling. The changes improve both the reliability and the performance when dialing with delayed ACKnowledgements.

* PCVT removal

• The old PCVT code has been completely removed. It hasn't been used for ages and syscons provides most, if not all of the functionality.

* GCC 4.0 compatibility

• A lot of smaller changes have been put into the tree to make GCC 4.0 happy. Since these often fix GNUisms, it also brings us nearer to ISO C.

* Libc work started

• The major libc revamp started with the switch to Citrus for the I18N support. This means full wide char support, integrated iconv support and more.

* DragonFly 1.2.0-RELEASE

• DragonFly 1.2 was officially released on Apr 8.

2005 Mar

* TLS support

• The support for Thread Local Storage in libc, libthread_xu and rtld has been finished. The pieces needed for libc_r will be integrated after the release. One important use of TLS is errno in multithreaded programs. Traditionally it had to be implemented as function in libc_r, but now it can be made a thread local variable, which is faster and easier to access.

* UDP sockbuf optimization

• DragonFlyBSD now keeps a hint of the last packet in a record-oriented socket buffer. This turns what was once a O(n) operation to append a new incoming packet to the end of the socket buffer into a constant-time O(1) operation.

* MSFBUF API Update

• The MSFBUF API has been updated by Hiten Pandya and Matthew Dillon so various types of data holders can be mapped. The new API contains functions for mapping UIOs, arbitrary buffers, raw list of VM pages and XIOs.

* Ncurses

• Ncurses has now been updated to version 5.4. See http://www.gnu.org/software/ncurses/ncurses.html for changes.

* More make(1) Updates * 4.4BSD papers

• The old 4.4BSD papers will not be built any more when doing buildworld. This removes dependancy on Groff from the list of build tools.

* Removal of PC98 and Alpha Support

• Sascha Wildner is slowly and steadily removing support and conditionals for the PC98 and Alpha platforms from the kernel code and also userland utilities.

* ACPI Drop 20050211

• Yonetani Tomokazu has integrated the latest ACPI drop, 20050211.

* I18N Framework

• Joerg Sonnenberger is working on integrating code from the Citrus project. In summary, the Citrus project has a state of the art Internationalisation framework for *BSD OSes.

* Journalling Update

• The journalling code has undergone major updates from Matthew Dillon. A userland utility called jscan has been added which understands journalling streams and it is able to read logs.

* Brooktree Driver Update

• The bktr driver has been updated by Steve O'Hara-Smith to match the driver in FreeBSD 6-CURRENT.

* TCP/IP stack Updates

• The TCP/IP stack has undergone various updates and bug-fixing from Jeffrey Hsu and Noritoshi Demizu.

* Namecache API

• A critical fix has been applied to the nlookup API. The wrong link field in the nlookup structure was being traversed.

* CVS 1.12.11

• CVS has been updated to version 1.12.11 by Simon 'corecode' Schubert.

2005 Feb

* Linuxulator DRM fix

• Brock Johnson fixed issues with the IOCTL wrapping, so that Linuxulator's DRM works again. Happy Gaming on DragonFly!

* Modules with kernel options

• Simon 'corecode' Schubert committed a patch that allows modules to be built with the corresponding kernel options if building them both together (which is what usually happens on buildkernel)

* ALTQ QoS for networking

• DragonFlyBSD has incorporated ATLQ into the base system. ALTQ provides Quality of Service for IPv4 and IPv6, with several user-selectable policies, including Class-Based Queueing (CBQ), Weighted Fair Queueing (WFQ), the H-PFQ and H-FSC Hierarchical Packet Schedulers, and RED (Random Early Detection) Queue Management. Among its many applications are giving priority to interactive traffic on ADSL lines, which makes them useable even with a bulk data transfer running in the background, and assigning different priorities to different classes of customers for a network service. DragonFlyBSD's version was merged from KAME. ATLQ is currently controlled through the PF userland utility. Support for administrating ALTQ from ipfw will be added soon.

* Jail support

• jls(8) and jexec(8) from FreeBSD have been ported.
• Only filesystems mounted inside the jail or the filesystem containing the jail root are visible in the jail.
• Kernel threads are hidden.

2005 Jan

* DFWiki

• We are now official. wiki.dragonflybsd.org Yea!!!

* WARNS6

• Currently a group of people are cleaning up sources in bin, sbin, usr.bin and usr.sbin to compile with WARNS=6

  See Base Cleanup for more information.

* Apache support for DragonFly

• APR (the Apache Portable Run-time (http://apr.apache.org/)) will be DragonFly supporting per 1.1.0. As reported back: "Commited to trunk in r125880."

* usr.bin/make (Max Okumoto)

• Sync with FreeBSD make completed.
• Cleaning up the code so we can find bugs.
• Find out why turning on debugging changes the behavior of make.
• Start pushing around Buffer objects instead of char *.
• Started add regression test infrastructure.

* Rewrite of BSD routing code

• The crufty hard-to-read and buggy BSD routing code has been revamped for clarity. This will make it much easier to use DragonFlyBSD as the basis for a router or a network appliance.

* 1:1 thread support

• Work has started on kernel support for user threads. This will allow a multithreaded application such as Mozilla or Apache2 to run on two or more CPUs simultaneously. Without kernel support, a multithreaded program can only multiplex all its threads onto one CPU at a time.

  In contrast to the M:N threading model adopted in FreeBSD, our thread library adopts the same 1 kernel thread for 1 user thread model as in Solaris and Linux. (Through the DragonFly variant symlink feature, in the future, we will be able to easily switch between multiple thread libraries on the same system without recompiling or relinking.) According to the original author, the 1:1 thread library we're using is faster than anything in FreeBSD. Update: The thread library was committed on Feb 1, 2005. Work continues on the kernel support for signals and per-thread pids.

* New commiter Hiroki Sato

• We are pleased to announce that Hiroki Sato has joined the ranks of DragonFly committers. Sato-san has long been active in Open Source projects as a FreeBSD committer. Sato-san will be working on keeping the quality of our IPv6 stack high by merging or implementing any missing features from KAME and other BSDs. He is also interested in i18n framework and documentation.

* O(log n) search for vm

• Eirikn just submitted a change to the DragonFly BSD VM code to convert a linear search to O(log n). This helps out large applications such as databases which mmap() thousands of regions.

* Trolltech adds supports for DragonFly to Qt

• As indicated in the release notes for Qt 3.3.4, Trolltech has rolled native support for DragonFly BSD into their Qt graphical toolkit that is used by KDE. http://www.trolltech.com/developer/changes/changes-3.3.4.html