Managing a project well when all the participants are volunteers presents some unique challenges. This is too large a topic to cover in a HOWTO. Fortunately, there are some useful white papers available that will help you understand the major issues.
For discussion of basic development organization and the release-early-release-often `bazaar mode', see The Cathedral and the Bazaar.
For discussion of motivational psychology, community customs, and conflict resolution, see Homesteading the Noosphere.
For discussion of economics and appropriate business models, see The Magic Cauldron.
These papers are not the last word on open-source development. But they were the first serious analyses to be written, and have yet to be superseded (though the author hopes they will be someday).