Hosting a Server

freenode urgently needs solid, well-connected hidden hub and public leaf servers. We look for multihomed T3 connections or better, in order to ensure high reliability and low latency. Actual bandwidth usage runs about 45-75Kbits/sec average for hubs, and 450-750Kbits/sec average for leaf servers. This comes to 15-25 gigabytes per month for hubs and 150-250 gigabytes per month for main rotation leaf servers.

We appreciate your interest in helping the network grow and your willingness to contribute to that process. But it's important to realize that freenode is not a conventional IRC network. Servers are hosted, not linked. They are maintained remotely from central sites using ssh and scp. New configuration files are generated via an automated script at a central location, and new servers are either staged as binary files or recompiled on the server account, depending on architecture and other factors.

We do not provide O-lines or operator privileges to our server hosts, though we happily acknowledge your contribution publicly. Facilities host staffers may apply for freenode staff duty; their applications receive the same consideration as those of other users. When a staff application is accepted, no minimum level of privilege is guaranteed or offered. Information on network operations is provided here. Hosting candidates are reminded that the utility of any given server depends on ping latencies and other factors which may be beyond our control. Servers may need to be delinked or rehubbed at any time. We cannot always provide notice before a server is delinked.

Regardless, hosting a freenode server helps the network and the community and we appreciate that you're considering it.

Hardware and Bandwidth Requirements.

The listed hardware and bandwidth requirements are extremely generous. However, the network has expanded since its inception and will likely do so for the forseeable future. Our goal is to ensure that each server exhibits the best possible connectivity and responsiveness.

Minimum bandwidth. Multihomed servers with T3 or better bandwidth are preferred.
Processor. A 1GHz processor or better. Generally, Intel or compatible hardware makes life a bit easier for us, but it's not a hard-and-fast rule.
Memory. 768MB or more of RAM is required.
Disk space. Space usage for logs may vary. In general, it's best to leave appreciable amounts of free space.
Hours of operation. 24-7, reliable operation is a must.
Multiple uses. Dedicated servers are generally preferred. In particular, we advise against running critical applications on the same host; if you do, please plan to closely monitor resource usage (including allocation of memory and processor time). We attempt to provide timely notification of problems, but we're a volunteer organization running a 24/7 network and staff are not always available as needed.

Software-Related Requirements.

The following list details the software-related requirements for freenode servers. Again, some attention has been paid to providing room for expansion with minimum effort on the part of hosting organizations.

FreeBSD or GNU/Linux. Current servers run either FreeBSD or GNU/Linux. Other excellent open source / free software operating systems exist, but the use of these two operating systems simplifies network operations. FreeBSD 4.4 or later or the Linux 2.4.23+ kernel are required. Unfortunately, we cannot currently accept servers which use jails under FreeBSD.
Vixie crond. The sometimes-documented Vixie crond @reboot time specification is used to restart the server at boot time, when crond is started.
Core files. Expect extremely large core files on occasion. Core file size limits should be generous. Our ircd code base continues to evolve so we will need this debugging resource.
A running network time daemon. Preferred daemons are either xntpd or ntpd, with local access via {,x}ntpdc to get server status. The server host should be synced to a time server at all times and should have backup servers. It's recommended you specify groups of three servers (with a minimum of at least one such group) to sync time from.

The ntp daemon uses port 37 (UDP and/or TCP) to sync with external time servers. We use port 123 on loopback to examine local server status. You'll probably want to work from this list of public stratum-2 time servers which serve your area. You'll need to follow the rules of engagement and notify the server owners you've started using their servers.

It's essential that you start up ntpd in your init scripts. It's also essential that you run ntpdate, part of the package, at startup before starting up ntpd. Use the same list of time servers for ntpdate as you use in your config file for ntpd. Without running ntpdate before starting up ntpd, ntpd will use an extremely slow software-PLL algorithm to bring your clock to an accurate time, which means that your server may not be able to connect to the network for some time after reboot.
Account access. The server daemon runs as a user program under a vanilla end-user account (frequently called "freenode"), with no setuid or special privilege. Our admins must have access to the account via password and ssh public key and must also have the ability to change the account password via the passwd command. Root access to the machine is highly desirable, since it makes it easier for us to double-check the initial configuration and resolve problems in an emergency.
Recent opensshd daemon, ssh2 protocol only; account access via ssh and scp. All access to the server account will use ssh. The scp program is required to transfer files. Access to the account by public key should be allowed, as should access by password.

Note: Recent opensshd provides a feature called privilege separation. This feature will not currently work properly with GNU/Linux PAM implementations. The symptom is that the method described below to set hard and soft maximum file descriptor limits will not work properly from an ssh session whose daemon is running privilege separation. So, when running on GNU/Linux, be sure to include the configuration statement "UsePrivilegeSeparation no" in your sshd configuration file (usually found in /etc/ssh/sshd_config).
The rsync program. File transfers may use this program.
Hard and soft file descriptor limits of 8192. At least 8192 spare file descriptors should be allocated system-wide for the server's use, if the operating system requires preallocation. Hard and soft process file descriptor limits of 8192 should be applied to processes started via crond, screen and ssh on the freenode account, and on GNU/Linux systems the easiest way to accomplish this in a selective fashion is to have PAM installed.

Setup for GNU/Linux:

First, allocate enough file descriptors systemwide so that there is no way you'll run out. The default on current kernels is 8192, and you should probably set yours to 16384 or 32768 to be safe, so we won't have to revisit this topic in the future. We need do the equivalent of this at startup:

echo 32768 >/proc/sys/fs/file-max

Many distributions use the sysctl(8) program at startup to automatically make changes to /proc/sys. If it's in use at your site, you can simply add the following line to /etc/sysctl.conf:

fs.file-max=32768

Second, set up larger file descriptor ulimits on the freenode account. There are several approaches. First check and see if you've got a file /etc/security/limits.conf. If so, add these two lines to it:

freenode soft nofile 8192
freenode hard nofile 8192

If you're running a distribution and release that are completely PAMized (such as Debian Woody), go to /etc/pam.d and add the following line:

session required pam_limits.so

to the files ssh (or sshd), cron (or crond), screen, login, su and sudo (the last three are purely for your convenience, as we won't use them). This causes PAM to honor the /etc/security/limits.conf statements for the specified applications.

If one of these files is missing from /etc/pam.d, chances are the corresponding application is not PAMized. In that case, the /etc/security/limits.conf changes should take care of it, with one known exception: Cron under recent Red Hat releases does not seem to honor these limits. In the case of cron, add the following to /etc/init.d/crond just before it starts the cron daemon to force it to provide more file descriptors to cron-started processes:

ulimit -Hn 8192
ulimit -Sn 8192

Warning: There seems to be no secure way to increase file descriptors to cron-started processes from recent Red Hat releases. The above approach is a horrible, insecure kludge and may allow users to launch resource attacks from shell accounts. Please try not to run a hyperion server on an open shell box, particularly in this case.

Setup for FreeBSD:

Thanks to Aaron for the following:

"I noticed the section on setting up maxfiles on FreeBSD was empty and that you requested help filling it in. I have some information for you. FreeBSD uses the sysctl oid kern.maxfiles and kern.maxfilesperproc.... [It] uses /etc/sysctl.conf, and reloads values from there at boot time.

"For per-user settings, you can modify /etc/login.conf. The default is openfiles=unlimited, but you can create a class for freenode and specify a specific number for that. From this file, you can also control the max size of core dumps. Something like this would need to be added:

freenode:\
        :openfiles=8192: \
        :coredumpsize=unlimited: \
        :tc=default:

"Then simply add the freenode account to this class:

# pw usermod freenode -L freenode

And you should be set. Hope this helps."
The bash shell. Shell scripts for server operations assume the availability of this shell.
Screen. The GNU screen utility is frequently used to run sessions which can be detached automatically at disconnection.
Perl 5.x. Perl 5.6+ are preferred.
Compile toolchain facilities. These should include gcc and gdb, as well as any required binary utilities. We currently require gcc 3.2.
A selection of editors. Joe, ae and vi are highly preferred. Emacs may be helpful.
At least one IRC client for testing. If no other clients are available, ircII is an acceptable minimum.

Applying to Host a Server.

Before applying to host a server, you may find it helpful to consult with network staff. Regardless, the formal application process is begun by submitting an email containing the following information to email address infra at freenode dot net:

Location of the server. Where is this server located geographically? City, principality and country. Include any "local color" (nearby landmarks and sites) that seems pertinent.
Hosting entity. Who should we credit for the server? Provide names of individuals and/or organizations, as appropriate.
Contact information. Please include names and email addresses of contacts, as well as telephone numbers, snail mail addresses and contact hours (including time zones).
Hardware, software, bandwidth and hosting. Describe the system you're offering for hosting.
IP address. This will help us to perform latency and route checking.
Server name preference. The server naming motif is "science fiction and fantasy authors". Vanity names may be provided, but canonical server names and standardized geographical names will be used exclusively by freenode. If you have preferred authors, let us know and we'll try to honor your preference.
Dancer account password. If you've spoken to us already and we've verified that we are interested in the server, you may include a password in your email to save time. Please make sure the password is not easily crackable.