Creation of an operating system based on the linux kernel. From scratch

31.08.2019 Windows 10

This series of articles is devoted to low-level programming, that is, computer architecture, operating system design, assembly language programming and related areas. So far, two habrauzera are engaged in writing - and. For many high school students, students, and professional programmers, these topics turn out to be very difficult to learn. There is a lot of literature and courses on low-level programming, but it is difficult to get a complete and comprehensive picture from them. It is difficult, after reading one or two books on assembler and operating systems, to at least in general terms imagine how this complex system of iron, silicon and many programs - a computer - actually works.

Everyone solves the learning problem in their own way. Someone reads a lot of literature, someone tries to quickly switch to practice and understand along the way, someone tries to explain to friends everything that he is studying. And we decided to combine these approaches. So, in this course of articles, we will demonstrate step by step how to write a simple operating system. The articles will be of an overview nature, that is, they will not contain exhaustive theoretical information, however, we will always try to provide links to good theoretical materials and answer all questions that arise. We do not have a clear plan, so many important decisions will be made along the way, taking into account your feedback.

Perhaps we will deliberately lead the development process to a standstill in order to allow you and ourselves to fully understand all the consequences of a wrong decision, as well as hone some technical skills on it. So you shouldn't take our decisions as the only correct ones and blindly trust us. Once again, we emphasize that we expect readers to be active in discussing articles, which should strongly influence the overall development and writing of subsequent articles. Ideally, we would like to see some of the readers join in the development of the system over time.

We will assume that the reader is already familiar with the basics of assembly and C languages, as well as basic concepts of computer architecture. That is, we will not explain what a register or, say, random access memory is. If you do not have enough knowledge, you can always refer to additional literature. A short list of references and links to sites with good articles are at the end of the article. It is also desirable to be able to use Linux, since all compilation instructions will be given specifically for this system.

And now - more to the point. In the rest of the article, we will write a classic "Hello World" program. Our Hello World will turn out to be a little specific. It will not run from any operating system, but directly, so to speak, "on bare metal." Before proceeding directly to writing the code, let's figure out how exactly we are trying to do this. And for this you need to consider the process of booting your computer.

So, take your favorite computer and press the largest button on the system unit. We see a cheerful splash screen, the system unit beeps happily with a speaker, and after a while the operating system is loaded. As you understand, the operating system is stored on the hard disk, and here the question arises: how did the operating system magically boot into RAM and start executing?

Know this: the system that is on any computer is responsible for this, and its name - no, not Windows, pips your tongue - it is called BIOS. Its name stands for Basic Input-Output System, that is, the basic input-output system. The BIOS is located on a small microcircuit on the motherboard and starts immediately after pressing the large ON button. BIOS has three main tasks:

Detect all connected devices (processor, keyboard, monitor, RAM, video card, head, arms, wings, legs and tails ...) and check them for operability. The POST program (Power On Self Test) is responsible for this. If vital hardware is not found, then no software will be able to help, and at this point the system speaker will squeak something sinister and the OS will not reach the OS at all. Let's not talk about the sad, suppose we have a fully working computer, rejoice and move on to examining the second BIOS function:
Providing the operating system with a basic set of functions for working with hardware. For example, through BIOS functions, you can display text on the screen or read data from the keyboard. Therefore, it is called the basic I / O system. Typically, the operating system accesses these functions through interrupts.
Launching the operating system loader. In this case, as a rule, the boot sector is read - the first sector of the information carrier (floppy disk, hard disk, CD, flash drive). The order of polling media can be set in BIOS SETUP. The boot sector contains a program sometimes called the primary boot loader. Roughly speaking, the bootloader's job is to start the operating system. The boot process of an operating system can be very specific and highly dependent on its features. Therefore, the primary bootloader is written directly by the OS developers and is written to the boot sector during installation. At the moment the bootloader starts, the processor is in real mode.

Sad news: the size of the bootloader should be only 512 bytes. Why so few? To do this, we need to familiarize ourselves with the device of the floppy disk. Here is an informative picture:

The picture shows the surface of a disk drive. The floppy disk has 2 surfaces. Each surface has ring-shaped tracks (tracks). Each track is divided into small, arched pieces called sectors. So, historically, a floppy sector has a size of 512 bytes. The very first sector on the disk, the boot sector, is read by the BIOS into the zero memory segment at offset 0x7C00, and then control is transferred to this address. The boot loader usually loads into memory not the OS itself, but another loader program stored on the disk, but for some reason (most likely, this reason is the size) that does not fit into one sector.And since so far the role of our OS is played by a banal Hello World, our main goal is to make the computer believe in the existence of our OS, even if only on one sector, and run it.

How does the boot sector work? On a PC, the only requirement for the boot sector is that its last two bytes contain the values 0x55 and 0xAA - the boot sector signature. So, it is already more or less clear what we need to do. Let's write the code! The above code is written for the yasm assembler.

section. text

use16

org 0x7C00 ; our program is loaded at 0x7C00

start:

mov ax, cs

mov ds, ax ; select the data segment

mov si, message

cld ; direction for string commands

mov ah, 0x0E ; BIOS function number

mov bh, 0x00 ; video memory page

puts_loop:

lodsb ; load the next character into al

test al, al ; null character means end of line

jz puts_loop_exit

int 0x10 ; call the BIOS function

jmp puts_loop

puts_loop_exit:

jmp $; eternal cycle

message:

db "Hello World!" , 0

finish:

times 0x1FE - finish + start db 0

db 0x55, 0xAA ; boot sector signature

This short program requires a number of important explanations. The org 0x7C00 line is needed so that the assembler (I mean the program, not the language) correctly calculates the addresses for labels and variables (puts_loop, puts_loop_exit, message). So we inform him that the program will be loaded into memory at the address 0x7C00.
In lines

mov ax, cs

mov ds, ax

the data segment (ds) is set equal to the code segment (cs), since in our program both data and code are stored in one segment.

Then the message “Hello World!” Is displayed character by character in the loop. The function 0x0E of interrupt 0x10 is used for this. It has the following parameters:
AH = 0x0E (function number)
BH = video page number (don't bother yet, specify 0)
AL = ASCII character code

At the line "jmp $" the program hangs. And rightly so, there is no need for her to execute extra code. However, in order for the computer to work again, you will have to reboot.

In the line "times 0x1FE-finish + start db 0", the rest of the program code (except for the last two bytes) is filled with zeros. This is done so that after compilation the boot sector signature appears in the last two bytes of the program.

We sort of figured out the program code, let's now try to compile this happiness. For compilation, we need, in fact, an assembler - the above mentioned

First, learn programming. Knowledge of assembler is essential; It is highly recommended that you also be aware of other additional lower-level programming languages such as C.

Decide on which device you want to boot the operating system. This can be a CD, DVD, flash drive, hard drive, or other computer.

Decide how you want your operating system to appear. Should it be a full version of the OS with a graphical user interface (GUI), or maybe something more minimalistic? You need to know which direction to go in before starting the process.

Check which processor platform your operating system will support. AI-32 and x86_64 are the two most common versions for personal computers, so they can be considered the best choice.

Decide if you prefer to do everything yourself from scratch, or if there are kernels on the basis of which you would like to build on the system. Linux from scratch is a project for those who wish, for example, to create their own Linux distribution.

Choose whether you are going to use your own bootloader or the pre-built Grand Unified Bootloader (GRUB). Because coding your own boot program requires extensive knowledge of computer software and BIOS, it can push back the programming schedule for the actual kernel.

Decide on the programming language you are going to use. Of course, it is quite possible to develop an operating system in a language such as Pascal or BASIC, but it is preferable to write in C or assembler. The assembler is absolutely necessary, since some important parts of the operating system require knowledge of this particular language. C ++, on the other hand, contains the keywords required to run the full OS.

To build an OS using C or C ++ code, you will, of course, use one compiler or the other. This means that you must read the manual / instructions / documentation for the C / C ++ compiler of your choice that comes bundled with the software or is available on the distributor's website. You will have to learn a lot of complicated things about the compiler, and you will also need to learn the schema and ABI to improve C ++. You are expected to understand the different execution formats (ELF, PE, COFF, regular binaries, etc.) and notice that Windows' native PE format (.exe) is copyrighted.

Select Application Programming Interface (API). One collection of good APIs is POSIX, as it is well documented. All Unix systems have at least partial POSIX support, so it would be trivial to add Unix programs to your operating system.

Decide on the design. There are monolithic kernels and microkernels. Monolithic kernels perform all services in the kernel, while microkernels have a small kernel combined with a custom service implementation. In general, monolithic kernels are faster, but microkernels have better isolation and protection against possible faults.

Consider development and teamwork. This way, you will need less time to resolve big problems, which will allow you to create a better operating system in a shorter time frame.

Don't erase your hard drive completely. Remember, formatting your drive will permanently erase all of your data! Use GRUB or another manager to duplicate booting your computer from another OS until your version is fully functional.

Start small. Pay attention first to the little things like text display and interrupts before moving on to complex elements like memory management and multitasking.

Keep a backup copy of the latest working version. This gives you a certain amount of peace of mind in case something goes completely wrong with your current version of your OS or subsequent add-ons. In the event of a breakdown of your computer and the inability to boot, as you yourself understand, having a second copy for work will be an excellent opportunity, so that you can fix the existing faults.

Test your new operating system in a virtual machine. Instead of restarting your computer every time you make changes or transfer files from your production computer to a test machine, you can use an application to run the OS in a virtual machine while your current OS is still running. VM applications include VMWare (which also has a freely available server), alternative open source, Bochs, Microsoft Virtual PC (not Linux compatible), and XVM VirtualBox.

Release the release version. This will allow users to tell you about possible flaws in your operating system.

The operating system also needs to be user-friendly, so be sure to add useful features that will become an integral part of your design.

When the development is finished, consider whether you want to make the code freely available or to establish private rights to it.
Make sure to make security features your top priority if you want your system to be viable.
Don't start an operating system development project for the purpose of teaching programming. If you do not know C, C ++, Pascal, or any other suitable languages and properties, including pointer types, low-level bit operations, bit switching, inline assembler, etc., then you are not ready to create OS.
Browse portals such as OSDev and OSDever to help you improve your own operating system. Please note in particular that for most of the issues, the OSDev.org community prefers that you consult the site content yourself rather than joining the forum. If you nevertheless decide to join the ranks of the members of the forum, there must be certain prerequisites for this. You must have a thorough knowledge of C or C ++ and the x86 assembly language. You should also understand general and complex programming concepts like Linked Lists, Queues, etc. The OSDev community in its rules explicitly states that no one is going to babysit new programmers. If you are trying to develop an OS, it goes without saying that you are a "god" in the field of programming. You are also required to read the processor manual for the architecture of your choice; for example x86 (Intel), ARM, MIPS, PPC, etc. Such a reference to processor structure can be easily found by searching Google ("Intel Manuals", "ARM manuals", etc.). Don't sign up on the OSDev.org forum to ask obvious questions. It will just lead to answers like "Read the f *** ing Manual". To get started, you should try reading Wikipedia, tutorials for the various tools you intend to use.
Check for potential blind spots and other bugs. Shortcomings, dead ends, and other issues can affect the design of your operating system.
If you want an easier way, imagine Linux distributions such as Fedora Revisor, Custom Nimble X, Puppy Remaster, PCLinuxOS mklivecd, or SUSE Studio and SUSE KIWI. However, the OS you create is owned by the company that first introduced the service (although you have the rights to freely redistribute, modify and run it as you like under the GPL).
A good solution is to create a completely new partition for the operating system under development.

Warnings

Careless rewriting of the OS to the hard drive can damage it completely. be careful
You won't have a complete system in two weeks. Start with a bootable operating system and then move on to more interesting material.
If you do something reckless, like write messy bytes in random I / O ports, then destroy your OS and could (in theory) burn your hardware.
Don't expect it to be easy to build a quality operating system. There are many complex interdependencies. For example, for an OS to be able to handle multiple processors, your memory manager must have "locking" mechanisms to prevent unnecessary processors from accessing the same resource at the same time. The "blocks" used assume a scheduler to make sure that only one processor is accessing a critical resource at any given time, and all others are idle. However, the scheduler depends on the presence of the memory manager. This is an example of a deadlock dependency. There is no standard way to solve these kinds of problems; each operating system creator is expected to be skilled enough to come up with their own solution.

Sources of

What do you need

Computer
The processor on which you are going to build
Sufficient random access memory (RAM) for the virtual machine
Primary OS (used for developing assembler source code (and others), and early assembly and packaging; your own OS will eventually become primary)
Syntax color code editor (used in the absence of Integrated Development Environment)
Compiler
CD / DVD drive

Ilya Alexandrov

We create our own Linux-based OS

There are hundreds of Linux distributions, and it is not known how many more will appear. Dozens of companies and thousands of programmers compete to create the best Linux project, while any experienced user can become the author of a home PC system that rivals the products of the IT giants.

Over the years working with Linux, I have used a huge number of different distributions: Mandriva, Fedora, SlackWare, Debian, Ubuntu and many others. I liked some project more, some less. But in all distributions one inevitably had to deal with serious flaws that made work very difficult. One is too demanding on resources, the other does not support all the necessary equipment, the third lacks various software. It was then that I remembered the well-known Eastern wisdom: if you need to do something well, do it yourself.

Linux from Scratch

I'm not the only one who decided to start building their own version of Linux - an OS in which the basic part of the system and the kernel will be taken as a basis, but where there will not be a single extra kilobyte from the developer, that is, from you. A large number of Linux distributions that do not meet user requirements pushed Gerard Beekmans to create a distribution that will give everyone the opportunity to build a system with only the components and functions they need.

The talented programmer's aspiration resulted in the Linux from Scratch project (www.linuxfromscratch.org), or LFS for short. This project allows you to construct from scratch, from source codes, your operating system based on Linux. LFS is compiled on a computer with a Linux system already installed, however, an "advanced" Live-CD, for example, Knoppix, will do.

In this case, the Linux-system used for building can be any - only the presence of a compiler and system libraries is required. Linux From Scratch can hardly be called a distribution kit in the usual sense of the word - it is something like auxiliary software that, together with the base part of the operating system, will allow you to create your own, unique version of the OS.

As you know, Linus Torvalds developed his operating system under the motto "Just for fun!" - that is, only for the sake of pleasure. I must admit that LFS is really not often found on servers, this system is usually used by computer enthusiasts. Installing and working with Linux from Scratch will help you understand the interconnection of OS components, which will come in handy for your own development of a Linux distribution, and not only based on LFS. Therefore, LFS is largely designed for those people for whom the process of building their own distribution is fun and interesting - and there are many such people, believe me.

So, if you are ready to spend a whole day (or even more) on constructing a system, then I recommend downloading LFS-packages-6.0, LFS-book from the site (2), and continue reading this article.

Partitioning a disk and creating a directory tree

For a better understanding of the material, we will describe the entire process in general terms (see Fig. 1).

At the first stage, using the already installed distribution kit or LiveCD, the disk is partitioned. A partition is allocated on the hard disk for the new system. After that, on this section, you will need to statically compile all the necessary programs and the system kernel. Next, the root directory is changed to a hard disk partition allocated for our new OS. You will need to re-compile, but this time the software must be compiled dynamically (the difference between dynamic compilation and static compilation will be described below). The last step involves building the essential glibc library and configuring the installed OS. As you can see, you don't have to do anything particularly difficult.

Throughout the entire process, your main assistant is the documentation from the LFS-book package, the Russian translation of which can be found here: http://multilinux.sakh.com/download/lfsbook.tar.bz2. The book covers every step of building an OS in detail, so be sure to refer to this guide in case of problems (this article is not intended to replace such extensive documentation).

Create a new partition - in my case / dev / hda5, since the / dev / hda1 partition is already occupied by the Slackware Linux hard disk installed. It is recommended that you first make a backup of the system so that you can restore it in case of damage, although the likelihood of this is close to zero. And here, I think, everything is clear: we allocate the required amount (23 GB is enough) for the root directory, a space equal to twice the amount of RAM - for a swap partition, if you wish, you can create separate partitions for the home directory (/ home) and for / boot. However, the popular partitioning option - to allocate all available space minus swap for the root directory, and the subsequent creation of the swap itself - is also quite acceptable when building LFS. On the author's computer and Linux, Slackware, which is the parent OS, and LFS use the same hard disk, however, installing LFS on another hard drive is also not difficult.

Choose the file system at your discretion: there were no problems with both Ext3 and ReiserFS under LFS. But XFS fans will have to upset - attempts to make Linux From Scratch work with this FS have not been crowned with success.

Now we mount the partition dedicated to the new OS:

$ mount / dev / hda5 / mnt / mylin

For convenience, let's define the MYLIN variable:

$ export MYLIN = / mnt / mylin

Great, for further work it is better to create a separate user mylin, which we will assign the owner of the mounted partition.

$ useradd mylin

$ chown –R mylin $ MYLIN

We need to create a directory tree at the root of the new section:

$ cd $ MYLIN

$ mkdir –p bin boot dev etc home lib mnt opt root sbin usr / (X11R6, local) var

Create the necessary structure in the usr, usr / X11R6, usr / local directories: subdirectories bin, etc, include, lib, sbin, share, src.

Then we will do the same for the / var and / opt directories of the future system:

$ mkdir var / (cache, lib, local, lock, log, opt, run, spool)

$ mkdir opt / (bin, doc, include, info, lib, man)

Let's not forget that there are deeper hierarchies, for example / usr / share / man / man1. But the volume of the article does not allow to give here all the information about the structure of the file tree, so you either need to use the Filesystem Hierarhy Standard document (can be found at: http://linux-ve.net/MyLDP/file-sys/fhs-2.2-rus) , or carefully study the structure of the OS of the Linux family you have already installed. After preparing the hard drive, we proceed to static assembly.

Static build

Why are we using static build? Static compilation attaches the source code of the library to the application code, which increases its size, but at the same time maintains integrity. With dynamic compilation, the library is located in a separate file, which is accessed by applications as needed. As a result, all programs work with one version of the library.

But when we use the chroot command to set the root directory for the newly built system, the libraries of the "parent", the installed system located in / lib, / usr / lib, and others, will no longer be available, so dynamically compiled programs will refuse to work, in addition, version compatibility by anyone not guaranteed.

To avoid this, we will first collect all the necessary software for our future system statically. Let's start with the bash shell. (Fans of ZSH or TCSH can install their favorite interpreters after installing the system, but at the stage of building their use is not provided by the author of LFS). You should check if you have the /usr/lib/libcurses.a file and if not, install the nсursesdev package. All packages must be compiled with static build flags: "--enable-static-link", "--disable-shared", or "--static". Which one is right for each particular case, you can find out from the documentation for a particular package or from the output of the configuration script launched with the "--help" parameter.

$ ./configure –-help

In order not to confuse later statically compiled programs with "dynamic" ones, create a special directory for them:

$ mkdir $ MYLIN / stat

When building and installing packages, do not forget to add the "--prefix = $ MYLIN / stat" parameter to move files to this particular directory. Finally, we install bash:

$ ./configure –-enable-static-link --prefix = $ MYLIN / stat

$ make

$ make install

We collect the rest of the necessary packages in the same way. : binutils, bzip2, textutils, texinfo, tar, sh-utils, gcc, grep, gzip, gawk, diffutils, fileutils, make, patch, sed, and, in fact, linux-kernel.

Yes, when compiling the kernel, do not forget that for older versions of kernels (2.2.x-2.4.x) you need to use gcc 2.95, and for the current version 2.6.x it is recommended to use gcc 3.x so that there are no problems.

Do not forget to look at the corresponding sections of the LFS-book, it says about this and many other nuances. In general, compiling the kernel in LFS does not differ from a similar procedure carried out when using the distribution kit installed on the HDD. Unzip the kernel sources into $ MYLIN / usr / src / linux-2.6.xx, then configure it by running:

$ make menuconfig

The process of configuring kernel parameters has been described many times on the Internet (6), there is hardly a need to dwell on this in more detail. Next, we give the following commands in the directory with the Linux-kernel sources:

$ make bzImage

$ make modules

That's it, there is a new kernel at $ MYLIN / usr / src / linux-2.6.xx / arch / i386 / boot / bzImage.

Next, we create the files $ MYLIN / etc / passwd and $ MYLIN / etc / group. In the first, we register the only user so far - root with any password, and in the second user group (to start, one root group will also be enough).

This concludes our preparations for the next step, and we move on to a more subtle dynamic assembly.

Dynamic build

Now we need to change the root directory to / mnt / mylin, where we will use only statically compiled utilities - we will no longer be able to use the tools from the "parent" OS. We give the command in the console:

$ chroot $ MYLIN / usr / bin / env –i

> HOME = / root TERM = $ TERM PS1 = ’u: w $’

> PATH = / bin: / usr / bin: / sbin: / usr / sbin: / stat / sbin

> / stat / bin / bash --login

With this command, we specified the paths to the executable files, the terminal type, the interpreter and the type of the command line prompt.

For some programs to work, the proc filesystem needs to be installed on the new system.

$ mount proc / proc -t proc

The most crucial moment has come. Building the glibc library. He is the most responsible because most of the necessary programs will not work without it, and there is no point in using Linux without the main library. Building glibc is often problematic.

When building, we specified the "--prefix = $ MYLIN / stat" parameter, so when changing the root, all statically built packages will appear in the / stat directory of the new OS partition.

So, unpack the glibc-2.x.x.tar.gz archive (for example, into the / usr / src / directory) and go to the glibclinuxthreads directory. We'll have to slightly tweak the source code due to the fact that at this stage in the system it is impossible to identify the user by name (just due to the lack of glibc and other libraries), and the fact that to install glibc you need a Perl interpreter, which we do not have.

We replace the username root in the login / Makefile with its uid, which is 0, and the $ PERL variable in the malloc / Makefile should be replaced with the path to the interpreter - / usr / bin / perl - and it will simply be ignored during configuration.

$ /usr/src/glibc-2.x.x/configure --prefix = / usr --enable-add-ons --libexecdir = / usr / bin &&

& make

& make install

$ make localedata / install-locales

$ / stat / bash --login

If you did everything correctly, glibc will compile, "root" will finally appear at the prompt, and all programs can be dynamically recompiled.

Finish installing the kernel:

$ make modules_install

$ make install

To move the new kernel to the / boot directory, execute one more command:

$ make unstall

We collect all installed and some new programs, now without static compilation flags. We will need (at this stage it is very important not to forget to compile all of the following) (see table 1).

Table 1. Required set of packages for assembly

autoconf	grep	perl
automake	groff
bash	gzip	procinfo
bin86		procps
binutils	less	psmisc
bzip2		reiserfs-progs
diffutils	libtool
e2fsprogs	lilo	sh-utils
		shadow
file	make	sysklogd
fileutils	makedev	sysvinit
findutils	man-pages
flex	modutils	texinfo
gawk	ncurses	textutils
	netkitbase	util-linux
bison	net-tools
gettext	patch

After performing dynamic recompilation, you can delete the directory with statically built packages:

$ rm -rf / stat

You can breathe a sigh of relief and drink coffee - the most difficult, routine is left behind. Let's move on to the next stage - the initial configuration of our system.

Initial system configuration

Before proceeding with the configuration, I note that all changes must be made in the files of the directory of the new OS, and not the parent system.

To set the system time, create the / etc / sysconfig / clock file, which contains just one line:

UTC = 0

Now the computer clock will display the time in your time zone - provided that the time in the BIOS is set correctly.

Let's give the computer a name:

echo "HOSTNAME = my_linux"> / etc / sysconfig / network

Now we will specify the partitions that the system should mount at boot time in / etc / fstab:

# filesystem mount-point fs-type options dump fsck-order

/ dev / hda5 / ext3 defaults 1 1

/ dev / hda3 swap swap pri = 1 0 0

proc / proc proc defaults 0 0

Instead of / dev / hda3 and / dev / hda5, write your partitions (root and swap), add mount points for other hard disk and CD-ROM partitions if necessary.

Now let's make our system bootable.

If in addition to lFS you use other Linux distributions, then now you need to log into the old system - for this we run the command:

$ exit

Already in the parent OS, add the following to the /etc/lilo.conf file:

# LFS

image = / boot / bzImage

Label = lfs

Root =

Read-only

It is clear that "/ boot / bzImage" is the path to the kernel you compiled for the system, and "partition" is the disk partition where the root directory is located.

If you do not plan to use other operating systems and Linux distributions, then go straight to configuring LILO in LFS.

In this case, lilo.conf will look something like this:

boot = / dev / hda

Delay = 40

Compact

Vga = normal

Root = / dev / hda1

Read-only

Image = / boot / zImage-2.6.12

Label = Linux

Make the necessary changes depending on your configuration. We update the bootloader with the command:

$ / sbin / lilo –v

And, if all the previous steps were performed correctly, we will find ourselves in a new system. However, a long stage of "fine-tuning" tuning (special attention should be paid to the security of the new system, because LFS by default looks rather unprotected, like any newly installed OS) is still ahead. But you already have a self-assembled version of Linux.

P.S

Gerard Bickmans isn't the only one who has the idea to create his own Linux. Another project, BYOLinux, headed by Jonatan Thorpe, has stopped its development today, although the written documentation remains relevant now, but it is not as detailed as the LFS-book and has not been translated into Russian. The main difference in John's method is that glibc is ported from parent to child without recompiling, which is not as efficient, but avoids many build problems. Some FreeBSD users also want to feel like an OS designer.

Now this is quite possible - at http://ezine.daemonnews.org/200302/fbsdscratch.html there is an article about building FreeBSD from source as a whole - from distributions to ports, and the method is not similar to the usual "rebuild" of the system, but similar to by the Gerard Beekmans method. Well, now you have your own unique Linux-based system. In case of problems, look for their solution in the LFS-book, everything is described in detail there. I also recommend downloading the Linux Network Administrator's Guide from the portal http://www.tldp.org, although it does not relate directly to LFS, it will come in handy at the stage of setting up the system. Do not forget that each program also comes with various man and info pages, which are also designed to make life easier for Linux users.

LFS-book in Russian - http://multilinux.sakh.com/lfs.
The official portal of the LFS project is http://www.linuxfromscratch.org.
ByoLinux Portal - http://www.byolinux.org.
FreeBSD from scratch article - http://ezine.daemonnews.org/200302/fbsdscratch.html.
An article on compiling the Linux kernel - http://vikos.lrn.ru/MyLDP/kernel/kompil-2-6.html.
Bayrak A. Review of Knoppix 3.7 Russian Edition. - Journal "System Administrator", No. 3, March 2005 - 4-6 p. ().

This series of articles is devoted to low-level programming, that is, computer architecture, operating system design, assembly language programming and related areas. So far, two habrausers are engaged in writing - iley and pehat. For many high school students, students, and professional programmers, these topics turn out to be very difficult to learn. There is a lot of literature and courses on low-level programming, but it is difficult to get a complete and comprehensive picture from them. It is difficult, after reading one or two books on assembler and operating systems, to at least in general terms imagine how this complex system of iron, silicon and many programs - a computer - actually works.

Detect all connected devices (processor, keyboard, monitor, RAM, video card, head, arms, wings, legs and tails ...) and check them for operability. The POST program (Power On Self Test) is responsible for this. If vital hardware is not found, then no software will be able to help, and at this point the system speaker will squeak something sinister and the OS will not reach the OS at all. Let's not talk about the sad, suppose we have a fully working computer, rejoice and move on to examining the second BIOS function:
Providing the operating system with a basic set of functions for working with hardware. For example, through BIOS functions, you can display text on the screen or read data from the keyboard. Therefore, it is called the basic I / O system. Typically, the operating system accesses these functions through interrupts.
Launching the operating system loader. In this case, as a rule, the boot sector is read - the first sector of the information carrier (floppy disk, hard disk, CD, flash drive). The order of polling media can be set in BIOS SETUP. The boot sector contains a program sometimes called the primary boot loader. Roughly speaking, the bootloader's job is to start the operating system. The boot process of an operating system can be very specific and highly dependent on its features. Therefore, the primary bootloader is written directly by the OS developers and is written to the boot sector during installation. At the moment the bootloader starts, the processor is in real mode.

section. text

use16

org 0x7C00 ; our program is loaded at 0x7C00

start:

mov ax, cs

mov ds, ax ; select the data segment

mov si, message

cld ; direction for string commands

mov ah, 0x0E ; BIOS function number

mov bh, 0x00 ; video memory page

puts_loop:

lodsb ; load the next character into al

test al, al ; null character means end of line

jz puts_loop_exit

int 0x10 ; call the BIOS function

jmp puts_loop

puts_loop_exit:

jmp $; eternal cycle

message:

db "Hello World!" , 0

finish:

times 0x1FE - finish + start db 0

db 0x55, 0xAA ; boot sector signature

mov ax, cs

mov ds, ax

the data segment (ds) is set equal to the code segment (cs), since in our program both data and code are stored in one segment.

At the line "jmp $" the program hangs. And rightly so, there is no need for her to execute extra code. However, in order for the computer to work again, you will have to reboot.

We sort of figured out the program code, let's now try to compile this happiness. For compilation, we need, in fact, an assembler - the above mentioned

Sooner or later, every Linux user thinks about creating his own distribution kit. Some argue that you can "customize everything for yourself." Others complain that there is no ideal among the distributions already presented in the Branch. And they supposedly have superconceptual ideas for their own system. Why did I start all this psychology? In order to immediately cut off the oxygen for beginners who are playing with Linux, who have nothing to do. If you are already thinking about creating an OS, think to the end. So,

I want to create a Linux based OS.
I warn you right away: it would have been the 18th century, all those who, for the basis of their future system, choose another developed distribution kit (and, God forbid, a popular one ...) would be hanged. The post is about creating a system from scratch, which means that we will not touch any Slax and Linux Mint.

Step 1. Select media
There are few options: either your OS starts from a LiveCD, or from a hard disk, or from a flash device. I'll make a reservation right away: I won't say a word about the hard drive in the post, because it is much more convenient to create a flexible distribution kit from the "I carry everything with me" series, or a locked distribution kit on an optical disk. If you learn how to create a LiveCD or LiveUSB system, there will be no problem installing on a hard drive.

Just in case, prepare a blank USB stick, CD-ROM, and finally install Virtualbox.

Step 2. Compiling the kernel
Regarding the release of the third Linux kernel, this step encourages further development ... So, we need the kernel sources. Every user knows that they can be obtained from kernel.org. Under no circumstances, do you hear ?, never screw a third-party kernel that was not compiled by you to your system!

Since my laziness was off the charts, I created the / linuxkernel folder and unpacked the archive with the sources there. After logging in as root, I did the following:

cd / linuxkernel
make menuconfig

In principle, the kernel can be configured in three ways: make config (dialog configuration), make menuconfig (pseudographic configuration via ncurses), and make xconfig (graphical configuration). The bottom line is that make config will ruin your mood for a long time, because he will ask all possible questions on all aspects of all topics. The problem with make xconfig is not encountered by everyone, but I have encountered it and still does it. If you want to do it through X, figure it out for yourself. The best option is make menuconfig. This thing will open you a pseudo-graphical interface through which you can customize the kernel in your own way. The thing requires the ncurses library, which is easy to install.

Basically, if your brain understands Linux at all, you can figure out the configuration. The process is interesting, there are really many options, and the help, although in English, still pleases with its accessibility and simplicity.

However, you still have to guide you. Go to File Systems ---> and add the required asterisks. The letter M means that this or that driver is supported by connecting to the core of an external module (I hate them!). We also need isofs support to read disks. File Systems ---> CD-ROM / DVD Filesystems ---> ISO 9660 CDROM file system support. You can still support the ancient Dosov systems.

The freaky Mandriva developers forgot to enable File systems ---> DOS / FAT / NT Filesystems ---> NTFS write support, and on one of their distributions I was struggling with access to the ancient Wind partition.

Have a look at Processor type and features ---> Processor family, I was advised to go for Pentium-MMX.

Also rummage in Device Drivers, it's helpful. For fun, you can select everything there and compile a kernel with a weight> 50 MB.

Further. After loading itself, the kernel must load the system itself. Either from compiled files (used in embedded systems), or from a CPIO archive compressed with something, or from Initrd. Here you are not DOS, here you will not be able to immediately refer to some init "new file in the root directory of a disk or flash drive. In fact, it will work, do not listen to Uncle Annix! This is wrong, even though there is already a sickly controversy on the Internet about this. we will use initrd on our system, because it is convenient and will not cause obscene expressions from third-party developers, unlike the CPIO archive.

Oh yes, compile the kernel with the command

If you have x86, you can find it at / linuxkernel / arch / x86 / boot / bzImage.

For harsh Chelyabinsk programmers, you can use Cross-compiling ...

Creation of Ramdisk.

Now we need an initrd with the simplest shell installed there. We'll be using busybox because it can do anything. We will steal the method from Roberto de Leo, the creator of Movix (I would even start to respect him if it were not for his outrageous love for Perl):

dd if = / dev / zero of = / dev / ram0 bs = 1k count = 5000 - Create a Ramdisk in the RAM of our computer.
mke2fs -m0 / dev / ram0 5000 - Format Ramdisk on Ext2
mkdir / distro - Create a folder
mount / dev / ram0 / distro - Mount to the / distro folder

That's it, now we have a Ramdisk with a capacity of 5 MB. More is possible, just not necessary. Unlike Thomas Matejisek, I'm not going to stuff the initrd with LZMA compressed modules in Squashfs. Anything that is needed will be compiled with the kernel. Yes, this is not very logical and correct, but the hassle is a hundred times less. And especially for those who condemn this approach, you can enable the modularity option in the kernel: Enable loadable module support.

In our Ramdisk, mounted in / distro, there is such a folder, lost + found. This is because we formatted it in ext2. In no case should you delete it, although it will hardly help here, the image is fixed. would busybox first put ...

Installing Busybox
Is that why such awesome projects have such lame sites? Although ... this is no longer important if the sources are downloaded and successfully unpacked into the / busybox folder.

You can configure busybox in the same way:

cd / busybox
make menuconfig

If you have not yet understood what it is, I will explain. Busybox replaces tons of UNIX applications stored in / bin, / sbin, / usr / bin, / usr / sbin folders. Instead, only one application is created: / bin / busybox, and a bunch of links are created to it in the above folders. Install busybox with the following command:

make CONFIG_PREFIX = / distro install

Busybox will also create the files / sbin / init and for some reason / linuxrc so that your system starts correctly. However, not all required folders have been created. So we finish everything with our hands and create:

/ distro / etc
/ distro / lib
/ distro / dev
/ distro / mnt
distro / proc
/ distro / root
/ distro / tmp
/ distro / root

If you forgot something - remember, tk. it is difficult to forget these directories.

Everything would be fine, but busybox requires libraries to work, which need to be copied into our distribution kit. It is very easy to find out which:

ldd / distro / bin / busybox

The program will show us the libraries required for our shell. I say right away: linux gate is created by the kernel and cannot be copied.

When copying libraries, you can cut off debug information (as Roberto advises):

objcopy --strip-debug from where to where

We make Linux from Linux

Several system text files need to be created:

We need / etc / inittab. I will surprise you: at the beginning of life, the system does not even know what Root is. We even have an unnamed user, but the system-wide low-level features (ONF) file must be present. The pilot content of the file is as follows:

# The very first file to run, / sbin / init later.
:: sysinit: /etc/rc.d/rc.S

# Run shell in console.
:: respawn: - / bin / sh

# Commands to execute before shutdown and reboot.
:: shutdown: / sbin / swapoff -a> / dev / null 2> & 1
:: shutdown: / bin / umount -a -r> / dev / null 2> & 1

The next file is / etc / fstab. This is a table that describes what and where to mount at boot. Useless thing! We need to mount proc, otherwise nothing will work at all, so in the file we write:

none / proc proc defaults 0 0

The mount also needs the / etc / mtab file. Create it and leave it blank.

But mount will only do whatever is necessary when we explicitly ask it to do so. And we will ask in the same initial boot file /etc/rc.d/rc.S (rc.d is the folder). We ask politely:

/ bin / mount -av -t nonfs

We also need a profile file (b) (a) sh, there is generally scope for imagination. Create the / etc / profile file and fill it with the following:

PATH = "$ PATH: / bin: / sbin: / usr / bin: / usr / sbin:"
LESS = -MM
TERM = linux
HOME = / root
PS1 = ">"
PS2 = ">"
ignoreeof = 10
export PATH DISPLAY LESS TERM PS1 PS2 HOME ignoreeof

You will also need the / etc / shell file, which indicates that there is a shell:

/ bin / sh
/ bin / ash
/ bin / bash

That's all. You can write our Ramdisk to a file.

mkdir / os - folder for "finished".
umount / dev / ram0 - unmount a piece of RAM.
dd if = / dev / ram0 of = / os / initrd bs = 1k count = 5000 - create a file.
gzip / os / initrd - compress the initrd file

Creating a bootable USB drive

The "home stretch" of our little design. We take a USB flash drive, insert it, format it in vfat (you can also use ext, but do not forget that not all Windows users have shot themselves yet).

On the USB flash drive, create a boot folder, in it the initrd and kernel folders.

Copy the compressed Ramdisk from the / os folder to the boot / initrd folder on the flash drive, call it "main.gz". Copy bzImage from the folder with the kernel sources to the boot / kernel folder on the USB flash drive, call it "main.lk". We get the Syslinux bootloader files (on the Internet, or from another distribution kit: it doesn't matter here), namely syslinux.bin, syslinux.boot, syslinux.cfg. We copy them to the root directory of our flash drive. We write something like this in the syslinux.cfg file:

default mm
prompt 1
timeout 100
label mm
kernel /boot/kernel/main.lk

label mc
kernel /boot/kernel/main.lk

label cm
kernel /boot/kernel/custom.lk
append initrd = / boot / initrd / main.gz load_ramdisk = 1 ramdisk_size = 5000 rw root = / dev / ram0
label cc
kernel /boot/kernel/custom.lk
append initrd = / boot / initrd / custom.gz load_ramdisk = 1 ramdisk_size = 5000 rw root = / dev / ram0
label hd
localboot 0x80

Thus, we supported custom initrd and kernel, which, for the sake of experiment, can be connected to our distribution.

Let's find out what device our flash drive is in the system (you can run mount without parameters and see). This is either / dev / sdb1, or / dev / sdc1, or / dev / sdd1. It is worth unmounting the USB stick before starting the installation.

Install syslinux (if the package is not on the system, apt-get install syslinux):

syslinux -d device_path

The ldlinux.sys file should appear in the root directory of the flash drive. If it is, then syslinux.bin, syslinux.boot are no longer needed.

I won't tell you how to configure the BIOS to boot from a USB flash drive - it's easy. I will only say that it is very convenient to create a / boot / initrd / init folder, into which / boot / initrd / main can be mounted, for subsequent work with it. Just remember to decompress and compress it with gzip.

OK it's all over Now.

As if I just told you how to create a Linux system from scratch. Easy, isn't it? Then you can edit the / sbin / init script, because you still have a lot of work to do! You will need to write a script to mount the stick that chroot to the root directory. Otherwise, you will have to work with the ReadOnly partition, 5 MB in size. But that's a completely different story.

Creation of an operating system based on the linux kernel. From scratch

Warnings

Sources of

What do you need

Top related articles