Linux Kernel Hacking

By Timothy Hall,2014-08-10 20:25
11 views 0
Project 1 -- Linux Kernel Hacking(1)

CS-502 & CS-3013 Operating Systems WPI, Summer 2007

    Hugh C. Lauer Project 3 (16 points)

    Assigned: Thursday, July 5, 2007 Due: Thursday, July 26, 2007


    In this project, you will study and analyze a Linux file system. You will need to use your vir-tual machine to experiment with the file system, and you may need to tinker with the code a bit, but no new design or implementation is required.

    The purpose of this project is twofold:

    1. To expose you to how a file system is added to the Linux environment, and

    2. To help figure out a problem that will affect the usage of this particular file system in

    future versions of the CS-502 Operating Systems course.

    The VMware Host-Guest File System

    As you know, Linux and other modern operating systems support the concept of Virtual File System i.e., an interface for implementing real file systems on a variety of media and connecting them to the operating system kernel at run time. This concept is described in

    th?11.2.3 and ?21.7 of Operating System Concepts (7 ed.) by Silbershatz, Galvin, and Gagne. The

    Linux implementation is described in great detail in Chapter 12 of Linux Kernel Development nd(2 ed.) by Robert Love.

    VMware provides a special file system that allows the host and guest operating system to share a folder. This is called the Host-Guest File System, or HGFS for short. A copy was in-

    stalled in your Linux virtual machine as part of the VMware Tools installation prior to distri-

    bution to the class. Instructions for enabling this file system are given below. When the file system and shared folders are enabled, the HGFS is automatically mounted on the directory /mnt/hgfs in the guest (Linux) operating system. When you open this direc-tory, you can see the contents of the host folder that you specify during the configuration step below. You can then transfer files back and forth between the host and the guest by

    1copying or dragging and dropping.

    Problem Statement

    The problem is that when a new kernel is installed and booted, the HGFS stops working. Moreover, if the Linux system is rebooted with the original kernel, the HGFS does not start working again. Your challenge in this project is to figure out why this happens and to rec-ommend a way to get around the problem. The instructor would like to use this feature in future versions of CS-502.

    1 Unfortunately, the instructor had not worked out the usage of the HGFS prior to the start of the term.

    Otherwise, it would have been a useful tool for getting files into and out of your virtual machines.


Enabling HGFS in your Virtual Machine

    To enable the Host-Guest File System in your virtual machine, you need to do several things:

    ; Add some lines to the VMware Configuration File that describes your virtual machine and

    start your guest operating system

    ; Enable ―Shared Folders‖ in VMware Player and specify the folder in the host system that

    you wish to share.

    ; Reboot your guest operating system (if necessary).

    ; If you are using (or have used) some other kernel than the default Linux kernel that was

    distributed with your virtual machine, rebuild the HGFS by reconfiguring VMware Tools.

    These steps are described below.

    Adding lines to the VMware Configuration File

    The following lines must be added to the VMware Configuration File describing your virtual


    sharedFolder0.present = "TRUE"

    sharedFolder0.enabled = "TRUE"

    sharedFolder0.readAccess = "TRUE"

    sharedFolder0.writeAccess = "TRUE"

    sharedFolder0.hostPath = "C:\Virtual_Machines\Shared_Folder"

    sharedFolder0.guestName = "CS-502"

    sharedFolder0.expiration = "never"

    sharedFolder.maxNum = "1"

    A patch file to add these lines can be found on patch_VMware_Config_File, but it is just as

    easy to navigate to the directory where your virtual machine is stored, open the file named ―SUSE Linux.vmx‖ using Notepad, and paste these lines onto the end of the file.

     (The hostPath and guestName values will be changed in the next step.)

    Enabling Shared Folders

    Start your virtual machine and boot the guest operating system. Use CTL-ALT to switch the input focus back to the host system, pull down the ―VMware Player‖ menu in the title bar of VMware Player, and select Shared Folders. A dialog box will appear with three radio buttons and a large box labeled Folders. Select the radio button marked Always enabled. Next, select the

    folder in the Folders area and click the Properties button. This will let you change the folder

    name and also with let you navigate to the host folder that you want to share. Finally, click OK to the Properties dialog and then again to the Shared Folders dialog.

    If you did this before your guest operating system completed booting, simply log in and open the /mnt/hgfs directory. You should now see the contents of the shared folder.


    If your guest operating system was already running when you enabled the shared folders fea-ture, you will probably have to reboot. If you still cannot see the contents of the shared fold-er, you will have to rebuild HGFS as described in the next section.

    Rebuilding HGFS

    HGFS is built and installed as part of installing VMware Tools, something that was done for

    you before your virtual machine was distributed at the start of the term. VMware Tools are a

    set of drivers and other objects installed in the guest operating system to make communica-tion with the host easier. Included in these are a virtual network card, a virtual display, and the HGFS.

    All of the VMware Tools seem to be dependent upon the version of the kernel. If you change the kernel, some of the tools stop working. For example, some students notice that they cannot access the network after installing a new kernel, or perhaps they have to reconfigure their network card with YaST. Likewise, HGFS seems to stop working; nothing is mounted

    at boot time, so the /mnt/hgfs directory contains whatever had been there before HGFS

    was enabled.

    To rebuild the HGFS, you need to reconfigure all of VMware Tools. This can be done by ex-

    ecuting the following command

    sudo /usr/bin/

    This PERL script asks you a number of questions and then rebuilds the various tools. This Project

    For this project, you need to do the following:

    ; Read and document HGFS, the Host-Guest File system. You can find the source code

    for the HGFS in the following directory:


    ; Study and document the PERL script to discover how

    the HGFS gets built and installed.

    ; Figure out and document what happens during system initialization time or mount time

    to cause HGFS to disable itself when you boot a different kernel.

    ; Recommend a practical approach to use in future CS-502 courses so that the HGFS is

    always present whenever a new Linux kernel is built and installed.

    Each of these four bullets should result in one or more paragraphs in your project write-up. Brevity is a virtue, provided that you cover the key points in an understandable way. Your answer to each bullet is worth 4 points toward the project grade.

    For extra credit worth 4 additional points, implement your practical approach and submit a patch file and any other files needed for the implementation.


Submission of this Assignment

    Be sure to put your name at the top of every file you submit and/or at the top of every file that you edit! Submit your assignment for grading via the web-based turnin tool developed by Professor

    Fisler’s group. A brief introduction can be found at

    and access to the turnin system itself can be found at

    For purposes of turnin, this assignment is Project 3.

    Individual Assignment

    This is an individual project, not a team project. Each student should submit his/her own work, not copies of jointly developed code.

    Nevertheless, if you are puzzled or unsure of some aspect of this assignment, you should consult your friends, colleagues, or the instructor to help clarify your understanding and/or derive an approach to the problem.


Report this document

For any questions or suggestions please email