Running Inline Monitor Applications for Microbenchmarking Experiments ======================================================================= This page describes how one can set up a mOS inline middlebox for microbenchmarking experiments. In this setup, we assume that the client(s) and the server(s) are only two network hops away from each other and the mOS middlebox application serves as a proxy of each peer node (whether it is a client or a server). The following figure explains the setup with more clarity. Environment ---------------------------- .. figure:: images/inline_micro_config.* :align: center :scale: 50 % `where:` * The client is connected to switch port #1, * The middlebox is connected to switch port #2, * The server is connected to switch port #3. All 3 hosts are considered to be set up in local area network (LAN) settings with an Ethernet switch. Each endpoint (client and server) exchanges traffic by sending Ethernet frames with destination MAC address of the middlebox's NIC. The mOS networking core transparently updates the destination MAC addresses of either node as the packet is forwarded in both directions. The startup mOS configuration file needs to be adjusted for Ethernet maquerading accordingly. Please follow the directions in :doc:`../walkthrough/03_setup` to set up the mOS net library & the environment and refer to :ref:`sample_app` to ensure that you successfully compile the mOS application. * In the middlebox, please compile `simple_firewall`. Static MAC address table setup ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ We need to carefully set up static ARP table entries to make all traffic between the client (node A) and the server (node B) propogate via the middlebox (node M). You can use 'arp' command to set a static ARP table entry of A, B and M. mOS networking core will read Linux kernel's ARP entry during its initialization phase. For A: .. code-block:: console $ arp -s For B: .. code-block:: console $ arp -s For the middlebox: .. code-block:: console $ arp -s $ arp -s Switch's forwarding table setup ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ========= ================= **Port#** **MAC address** 1 ``A_MAC_ADDR`` 2 ``M_MAC_ADDR`` 3 ``B_MAC_ADDR`` ========= ================= Setup mOS application configurations ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Before running the ``simple_firewall`` application, you need to update the configuration files in the middlebox. * In M, specify the L3/L4 firewall rules in ``simple_firewall.conf`` (please refer to the example configration file below). .. code-block:: console # Simple firewall rules #(act) (src) (dst) (port) DROP 10.0.0.0/24 10.1.0.0/24 dport:80 ACCEPT 10.0.1.7 10.1.1.9 sport:1024 ACCEPT 10.0.2.7 10.1.2.9 ACCEPT 10.0.3.0/24 10.1.3.0/24 * In the middlebox, please configure ``mos.conf`` as below. We assume that the node uses CPU core 0~7 for running mOS ``simple_firewall``. Please make sure that you set ``forward = 1`` to allow the ``simple_firewall`` application to forward the traffic if the ruleset permits. Also set the ``arp_table`` and the ``route_table`` entries appropriately. Please note that the ARP learning submodule of mOS networking core is disabled for this setup. .. code-block:: console ####################### # MOS-RELATED OPTIONS # ####################### mos { forward = 1 ####################### ##### I/O OPTIONS ##### ####################### # number of memory channels per socket [mandatory for DPDK] nb_mem_channels = 4 # devices used for MOS applications [mandatory] netdev { dpdk0 0x00FF dpdk1 0x00FF } ####################### ### LOGGING OPTIONS ### ####################### # NICs to print network statistics per second # if enabled, mTCP will print xx Gbps and xx pps for RX and TX stat_print = dpdk0 dpdk1 # A directory contains MOS system log files mos_log = logs/ } Run simple_firewall ------------------------- In M, run ``simple_firewall`` with sudo privileges. You would see the application block TCP flows according to the rules specified in ``simple_firewall.conf``. .. code-block:: console $ sudo ./simple_firewall Troubleshooting ----------------------- If your mOS program crashes, search through the printed log messages to check if you missed any of the following requirements: * You should load only the dpdk-registered interfaces (after loading the ``igb_uio`` driver) and configure hugepages as described in :doc:`../walkthrough/03_setup`. * You should create a log folder specified in ``mos_log`` parameter in ``mos.conf`` (``logs/`` in our example) within the same directory that contains the mOS application binary. * Please check if your system is running out of memory by looking at the memory footprint (OOM errors). If you have memory deficiency issues, please reduce the number of socket buffers in ``mos.conf``. If the traffic fails to pass through Host 1, check the following issues: * Check whether you achieve connectivity by reverting back to kernel network driver for the interfaces (see option 5 in :ref:`kernel_module`). You can try testing your setup as a network bridge with default Linux kernel network drivers. .. Inline Mode for Microbenchmarks/Debugging .. --------------------------------------------- .. In this section, we show how one can set up a mOS middlebox .. within a LAN in an inline mode but with a different configuration. .. figure: images/test_setup_white.* .. :align: center .. :scale: 50 % .. In this example, we place the middlebox between two endpoints so .. that it can monitor traffic between them using only **one network .. interface**. This setup requires at least three machines, two for .. traffic generation (A and B) and one for middlebox (M). End-host .. A and B do not have to run the networking applications on the .. mOS stack. .. * A, B: End-host machines .. * M: mOS middlebox between A and B .. * NIF(x): Network interface (e.g. dpdk0, dpdk1, ...) of a machine 'x' .. * MAC(x): MAC address of network interface 'x' .. * IP(x): IP address of network interface 'x' .. Static MAC address table setup .. ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ .. We need to carefully set up static ARP table entries to make all traffic .. between A and B travel via M. You can use 'arp' command to set a .. static ARP table entry of A, B and M. mOS will read Linux kernel's ARP .. entry in its initialization phase. .. For A: .. code-block: console $ arp -s IP(NIF(B)) MAC(NIF(M)) .. For B: .. code-block: console .. $ arp -s IP(NIF(A)) MAC(NIF(M)) .. For M: .. code-block: console $ arp -s IP(NIF(A)) MAC(NIF(A)) $ arp -s IP(NIF(B)) MAC(NIF(B)) .. Switch's forwarding table setup .. ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ .. ======= ================= .. Port MAC address .. 1 NIF(A) .. 2 NIF(M) .. 3 NIF(B) .. ======= ================= Compile and build mOS library and application (on M) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Please refer to :doc:`../walkthrough/03_setup` and :doc:`../walkthrough/04_run`. Run 'simple_firewall' (on M) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ .. code-block:: console $ sudo ./simple_firewall Run web server (on A) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ You can setup Nginx or Apache web server for this test. See http://nginx.org or http://www.apache.org for further information. Run Wget (on B) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ .. code-block:: console $ wget http://ipaddress:port/ .. _Sample Applications User Guide: ../samples/index.html