Cisco
Note: The development effort and documentation for the Linux-on-Cisco hardware has moved to Linux-Cisco.org, predominantly because routers with non-MIPS CPUs are starting to be supported. The content of this page is largely obsolete.
Cisco Systems Routers
Cisco Systems manufactures several routers with MIPS processors. Most of these routers are relatively modern, and sometimes are still prohibitively expensive on eBay or even via alternative channels of purchase. However, as many of these routers get decomissioned, they do appear on eBay at reasonable prices. A popular model with Linux/MIPS developers is the Cisco 3600-series.
History
Cisco has a very bland history when it comes to the class of CPUs in their hardware, often known for their (ab)use of m68k CPUs for their routers, as well as x86 CPUs for their firewalling and VPN solutions.
Cisco later switched to MIPS-based solutions from a variety of vendors, most notably IDT, PMC-Sierra and Broadcom, as well as some higher-end models being based on PowerPC processors.
Models
| Model Number | CPU | CPU Clockspeed | Controller | Slot/Bus |
|---|---|---|---|---|
| Aironet 1100 | IBM PowerPC 405GP | 200Mhz | - | Mini-PCI |
| Aironet 1200 | IBM PowerPC 405GP | 200Mhz | - | Mini-PCI/Cardbus |
| Aironet 1300 | IBM PowerPC 405GP | 200Mhz | - | Mini-PCI |
| Cisco 851/857/871/877 | Freescale MPC8272 (Motorola PowerQUICC II) | 266MHz | on-chip | Mini-PCI 66MHz |
| 1600 | Motorola QUICC 68360 | 33MHz | - | WIC |
| 1720 | Motorola PowerQUICC MPC860 | 40MHz | - | WIC |
| 1750 | Motorola PowerQUICC MPC860 | 40MHz | - | WIC |
| 1801-1812 | Freescale SC8517 (Motorola PowerQUICC III MPC8500 Family) *CISCO CONFIDENTIAL* (no datasheet publicly available) |
? | on-chip | Mini-PCI |
| 1841 | RM5261A-256H | 250MHz | Marvell GT96103A | 2 x HWIC, AIM |
| 2500 | Motorola 680EC30 | 20MHz | - | - |
| 2600 | Motorola PowerQUICC MPC860 | 40MHz | - | NM (PCI) |
| 2610 | Motorola PowerQUICC MPC860 | ? | - | NM (PCI) |
| 2610XM | Motorola PowerQUICC MPC860 | ? | - | NM (PCI) |
| 2611 | Motorola PowerQUICC MPC860 | ? | - | NM (PCI) |
| 2611XM | Motorola PowerQUICC MPC860 | ? | - | NM (PCI) |
| 2620 | Motorola PowerQUICC MPC860 | ? | - | NM (PCI) |
| 2620XM | Motorola PowerQUICC MPC860 | ? | - | NM (PCI) |
| 2621 | Motorola PowerQUICC MPC860 | ? | - | NM (PCI) |
| 2621XM | Motorola PowerQUICC MPC860 | ? | - | NM (PCI) |
| 2650 | Motorola PowerQUICC MPC860 | ? | - | NM (PCI) |
| 2650XM | Motorola PowerQUICC MPC860 | ? | - | NM (PCI) |
| 2651 | Motorola PowerQUICC MPC860 | ? | - | NM (PCI) |
| 2651XM | Motorola PowerQUICC MPC860P | 80MHz | - | NM (PCI) |
| 2691 | RM7061A | 160MHz | ? | NM (PCI) |
| 2801 | RM5261A | 250MHz | Marvell MV96103A | AIM, HWIC, PVDM |
| 2811 | RM5261A | 350MHz | Marvell MV96340 | NM (PCI), AIM, HWIC, PVDM |
| 2821 | RM7065C | 466MHz | Marvell MV96340 | NM (PCI) |
| 2851 | RM7065C-466T | 466MHz | Marvell MV96340 | NM (PCI) |
| 3620 | IDT R4600 | 80MHz | Galileo | NM (PCI) |
| 3631 | RM7061a | 240MHz | ? | NM/WIC |
| 3640 | IDT R4600 | 100MHz | Galileo | NM (PCI) |
| 366x | RM5271 | 225MHz | Galileo | NM (PCI) |
| 3725 | RM5200 | 240MHz | GT96100 | NM (PCI) |
| 3745 | RM5200 | 350MHz | ? | NM (PCI) |
| 3825 | BCM1125H | 500MHz | AMD1813 | NM (PCI) |
| 3845 | BCM1250 | ? | ? | NM (PCI) |
| 4000 | Motorola 68030 | 40MHz | ? | NP ? |
| 4500M | R4700 | 100MHz | ? | NP ? |
| 4700M | R4700 | 133MHz | ? | NP ? |
| AS5300 | R4700 | 150MHz | ? | ? |
| AS5350 | RM7000 | 250MHz | ? | ? |
| AS5350XM | BCM-1250 | 750MHz | ? | ? |
| AS5400 | RM7000 | 250MHz | ? | ? |
| AS5400HPX | RM7000 | 390MHz | ? | ? |
| AS5400XM | BCM-1250 | 750MHz | ? | ? |
| 7000 | Motorola 68040 | 25MHz | ? | ? |
| 7010 | Motorola 68040 | 25MHz | ? | Cbus |
| 7120 | RM5271 | 225MHz | ? | PA ? |
| 7140 | RM7000 | 262MHz | ? | PA ? |
| 7200 NPE100 | R4700 | 150MHz | ? | PA ? |
| 7200 NPE150 | R4700 | 150MHz | ? | PA ? |
| 7200 NPE175 | RM5270 | 200MHz | ? | PA ? |
| 7200 NPE200 | R5000 | 200MHz | ? | PA ? |
| 7200 NPE225 | RM5271 | 225MHz | ? | PA ? |
| 7200 NPE300 | RM7000 | 262MHz | ? | PA ? |
| 7200 NPE400 | RM7000 | 350MHz | ? | PA ? |
| 7200 NSE-1 | RM7000 | 263MHz | ? | PA ? |
| 7200 NPE-G1 | BCM-1250 | 800MHz | ? | PA ? |
| 7200 NPE-G2 | PowerPC 7448 | 1.67GHz | ? | PA ? |
| 7500 RSP1/2 | R4600 | 100MHz | ? | CyBus |
| 7500 RSP4 | R5000 | 200MHz | ? | CyBus |
| 7500 RSP8 | RM7000 | 250MHz | ? | CyBus |
| VIP4-50 | RM7000 | ? | ? | ? |
| VIP4-80 | RM7000 | ? | ? | ? |
| 12000 GRP | R5000 | 200MHz | ? | ? |
| 12000 PRP-1 | PowerPC 7450 | 667MHz | ? | ? |
| 12000 PRP-2 | PowerPC 7455 | 1GHz | ? | ? |
The Boot ROM
The Cisco Boot ROM firmware contains startup diagnostic code (ROM monitor, or ROMmon) as well as the boot loader for the Cisco Internetworking Operating System, or IOS for short.
During early boot, the code in the boot ROM performs a Power-on Self Test (POST) and, if all tests are passed, boot into IOS from the flash.
The Boot ROM CLI
In order to gain access to the monitor in the Boot ROM, send a break sequence to the device early on in the boot process. This can be done with a C-a f (Ctrl+a f) in Minicom, for example.
Recently boot ROMs have had an undocumented priv command. This command then can be used to gain access to several additional commands, including a debugger, disassembler and additional hardware tests. To use this command, a secret password is required, known only to Cisco representatives. However, in recent times this has been cracked and the password for many models of Cisco routers can be calculated using the tool at http://ers.pp.ru/cisco/priv.html.
This is the list of commands available on the C3640 :
rommon 6 > help addrloop walk 1 thru range of addresses alias set and display aliases command alter alter locations in memory berrscan scan range of addresses for bus errors boot boot up an external process break set/show/clear the breakpoint call call a subroutine at address with converted hex args cat concatenate files checksum checksum a block of memory clrerr clear the error log compare compare two blocks of memory confreg configuration register utility cont continue executing a downloaded image context display the context of a loaded image cookie display contents of cookie PROM in hex cpu cpu / system information and control cycles excercise the hardware with all possible cycles dev list the device table dir list files in file system dis disassemble instruction stream dnld serial download a program module dram verify DRAM dump display a block of memory echo monitor echo command errlog display the error log fdump file dump utility fill fill a block of memory flash flash services command frame print out a selected stack frame help monitor builtin command help history monitor command history ifill fill a block of memory w/incrementing pattern initfs re-initialize the file system access structures jump call a subroutine at address with argc/argv launch launch a downloaded image leds check out the error LED memdebug write/read/verify scope loop meminfo main memory information memloop write or read scope loop memtest simple memory test menu main diagnostic menu move move a block of memory partest memory parity test repeat repeat a monitor command reset system reset set display the monitor variables sleep millisecond sleep command speed timed performance loop stack produce a stack trace sync write monitor environment to NVRAM sysret print out info from last system return tcal timer calibration test tlbdump display the cpu TLB tlbflush flush the TLB tlbmap initialize a TLB mapping tlbpid set/display process ID number tlbphy search TLB for physical translation tlbtest test the TLB tlbscan scan for TLB exceptions tlbvir search TLB for a virtual translation tscope timer scope loop unalias unset an alias unset unset a monitor variable watchdog test watchdog rebooting of the box xmodem x/ymodem image download
Environment variables
Boot ROM maintains certain system configuration parameters in environment variables. For example, the $MONRC can contain a starup command sequence, $PS1 contains the command prompt, and so forth. As well, the BootROM supports basic command aliasing. Both the environment variables and the alias table are stored in NVRAM so that their values persist, even when power is off.
The Boot ROM API
The Boot ROM API provides some simple APIs for IOS (for example, putchar and version information commands). Unlike many other firmwares in the world of MIPS, the syscall opcode is used to call these firmware APIs. Note that register a0 must contain the syscall number.
A sample "Hello, world!" program can be found here.
To load this hello world program, you will need to enter in the private mode of the rommon. To accomplish this, follow the instructions from http://ers.msk.ru/cisco/priv.html .
Once done, you can test sending the file using xmodem :
rommon 8 > xmodem -r Do not start the sending program yet... Invoke this application only for disaster recovery. Do you wish to continue? y/n [n]: y Ready to receive file help ... Download Complete! program load complete, entry point: 0x80008000, size: 0x4c Hello World!
Note that testing your programs with Dynamips works as well :
./dynamips -P 3600 ciscohello/hello.bin Cisco Router Simulation Platform (version 0.2.7-x86) Copyright (c) 2005-2007 Christophe Fillot. Build date: Aug 8 2008 09:02:48 IOS image file: ciscohello/hello.bin ILT: loaded table "mips64j" from cache. ILT: loaded table "mips64e" from cache. ILT: loaded table "ppc32j" from cache. ILT: loaded table "ppc32e" from cache. CPU0: carved JIT exec zone of 64 Mb into 2048 pages of 32 Kb. NVRAM is empty, setting config register to 0x2142 C3600 instance 'default' (id 0): VM Status : 0 RAM size : 128 Mb NVRAM size : 128 Kb Chassis : 3640 IOS image : ciscohello/hello.bin Loading ELF file 'ciscohello/hello.bin'... ELF entry point: 0x80008000 C3600 'default': starting simulation (CPU0 PC=0xffffffffbfc00000), JIT enabled. ROMMON emulation microcode. Launching IOS image at 0x80008000... Hello World! Image returned to ROM. % No memory map for code execution at 0x0 % Unable to create instruction block for vaddr=0x0 insn_page_compile: unable to create JIT block. VM 'default': unable to compile block for CPU0 PC=0x0 Shutdown in progress... Shutdown completed.
Be forewarned -- the Dynamips emulation of hardware is far from complete and it doesn't quite emulate all the quirks of the firmware. For example, the Dynamips loader is actually capable of loading ELF files with multiple segments.
Binary Format
IOS executables are shipped in a raw binary format (known as a .bin file to many). For MIPS-based devices, this is just conventional MIPS Big-Endian ELF, however Cisco does play a dirty trick in using a non-standard e_machine value in the ELF header. This seems to be based on the router model. For example, the Cisco 3620/40 routers have an e_machine value of 0x1e, while the 3660 routers have a value of 0x34. Cisco likely does this so people don't attempt to run IOS images intended for different models of routers.
In order to alter the e_machine value, a recent version of objcopy can be used with the switch --alt-machine-code 0x1e (in this example for a Cisco 3620/3640 series router).
As well, there are limitations placed on the binary format due to the behaviour of the software in the Boot ROM. ROMMON cannot load multiple ELF program headers, as it is very broken. Thus, to work around this problem, toolchains must be built with --target=mips-elf. CiscoLoad also helps to work around this issue, by allowing you to chainload ELF images with multiple program headers.
Finally, all symbol tables must be removed.
Boot sequence
The Boot ROM can load and execute a block of executable code (such as IOS) from various internal locations: the internal FLASH module(s), a PCMCIA Linear Flash card and (unofficially) TFTP. More recent models also support PCMCIA IDE, CompactFlash and USB flash. Also the Boot ROM supports compressed images (.gz) with embedded helper and text files that contain a command sequence (like a shell script). Nowadays -mz- IOS distributions are compressed with ZIP and have a built-in ELF ZIP decompressor "piggybacked" on top, since ZIP provides better compression.
To boot from the PCMCIA ATA or CompactFlash the media should be formatted on the CISCO router with format disk0: command. This command creates a FAT structure with second bootloader (MONLIB) resides in the hidden FAT area. More inforamtion may be found in the ATA Monlib Enhancements article. Note that this does not apply to devices with Linear Flash, which use the Cisco IFS filesystem.
Cisco 3600
The Cisco 3600-series were entry-level edge routers built on the IDT R4700 MIPS CPUs. The system controller is a Galileo GT-64010.
The Cisco 3600-series routers were designed with limited expandability in mind. As such, the Cisco 3620 has two expansion slots; the 3640 has four (as is reflected in the size of the units). These expansion modules sit on the NM expansion bus, which is essentially a modified PCI 2.1 bus with some proprietary extensions relating to EEPROM identication and what Cisco calls OIR, or Online Insertion and Removal.
Deeper technical information on the Cisco 3600-series can be found at http://www.cisco.com/warp/public/63/36xx-arch.pdf. Cisco also has made available a document describing the memory map of these routers, which can be found at [1].
NM Cards
Many NM cards used in the Cisco products contain a conventional PCI ICs. As an example, NM-1FE is built arount AMD PC-Net PCI Ethenet controller, NM-2FE contains two of them. Another "non-intellectual" NM- cards has some sort of PCI bridge and peripheral ICs (serial NMs, ISDN NMs e.t.c.)
More complicated NM cards have a Motorola QUICC on them. While the purpose of this is unknown at this time, it is likely that the protocol decode core within the QUICC is used to assist in protocol functions; firmware appears to be loaded into the QUICC at startup time by IOS (via the IOFPGA on the board), but I have yet to capture the firmware being loaded.
One possiblity is that once Linux/MIPS is booting on this hardware, the QUICCs can be leveraged to improve routing/networking performance of the 3600-series devices if they are to be used in a networking context.
Cisco IOS Flash Filesystem (IFS)
The Cisco IOS Flash Class B Filesystem[1] file entries start at the first byte of flash (no additional structures). Each file is denoted by a magic number, 0xbad00b1e.
A file is denoted by a 64 byte header with:
- 32-bit magic number (so it seems, I've never seen it change)
- 32-bit file length (confirmed)
- 64 bits of flags/modification time/crc (haven't bothered to figure out which byte(s) is/are which)
- 48 bytes of filename
This is very primitive, however effective and means that code can be executed in place from the flash; the CiscoLoad bootloader actually takes advantage of this with the second stage bootloader essentially running straight from flash.
Current Status
See [2].
External Links
- Cisco 7200 Simulator
- Cisco 7200 Series Router Architecture
- uClinux port for the Cisco 2500
- uClinux Ports
