Repository navigation
ip27: clean up the IP27 machines, and fixes found bringing up the Onyx2 - #481
Draft
jamesbraid wants to merge 44 commits into
Draft
jamesbraid wants to merge 44 commits into
jamesbraid wants to merge 44 commits into
Conversation
A device that makes the processor wait answers a read busy and gets it
again. The Hub's processor path took any answer but OK for a failed read
and latched its address in the error status, so a pipe that answers
busy while its command ring drains left "Read Access Error" behind, and
IRIX panicked on it at start-up ("Invalid RRB in error stack") when it
read the Hub's error state. Only a failed read is recorded now.
The Onyx2 read its node count, flash and controller store from [origin2000], so its guide had to explain that it was configured under another product's name, and the Origin 200 and Origin 2000 each had a section struct with the same keys. No other machine does either: [o2], [octane], and the Fuel and Tezro each have one section named for the product. [origin200], [origin2000] and the new [onyx2] now deserialise into one Ip27Section, and MachineConfig::ip27_section returns the one the profile reads, so board code no longer names another product's table. validate() refuses a table that is not the profile's own, naming the one to rename it to, which turns an Onyx2 file that still says [origin2000] into a clear error rather than silently ignored keys. It also refuses nodes on the Origin 200, which has one node board. The keys landed only days ago, so the rename is cheapest now, before users have files. FEATURES.md gains the missing [origin200] table and an onyx2.toml.example is added.
The node PROM copies the I/O PROM from the BaseIO's flash at "Loading BASEIO prom" and, finding nothing, runs its own reduced copy, which has no graphics console: with console set to g it prints "Cannot open /dev/graphics/textport" and stays on the serial port. io_prom, in [origin2000] or [onyx2], names the image the IRIX 6.5.30 media ship beside the node PROM. It is fitted as a 1 MiB x16 part on the Bridge's flash port that answers reset and autoselect with the codes the loader accepts, and refuses program and erase. Without the key nothing is fitted, as before. The Origin 200 has no BaseIO card and refuses the key.
The three SN0 machines were spelled out as an or-pattern about ten times in config.rs and three times in machine.rs, which also repeated the same word list for the Origin 2000 and the Onyx2. MachineProfile::is_ip27 says it once, ip27_name gives the product's name for a sentence, and the machine.rs lists and tests walk MachineProfile::EVERY, so a fourth product cannot be left out of one of them. Several messages named the wrong machine along the way. The DaynaPort refusal said "not available on the O2" for all three SN0 machines. network.mode = "off" listed "the Origin 200 and 2000" and left out the Onyx2. The second-SCSI-controller refusal listed only the Indigo2 and the O2, although the Octane and the SN0 machines also have two; it now derives the list from scsi_controllers.
The Origin 200, Origin 2000 and Onyx2 test modules each opened with the same 60 lines: a scratch directory, a PROM stand-in and a base configuration that differed only in the profile, plus a copy of big_stack that display_stop_tests already provides. They now share one test_support module. The configuration also bound three listeners on port 0 and dropped them before the machine started, so another test thread could take a port between the drop and its use. The serial backend binds the port it is given and reports the one it got, so the serial ports are 0 and the machine finds its own free ones. The monitor port stays unset, since no test starts a machine. Scratch directories take their product as a prefix, because the three modules now share one name space inside one process.
The machine module and the Hub's own documentation both called Hub::inbound and Hub::interrupt_target "the smallest adapter the Bridge needs, meant to be replaced by the shared host interface". That interface, xtalk::HostPort, landed in the same series and takes exactly those two parts. Hub::host_port(widget) builds one, with the Hub's 128 interrupt numbers, and all four places that wired a Bridge to a Hub by hand use it. The two stale paragraphs go. The Origin 2000's Bridge sits behind a Crossbow but was built with Upstream::single, so its transfers and interrupt packets ignored the Crossbow's link. It now takes xbow.upstream, as the Onyx2 does, and the port's link gates them. The model has no way to take a fitted port down (a link reset brings it back, and only a snapshot restore clears an alive bit), and the PROM is not known to try, so nothing a guest can do shows the difference yet; the new test pins the wiring instead. A Bridge on the first node's Crossbow reaches that node's Hub at widget 9, and a packet naming the second node's widget 10 finds no host.
The Octane, the Origin 2000 and the Onyx2 each put a Bridge behind a Crossbow, and each repeated the same wiring: tell the Bridge which widgets have a live link, and reset the Bridge when the Crossbow resets its link. Xbow::serve_bridge does both for the port the Bridge sits on. The Octane's behaviour is unchanged; its wiring moved from two places to one call. The existing Octane tests that drive a link reset (ip30::tests and the Crossbow's link-reset tests) pass as they were, and a new Crossbow test shows a reset of the Bridge's port resets the Bridge while a reset of another port does not.
Onyx2Deskside::specs and Origin2000::specs were the same 25 lines, apart from the slot identity, and each file declared its own widget_hub, slot_bits and hub_host and built the BaseIO's Bridge the same way. The Indy and Indigo2 are one machine with the product as data; the IP27 machines already had one Ip27Machine over a board, so the desksides become one board, Deskside, and the product a DesksideLayout read once when the machine is built: each node slot's Hub widget and slot identity, the BaseIO's widget, board record and keyboard ports, whether the node boards share one Crossbow, and the graphics slot's widget if the product has one. Nothing tests the product on an access. The two topologies stay two build-time functions, wire_per_node for the Origin 2000's Crossbow on each node board and wire_shared for the Onyx2's one Crossbow; they and specs, controller and the Bridge builder share everything else. The graphics slot is a field of the layout, so the InfiniteReality pipe fits where the test widget is now. Ip27Board loses its NAME constant, since one board type serves two products; the product's name comes from the profile. Both products are reached through DesksideMachine, and the two test modules become one that runs the shared cases against both layouts. No behaviour changes: the tests that read the flash words, slot identities and widget map for one and two nodes pass as they were. FlashValues::ORIGIN200_R10000_180_1M was the base for the Origin 2000 and the Onyx2 as well as the Origin 200, each overriding the machine type and the processor module's words. The values are the stock PROM image's, not one product's, so the constant is STOCK_R10000_180. Asking Deskside for the layout of a profile that is not a deskside is an error on the config path, not a panic.
The Origin 2000 module reported its power, reset and NMI lines as ModuleEvent. The machine's ModuleController wanted the Origin 200 controller's ElscEvent, so the deskside translated one into the other and dropped the reset line's falling edge on the way. Both controllers now emit one type, renamed ModuleOutput because two controllers send it, and ModuleManagement hands the module's outputs on as they are. controller_output remains the one place that maps an output to a MachineEvent. T 8000 he module still holds its reset line high for the 100 ms of a pulse, but reports only the rising edge, as the translation already did; no machine reads the fall. The reset test checks that the fall is silent.
The Onyx2's graphics slot held a TestWidget in production code, a second "log and fail" bus device beside ip27_machine's Unmodelled, with its own log text. Unmodelled takes an optional widget identification word that it answers at offset 4, and the slot uses that; TestWidget goes. The log line still says the access was to the graphics slot, which holds a test widget.
pci_devices took nine positional arguments, three of which (the scsi name, the product name and the Bridge) every caller derived from the configuration or built just before. PciPopulation holds the five things a board actually chooses (the Bridge, the IOC3 and Ethernet it fits, the module controller and the SuperIO pin), and the two names come from the profile inside pci_devices. BoardIo's test-only bridge field stays. The tests attach a PCI function to the Bridge's bus and check that dropping a machine frees it, and the Hub's widget table holds the Bridge as a dyn BusDevice behind a Crossbow port, which gives them neither.
Three places walked the banks the same way: skip the empty ones, number the RAM of the fitted ones, split a bank into its runs and track where each run starts in its RAM. NodeMemory::new built the decode table that way, Home built its lookup table that way, and NodeZero::apply repeated the walk every time a node's number changed. A change to the stride rule needed three edits. layout() makes the walk once, as a list of Runs stored in NodeMemory and shared by Home, NodeZero and memory_map. The memory map's sixteen-entry name table, which existed only to give each run a &'static str, becomes one name per bank.
The module notes said unfitted memory reads zero and drops writes, "logged under phys hole on". It was not: the default device had no log path, and phys hole on only toggles BlackHoleRegion, which reads all ones. A guest reading a bank that was never fitted left no trace. Unfitted memory now fails every access with a bus error and logs it under the hub category, which is the one the SN0 guides tell a user to turn on. The zero was an approximation nothing supported. The basis for the fault is a software expectation: the IRIX kernel's handler for an uncached bus error lists memory that does not exist as a case, and SGI's Embedded Support Partner event catalogue carries "Access to non-existent memory address". No capture from a physical machine shows it. The PROM's memory sizing does not need zero: it reads the directory back door, which answers its own fixed pattern for an absent bank, so it never reaches unfitted memory. Boots to an IRIX shell with the hub log on, with one 128 MiB bank, two, and a 256 MiB bank on the R12000 module (four 64 MiB runs with three holes between them), log no unfitted access. Only addresses outside the fitted runs reach the default device, so fitted memory pays nothing for the change. The test checks the contract at every width and no longer drives the process-wide log with retries.
The module controller's logs (the Origin 200's ELSC, the Origin 2000 module's controller, their configuration NVRAM and the I2C controller) went out under LogModule::Bedrock, which debug.md describes as the Fuel's devices on its Bedrock hub. They are SN0 parts, so a user turning on bedrock to watch a Fuel got controller lines, and a user trying to watch an Origin 200's controller had to know to turn on a Fuel category. They now log under a new module category, and debug.md lists it with hub, which the SN0 guides tell users to enable and the list left out. bedrock stays for the Bedrock devices. An unknown name in IRIS_DEBUG_LOG prints "unknown log module" at startup and is otherwise ignored, so debug_log = "bedrock" on an SN0 machine still starts and now enables nothing. A round-trip test pins the name table, the category list and the count together.
The machine woke every 10 ms for the life of the guest to bring the module controller up to date, even when the controller had nothing pending. The controller keeps its own time, and a guest's bus access already replays what is overdue, so the visit only matters when the bus is idle while a countdown or a pulse ends: an operator typing at the AUX port, a power-down running out. Elsc and Origin2000Module now say when they next have an output to give or a committed action to run (next_due), and tell the machine when an access may have moved that (set_wake). ControllerWatch holds one one-shot timer, armed for that time, re-armed after each visit and each wake, and removed when the machine stops, as the Hub's real-time comparator does. The grant ring's ticks and a message's expiry change only what a bus access sees, and the access replays them first, so they are not due times; an idle controller arms nothing. New tests drive the controllers on their explicit clock: an NMI wait and pulse, a power-down countdown and the module's reset wait and pulse report the right deadline, and an access wakes the machine only when the schedule moved. The machine test uses a scripted controller to check that a visit happens when due, a moved schedule replaces the timer, and a stopped machine holds none. A bus access can pass a deadline before the machine's visit. It then queues the output itself and clears the deadline, and with nothing due the machine would arm nothing: a power-off, reset or NMI would wait for some later access to move the schedule. The controller and the module therefore also report themselves due, at once, while an output they queued has not been taken, and the watch never arms a zero-length timer, so a controller that stays due cannot spin the timer thread. A test lets an access beat the visit and checks that the queued outputs are due.
Iris had three hand-written AMD command state machines: AmdCore (landed for the Fuel, no consumer upstream), the node flash's, and the BaseIO flash's. AmdCore now takes a Geometry: bus width, sector count and size, the unlock addresses and how many bits of them the part compares, the autoselect codes, whether it answers a CFI query, whether program and erase are refused, whether erase takes time, and where an unsupported cycle is logged. The default is still Fuel's 8 MiB x16 part, and its public names (Default, Effect, SECTOR_SIZE, PART_SIZE, the erase times) are unchanged, so a user of the engine compiles as it did. NodeFlash (1 MiB x8, unlock 0x5555 and 0x2aaa, codes 0x01 and 0xd5) and BaseioFlash (1 MiB x16, unlock matched on 11 bits, codes 0x01 and 0x22da, read-only) become owners of an AmdCore and keep only their storage: the node flash applies a program or a selected erase and writes it through to its file, the BaseIO flash serves its image. Their tests pass unchanged. The file format is the raw 1 MiB image as before. One behaviour follows from sharing the engine. The node flash treated 0xf0 as a reset even where it was the data cycle of a program, so a programmed byte of 0xf0 never reached the array; the engine, like the part, takes every bit of the data cycle, and a test pins it. The node flash still allows a new command sequence from autoselect without a reset, which the others do not, as a geometry flag. The part's data sheet agrees (AMD Am29F016D, publication 21444 rev E+2, page 16, "Reset Command" and "Byte Program Command Sequence"): a reset is accepted between the cycles of a sequence before programming begins, and the program address and data are the fourth bus cycle, so a data byte of 0xf0 is data. The data sheet checked is the Am29F016D's; the node PROM's part is an Am29F080 of the same AMD command family, which was not available to check. The shared engine also logs and cancels a sequence that gets a wrong address on an unlock cycle, where the node flash's own code silently went back to array mode.
The Hub is one ASIC with four register sections, but it read as five
devices: hub.rs and hub_{pi,md,io,ni}.rs were flat public modules, and
machines reached into the units with hub.io().connect_widget(..),
hub.pi().set_timer_manager(..), hub.md().set_onewire(..) and a raw
register offset to enable a processor. The units are plain parts of one
chip, as the mgras board's are.
They now live in src/dev/hub/{mod,pi,md,io,ni}.rs with their tests beside
them, the unit modules private, and drop the chip prefix inside the
module (hub::pi::Pi, not HubPi). The Hub gets the methods a machine
needs (connect_widget, connect_packets, set_timer_manager, set_onewire,
enable_cpu), and SlotSignals and the register offsets machines and their
tests name are re-exported from hub. Log names and the locks monitor's
names are unchanged. The commit is the move and the narrowed surface; no
behaviour changes.
The management interface of a transceiver is the PHY chips' contract, not the MAC's, and the PHYs reach back into the MAC's file for it. Give it a module of its own. The ICS1890 and DP83843 carried inherent methods that a forwarding impl re-exposed; they implement the trait directly, as the DP83840 already did. The ICS1890 had no tests of its own, so cover its reset values and writable masks first.
The Hub carried a mechanism for a processor-rendezvous hold that nothing builds: HubObserver and Store, the observe() calls in the processor, I/O and network units, Hub::set_observer, and the observer tests. It also had Hub::attach_uart and a UART overlay over the memory block, which no machine fits; Hub::set_powered, which no machine calls (the module controller's power output is a machine event, not a Hub input); and HubConfig::nasid, a node number preprogrammed for a synthesized firmware that does not exist here. Only tests used any of it, and the audit expected another branch to; that branch does not. All of it goes, with the tests that exercised only it. Every register test stays. The network unit's enable switch, which two unit tests use to power a peer down, is now built for tests only.
The Hub's four units were joined to the Hub, and to each other, by back-references: the processor and network units each held the machine's hooks through a forwarding shell that existed to copy the cached-alias size into the Hub; the network unit told the Hub to rebuild a 256-entry claim map through a second trait object; and the I/O unit's block transfer engines reached memory and interrupts through a stored Weak<dyn NodeFabric> to the Hub. Machines also saw hooks for the processor enable and NMI stores that only logged. A unit's register write now returns what it changed, as the AMD flash engine returns its program effect, and the Hub acts on it once the unit's lock is released: the node number and cached alias reach the machine, a soft reset reaches the enabled slices, a restart reaches the machine, and a launched transfer runs with the Hub passed as its fabric. Writes through the node's I/O space take the same path, through a weak handle the space holds only to write. The enable and NMI stores log in the processor unit, which is where the log line belongs. The claim map goes. The network unit mirrors its node number and its neighbour's in two atomics when either changes, and the Hub's decode compares a node number with them, so a decode still takes no lock and the rebuild, its trait and the 256 stores per renumbering are gone. A vector request that renumbers the neighbour comes back to the launching Hub as an effect, which tells the neighbour's machine through the hooks the two share. The XIO link's reset closure, which no machine ever connected, goes too: a warm reset is logged, as it was. Unit tests drive the units directly and read the returned effects; a new Hub test checks that a write through the I/O space acts like one on the window.
One 3000-line chip behind a single lock becomes the shape Linux's drivers/mfd/ioc3.c drives: a parent that owns the PCI function, the register window and DMA, and one child per function that the parent dispatches to by offset. The children follow what the public drivers show of the hardware, not just the register map. The 16550s and the parallel port's ECP registers sit in the 0x20000 window and are the external Super I/O part: Linux drives the UARTs with the stock 8250 driver at offset^3 and OpenBSD attaches com and lpt there. The ring-DMA engines, the PS/2 controller (main register block at 0x9c) and the parallel DMA engine are the IOC3's own. So superio.rs holds the UARTs and the ECP file under one lock, the serial and parallel engines reach them through it as the part's own bus master would, and kbd.rs, w1.rs and gpio.rs each hold one function. A board that lacks a function has no child; its sub-range reads zero. Children raise their interrupt bits through the parent, which keeps the status and enable words and the level it last drove onto the SuperIO line under one small lock. A change to the words and the drive of the line happen inside that lock, and the Bridge is called only when the output (status AND enable) changes. The lock orders the edges, so the Bridge, which sends a packet on each rising edge, sees only edges the registers made and in the order they made them. The Bridge ignores a level it already has and drops its own locks before it calls a device, so the order is child lock, then this lock, then the Bridge's, and never the reverse. A character the host types while an access holds the PS/2 controller is still picked up by the holder. Snapshot keys and the NVRAM file are unchanged. The Ethernet MAC, the DS1685 and the board configuration are not touched here.
Every store to PI_INT_MASK*, every device interrupt the Bridge sends (PI_INT_PEND_MOD) and every real-time pending or enable change published the processor's three lines under a reentrant lock, then the controller mutex, then one mutex per line: eight lock acquisitions for a change, whether or not any line moved. The controller behind them, CpuIrq, only set or cleared a bit of the processor's interrupt word, so neither mutex protected anything the dispatch lock did not. The Hub now writes the word itself (MipsCpu::interrupts_ptr passed to set_cpu_interrupts), under one lock for the whole PI. The rule: a device changes the state its output comes from, computes the output and drives the line while holding the one lock that guards that state, and writes only what changed. Every writer of the pending banks, the masks, the comparator's pending and enable bits, and the reset, takes the lock, stores its input, computes the three lines for each processor and writes the lines that differ from the word. Because the input changes under the same lock as the sample, no input moves while the lines are computed, a level is always one the registers held at one instant, and each edge in the word is a change of the registers, in order. This follows HACKING.md's Interrupts section (a controller updates its state and drives the next one up under its own lock, so two threads cannot deliver levels out of order) over Locks rule 2, which says to raise interrupts outside your state lock. Outside the lock, a writer that sampled the lines and was then descheduled writes its stale level after another writer's fall and rise: the word shows one edge where the registers made two, or a level no register held. Holding the lock across the write keeps the edges in the order the registers changed. That is safe only while nothing calls down into the Hub with `lines` held. The word is a leaf (an atomic nobody calls out of), and the lock is taken inside the state mutex, never the other way round, so nothing in the Hub is called while it is held. The banks, masks and bits stay atomics, written under the lock, so a processor that polls them as mailboxes takes no lock. An earlier version of this commit published without a lock: writers counted their arrival and one at a time sampled the inputs again until no one had arrived. A model check of it found a spurious rising edge from three threads, and its claim of at most one rising edge per pulse was false. That publisher, its arrival counts and the test hooks that planted a change between a sample and a write are gone. (HEART and CRIME drive their lines under their register locks the same way.) A booted Origin 200 under IRIX 6.5.30, from the PROM menu through a login and a minute of find, cat of the kernel and 40 process spawns, counted 23 stores to PI_INT_MASK*, 60211 interrupt-modify calls and 65051 publications. Before this change those were about 520000 lock acquisitions; now they are 65051 of one lock that a second thread rarely wants. Not remeasured: the cost of the lock against the lock-free version, or the host instruction count with it; the lock is one uncontended acquire and release per change, on a path that runs some thousands of times a second. The tests change a line and check the edges that were written. A write that changes no line makes no edge. Four threads pulse a pending bit and the comparator while two others rewrite the same mask and enable values and drive the lines over and over: each pulsed line rises exactly once and falls exactly once per pulse, and the line nobody pulsed never moves. A third test races an input, a mask and an enable on three threads and checks the word against the registers at rest. A fourth stalls a writer inside the write of the lines and has a second thread clear and set the same bit: the input must not change until the first writer lets go, and the word then shows a rise, a fall and a rise. It fails when an input is changed before `lines` is taken; the threaded tests above did not notice when that change was planted. The word must outlive every thread that can publish to it. A machine stops its processors, devices and the Hub's timers first, clears the words in Drop, and only then frees the processors, since a cabled Hub can still reach this one's units; a publisher that loaded the pointer before the clear and ran after the free would write freed memory, which the stop rules out. A test holds a Hub past its machine to check that nothing publishes into the freed processor.
Linux's ioc3-eth is a cell the IOC3 parent creates, not a device the board attaches to it. Here the machine built the MAC itself, handed it to the chip through an Ioc3EthPort trait that only it implemented, and had the chip hand back DMA and a closure for the PCI pin. The parent now builds the MAC from the board's transceiver choice and shares its chip state with it, so the MAC reads the chip's DMA and drives the PCI pin directly. The Ioc3EthPort trait, attach_eth and the three constructors named after one machine each go. The parent starts, stops and resets the MAC with the rest of the chip; a machine still reaches it through Ioc3::eth to attach the network link and save its state under its own snapshot key. The MAC's threads are ioc3-eth-rx and ioc3-eth-net on every machine; the Octane's were ip30-eth-*. Tests that stood a recording mock in for the MAC use the real one.
The Origin 200 guide still said the Origin 2000 was not built, the README's machine row listed none of the three SN0 machines, and the machine module's notes described a single-node, Origin-200-only machine. Several comments named the Origin 200 for things all three share: the processor and module tables, the bank count, and the cable between two node boards. HACKING.md and the BusDevice::write_narrow_from documentation said that only Bedrock's ECC view reads the source register's other lanes. No device in this tree overrides it yet, and the call is a dyn call where the uncached byte and halfword stores used to call write8 and write16; both now say so. A clanker note records two rules a reader would otherwise have to rediscover: a renumbered node sends node zero's cached window beyond its alias to the Hub that holds zero, which costs it the ppmem fast path, and a Bridge behind a Crossbow picks its host by the widget the guest programs, gated by that port's link. CHANGELOG gets one SN0 entry. The cost of the second node is measured and recorded in the Onyx2 and Origin 2000 guides: under IRIX 6.5.30 on two nodes, a CPU-bound awk loop took 28.1 s on average on the first node's processor and 61.8 s on the second's (five runs each, alternating in one boot), about 2.2 times as long. Nothing changes in the code for it.
The MAC drove the PCI pin through a queue of level changes, a lock that kept their delivery in order, a boxed callback and then the chip's own two more locks and level cache: six acquisitions to move a pin that reaches one Bridge input. The MAC now changes its status and enable registers and drives the pin inside its one register lock, and calls the Bridge only when the output changes, remembering the level it last drove. The lock orders the edges, so the Bridge, which sends a packet on each rising edge, sees exactly the edges the registers made and in the order they made them. The one case that needs two edges, an acknowledge followed by a completion that landed on the same bit, drives the pin low and then high inside the register write, as the queue did. The Bridge ignores a level it already has and drops its own locks before it calls a device, so the lock order is the MAC's, then the Bridge's, and never the reverse. With the SuperIO line already on the same rule, the chip's line_order, pin relay and level caches and the MAC's irq_order, callback and event queue are gone. Tests: the SuperIO line against a host key racing writes of the enable words, four threads changing one source and asserting no repeated level reaches the parent, and a MAC reset, raise and guest acknowledge racing on three threads asserting the pin's calls alternate.
Each machine that carried an IOC3 built it from a literal of its own: an eleven-field Ioc3Config for the Origin 200, another for the Origin 2000 and a copy of that with the keyboard flag flipped for the Onyx2, plus the Octane's and Fuel's constructors, with the subsystem ids and the Bridge input the SuperIO interrupt uses scattered as constants through the machine files. Linux keeps this as a table, ioc3_infos, keyed by subsystem id. Do the same: board.rs has a row per board (IP30, IP29, BASEIO, BASEIO6G, IP34) with the children it fits, its transceiver, the SuperIO Bridge input, what is wired to the GPIO pins (straps, and the LEDs a status-bar bit mirrors) and the PCI header policy, and Ioc3::new takes the row, the station address and the clock the machine built for the byte bus. The one-wire board records leave the chip: the chip's wire carries the station-address part, and a machine that has identity records puts them on it. The GPIO block is the chip's and its wiring is the row's, so the chip no longer tests for the Octane's subsystem id. The Onyx2's BaseIO-G IOC3 takes Linux's id for part 030-0734 (0xc300) in place of the BaseIO's 0xc302. The chip decodes only 32 bytes of configuration space, so no guest reads the id; it is the row's key. The Octane still gets the Octane's records, now from its machine. A test per row reads back the id, children and straps, and another checks each row's pins write, read back and light the bits it lists.
The M48T35-compatible Timekeeper is the clock of the Origin 200, the Origin 2000, the Onyx2 and the Fuel, but its constructors were named after the Fuel. Rename fuel, fuel_at and fuel_with_nvram_file to new, at and with_nvram_file, and say in the module doc which boards use it.
The DS1685 on the Octane's byte bus is a separate chip, but its model, its 512-byte NVRAM image, the debounce that rewrites the file and the writer thread lived inside the IOC3, along with the power-off event and a snapshot of the clock. A board that fits the other clock, the M48T35-compatible Timekeeper, attached it from outside after the chip was built, so the byte bus had two unlike owners. Both clocks are now built by the machine, with the file they persist in and the event channel, and passed to Ioc3::new as the byte bus. The DS1685 moves to dev/ds1685.rs with its writer thread and sends the power-off event itself as the guest writes it. The IOC3 decodes the two selects, starts and stops the part, and arms the debounce deadline; the snapshot key (ioc3.chip.rtc), the file format and the IOC3-NVRAM thread name are unchanged.
A machine with a V10 or V12 graphics board is an Octane2: it already took the Octane2's Crossbow and Bridge revisions. But the four one-wire identity records the PROM and IRIX can read, the processor module on HEART, the system board on the Bridge, and the front plane and power supply on the IOC3, were the Octane's for every machine, so an Octane2 presented an Octane's front plane, power supply and system board. The records are now product data. The Octane keeps the ones it had; the Octane2's come from one reference unit's inventory: system board 030-1467-001 rev B, front plane 030-0891-003 rev H, power supply PWR.SPPLY.ER 060-0035-002 and processor module PM10360MHZ 030-1590-002. That module is the 360 MHz single-processor one and is used for any processor count, as no inventory of a dual-processor Octane2 is at hand. Barcodes and ROM serials stay the marked-synthetic ones. This changes what an Octane2 guest reads from the one-wire parts. Whether the PROM or IRIX acts on any of these records was not checked.
The Onyx2's MediaIO daughtercard carries a second IOC3 that has both serial ports, a keyboard and mouse controller and a parallel port but no Ethernet, clock or one-wire part (Linux's ip27_mio_setup; its SuperIO interrupt is wired to Bridge input 6). Add its row, with Linux's subsystem id 0xc301. The MicroLAN master is the chip's, not the board's: it is in the register block whatever sits on the wire. A row without the one-wire part therefore keeps the master and gets an empty wire, whose reset pulse draws no presence answer. (The PROM polls the master of every IOC3 it finds; a master that never finishes a slot hangs it.) The per-row test checks the slot finishes and the presence answer against the row.
A published hinv -vm of a BaseIO-G lists two boards under the BaseIO slot: the base board, assembly 030-0734-002 revision N, and the MediaIO daughtercard, 030-0880-003 revision G. The model carried only the base board's record on the Bridge's one-wire line. IRIX reads the MediaIO record to decide what its IOC3 has; without it the kernel falls back to a default that turns Ethernet on for a chip that has none. Put the second record on the line as a separate part with its own ROM serial, so a one-wire search tells the two apart. Barcode and serial are synthetic as before.
The BaseIO-G's MediaIO daughtercard carries a second IOC3: a published hinv -v of one lists IOC3s at slots 2 and 6 on the BaseIO's Bridge, and Linux's Bridge setup for the board has an IOC3 at slot 6 whose SuperIO interrupt goes to input 6. IRIX also finds the keyboard, mouse and parallel port there. Fit it from its own board row: both serial ports, keyboard and mouse, the parallel port, and no Ethernet, clock or one-wire part. The board is reset, started and stopped with the rest of the BaseIO. Its serial ports have no host backend (they are peripheral lines; the board IOC3's first is the console), and the host's keys and pointer stay with the board IOC3's controller; which keyboard connector reaches which chip is not known.
The MediaIO daughtercard carries the RAD1 audio chip: a published hinv -v of a BaseIO-G lists 0x10a9:0x0005 at PCI slot 7 of the BaseIO's Bridge, beside the SCSI controllers at 0 and 1 and the IOC3s at 2 and 6. The model had no audio on the Onyx2. Reuse the Octane's RAD1 model: it takes its timers from the node, its host output from the machine's audio setting, and is started and reset with the rest of the board. It raises no interrupt, as the audio driver registers no data-path interrupt. An existing test attached a stand-in card at device 7; it uses device 4.
Add one page for the chip: the parent and each child against the Linux cell it stands for, which parts are the IOC3 and which the external Super I/O, how the interrupt words and DMA work, what each lock covers, each board's row and wiring (including the Octane2's records and the Onyx2's MediaIO and RAD1), what is modelled and what is approximated, and which snapshot keys remain. Point the older notes that named src/dev/ioc3.rs and ioc3_eth.rs at the new paths, and retitle the PS/2 page, which is not Octane-only. The clanker notes keep only the traps (what the PROM insists on, the pacing rule, scan code set 3).
The R10000 family puts a two-bit uncached attribute beside the address of every uncached reference. An unmapped one takes it from XKPHYS bits 58:57, which already reach the board in those bits; a mapped one takes it from EntryLo bits 63:62, which the TLB stored and then dropped when it made the physical address. Every mapped uncached reference therefore went out with attribute 0. The Onyx2's Hub picks node memory, its own registers or a crossbar widget by that attribute. IRIX maps the InfiniteReality pipe's command page into the kernel and into the graphics start-up and X server as an uncached accelerated page with attribute 1, I/O. With attribute 0 the stores landed in the Hub-special view at node memory past the fitted 128 MiB and vanished: the kernel's pipe start never reached the pipe, and the driver gave up with "IR0: mopup timeout" and death reason 17. A TLB entry with cache algorithm 2 (uncached) or 7 (uncached accelerated) now carries its attribute in bits 58:57 of the physical address it gives, the same place an unmapped reference carries it. Cached entries are unchanged. The attribute is folded in when the entry is written, so translation does no extra work. The other processors cannot store those EntryLo bits, and the boards that do not decode the attribute cut it off with the executor's physical mask.
The Hub still asked the machine four things through a hooks trait: the node number, the cached alias, a processor soft reset and a restart. A trait that a board implements is the wrong direction for a chip, and each call hid ordering: the effect carried the new value, so two writers that raced could leave the machine's routing at the older one. The node number and the cached-alias size are now published values the owning unit stores under its own lock, as the register changes. The Hub's address decode reads them from there with no lock, as before and through one pointer. The machine's node-zero routing is handed to the Hub once and re-reads both under its own lock when a store, a neighbour's vector write or a reset has changed one, so the last value written wins. A soft reset is a line, not a call: the Hub sets bit 63 of the processor's interrupt word, which is what the processor's reset signal did, without a lock and without the machine. A restart goes out on the machine's event channel as a hard reset, wired the way the Indy IOC's power events are. Enable and NMI stay stored and logged. The node number stays 16 bits because the register field is 9 bits wide. No per-access cost is added: the claim decode loads the number and the alias through a pointer it already followed.
src/ip35.rs held the bounded MMIO windows, their sparse map and map_device, but the only code that uses them upstream is the SN0 node bus, so IP27 imported its bus library from a module named for the Fuel's software platform. The file becomes src/window_map.rs, named for what is in it, and takes the physical address masks that ip27.rs and a Fuel bus both define (PA_MASK and CPU_PHYS_MASK), so there is one definition. A bus that wraps the four views differently stays with the machine that has it. The move is a rename; a test now pins what the window map did without one: a window answers exactly its bounds, the rest of the page is refused, and a later map over a page replaces the earlier.
On a two-node Onyx2 or Origin 2000 the PROM numbers one node 1. That
node's software addresses its own memory by its number (bit 32 up), and
node zero's memory beyond the cached alias belongs to the other node.
The processor's cache and the JIT reach RAM through the node's ppmem
window, but the window covered only node-relative offsets below 4 GiB,
and NodeZero cleared it entirely once the node held a nonzero number.
Every cached access of that node's processor then went through the
decode table, the Hub's node view and its bank lookup, and every
instruction fetch missed the decode cache's window pointer and re-read
its word over that path. A per-thread count in a throwaway build showed
it: during `runon 1 awk 'BEGIN{for(i=0;i<1000000;i++)s+=i}'` on the
Onyx2, all 912 million instruction fetches and 458 million data
accesses of that processor took the bus path, 86 percent of the
fetches to its own memory at its number and the rest to node zero's
memory. During runon 0 the other processor fetched 665 million of its
804 million instructions from node 1's memory the same way.
Windows now lays every node's window from the numbers and aliases the
Hubs publish, as the decode table routes them: node zero's window from
the node's own banks below the alias and the zero holder's above it,
and each node's memory at its number, the other node's included,
sharing its storage and generation counters. A two-node board gets an
8 GiB window, so numbers 0 and 1 fit. Windows lays only the difference:
a page that stays mapped is replaced in one mapping call, and a range
that goes leaves the region bitmap before it is scrubbed, so a
renumbering never clears a window the other processor is executing from.
The access path is unchanged.
A store to a Hub's number or alias can come from the other node's
processor (the neighbour's vector write), and the hardware takes the new
number at once, so the store lays every node's window from the storing
thread. The other node's processor meanwhile holds and releases line
pages at CACHE instructions in its own window's PpMemSpace, which was
single-writer by contract. Laying from another thread rebuilt the
mapping list under hold_line_page, which then found its page unmapped.
PpMemSpace now has one small mutex, taken by the remaps (map_range,
map_object, unmap_range, map_bank, map_alias, clear_mappings) and by
add_cached_view, hold_line_page and release_line_page. A load, store or
fetch never takes it: they read the window through the processor's
inline bitmap and its view's base pointer. Windows' own lock is always
taken first, so the two cannot deadlock.
An earlier version of this change avoided the lock by marking every
window stale and letting each processor lay its own at its next batch
boundary, which left a window up to a thousand instructions behind the
decode table. Nothing shows that guests tolerate that, so it is gone.
A test renumbers on one thread while another holds and releases line
pages. Without the mutex it kills the test process with SIGILL (three
runs of three); with it, it passes. Per-access cost is unchanged: 16
runs of `iris-bench run --quick --cpu r10000` (ship profile, A B B A
order, separate build targets) counted a mean of 271.48 G user
instructions on the previous tip and 271.98 G with the mutex (+0.2
percent, against a run-to-run standard deviation of about 1.5 percent).
With the windows laid, the same count shows no cached access on the bus path on
either processor, from power-on to the end of the runs. On the Onyx2
at this branch's previous tip, the awk loop took 29.9 s on processor 0
and 57.9 s on processor 1 (means of five runs each, alternating in one
boot). With the windows it takes 27.9 s and 28.2 s (measured with the
earlier batch-boundary arrangement; the mutex tip ran the loop in 33.5,
34.3, 33.7 and 30.6 s on a busier host, the processors equally fast).
Boot to the login prompt went from 293 s to 208 s. The two-node Origin 2000 runs the
loop in 35.2-36.5 s on processor 0 and 37.7-39.6 s on processor 1.
An 8 GiB window has 128 MiB regions in its bitmap, so on two nodes a
64 or 256 MiB bank, which fills no region whole, stays on the bus path
for both processors. A number above 1 is left to the bus path too.
Shipped machines: the Indy, Indigo2, Octane and O2 are not rewired, but
their ppmem changes. A mapping now keeps a reference to its backing
memory instead of a bank number, and recording a mapping trims one it
only partly overlaps, where it used to drop only mappings it covered
whole. That runs when firmware remaps memory, not on a load, store or
fetch, whose path is unchanged.
The Origin 2000 guide said a two-node machine stops in IRIX with "Processor #1 did not start", the second node's processor taking a TLB exception in the PROM. It no longer does. At this branch's tip, with and without the node window change, a two-node Origin 2000 (180 MHz R10000, 128 MB per node, stock IP27 PROM 6.156 and I/O PROM) boots IRIX 6.5.30 from the BaseIO's SCSI disk to a root login on the serial console, and hinv lists two IP27 processors. runon 0 and runon 1 both complete.
The two-node SN0 window was 8 GiB, so its one-word region bitmap counted 128 MiB regions. The bitmap only names a region that is RAM end to end, and a 64 MiB bank fills half of one, as does each 64 MiB run of a 256 MiB bank. On a two-node Origin 2000 or Onyx2 with those banks every cached access of both processors took the bus path again, a regression for the first node, which had used 64 MiB regions before. A node number above 1 fell outside the window altogether. The bitmap test runs on every load, store and fetch, so it stays as it is. A space now keeps a fine table beside it, one bit per 64 MiB region of the whole window, and the shadow cache consults it on the bitmap's miss branch before taking the bus. A space's bitmap region size can be set apart from its window size. A cabled SN0 board gets a window spanning the node numbers up to the next power of two above its node count, 8 GiB for two nodes and 16 GiB for three or four, with 64 bitmap regions of 128 MiB over numbers 0 and 1. One node keeps its 4 GiB window and 64 MiB regions. The window is reserved address space that commits nothing (mmap PROT_NONE and MAP_NORESERVE on Unix, a VirtualAlloc2 placeholder on Windows, read from the code and run only on Linux), and sizing it by the node count rather than for all 256 node numbers keeps the reservation at 8 GiB per node on a two-node board. A node a guest gives a number past the window still works: the bus path answers every number, at the speed it had before the window. The decode cache keeps window pointers for pages either test finds. It used to key them on the bitmap and start over when it moved, which misses a change only the fine table sees, so it now keys them on a value the space bumps whenever either changes (PpMemSpace::add_key_sink). Every machine that wires a shadow cache to a window registers it. With a throwaway per-thread counter, a two-node Onyx2 made no cached access on the bus path on either processor, from power-on through a runon 0 and a runon 1 awk loop, with two 64 MiB banks per node and with one 128 MiB bank. The loop ran at the same speed on both processors in both layouts (means 19.3 s and 19.8 s at 64 MiB, 19.4 s and 18.1 s at 128 MiB, five runs each, alternating in one boot; the spread within a boot is 17 to 24 s, so differences under about 20 percent are noise). Shipped machines: this changes the Indy and Indigo2 (machine.rs), the Octane (ip30_machine.rs) and the O2 (ip32_machine.rs) as well, since they share the shadow cache and the decode cache. Each registers its CPU's key sink with add_key_sink(view, ptr). On the bitmap's miss branch the shadow cache makes one more compare, against the fine table's length, which is zero on these machines because only SN0 sets a table, so the branch falls through to the bus as before. The decode cache's hit compares the window key where it compared the bitmap, one compare as before. The bitmap test itself and the window pointer are untouched. One node, `iris-bench --deterministic --cpu r10000`, host instructions:u, A B B A, where A is upstream main at the branch point and B is this branch's tip: 343.10e9, 339.47e9, 339.14e9, 342.87e9, with identical guest work (621.45 million instructions) on every run. The tip runs about 1.1 percent fewer host instructions. That compares the whole branch, so it bounds the combined effect of its commits, not this one alone, and it was taken before processors laid their own windows, code only a board with several nodes runs. On a one-node Origin 200 under IRIX 6.5.30, the processor thread's user instructions over one awk loop, base and branch alternating over six boots (eleven runs), were 243e9 to 279e9 for the base and 239e9 to 292e9 for the branch, means 264e9 and 251e9. That count is not fixed work, since the guest's timers run on the host clock, so it shows no regression beyond its own spread and nothing finer.
A 512 MiB bank 0 spans the whole 512 MiB stride, boot flash's reset
address included. The node memory mapped the flash over the bank in the
cached decode and listed both in the memory map, so MemoryMap::new
panicked ("bank 0 overlaps boot flash") for any machine built with one,
such as a two-node deskside with 512 MiB in bank 0.
Cached references now reach the bank, as the PROM's memory test expects
when it walks that address as RAM, and the reset fetch, which is
uncached, reaches the flash through the Hub-special space, where it
already is. With a smaller bank 0 the flash stays in the cached decode
as before. The QEMU sgi-origami node model makes the same split. That a
real machine does is inferred from the PROM's memory test, not measured.
A test builds every size bank 0 can be, and checks that a 512 MiB one
keeps its RAM at the reset address and every smaller one the flash.
After a restore the combiner assumed the host had every output low. The O2 redrives its bus after loading, so CRIME is told every level again. The Octane's Bridge restores its own pin levels instead and is not redriven. A line saved high that fell after a restore was then never delivered, because the combiner thought the host already had 0. The Bridge kept the pin high, so the line's next rise was no edge and sent HEART no interrupt packet. load() now records each output's loaded level as what the host has. That matches a host that restores its own copy, and a redrive still tells one that does not. Fixes: 2cd0589 ("pci: add PciBus, with its decod 10BC8 e table, BAR runs and interrupt combiner")
jamesbraid
force-pushed
the
ip27-fixes
branch
from
November 10, 2026 17:00
ef7758d to
434abe4
Compare
The combiner kept every line's level in three shared words and ran a separate four-flag handshake per output (DIRTY, HELD, FELL, FORCE) so that no lock was held while it called the host. A lower worked out whether its output fell by reading the other line words again, unordered, a limit its own comment admitted. It took 183 lines of code and two test-only hook points. Holding one mutex across the host call would be smaller, but HACKING.md's second lock rule says to raise interrupts outside the lock. Each output now owns one word: a bit per line routed to it, plus HELD, FELL and FORCE. A change is one compare-exchange of that word, so a lower knows exactly whether the output fell, and a set that changes nothing writes nothing. A change that finds HELD clear delivers. The holder gives HELD up with a compare-exchange against the word it delivered from, and goes round again if anything landed meanwhile. Deliveries to an output are therefore one at a time and in order, the last is the OR after the last change, and a fall and rise during a delivery still reach the Bridge as two edges, so HEART gets its packet. A clear HELD means the host has the word's OR, which removes DIRTY and the stored copy of the host's level. The loom tests now model this word. They include the two ways it could break: a holder that gave HELD up without checking the word loses a change made during its delivery, and a fall and rise without FELL reach the host as a steady 1. Both bugs, planted by hand, fail the tests. Code drops from 183 to 141 lines (not counting blanks and comments). Each output's word sits on its own cache line. Packed, two threads driving different outputs cost 166 ns per change in one run. Per change, release build without LTO, best of 7 x 10 million, three interleaved runs each on a shared host: case before after set that changes nothing 28-30 ns 6-7 ns toggle 27-30 ns 17-19 ns two threads, two outputs 379-430 ns 65-75 ns two threads, one output 204-315 ns 56-66 ns The two tests that drove the old handshake through its hook points go, since those windows no longer exist.
The chip, its Ethernet MAC and their tests were four sibling files in dev/. They are one device, and the next commits split it into children that live beside each other, so give it a directory. Only a move: paths and module declarations change, no code does.
jamesbraid
force-pushed
the
ip27-fixes
branch
from
November 10, 2026 19:17
434abe4 to
a3a9bbd
Compare
jamesbraid
force-pushed
the
ip27-fixes
branch
2 times, most recently
from
November 11, 2026 00:24
2d56631 to
7db2e12
Compare
jamesbraid
force-pushed
the
ip27-fixes
branch
2 times, most recently
from
November 11, 2026 02:31
3fac6c4 to
ca677fd
Compare
jamesbraid
force-pushed
the
ioc3-mfd
branch
2 times, most recently
from
November 11, 2026 04:29
b70ef08 to
67b2759
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Builds on #546 (the IOC3 refactor), which builds on #512.
The IP27 machines from #455 boot, but the code carried shims, a lock-heavy Hub, two config sections for three machines and some bugs found bringing up graphics on the Onyx2. This cleans that up and fixes them.
Fixes
io_promfits the I/O PROM on the BaseIO's flash, without which the node PROM runs a copy with no graphics console.Cleanup
[origin200],[origin2000]and a new[onyx2]read into one struct. An Onyx2 file that still says[origin2000]gets an error naming[onyx2], so existing Onyx2 configs need that table renamed.src/dev/hub/) with private units. Units return what they changed instead of calling hooks; restarts go on the machine's event channel; soft reset is a line on the processor's interrupt word. Each processor interface changes its interrupt inputs and drives its lines under one lock, only on a change (it took abo 8000 ut eight locks per mask write).xtalk::Upstreamreplaces, and the Origin 2000 Bridge that ignored the Crossbow link.src/ip35.rs's window map moves tosrc/window_map.rs.Worth a look: the Hub's interrupt lock (
hub/pi.rs) and the node windows (ip27.rs,ppmem).Tested with
cargo test --workspace(also the IP27 tests underjitv2andlightning,jitv2,idle-pause), every commit building. IRIX 6.5.30 boots to a shell on an Origin 200, a two-node Origin 2000, a two-node Onyx2 (with the MediaIO's serial ports and audio inhinv), an R12000 with a 256 MiB bank and a 512 MiB first bank; an Octane still boots. Not built on macOS or Windows here.