PIF-NUS

The PIF-NUS (or PIF, or PIF(P)-NUS on PAL) manages multiple critical functions of the N64 console. It is a physical microchip found on the console's motherboard. It is not clear whether SGI or Nintendo intended this to stand for "Peripheral InterFace" or not. While the naming is unintuitive, the Peripheral (or Parallel) Interface is used to read/write to the game ROM and devices like the 64DD; whereas, the PIF handles the following:
 * Console startup and piracy protections
 * Stores the first 2 stages of the Initial Program Load (IPL) that is executed by the VR4300 CPU
 * Console reset button to avoid corrupting save game data
 * Controller and EEPROM read/write via JoyBus protocol

Console startup

 * 1) PIF holds all of the console components in reset mode, listening for cartridge CIC
 * 2) Cartridge sends 4 bits (nibble) including region encoding
 * 3) Cartridge sends 4 bit encrypted seed value
 * 4) Cartridge sends 4 bit checksum
 * 5) PIF checks that these are the expected values
 * 6) If the values don't match (same scenario if no cartridge is inserted)
 * 7) The user would generally reset using the button or power off
 * 8) Remove/Re-insert cartridge (or change games)
 * 9) Power on (go to step #1)
 * 10) PIF writes the encrypted seed value to memory address 0xBFC0'07E4
 * 11) PIF releases the reset pin for the whole console
 * 12) The PIF (console) and CIC (cartridge) communication continues as long as the console is powered on
 * 13) If there is ever a failure in the data exchange the console will be reset.
 * 14) VR4300 requests RAM address 0xBFC0'0000 from the Memory Management Unit inside the RCP
 * 15) IPL1: The MMU then loads the MIPS code from a storage area of the PIF chip
 * 16) These instructions are executed directly in this very slow manner.
 * 17) Thankfully IPL1 is only 52 instructions + some looping
 * 18) Some really basic hardware initialization
 * 19) Copy IPL2 to the RSP IMEM address range
 * 20) IPL 2 is executed by the VR4300 but reading the instructions from the RSP's IMEM
 * 21) More general hardware initialization
 * 22) If it's determined to be a 64DD disk it will jump to 0xA600'0000
 * 23) Load IPL3 from the cartridge into the RSP's DMEM
 * 24) The IPL3 code is CRC checked
 * 25) Set 0xBFC0'0000 range to "invisible"
 * 26) Jump to RSP DMEM to execute IPL3
 * 27) IPL3 is executed by the VR4300 but reading the instructions from the RSP's DMEM
 * 28) Initialize RDRAM
 * 29) Depending on reset type
 * 30) Power On: Invalidate VR4300 ICache & DCache
 * 31) Reset : Writeback VR4300 ICache & DCache
 * 32) Move IPL3 execution from DMEM to RDRAM
 * 33) DMA 1 MB of Game code to RDRAM
 * 34) Authenticate 1 MB of Game Program
 * 35) Reset RSP
 * 36) Clear Interrupts
 * 37) Clear IPL3 from DMEM
 * 38) Clear IPL2 from IMEM
 * 39) Jump to Game code in RDRAM

Console Reset

 * 1) User presses Console Reset button
 * 2) PIF reads the button state
 * 3) PIF Toggles VR4300 Interrupt 2 (INT2)
 * 4) PIF Waits 0.5 seconds
 * 5) This is the time and opportunity for the game to finish saving game data to avoid corruption
 * 6) PIF Toggles VR4300 Non-Maskable Interrupt (NMI)
 * 7) Which resets the console components

Controller and EEPROM communication
In addition to managing the Joybus Protocol for connected hardware like controllers and connected accessory PAKs

Sharp SM5 4-bit Microcontrollers
The PIF and CIC (inside the cartridge) are both custom versions of Sharp branded SM5 4-bit microcontrollers. These microcontrollers were also used in the Game & Watch handheld games, so Nintendo already had developers that were familiar with them. While the core functionality of the PIF and CIC are generally understood, the microcontroller model is custom and therefore not well known.

There has been some effort to reverse engineer the PIF and CIC communication to ease the process for creating compatible flash carts. At least 2 projects went through the time effort and money to decap the chips and view the internals of to better understand what they are doing.

CIC
'''Disclaimer: All of the CIC actions described below have been done successfully as part of a University project. Some of the details are missing whether it was to avoid encouraging piracy, simply not required for the core of the paper or lost in the language translation (authors live in Germany) is unknown.''' The test modes of the CIC are available on most (all?) retail cartridges.

To Enable Test Mode(s)
Pulling TS:0 and/or TS:1 high before power on will place the SM5 controller in one of 3 test modes. (Which test modes and which pin states are unknown) It's also unclear if you can change between test modes while the unit is powered on.

The fourth state is standard usage with TS:0 and TS:1 tied to ground.

It's unknown how slowly you can clock the CIC.

In Test Mode
Which mode is still unclear but the following functionality is available.

Arbitrary Code Execution
Instructions can be set 1 nibble at a time on Port 5 pins, most instructions are 1 byte long so they must be entered 1 nibble at a time then toggle the clock line.

Halt Instruction
The Halt instruction is encoded as 0x77 which is nice because it doesn't matter which nibble you send first. This instruction also has a nice benefit of causing a clear external change, the TIO line defaults to the Clock signal but after the Halt instruction it stops outputting a clock signal.

Stop Instruction
The Stop instruction is encoded as 0x76 which will assist in determining if the high or low nibble should be input first. This instruction also stops the TIO clock upon execution, so we have a clear external indication of success.

Output Data
Port 2 of the CIC can be used to output either the AREG register or the Program Counter (PC), it's unclear at this time if the difference is achieved with different test modes or by modifying internal configuration registers.

NOTE: On power up Port 2 is configured for Input, an internal configuration register must be modified to make it output.

Load Constant into Accumulator LDX
The LDX instuction can be used to populate the AREG with a known value that can be checked on Port 2

Dumping the CIC code
This process is very confusing so it may take some experimentation to work out the exact steps and details, the original document tries to explain but it feels like some details are missing.

Jump Instructions are 2 bytes, the first nibble is the instruction and the following 12 bits are the destination address.

Being in test mode seems to have a side effect on this instruction, inputting only the Jump instruction followed by a zero nibble, the second byte is loaded from the ROM, which is an instruction but is treated as data. The Jump instruction is then executed and the PC can be viewed on Port 2, as well

PIF
The PIF handles a lot of very core functions in the console. While the PIF chip is clearly a custom part, it appears to be heavily based on the SM5K (4/5/6) series. The instruction set and features match closely.