PCIe Link Configuration at Runtime (AN38)

Affected Products	All products featuring PCIe root ports

Preface

This application note explains how to configure a PCI Express (PCIe) link during runtime. This is especially helpful if a PCIe link is trained incorrectly during the Power On Self Test (POST). A PCIe link could be trained with the wrong link width, speed or not trained at all.

Introduction

This application note explains how to configure a PCI Express (PCIe) link during runtime. This is especially helpful if a PCIe link is trained incorrectly during the Power On Self Test (POST). A PCIe link could be trained with the wrong link width, speed or not trained at all.

Prerequisites

PCI Config Space registers can be modified with several tools. Use PCI.EXE under DOS if possible. Contact your congatec FAE if you do not have access to this DOS tool. In Windows, RWEverything (www.rweverything.com) can be used.

A DOS bootable USB stick with PCI.EXE is required for the purpose of this application note. Follow the steps described in the CTN 20132507 001 “How to create a DOS bootable USB stick and use PCI.EXE” to create it. This CTN is available in the restricted area of the congatec website at www.congatec.com

It is recommended to perform the manual link configuration steps in a pre-OS environment like DOS or the EFI shell. This ensures that other software (OS or device driver) does not access these registers.

If the PCIe link is not trained at all during POST, enable hot-plug functionality for the PCIe root port in BIOS Setup. This prevents the BIOS from disabling the root port. If the root port is disabled, it is not possible to retry link training at runtime.

Important PCI Config Space Registers

Link training can be initiated with the help of the Link Control Register. The current link status is shown in the Link Status Register. The Link Capabilities Register identifies PCI Express link specific capabilities of the device, All three registers are in the PCI configuration space of the PCI Express device and part of the PCI Express Capability structure.

The structure can be located by following the Capability Pointer in PCI configuration register offset 34h. This register points to a linked list of capabilities implemented by a PCI function, in this case a PCI Express root port.

The first byte the pointer points to is the capability ID, the byte following the capability ID is the pointer to the next capability structure. Follow the linked list until you find the PCI Express Capability structure with the ID of 10h. See example in section Finding the PCI Express Capability Structure.

Link Control Register (Offset 10h)

The Link Control Register controls PCI Express Link specific parameters. Figure 1 below details allocation of register fields in the Link Control register.

Figure 1: Link Control Register Bit Description

Link Status Register (Offset 12h)

The Link Status Register provides information about PCI Express specific parameters. Figure 2 below details allocation of register fields in the Link Status register.

Figure 2: Link Status Register Bit Description

Link Capabilities Register (Offset 0Ch)

The Link Capabilities Register identifies PCI Express link specific capabilities, Figure 3 below details allocation of register fields in the Link Capabilities register.

Figure 3: Link Capabilities Register Bit Description

When debugging PCI Express link problems, the Max Link Speed and Maximum Link Width indicated in this register are of special interest.

The Max Link Speed [bits 3:0] indicates the maximum link speed this device can support. Although the encoding of the bit value changed in a newer version of the PCI Express specification (it specifies a bit location in the Supported Link Speeds Vector in the Link Capabilities 2 register), the bit value can still be interpreted as the supported link speed:

Gen1	Gen2	Gen3
0001b	0010b	0011b

The Maximum Link Width [bits 9:4] indicates the maximum link width (number of lanes) this device can support:

x1	x2	x4	x8	x16	x32
000001b	000010b	000100b	001000b	010000b	100000b

The maximum supported link speed and width of the two link partners (usually a root port and an endpoint) is important to judge whether the link is trained properly. A link should train in the highest common speed and width supported by both link partners.

For example: If the PCIe root port supports Gen3 x8 and the endpoint supports Gen2 x4, the link should train in Gen2 x4. This is indicated in the Link Status Registers Negotiated Link Width and Current Link Speed fields.

Compare the status information from the link Status Register with the capabilities of the two link devices.

Finding the PCI Express Capability Structure

Start at the pointer in register offset 34h and follow the linked list until you reach the capability ID 10h. The two examples below show PCI Express root ports of an Intel PCH and a System Agent (PEG Port).

Figure 3: PCI Config Space of Intel PCH PCI Express Root Port

Figure 4: PCI Config Space of Intel PEG PCI Express Root Port

Disabling / Enabling a PCI Express Link

1. Navigate (F1 in PCI.EXE) to the PCI Express root port your device is connected to.
2. Find the Link Control Register as explained in section Finding the PCI Express Capability Structure.

Note:

Bit 6 (common clock configuration) of the Link Control Register must be set if bit 12 (slot clock configuration) of the Link Status Register is set. This is the case for most PCI Express links since the downstream device usually uses the same reference clock as the root port. Bit 4 in the Link Control Register disables or enables the link.

3. Modify (F2 in PCI.EXE) the low byte of the Link Control Register with the following values:

• 50h to disable the link
• 40h to re-enable the link

4. Check the low byte of the Link Status Register to see if the link is trained:

• 12h for example indicates an x1 link width and Gen2 speed
• 83h for example indicates an x8 link width and Gen3 speed

5. Go to the PCI device list overview. If the link has been trained, the device connected to the root port should be visible.

Retraining a PCI Express Link

To retrain a PCI Express link, set bit 5 of the Link Control Register.

Modify (F2 in PCI.EXE) the low byte of the Link Control Register with the following value:

• 60h to retrain the link (bit 6 and bit 5 set)

A link that was trained with incorrect width and/or speed should be trained with the correct configuration after link retraining.

Check the low byte of the Link Status Register to see if the link is trained correctly.

Summary

The Link Control and Link Status registers of a PCI Express root port can be used to debug link problems at runtime. This can be especially helpful if power-up sequencing or asynchronous reset behavior is the root cause of a link problem.

Refer to the PCI Express specification or Mindshare PCI Express book for more details about this topic.