PCIe Card – PCB Layout and Thermal Analysis

High Component Density and Little Space

The market for FPGA-based PCIe cards for server applications is growing at an immense rate.  

Our customer, well versed in designing SW/FW for off-the-shelf PCIe cards, realized that the market did not cater to all the high-speed functionality required. At the same time, Xilinx released new Ultra-Scale FPGAs that were the first that broke the magical barrier of 100 Gbps per transceiver.

To gain a market advantage, the customer decided to design a custom-made card.  

They felt confident in HW design, but the layout and thermal management of such a card is a challenge even for experienced teams. 

Consilia has a long track record in the design of multi-layer high-speed digital cards. 

This fact has led to us being approached as a partner who can carry the product through the industrialization process to the running assembly line.

The layout and heat sink management posed several challenges to the design team.  

As the board was designed to cover several market segments, the component density was high, and the FPGA was in one of the most extensive packages for that family.

The heat sink needed to cool several components with different hights and profiles and dissipate almost 63 W of heat.  

Mounting the heat sink while maintaining good contact with the cooled components and limiting the fixing elements that take up precious space on the board turned out to be another mechanical challenge.

Close Cooperation Was Required Between HW Design Team,  Layout Team, and Thermal Engineer

The HW team initially identified components with the highest heat dissipation and determined their expected power loss. 

A list of these components, together with the expected airflow in the server, served as an input for thermal analysis.

Regarding placement, there are many, usually conflicting, requirements of HW designers, layout engineers, and thermal engineers. It takes time and a lot of experience to find compromises by weighing up which requirements are “must have” and which are “should have.”

The Consilia team has been building this expertise for decades and is a reliable and qualified partner for the design of ultra-high speed, complex, multi-layer digital boards. 

After weeks of design and simulations and several placement iterations, we have created a placement that fulfilled the requirements of all involved parties, selected proper material for heat transfer from the packages to the heatsink, and designed the heatsink and its mounting. The heatsink design was unique because cooling SFP cages was not possible with standard fins, so the thermal design had to take this constraint into consideration.

Compact PCIe Card with Advanced Features 

In the network segment, it is essential to save the server spaces and minimize failure by not using moving components. We designed a low-profile single slot PCIe 2 x 100Gbps communication card with a passive heatsink.

Complete Hardware and Mechanical Design 

The first challenge when we won the project was selecting the material for the PCB. Although the situation has improved since we finished this project, finding a PCB manufacturer that can manufacture boards designed for 100 Gbps transceivers is still challenging. The second challenge was the placement of components because the board coverage density approached physical limits.

The result was a low-profile PCIe card with the following parameters: 

• PCB core from Megtron 6, 
• 16-layer PCB,  
• blind vias used, 
• key component packages: 2106 pin BGA and 324 pin BGA, 
• component board coverage ratio exceeding 65 % on both sides,  
• transceiver pairs running at 25 Gbps, 
• copper heatsink with black oxidizing with custom-designed fins.

Innovative Producer in Infrastructure Field

Our client operates and develops the national e-infrastructure for science, research, and education which encompasses a computer network, computational grids, data storage, and a collaborative environment. It offers a rich set of services to connected organizations.