Re: [PATCH v9 1/4] pci:host: APM X-Gene PCIe host controller driver

[Bjorn Helgaas]

On Mon, Sep 22, 2014 at 3:33 PM, Tanmay Inamdar <tinamdar@xxxxxxx> wrote:

>>> +static void xgene_pcie_fixup_bridge(struct pci_dev *dev)
>>> +{
>>> +     int i;
>>> +
>>> +     /* Hide the PCI host BARs from the kernel as their content doesn't
>>> +      * fit well in the resource management
>>> +      */
>>
>> This needs a better explanation than "doesn't fit well."
>>
>> I *think* you're probably talking about something similar to the MVEBU
>> devices mentioned here:
>> http://lkml.kernel.org/r/CAErSpo56jB1Bf2JtYCGKXZBZqRF1jXFxGmeewPX_e6vSXueGyA@xxxxxxxxxxxxxx
>>
>> where the device can be configured as either an endpoint or a root port,
>> and the endpoint BARs are still visible when configured as a root port.
>>
>
> It is true that X-Gene PCIe port can be configured as EP, however the
> the FIXUP is required not because of the BARs are still visible when
> configured as EP in past. X-Gene PCIe port, when configured as RC,
> uses BAR0-BAR1 of RC's configuration space as inbound BARs. Entire DDR
> region is mapped in these BARs so that it is accessible for EP devices
> for DMA. So if the fixup is not done during enumeration, whole
> outbound memory resource space gets assigned to RC and nothing is left
> EP devices.

I'm not familiar with the concept of an "inbound BAR"; at least I'm
not aware of anything like this in the PCI specs.  I think it might
reduce confusion if we avoided calling them "BARs".  They happen to be
at the same addresses where real BARs would be, but they certainly
don't behave like real BARs.

It sounds like this is basically a trivial IOMMU that determines which
DMA accesses can reach main memory.

>> In any event, I'd like a description of exactly what these BARs are and wha
>> the problem is.
>
> Is it ok to put above description in comment to explain the fixup?
>
>> Presumably the BARs exist and were sized by the PCI core
>> in __pci_read_base().  That will generate some log messages and possibly
>> some warnings, depending on how the host bridge windows are set up.

Instead of doing this in a fixup, can you change the PCI config
accessors so that when accessing these inbound DMA mapping registers,
they drop writes and return zeros for reads?  Then the PCI core won't
think there are BARs there, and it won't mistakenly corrupt them when
it tries to size them.

Bjorn
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/