Linux Virtualization - Aug 2013 - [PATCH 0/5] Enable Drivers for Intel MIC X100 Coprocessors.

On Tue, 2013-08-13 at 14:43 +0200, Pavel Machek wrote:
> Hi!
> 
> > Since it is a PCIe card, it does not have the ability to host hardware
> > devices for networking, storage and console. We provide these devices
> > on X100 coprocessors thus enabling a self-bootable equivalent
environment
> > for applications. A key benefit of our solution is that it leverages
> > the standard virtio framework for network, disk and console devices,
> > though in our case the virtio framework is used across a PCIe bus.
> 
> Interesting...
> 
> >  Documentation/mic/mic_overview.txt   |   48 +
> >  Documentation/mic/mpssd/.gitignore   |    1 +
> >  Documentation/mic/mpssd/Makefile     |   20 +
> >  Documentation/mic/mpssd/micctrl      |  157 +++
> >  Documentation/mic/mpssd/mpss         |  246 +++++
> >  Documentation/mic/mpssd/mpssd.c      | 1732
++++++++++++++++++++++++++++++++++
> >  Documentation/mic/mpssd/mpssd.h      |  105 +++
> >  Documentation/mic/mpssd/sysfs.c      |  108 +++
> >  drivers/misc/Kconfig                 |    1 +
> >  drivers/misc/Makefile                |    1 +
> >  drivers/misc/mic/Kconfig             |   56 ++
> >  drivers/misc/mic/Makefile            |    6 +
> >  drivers/misc/mic/card/Makefile       |   11 +
> >  drivers/misc/mic/card/mic_common.h   |   43 +
> >  drivers/misc/mic/card/mic_debugfs.c  |  139 +++
> >  drivers/misc/mic/card/mic_debugfs.h  |   40 +
> >  drivers/misc/mic/card/mic_device.c   |  311 ++++++
> >  drivers/misc/mic/card/mic_device.h   |  106 +++
> >  drivers/misc/mic/card/mic_virtio.c   |  643 +++++++++++++
> >  drivers/misc/mic/card/mic_virtio.h   |   79 ++
> >  drivers/misc/mic/card/mic_x100.c     |  253 +++++
> >  drivers/misc/mic/card/mic_x100.h     |   53 ++
> >  drivers/misc/mic/common/mic_device.h |   85 ++
> >  drivers/misc/mic/host/Makefile       |   13 +
> >  drivers/misc/mic/host/mic_boot.c     |  181 ++++
> 
> So... there are basically separate computers running on PCIe card
> plugged into host computer, right?
> 
They are PCIe form factor Coprocessors plugged into the host.
> Maybe we should have something more promintent than drivers/misc for
> this, then? Like drivers/multicomputer?
> 
multicomputer" is an interesting name for these kind of devices but has
several issues:
a) The definition I found for multicomputer online was "A computer made
up of several computers. The term generally refers to an architecture in
which each processor has its own memory rather than multiple processors
with a shared memory. A multicore computer, although it sounds similar,
would not be a multicomputer because the multiple cores share a common
memory." Intel MIC X100 devices typically have upto 244 CPUs (61 cores)
on the card sharing common card memory so multicomputer would not be
accurate based on this definition.
b) X100 MIC devices have always been referred to Coprocessors and never
as multicomputers in product specifications @
http://software.intel.com/en-us/mic-developer
c) multicomputer is a very long path name.

Given these issues, we would like to stick to drivers/misc/mic/ unless
you have objections to this approach.

Thanks for the feedback.
Sudeep Dutt
> 									Pavel
>

Hi!
> > > Since it is a PCIe card, it does not have the ability to host
hardware
> > > devices for networking, storage and console. We provide these
devices
> > > on X100 coprocessors thus enabling a self-bootable equivalent
environment
> > > for applications. A key benefit of our solution is that it
leverages
> > > the standard virtio framework for network, disk and console
devices,
> > > though in our case the virtio framework is used across a PCIe
bus.
> > 
> > Interesting...
> > 
> > >  Documentation/mic/mic_overview.txt   |   48 +
> > >  Documentation/mic/mpssd/.gitignore   |    1 +
> > >  Documentation/mic/mpssd/Makefile     |   20 +
> > >  Documentation/mic/mpssd/micctrl      |  157 +++
> > >  Documentation/mic/mpssd/mpss         |  246 +++++
> > >  Documentation/mic/mpssd/mpssd.c      | 1732
++++++++++++++++++++++++++++++++++
> > >  Documentation/mic/mpssd/mpssd.h      |  105 +++
> > >  Documentation/mic/mpssd/sysfs.c      |  108 +++
> > >  drivers/misc/Kconfig                 |    1 +
> > >  drivers/misc/Makefile                |    1 +
> > >  drivers/misc/mic/Kconfig             |   56 ++
> > >  drivers/misc/mic/Makefile            |    6 +
> > >  drivers/misc/mic/card/Makefile       |   11 +
> > >  drivers/misc/mic/card/mic_common.h   |   43 +
> > >  drivers/misc/mic/card/mic_debugfs.c  |  139 +++
> > >  drivers/misc/mic/card/mic_debugfs.h  |   40 +
> > >  drivers/misc/mic/card/mic_device.c   |  311 ++++++
> > >  drivers/misc/mic/card/mic_device.h   |  106 +++
> > >  drivers/misc/mic/card/mic_virtio.c   |  643 +++++++++++++
> > >  drivers/misc/mic/card/mic_virtio.h   |   79 ++
> > >  drivers/misc/mic/card/mic_x100.c     |  253 +++++
> > >  drivers/misc/mic/card/mic_x100.h     |   53 ++
> > >  drivers/misc/mic/common/mic_device.h |   85 ++
> > >  drivers/misc/mic/host/Makefile       |   13 +
> > >  drivers/misc/mic/host/mic_boot.c     |  181 ++++
> > 
> > So... there are basically separate computers running on PCIe card
> > plugged into host computer, right?
> > 
> 
> They are PCIe form factor Coprocessors plugged into the host.
> 
> > Maybe we should have something more promintent than drivers/misc for
> > this, then? Like drivers/multicomputer?
> > 
> 
> multicomputer" is an interesting name for these kind of devices but
has
> several issues:
> a) The definition I found for multicomputer online was "A computer
made
> up of several computers. The term generally refers to an architecture in
> which each processor has its own memory rather than multiple processors
> with a shared memory. A multicore computer, although it sounds similar,
> would not be a multicomputer because the multiple cores share a common
> memory." Intel MIC X100 devices typically have upto 244 CPUs (61
cores)
> on the card sharing common card memory so multicomputer would not be
> accurate based on this definition.
Well... you have your "host" computer, and than (potentially several)
Intel MIC devices, which are basically separate computers. So X100 is
not a multicomputer, but machine with several X100 cards certainly is
multicomputer.
> b) X100 MIC devices have always been referred to Coprocessors and never
> as multicomputers in product specifications @
> http://software.intel.com/en-us/mic-developer
Coprocessor sounds like i487, but why not.
> c) multicomputer is a very long path name.
Agreed it is long. drivers/coproc? 

[I guess we'll get similar hardware from different vendors in
future. It would make sense having it at common place.]

								Pavel

-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures)
http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html

On Thu, 2013-08-15 at 12:14 +0200, Pavel Machek wrote:
> Hi!
> 
Hi!
> > > > Since it is a PCIe card, it does not have the ability to
host hardware
> > > > devices for networking, storage and console. We provide
these devices
> > > > on X100 coprocessors thus enabling a self-bootable
equivalent environment
> > > > for applications. A key benefit of our solution is that it
leverages
> > > > the standard virtio framework for network, disk and console
devices,
> > > > though in our case the virtio framework is used across a
PCIe bus.
> > > 
> > > Interesting...
> > > 
> > > >  Documentation/mic/mic_overview.txt   |   48 +
> > > >  Documentation/mic/mpssd/.gitignore   |    1 +
> > > >  Documentation/mic/mpssd/Makefile     |   20 +
> > > >  Documentation/mic/mpssd/micctrl      |  157 +++
> > > >  Documentation/mic/mpssd/mpss         |  246 +++++
> > > >  Documentation/mic/mpssd/mpssd.c      | 1732
++++++++++++++++++++++++++++++++++
> > > >  Documentation/mic/mpssd/mpssd.h      |  105 +++
> > > >  Documentation/mic/mpssd/sysfs.c      |  108 +++
> > > >  drivers/misc/Kconfig                 |    1 +
> > > >  drivers/misc/Makefile                |    1 +
> > > >  drivers/misc/mic/Kconfig             |   56 ++
> > > >  drivers/misc/mic/Makefile            |    6 +
> > > >  drivers/misc/mic/card/Makefile       |   11 +
> > > >  drivers/misc/mic/card/mic_common.h   |   43 +
> > > >  drivers/misc/mic/card/mic_debugfs.c  |  139 +++
> > > >  drivers/misc/mic/card/mic_debugfs.h  |   40 +
> > > >  drivers/misc/mic/card/mic_device.c   |  311 ++++++
> > > >  drivers/misc/mic/card/mic_device.h   |  106 +++
> > > >  drivers/misc/mic/card/mic_virtio.c   |  643 +++++++++++++
> > > >  drivers/misc/mic/card/mic_virtio.h   |   79 ++
> > > >  drivers/misc/mic/card/mic_x100.c     |  253 +++++
> > > >  drivers/misc/mic/card/mic_x100.h     |   53 ++
> > > >  drivers/misc/mic/common/mic_device.h |   85 ++
> > > >  drivers/misc/mic/host/Makefile       |   13 +
> > > >  drivers/misc/mic/host/mic_boot.c     |  181 ++++
> > > 
> > > So... there are basically separate computers running on PCIe card
> > > plugged into host computer, right?
> > > 
> > 
> > They are PCIe form factor Coprocessors plugged into the host.
> > 
> > > Maybe we should have something more promintent than drivers/misc
for
> > > this, then? Like drivers/multicomputer?
> > > 
> > 
> > multicomputer" is an interesting name for these kind of devices
but has
> > several issues:
> > a) The definition I found for multicomputer online was "A
computer made
> > up of several computers. The term generally refers to an architecture
in
> > which each processor has its own memory rather than multiple
processors
> > with a shared memory. A multicore computer, although it sounds
similar,
> > would not be a multicomputer because the multiple cores share a common
> > memory." Intel MIC X100 devices typically have upto 244 CPUs (61
cores)
> > on the card sharing common card memory so multicomputer would not be
> > accurate based on this definition.
> 
> Well... you have your "host" computer, and than (potentially
several)
> Intel MIC devices, which are basically separate computers. So X100 is
> not a multicomputer, but machine with several X100 cards certainly is
> multicomputer.
> 
> > b) X100 MIC devices have always been referred to Coprocessors and
never
> > as multicomputers in product specifications @
> > http://software.intel.com/en-us/mic-developer
> 
> Coprocessor sounds like i487, but why not.
> 
> > c) multicomputer is a very long path name.
> 
> Agreed it is long. drivers/coproc? 
> 
> [I guess we'll get similar hardware from different vendors in
> future. It would make sense having it at common place.]
> 
drivers/coproc is a good, short suggestion. Intel MIC X100 devices are
likely unique since they can run a general purpose OS in a PCIe form
factor Coprocessor plugged to the host. This might result in MIC
potentially being the only driver under drivers/coproc till another
general purpose Coprocessor comes along. Is it recommended to create a
new driver directory (drivers/coproc) in anticipation of future devices?
Do you think it would be better if we revive this discussion, once
another Coprocessor with similar capabilities comes along?

Thanks,
Sudeep Dutt
> 								Pavel
>