> Interesting email address there :) > On Thu, 2004-01-08 at 01:18, =?koi8-r?Q?=22?=Valery > A.Khamenya=?koi8-r?Q?=22=20?= wrote:unfortunally some email parsers and email clients deny to work correctly with international conventions :( follow this URL for more details: http://www.python.org/doc/current/lib/module-email.Header.html> On the same machine, LLVM definitely needs to support both symmetric and > asymmetric multi-processing. I believe some primitives to support this > are being worked on now by Misha. However, I don't really consider this > "distributed systems" because distribution by a few inches doesn't > really amount to much. In my way of thinking distributed computing > *always* involves a network.you are right, but bigger steps after smaller ones? let's agree with a simple things and then we could proceed (and probably die) with networking case.> Okay, this kind of "distribution" (what I call parallel computing)(see above)> should also definitely be supported by LLVM. There are several > primitives that could be added to LLVM to enable compiler writers to > more easily write parallel programs. However, I don't think LLVM should > get involved in parallel programs that run on multiple computers, only a > single computer (possibly with multiple CPUs).Oh, we agreed to a single host so soon? :)> > Is it OK up to here and I could continue using this > > model example? > > Yes, but I think the confusion was simply one of terminology. What > you're talking about is what I call parallel computing or > multi-processing (symmetric or asymetric).(see above again).> This isn't really distributed computing although > one could think of the operations being "distributed" > across several CPUs on the _same_ computer.(people speaking about distributed computations even in more boring context: "one has transfered stand-alone application to a remote PC, executed it, and this already mean distributed computation"). My idea was to consider Fib example for a single PC with several CPUs and when you are agreed that it makes sense to bring notion of CPU in LLVM layer, i should ask you: "Raid, why should we restrict ourselves with _single_ PC only ?!"> > Okay, now you're talking about "hosts" which I take to mean separate > physical computers with the only way for the "hosts" to communicate is > via a network. Is this what you mean by "host"?We could restrict ourselves to notion of host as in http://www.faqs.org/rfcs/rfc1.html and later RFCs for TCP/IP.> No, we don't. Distributed computing would be built on top of LLVM (in my > opinion). But, we _do_ need support for parallel or multi-processing > computation as you described above (again, on a single physical > computer).wait, let's fix your elrier statement. As far as I understood you do agree that case of multiple CPUs on a _single_ host should be supported at LLVM layer. Right? as I see from here:> okay .. again, this isn't "distributed", just "parallel", and I agree > its needed.you call this case just "parallel" and agree.> I would tend to agree. I fully expect LLVM to fill the mission that Java > started: ubiquitous computing. I would expect to see LLVM programs > running on everything from gameboy to supercomputer, possibly with > kernel support."possibly with kernel support" is kind of crutch to get supercomputer with networking architecture ;) Reid, couldn't you agree that networking is only a particular interface to get an access to others CPUs? why should then LLVM be abstracted from suppercomputers with CPU distributed over network? All benefits, what one could obtain from "LLVM supporting multiple CPU at single host", one might obtaine from "LLVM supporting multiple CPU at multiple hosts". Isn't that logical?> I've toyed with the idea of a Linux kernel module for > LLVM already. [...]then even easier to speak :)> Nothing at this point. I think I realize where you're coming from now > and agree that _parallel_ computing is a very important next step for > LLVM. Let's see what happens with the current work on synchronization > and threading primitives. These things will be needed to support the > kind of parallelization you're talking about.Right. ...I almost hear the thoughts from Chris: "guys, instead of trolling about cool things make something cool!" ;) -- Valery
> I see more precisely what you mean, but I don't think it is that > straightforward to generalise the benefits multiple CPU on single host > programming to multiple CPU at multiple hosts. I don't think that both > cases involve the same techniques.you are right, just think of shared memory.> For example, in "single host" configuration you get a very low cost for > communicating data because you use memory instead of network. Memory is > [...]you are right, that's exactly the point. I'd never tell that the difference is tiny.> > What would you consider as the core primitives of single host, multi-CPU > programming ?if LLVM would be really "low-level", then I would consider a generalized CALL as such a primitive, i.e. where not only address of subroutine is supplied, but the address of host as well. something like call host:subr_address However LLVM is very high-level ! It is so high-level, that I'd propose ...not include any primitives at LLVM language at all ! (So, Se'bastien, sry for answering "rather yes" about UCP) Indeed, let's consider Fib example: //------------ int f(int n) { if(n<2) return 1; return f(n-1) + f(n-2); } //------------ in this LLVM byte code output: --------------------------------------------------- target endian = little target pointersize = 32 implementation ; Functions: int %f(int %n) { entry: br label %tailrecurse tailrecurse: ; preds = %entry, %endif %cann-indvar = phi uint [ 0, %entry ], [ %next-indvar, %endif ] ; <uint> [#uses=2] %accumulator.tr = phi int [ 1, %entry ], [ %tmp.9, %endif ] ; <int> [#uses=2] %cann-indvar = cast uint %cann-indvar to int ; <int> [#uses=1] %n.tr-scale = mul int %cann-indvar, -2 ; <int> [#uses=1] %n.tr = add int %n.tr-scale, %n ; <int> [#uses=2] %next-indvar = add uint %cann-indvar, 1 ; <uint> [#uses=1] %tmp.1 = setle int %n.tr, 1 ; <bool> [#uses=1] br bool %tmp.1, label %then, label %endif then: ; preds = %tailrecurse ret int %accumulator.tr endif: ; preds = %tailrecurse %tmp.5 = add int %n.tr, -1 ; <int> [#uses=1] %tmp.3 = call int %f( int %tmp.5 ) ; <int> [#uses=1] %tmp.9 = add int %tmp.3, %accumulator.tr ; <int> [#uses=1] br label %tailrecurse } --------------------------------------------------- BTW, it was quite impressive for me, that I've found only a one call in the LLVM output... :) Anyway, above we have: call int %f( int %tmp.5 ) and it is a quite high-level statement i'd say! Indeed, LLVM will allocate registers and regular memory for us. This output is free of any strategy of this allocation. We do *not* define strategy of this allocation and we are even quite happy with that! Therefore I could expect, that allocation of CPU could be also a internal back-end problem of the LLVM like registers. In other words, indirectly, I propose to treat CPUs as the same resource class as memory :-) Chris, LLVMers, could you allocate CPUs resources as you do it for registers and memory? :P Actually, maybe the problem is that scince is not ready arround phi-functions, SSA and so on, if we speak about calculating machine with several CPUs spread via network... What could LLVM-gurus say here? -- Valery.
Hello again Valery, Valery A.Khamenya wrote:> All benefits, what one could obtain from "LLVM supporting multiple CPU > at single host", one might obtaine from "LLVM supporting multiple CPU > at multiple hosts". Isn't that logical?I see more precisely what you mean, but I don't think it is that straightforward to generalise the benefits multiple CPU on single host programming to multiple CPU at multiple hosts. I don't think that both cases involve the same techniques. For example, in "single host" configuration you get a very low cost for communicating data because you use memory instead of network. Memory is a low-level primitive medium for communicating data, while network is not a primitive medium because it involves many communication protocols. Memory is simple to manage when it is on a single hardware, but when it turns to a "shared" (or "distributed") memory, things can get pretty complicated (because you have -among others- security, latency, integrity and fault-tolerance to take into account). Moreover, there are many, many ways to implement "distributed" or "parallel" computing. Some approaches are based on distributing data according to the available CPU resources, others on distributing the program control flow according to the proximity of data, and so on. What would you consider as the core primitives of single host, multi-CPU programming ? Cheers, -- Se'bastien