CMP EMBEDDED.COM

Login | Register     Welcome Guest  
HOME DESIGN PRODUCTS COLUMNS E-LEARNING CONFERENCES CODE FORUMS/BLOGS NEWSLETTERS CONTACT FEATURES RSS RSS

Making the transition from sequential to implicit parallel programming: Part 5
Implicit Parallel Programming: Declarative Programming Languages



Embedded.com
Implicit Parallelism Due to Lack of Anti-dependences
In functional languages, there is no such thing as a variable whose value can change during the computation. Rather, a variable behaves like a mathematical variable, having a unique value that is computed (or "solved") by the program.

Multiple definitions of a variable are simply not allowed and all computations are expressed in terms of computing new values from old. For example, given A, a vector of numbers, suppose we want to scale it by 5; that is, the result should be a vector whose i'th component should be 5A(i). In Fortran, we might say:

do i = 1,N
        A(i) = 5 * A(i)
end do

and A itself contains the result. Further, the old value of A is no longer available. In functional languages, such a concept does not exist; instead, we define a new vector based on the old. For example, in pH

b = array (1,n)
        [(i, 5 * a!i) | i <- [1..n]]

(In Haskell, array indexing is denoted by the infix "!" operator.) This defines b, a new vector with index range 1 to n, such that b(i) = 5a(i). Nothing has happened to a-it still means exactly what it used to mean. For example, we could now produce a third vector c that is the vector-sum of a and b:

c = array (1,n)
            [(i, a!i +b!i) | i <- [1..n]]

Matrix multiplication may be expressed in pH as follows:

matmult a b =
    matrix ((1,1), (n,n))
                [((i,j), ip A i B j) | i <- [1..n], j <- [1..n]]

ip A i B j =
    let
        s = a!(i,l) * b!(l,j)
    in
        for k <- [2..n] do
            next s = s + a!(i,k) * b!(k,j)
        finally s

The first part defines the function matmult, which takes two arguments-the matrices a and b. It produces a new n x n matrix whose (i,j)'th component is the inner product of the i'th row of a with the j'th column of b.

(pH permits us to extract size information from a matrix, but since this representation choice is orthogonal to whether a language is functional or not, and since we want to keep the example simple, we assume that a and b are n x n matrices.)

The second part defines the inner product function ip. The let statement binds the value of a(i,1) x b(1,j) to a variable s. Although the loop may superficially look like the corresponding Fortran loop, it is fundamentally different-it expresses the idea of n separate and concurrent computations:

sl = a!(i,j) * b!(1,j)
s2 = s1 + a!(i,2) * b!(2,j)
s3 = s2 + a!(i,3) * b!(3,j)

sn = s(n-1) + a!(i,n) * b!(n,j)

The first computation is the one outside the loop (the let-binding), and sn, the final s, is the value of the entire loop. In other words, there are n different values of s; the next notation simply relates each sk to its corresponding s(k-1); and the finally term just specifies which s is returned as the result value of the whole loop (and, in turn, the function).

The statements of a loop are all executed concurrently, limited only by data dependencies (although the data dependencies in this particular loop do not admit much parallelism other than doing the multiplications in parallel). Thus, it is not the case that s is a variable that is being updated, as in the Fortran loop-each variable still has a unique definition.

The pH program for matrix multiplication directly expresses the structures in original figures at the start of this series of articles, illustrating the mathematical definition. In particular, it takes no position on the order in which computations are done, beyond the natural ordering of data dependencies.

Since variables never change, there cannot ever be any antidependencies. It is for this reason that functional languages are considered to be "naturally parallel" languages.

1 | 2 | 3

Rate this article: Low High
Current rating
  • .
Embedded.com Career Center
Looking for a new job?
SEARCH JOBS

Browse all jobs

SPONSOR
RECENT JOB POSTINGS





 :