Utilities#

eltype(backend::Backend{VT}) -> Type
eltype(::Type{<:Backend{VT}}) -> Type

Return the coordinate and scalar element type of vector type VT configured in backend.

_gpu_functional(::Val{name}) -> Bool

Whether the GPU extension named by name (a Symbol, e.g. :metal) has a functional device, not merely that the corresponding package loaded.

Defaults to false for every name: loading a GPU package extension (using Metal, and in future using CUDA/using AMDGPU) succeeds on any platform, whether or not that host actually has the accelerator's hardware and drivers – it degrades gracefully rather than erroring. ext/BrambleMetalExt.jl overrides this for Val(:metal) with Metal.functional(), the one call that actually probes the device. gpu_backend requires both this and the extension being loaded.

It is declared with the generic ::Val signature (rather than ::Val{:metal} itself) so that BrambleMetalExt's own Val(:metal) method is strictly more specific than this stub – otherwise the extension's method would silently overwrite this one instead of adding to it, which precompilation reports as method overwriting.

MarkedIndices(mask::BitVector, offset::Int = 0)

Lazily iterates the 1-based positions where mask is set, each shifted by offset (so a leaf's mask, consulted at its offset into a global vector, yields global indices without copying). Walks whole 64-bit words at a time, skipping zero chunks entirely and extracting set bits via trailing_zeros, so the work is proportional to the number of set bits, not to length(mask).

No bounds guard against length(mask) is needed: BitVector guarantees the padding bits of its final chunk are zero, so the walk never yields an index past the mask's own length.

MarkedIndicesUnion(masks::NTuple{N,BitVector}) where N

Lazily iterates the 1-based positions where the union of masks is set.

The multi-marker counterpart of MarkedIndices: _combined_mask (space/inner_product.jl) used to materialize the union into a fresh BitVector via copy + .|= before walking it, one heap allocation per call (gpena/Bramble.jl#149). This ORs each mask's 64-bit chunk on the fly instead, so the union is never materialized – still proportional to the number of set bits, still a whole-word skip wherever every mask's chunk is zero, and allocates nothing.

All masks must have the same length: len is taken from the first, and no constructor check compares the others, so a longer or shorter mask reads indices outside 1:len.

_LastAxisChunks(rest::Tuple, ax::AbstractRange, n::Int)

Splits a CartesianIndices into n blocks along its last axis, each block itself a CartesianIndices over the leading axes rest. Indexable and lazy, so the split allocates nothing and Threads.@threads can partition it directly.

Blocks differ in length by at most one slice: the remainder is spread over the first of them rather than left on the last, so no thread receives a double-sized tail.

_band_count(len::Int, span::Int, nthreads::Int) -> Int

How many slabs to cut an axis of len points into for a stencil reaching span along it.

Enough for every thread to hold one slab per colour, never so many that a slab falls below span (which is what keeps alternate slabs free of each other's stencil footprints), and always even so the two colours are balanced. Returns 0 when the axis is too short to band at all, leaving the caller on its point-coloured path.

_band_range(ax::AbstractRange, nbands::Int, b::Int) -> AbstractRange

The b-th of nbands contiguous slabs of ax.

Slabs differ in length by at most one, the remainder spread over the first of them rather than left on the last. b indexes positions within ax, not values, so an axis carrying a stride keeps it, and so does a descending one.

The positions are Int whatever length(ax) returns: a UInt64 range has a UInt64 length, and Base indexes a descending StepRange by a UnitRange{UInt64} with a wrapped, unsigned step ((UInt(9):-1:UInt(1))[UInt(1):UInt(5)] is empty).

_dot(u::AbstractVector, w::AbstractVector, v::AbstractVector) -> Number

Compute the weighted trilinear dot product

Accumulates via fused multiply-add operations (muladd) with @simd vectorization.

The first argument is conjugated (gpena/Bramble.jl#644), and so is u in every policy and device counterpart below; the weight is the second argument, as in every caller. conj of a Real is the identity, so real data, Duals included, gives the same bits as the unconjugated product.

A same-eltype specialization used to sit alongside this one, skipping the promote_type call and the T(...) conversions on the (dispatch-favoured) assumption that they cost something. Compared by @code_llvm/@code_native with matching element types (gpena/Bramble.jl#71): identical generated code, since promote_type(T, T, T) === T and T(x::T) is an identity conversion the compiler elides. One method now covers both cases.

Arguments

• u: First vector, the one that is conjugated.

• w: The weights, second in the call as at every caller.

• v: Third vector.

Throws

• DimensionMismatch: If length(u), length(w), and length(v) do not match.

_dot(::DeviceLocality, ::GpuPolicy, u::AbstractVector, w::AbstractVector, v::AbstractVector) -> Number

The device counterpart of _dot(u, w, v): sum(conj.(u) .* w .* v), one broadcasted GPUArrays reduction, reached once a GpuPolicy derives DeviceLocality from itself.

Throws

• DimensionMismatch: If length(u), length(w), and length(v) do not match.

_dot(policy::ExecutionPolicy, u, w, v) -> Number

Policy-dispatched _dot: derives the locality from policy and dispatches on the pair. On the host, CpuSerial reaches the plain three-vector method, CpuThreaded reaches _threaded_dot and CpuPolyester reaches _batch_dot; a GpuPolicy reaches the device methods below.

_dot_band(u, w, v, ax, nchunks::Int, k::Int) -> Number

The k-th of nchunks partial sums of _threaded_dot, over the band _band_range(ax, nchunks, k). One compiled body serves both the threaded loop and the serial one _static_or_serial falls back to, so the two return the same partial sums bitwise. BramblePolyesterExt runs _dot_band and keeps a copy of _dot_masked_band over the mask's words (_masked_words_band), so a change here must be made there too (gpena/Bramble.jl#473).

The band sums through _fixed_order_sum, so its value does not depend on the optimisation level of the package image (gpena/Bramble.jl#942), and can differ in the last bit from _dot's @simd sum.

_dot_masked(u::AbstractVector, w::AbstractVector, v::AbstractVector, mask::BitVector) -> Number

Compute the weighted dot product restricted to indices where mask is true:

Walks MarkedIndices(mask), so the work is proportional to the number of set bits rather than to length(mask).

As with _dot, a same-eltype specialization used to sit alongside this one; @code_llvm/@code_native with matching element types (gpena/Bramble.jl#71) showed identical generated code, so one method now covers both cases.

Arguments

• u: First vector, the one that is conjugated.

• w: The weights, second in the call as at every caller.

• v: Third vector.

• mask: Boolean selection mask.

Throws

• DimensionMismatch: If vector or mask lengths do not match.

_dot_masked(::DeviceLocality, ::GpuPolicy, u, w, v, mask::BitVector) -> Number

The device counterpart of _dot_masked(u, w, v, mask): mask is copied onto u's own device once, then the masked sum is sum(conj.(u) .* w .* v .* md), one broadcasted reduction instead of a walk over MarkedIndices.

Throws

• DimensionMismatch: If vector or mask lengths do not match.

_dot_masked(::DeviceLocality, ::GpuPolicy, u, w, v, mask::MarkedIndicesUnion) -> Number

The multi-marker counterpart of the BitVector method above: mask is walked once, on the host, to build a plain BitVector (the same union _combined_mask would have materialized), which is then copied onto u's own device exactly as above.

Throws

• DimensionMismatch: If vector or mask lengths do not match.

_dot_masked(policy::ExecutionPolicy, u, w, v, mask) -> Number

Policy-dispatched _dot_masked: as _dot(policy, u, w, v), with the plain masked method, _threaded_dot_masked and _batch_dot_masked on the host.

_dot_masked_band(u, w, v, mask::Union{BitVector, MarkedIndicesUnion}, ax, nchunks::Int, k::Int) -> Number

The k-th of nchunks partial sums of _threaded_dot_masked, over the mask words _band_range(ax, nchunks, k). As _dot_band, one compiled body serves the threaded loop and the serial one. A SeparableWeights weight goes through it too. The band reads such a weight through its CartesianIndices (_masked_cart, _masked_term) and never through the linear getindex. BramblePolyesterExt runs _dot_band and keeps a copy of _dot_masked_band over the mask's words (_masked_words_band and _separable_masked_words_band), so a change here must be made there too (gpena/Bramble.jl#473).

_effective_parallel_policy(sp) -> CpuPolicy

The policy a forced-threaded sweep (assemble_parallel!, or the non-serial branch of assemble!/assemble_add!) actually runs under: CpuSerial is coerced to CpuThreaded – a threaded sweep threads even from a serially configured backend, which is the entire point of forcing it – while CpuThreaded and CpuPolyester keep their own policy, so a CpuPolyester backend still runs its own (Polyester) sweep, or errors clearly without it, rather than silently substituting Threads.@threads. A GpuPolicy is coerced to CpuThreaded as well (bilinear_traversal.jl): the colour-banded sweep is storage-generic, so a device backend runs the same Threads.@threads sweep.

_in_threaded_region() -> Bool

Whether a threaded region is running, so that a Threads.@threads :static loop started now would throw "@threads :static cannot be used concurrently or nested". This is exactly the condition Base.Threads itself tests before a :static loop (ccall(:jl_in_threaded_region, Cint, ()) != 0, base/threadingconstructs.jl): true inside a user's Threads.@threads loop, false at top level. The counter behind it is process-wide, so it is also true in a Threads.@spawned task while some other task's threaded region runs – and can turn true between this check and Base's own, which is why _static_or_serial also catches Base's error.

_is_static_nesting_error(err) -> Bool

Whether err is the error Threads.@threads :static raises when a threaded region is already running: an ErrorException carrying exactly Base's message. Base raises it before it creates or schedules a single task, so no iteration of the loop has run.

_last_axis_chunks(idxs::CartesianIndices{D}, n::Integer) -> _LastAxisChunks

Return idxs split into at most n blocks along its last axis, clamped to the length of that axis so no block is empty.

_serial_for!(v::AbstractArray, idxs, f::Function) -> Nothing

Iterate sequentially over idxs, writing v[idx] = f(idx) in place.

Arguments

• v: Destination array mutated in place.

• idxs: Iterable collection of indices.

• f: Kernel evaluating values at each index.

_static_bands!(f, nbands::Int, args...) -> Nothing
_serial_bands!(f, nbands::Int, args...) -> Nothing

Run f(args..., nbands, b) for every band b in 1:nbands: one band per thread under Threads.@threads :static, or in order on the calling task. The pair the banded stencil engines hand to _static_or_serial.

_static_or_serial(threaded!, serial!, args...) -> Nothing

Run threaded!(args...), a function whose only work is one Threads.@threads :static loop, or serial!(args...), the same loop body run over the same chunks or bands in order on the calling task, when a :static loop cannot start here.

Every CpuThreaded sweep goes through it. _in_threaded_region is checked first, so a call from inside a user's Threads.@threads loop goes straight to serial!. Because that counter is process-wide, a concurrent task can still enter its own threaded region between this check and Base's; Base then throws before running any iteration, and exactly that error (_is_static_nesting_error) is caught and answered by serial!. Anything else is rethrown. Either way the caller gets what a top-level call returns: reductions fill the same partial sums in the same order, so match it bitwise.

threaded! and serial! are isolated functions rather than closures, so no closure is boxed on the path that runs serially.

_static_partials!(partials, n::Int, band, args...) -> Nothing
_serial_partials!(partials, n::Int, band, args...) -> Nothing

Fill partials[k] = band(args..., k) for k in 1:n: one k per thread under Threads.@threads :static, or in order on the calling task. The pair every threaded reduction hands to _static_or_serial; both call the same @noinline band, so their partial sums agree bitwise.

_sweep_for!(loc::Locality, policy::ExecutionPolicy, v::AbstractArray, idxs, f::Function) -> Nothing
_sweep_for!(policy::ExecutionPolicy, v::AbstractArray, idxs, f::Function) -> Nothing

Apply f across indices idxs and write the result into v in place.

Legality first, strategy second (gpena/Bramble.jl#298): a HostLocality destination paired with a CpuPolicy dispatches to sequential iteration for CpuSerial or static work partitioning across threads for CpuThreaded; a DeviceLocality destination paired with a GpuPolicy reaches _gpu_for!. Any other pairing – a host v under a GpuPolicy, or a device v under a CpuPolicy – throws via _throw_locality_mismatch rather than indexing v element by element or reaching for a device sweep that does not exist.

The five-argument method is the seam; the four-argument method is the thin wrapper every caller actually uses, deriving loc from the destination array itself (locality(typeof(v))) so callers pass a policy alone and never compute a locality.

Arguments

• loc: Locality of the destination array (derived, not passed by callers).

• policy: Execution policy (CpuSerial, CpuThreaded, CpuPolyester or a GpuPolicy).

• v: Destination array mutated in place.

• idxs: Iterable collection of linear or Cartesian indices.

• f: Kernel mapping each index idx to the scalar value stored in v[idx].

_sweep_scatter_for!(loc::Locality, policy::ExecutionPolicy, mats::Tuple, idxs, g::Function) -> Nothing
_sweep_scatter_for!(policy::ExecutionPolicy, mats::Tuple, idxs, g::Function) -> Nothing

Evaluate tuple-valued kernel g across idxs and scatter results into destination arrays mats.

Legality first, strategy second (gpena/Bramble.jl#298), the same shape as _sweep_for!: a HostLocality destination paired with a CpuPolicy dispatches to sequential execution for CpuSerial or static multithreaded execution for CpuThreaded; a DeviceLocality destination paired with a GpuPolicy reaches _gpu_scatter_for!. Any other pairing throws via _throw_locality_mismatch.

The five-argument method is the seam; the four-argument method is the thin wrapper every caller actually uses, deriving loc from the first destination array (locality(typeof(mats[1]))) so callers pass a policy alone and never compute a locality.

Arguments

• loc: Locality of the destination arrays (derived, not passed by callers).

• policy: Execution policy (CpuSerial, CpuThreaded, CpuPolyester or a GpuPolicy).

• mats: Tuple of destination arrays mutated in place.

• idxs: Iterable collection of indices.

• g: Kernel mapping each index to a tuple of values matching length(mats).

_threaded_axis_for!(v::AbstractArray, idxs::CartesianIndices, f::Function) -> Nothing

As _threaded_for!, for a CartesianIndices: each thread takes one block of whole last-axis slices and walks it natively, never converting a linear index.

_threaded_dot(u::AbstractVector, w::AbstractVector, v::AbstractVector) -> Number

CpuThreaded's own counterpart of _dot: partitions 1:length(u) into Threads.nthreads() static bands via _band_range, each Threads.@threads :static task accumulating its own partial sum in a fixed order, landed in a fixed Threads.nthreads()-length buffer and summed serially once every task has returned. Where a :static loop cannot start (_static_or_serial) the same bands run in turn on the calling task, through the same _dot_band, so the value is unchanged bitwise. At one thread there is a single band, and it returns _dot(u, w, v) itself, so CpuSerial, CpuThreaded and CpuPolyester agree bitwise there.

The buffer's size depends only on the thread count, never on length(u), so allocation is independent of vector length rather than zero. No size threshold falls back to the serial kernel below a cutoff, and no new dependency is pulled in to avoid the buffer.

Kept in an isolated function, the same reason _threaded_for! is, to keep Threads.@threads from boxing a closure over u/w/v on paths that execute serially.

Throws

• DimensionMismatch: If length(u), length(w), and length(v) do not match.

_threaded_dot_masked(u::AbstractVector, w::AbstractVector, v::AbstractVector, mask::Union{BitVector, MarkedIndicesUnion}) -> Number

CpuThreaded's own counterpart of _dot_masked: partitions the mask's 64-bit words (not 1:length(u)) into Threads.nthreads() static bands, so the work stays proportional to the number of set bits rather than to length(u) – masks here are typically a mesh marker such as :boundary, O(perimeter) set bits inside an O(n^D) index range, and walking every index (as an earlier version of this function did) turned that sparsity into a slowdown under CpuThreaded relative to plain CpuSerial.

Each task loads its band's words once (_mask_word: mask.chunks[w] for a BitVector, joined across markers for a MarkedIndicesUnion), skips whole zero words, and walks a nonzero word's set bits with trailing_zeros/word & (word - 1), the same walk MarkedIndices/MarkedIndicesUnion use. No bounds guard against length(u) is needed for the padding bits of the final word, for the same reason MarkedIndices needs none: a BitVector's own invariant guarantees they are zero, and a MarkedIndicesUnion ORs only BitVectors of that same length together.

Throws

• DimensionMismatch: If vector or mask lengths do not match.

_threaded_for!(v::AbstractArray, idxs, f::Function) -> Nothing

Fill v[idx] with f(idx) across threads, statically partitioning idxs, or serially where a :static loop cannot start (_static_or_serial).

The Threads.@threads loop is kept in an isolated function to prevent closure boxing allocations on paths that execute serially. The threaded branch hands the kernel to _static_for! behind a Ref, as _sweep_for! hands it to _batch_for!: BrambleEnzymeExt's rule for _static_for! cannot take a by-value closure that captures an array and a plain value (Enzyme's calling convention).

_threaded_scatter_for!(mats::Tuple, idxs, g::Function) -> Nothing

Evaluate tuple-valued g across idxs and scatter the results into mats, statically partitioning idxs across threads.

Kept in an isolated function to prevent Threads.@threads closure boxing allocations on paths that execute serially. The threaded branch hands the kernel to _static_scatter_for! behind a Ref, as _threaded_for! does.

_throw_locality_mismatch(loc::Locality, policy)

Throw the ArgumentError _sweep_for!/_sweep_scatter_for! give when the destination's locality and the execution policy's locality disagree, in either direction (gpena/Bramble.jl#191, #298).

Stated here rather than left to a MethodError, which would name _sweep_for! and not say which half disagreed. Dispatched on the destination's own locality so each direction names what actually disagreed – the destination's locality and the policy's – rather than assuming a GpuPolicy is always the one out of place. Neither method advises building the backend with a CpuPolicy as a fix for device storage: that combination is itself rejected at construction (gpena/Bramble.jl#296), and is not a route this message may point to.

Arguments

• loc: Locality of the destination array.

• policy: The execution policy whose locality disagrees with loc.

Throws

• ArgumentError: always.

_throw_no_device_sweep(policy)

Throw the ArgumentError _gpu_for!/_gpu_scatter_for! give when no device sweep implementation is available at all – a case _throw_locality_mismatch never reaches, since it is only called once locality has already been checked to agree.

Arguments

• policy: The GpuPolicy with no device sweep loaded for it.

Throws

• ArgumentError: always.

_write_components!(mats::Tuple, vals::Tuple, idx) -> Nothing

Recursively unpack and write elements of vals into destination arrays mats at index idx.

Recursion on tuples unrolls at compile time with zero heap allocations.

_batch_axis_for!(v::AbstractArray, idxs::CartesianIndices, f::Function) -> Nothing

CpuPolyester's counterpart of _threaded_axis_for!, filled by BramblePolyesterExt. Its src/ method only throws, naming Polyester; BramblePolyesterExt defines the one that runs.

_batch_dot(u, w, v) -> Number

CpuPolyester's counterpart of _dot, filled by BramblePolyesterExt. Its src/ method only throws, naming Polyester; BramblePolyesterExt defines the one that runs.

_batch_dot_masked(u, w, v, mask) -> Number

CpuPolyester's counterpart of _dot_masked, filled by BramblePolyesterExt. Its src/ method only throws, naming Polyester; BramblePolyesterExt defines the one that runs.

_batch_for!(v::AbstractArray, idxs, f::Function) -> Nothing

CpuPolyester's counterpart of _threaded_for!, filled by BramblePolyesterExt (gpena/Bramble.jl#190). Its src/ method only throws, naming Polyester; BramblePolyesterExt defines the one that runs.

_batch_scatter_for!(mats::Tuple, idxs, g::Function) -> Nothing

CpuPolyester's counterpart of _threaded_scatter_for!, filled by BramblePolyesterExt. Its src/ method only throws, naming Polyester; BramblePolyesterExt defines the one that runs.

_gpu_for!(policy::GpuPolicy, v::AbstractArray, idxs, f::Function) -> Nothing

The GpuPolicy counterpart of _serial_for!/_threaded_for!: fills v[idx] with f(idx) across idxs with a KernelAbstractions.@kernel launch on v's own device, filled by ext/BrambleKernelAbstractionsExt.jl.

f runs on the device, so it must be GPU-compilable: no heap allocation, no boxed captures (a Ref, a mutable struct field, a global), and every function it calls must itself compile for the device. A closure over a plain number or over another device array works; one that allocates or calls a non-inlineable function fails at kernel-compile time with a GPUCompiler.InvalidIRError, not a Bramble-specific message – this is inherent to GPU execution.

Throws

• ArgumentError: no KernelAbstractions extension is loaded, so there is no device sweep to reach (_throw_no_device_sweep).

_gpu_scatter_for!(policy::GpuPolicy, mats::Tuple, idxs, g::Function) -> Nothing

The GpuPolicy counterpart of _threaded_scatter_for!: evaluates tuple-valued g across idxs and scatters the results into mats with a KernelAbstractions.@kernel launch, filled by ext/BrambleKernelAbstractionsExt.jl. Same GPU-compilability requirement on g as _gpu_for!.

Throws

• ArgumentError: no KernelAbstractions extension is loaded (_throw_no_device_sweep).

_batch_rebuild(skeleton, arrays::Tuple) -> rebuilt

The value _batch_split(x) split, rebuilt around arrays: the same struct types, with each array type parameter replaced by the type of the array now in its slot, each mesh by its walk state and each Ref by its value; a SpaceWeights is the caller's own object, so a space keeps its weights' vector type. arrays may hold any array type, such as the PtrArrays @batch passes, or the plain arrays it passes a single-iteration loop, provided every array split from one original array type arrives as one array type: the type parameters are substituted per original type, not per slot. @batch converts all arrays of one type alike, so it always meets this.

Throws

• ArgumentError: if a struct's declared field type does not accept the rebuilt field, as when two arrays of one original type arrive as different array types.

_batch_split(x) -> (skeleton, arrays::Tuple)

Split x into an isbits skeleton and the flat tuple arrays of the plain arrays it holds, so a Polyester.@batch loop can capture both without a heap-allocated argument box; _batch_rebuild is the inverse. x is any walk argument, typically the tuple (space, term) of one walk unit.

The walk goes through every field of every immutable struct and tuple in x, and each leaf takes one rule:

• an Array of an isbits element type becomes a slot of arrays;

• a mutable mesh is replaced by its walk state _walk_mesh, which is split in turn (a GhostMesh by its GhostMeshState);

• a Base.RefValue of an isbits value is replaced by that value, read once at split time on the host: a Ref coefficient is read per call at launch, not per point;

• a SpaceWeights crosses as its address, and the caller must keep x alive until the loop ends;

• an isbits value is kept.

Throws

• ArgumentError: naming the type and the path of any other value (a Dict, a Symbol, a mutable struct, a field of abstract type), so a new value that would box fails loudly. A hook that must accept such a value asks _batch_splittable first and, on false, captures it whole, which boxes.

_batch_splittable(::Type{T}) -> Bool

Whether _batch_split takes a value of type T apart and _batch_rebuild puts it back together around the arrays a Polyester.@batch task receives, rather than either throwing: true when every leaf of T takes one of the split's rules and every struct field's declared type accepts any array in place of the one it held (a field typed Vector{T} does not). Generated, so the answer is a constant of the type and a branch on it costs the caller nothing. A split hook that meets false (an array of BigFloats, a BigFloat coefficient, a struct holding a Vector{Float64} field) captures its arguments whole instead.

_ProfileKernel{T}

The kernel profile_backends sweeps: i -> sqrt(T(i)) * a + b. An isbits struct rather than a closure, so the same value is a legal device kernel argument, and cheap enough per point that the sweep is bound by the policy's memory traffic and scheduling, as the weight and scatter loops are.

_crossover(p::BackendProfile, j::Integer) -> Union{Int, Nothing}

Smallest size from which policy j stays at least _PROFILE_SPEEDUP times faster than the baseline (column 1) at every larger size of the sweep, or nothing if it never does.

_profile_eltype(policy::ExecutionPolicy) -> Type{<:AbstractFloat}

The widest element type policy can sweep. Float64 unless a device extension says otherwise; profile_backends times every row on the narrowest of the declared types.

_profile_time(policy::ExecutionPolicy, ::Type{T}, n::Integer) -> Float64

Seconds taken by the fastest of three sweeps of _ProfileKernel{T} over 1:n of a Vector{T} under policy, after one warm run that compiles and faults the pages in. The vector is allocated inside the call, so the timed region holds the sweep alone.

Device extensions add a method for their own policy type; see the note at the top of src/utils/backend_profile.jl.

_launch_spmm_csr!(C, rowPtr, colVal, nzVal, B, α, β) -> Nothing

Row-parallel sparse matrix-matrix product C .= α .* (A * B) .+ β .* C for a dense right-hand side B, where A's storage is given as the raw CSR arrays rowPtr, colVal, nzVal. See _launch_spmv_csr! for the extension contract and why the arrays are passed separately rather than as a struct.

Requires using KernelAbstractions; the real method is supplied by BrambleKernelAbstractionsExt.

Throws

• ErrorException: if KernelAbstractions is not loaded.

_launch_spmv_csr!(y, rowPtr, colVal, nzVal, x, α, β) -> Nothing

Row-parallel sparse matrix-vector product y .= α .* (A * x) .+ β .* y, where A's storage is given as the raw CSR arrays rowPtr, colVal, nzVal – never a struct wrapping them, since a struct nesting a device array fails KernelAbstractions kernel compilation. One work item owns one output row, so there are no write conflicts and no atomics.

Requires using KernelAbstractions; the real method is supplied by BrambleKernelAbstractionsExt.

Throws

• ErrorException: if KernelAbstractions is not loaded.

@forward T.field functions
@forward T.field (f, g, ...)

Generate delegating method definitions forwarding function calls on type T to x.field.

For each supplied function f, generates an inlined method:

@inline f(x::T, args...; kwargs...) = f(x.field, args...; kwargs...)

Examples

using Bramble: @forward

struct Container
    data::Vector{Float64}
end

@forward Container.data (Base.length, Base.size)

c = Container([1.0, 2.0, 3.0])
length(c) == 3 && size(c) == (3,)

# output
true