Skip to content

feat(bitnet): implement scaled dot-product attention - #210

Merged
thejhh merged 5 commits into
bitnetfrom
182-compute-scaled-dot-product-attention
May 22, 2025
Merged

feat(bitnet): implement scaled dot-product attention#210
thejhh merged 5 commits into
bitnetfrom
182-compute-scaled-dot-product-attention

Conversation

@thejhh

@thejhh thejhh commented May 22, 2025

Copy link
Copy Markdown
Contributor

Test Coverage

  • Current coverage: 87.1%
  • Coverage changes: 86.3% → 87.1%

Performance Metrics

Memory Usage

Tensor Operations

  • Allocations per operation:
    • New tensor creation: 120 allocs/op
    • Get/Set operations: 0 allocs/op
    • Parallel operations: 160748 allocs/op
    • BitLinear operations: 3548 allocs/op

BitNet Model Operations

CPU Performance

Tensor Operations

  • Operation timing:
    • Basic operations: 12.86 ns/op
    • Parallel operations: 97222 ns/op
    • Large tensor operations: 1287 ns/op
    • BitLinear operations: 24284088 ns/op

BitNet Model Operations

Areas for Improvement

High Priority

Medium Priority

Low Priority

Closes #182

@thejhh thejhh linked an issue May 22, 2025 that may be closed by this pull request
@thejhh
thejhh marked this pull request as ready for review May 22, 2025 16:18
@thejhh thejhh self-assigned this May 22, 2025
@thejhh
thejhh merged commit 44af36b into bitnet May 22, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Compute Scaled Dot-Product Attention

1 participant