Heterogeneous Tile
The NaplesPU project provides a heterogeneous tile integrated into the NoC, meant to be extended by the user. Such a tile provides a first example of how to integrate a custom module within the network-on-chip. The project comes along with a dedicated prototype in src/mc/tile/tile_ht.sv meant to be extended for custom logic.
The provided tile_ht module instantiates all modules typical of an NPU tile, except for the GPGPU core, and a modified version of the load/store unit wrapped into a simplified interface meant to be easily accessible for a custom component.
The HT tile provides to users two main interfaces to interact with the system: the Memory Interface and the Synchronization Interface further explained below. Such a tile provides the following parameters for user-specific configurations:
`include "npu_user_defines.sv" `include "npu_defines.sv" `include "npu_coherence_defines.sv" module tile_ht # ( parameter TILE_ID = 0, // Current tile ID parameter CORE_ID = 0, // Current core ID, not used in this type of tile parameter TILE_MEMORY_ID = 9, // ID of the memory controller tile parameter THREAD_NUMB = 8, // Supported thread number, each thread has a separate FIFO in the LSU and requests from different threads are elaborated concurrently - Must be a power of two parameter ADDRESS_WIDTH = `ADDRESS_SIZE, // Memory address width - has to be congruent with the system address width parameter DATA_WIDTH = `CACHE_LINE_WIDTH, // Data bus width - has to be congruent with the system parameter L1_WAY_NUMB = 4, // Number of way in the L1 data cache parameter L1_SET_NUMB = 32, // Number of L1 data sets parameter SYNCH_SUPP = 1 // Allocates barrier_core modules for synchronization support )
Contents
Memory Interface
The Memory Interface provides a transparent way to interact with the coherence system. The memory interface implements a simple valid/available handshake per thread, a different thread might issue different memory transaction and those are concurrently handled by the coherence system.
When a thread has a memory request, it first checks the availability bit related to its ID, if this is high the thread issues a memory transaction setting the valid bit and loading all the needed information on the Memory Interface.
Supported memory operations are reported below along with their opcodes:
LOAD_8 = 'h0 - 'b000000 LOAD_16 = 'h1 - 'b000001 LOAD_32 = 'h2 - 'b000010 LOAD_V_8 = 'h7 - 'b000111 LOAD_V_16 = 'h8 - 'b001000 LOAD_V_32 = 'h9 - 'b001001 STORE_8 = 'h20 - 'b100000 STORE_16 = 'h21 - 'b100001 STORE_32 = 'h22 - 'b100010 STORE_V_8 = 'h24 - 'b100100 STORE_V_16 = 'h25 - 'b100101 STORE_V_32 = 'h26 - 'b100110
A custom core to be integrate d in the NaplesPU system ought to implement the following interface in order to communicate with the memory system:
/* Memory Interface */ // To Heterogeneous LSU output logic req_out_valid, // Valid signal for issued memory requests output logic [31 : 0] req_out_id, // ID of the issued request, mainly used for debugging output logic [THREAD_IDX_W - 1 : 0] req_out_thread_id, // Thread ID of issued request. Requests running on different threads are dispatched to the CC conccurrently output logic [7 : 0] req_out_op, // Operation performed output logic [ADDRESS_WIDTH - 1 : 0] req_out_address, // Issued request address output logic [DATA_WIDTH - 1 : 0] req_out_data, // Data output // From Heterogeneous LSU input logic resp_in_valid, // Valid signal for the incoming responses input logic [31 : 0] resp_in_id, // ID of the incoming response, mainly used for debugging input logic [THREAD_IDX_W - 1 : 0] resp_in_thread_id, // Thread ID of the incoming response input logic [7 : 0] resp_in_op, // Operation code input logic [DATA_WIDTH - 1 : 0] resp_in_cache_line, // Incoming data input logic [BYTES_PERLINE - 1 : 0] resp_in_store_mask, // Bitmask of the position of the requesting bytes in the incoming data bus input logic [ADDRESS_WIDTH - 1 : 0] resp_in_address, // Incoming response address
The Heterogeneous Tile shares the same LSU and CC of a NPU tile, consequently the LSU forwards on the Memory Interface its backpressure signals, as follows:
// From Heterogeneous accelerator - Backpressure signals input logic [THREAD_NUMB - 1 : 0] lsu_het_almost_full, // Thread bitmask, if i-th bit is high, i-th thread cannot issue requests. input logic [THREAD_NUMB - 1 : 0] lsu_het_no_load_store_pending, // Thread bitmask, if i-th bit is low, i-th thread has no pending operations.
In particular, lsu_het_almost_full i-th bit has to be low before issuing a memory request for i-th thread.
The Memory Interface provides performance counters as part of its interface:
// From Heterogeneous LSU - Performance counters input logic resp_in_miss, // LSU miss on resp_in_address input logic resp_in_evict, // LSU eviction (replacement) on resp_in_address input logic resp_in_flush, // LSU flush on resp_in_address input logic resp_in_dinv, // LSU data cache invalidatio on resp_in_address
Those signals state when the L1 Data cache incurs in a miss, eviction (or replacement), flush, and data cache invalidation.
The LSU in the Heterogeneous Tile can be configured in two different modalities, namely Write-Through and Write-Back:
output logic lsu_het_ctrl_cache_wt, // Enable Write-Through cache configuration.
When lsu_het_ctrl_cache_wt is high the LSU acts as a Write-Through cache, dually when low it implements a Write-Back mechanism.
Finally, the Memory Interface provides error signals in case of address misaligned in the issued request:
// Heterogeneous accelerator - Flush and Error signals input logic lsu_het_error_valid, // Error coming from LSU input register_t lsu_het_error_id, // Error ID - Misaligned = 380 input logic [THREAD_IDX_W - 1 : 0] lsu_het_error_thread_id, // Thread involved in the Error
Synchronization Interface
The Synchronization Interface connects the user logic with the synchronization module core-side allocated within the tile (namely the barrier_core unit).
Such an interface allows user logic to synchronize on a thread grain, when parameter SYNCH_SUPP is high the tile implements the synchronization support allocating a barrier_core module which handles the synchronization events core-side:
// SYNCH_SUPP parameter allocates the following barrier_core, which
// provides core-side synchronization support.
generate
if ( SYNCH_SUPP == 1) begin
barrier_core # (
.TILE_ID ( TILE_ID ),
.THREAD_NUMB ( THREAD_NUMB ),
.MANYCORE ( 1 ),
.DIS_SYNCMASTER ( DISTR_SYNC )
)
u_barrier_core (
.clk ( clk ),
.reset ( reset ),
// Synch Request - Core Interface
.opf_valid ( breq_valid ),
.opf_inst_scheduled ( bc_inst_scheduled ),
.opf_fetched_op0 ( breq_barrier_id ),
.opf_fetched_op1 ( breq_thread_numb ),
.bc_release_val ( bc_release_val ),
...
);
end else begin
assign bc_release_val = {THREAD_NUMB{1'b1}};
assign c2n_account_valid = 1'b0;
assign c2n_account_message = sync_account_message_t'(0);
assign c2n_account_destination_valid = tile_mask_t'(0);
assign n2c_mes_service_consumed = 1'b0;
end
endgenerate
The synchronization mechanism supports inter- and intra- tile barrier synchronization. When a thread hits a synchronization point, it issues a request to the distributed synchronization master through the Synchronization Interface.
Then, the thread is stalled (up to the user logic) till its release signal is high again.
A custom core may implement the following interface if synchronization is requeried:
/* Synchronization Interface */ // To Barrier Core output logic breq_valid, // Hit barrier signal, sends a synchronization request output logic [31 : 0] breq_op_id, // Synchronization operation ID, mainly used for debugging output logic [THREAD_NUMB - 1 : 0] breq_thread_id, // ID of the thread perfoming the synchronization operation output logic [31 : 0] breq_barrier_id, // Barrier ID, has to be unique in case of concurrent barriers output logic [31 : 0] breq_thread_numb, // Total number - 1 of synchronizing threads on the current barrier ID // From Barrier Core input logic [THREAD_NUMB - 1 : 0] bc_release_val // Stalled threads bitmask waiting for release (the i-th bit low stalls the i-th thread)
Service Message Interface
The Service Message Interface connects the user logic with the service network. The heterogeneous tile can send message to other nodes through this interface, this is often used for host-tile communication. A custom core may implement the following interface if communication through messages is required:
/* Service Message Interface */ // From Service Network input logic message_in_valid, // Valid bit for incoming Service Message input service_message_t message_in, // Incoming message from Service Network output logic n2c_mes_service_consumed, // Service Message consumed // To Service Network output logic message_out_valid, // Valid bit for outcoming Service Message output service_message_t message_out, // Outcoming Service Message data input logic network_available, // Service Network availability bit output tile_mask_t destination_valid // One-Hot destinations bitmap
The Service Message interface is a standard valid/available interface, when message_in_valid is asserted an incoming message message_in is valid, and the user ought to assert for a clock cycle the n2c_mes_service_consumed bit, which signals to the Network Interface that the message has been correctly received and handled. The incoming message is declared as a service_message_t type, as shown in the following code snippet:
typedef struct packed {
service_message_type_t message_type;
service_message_data_t data;
} service_message_t;
Field data stores the information received, while message_type states the nature of the incoming message. In such cases this value might be HOST, in case of message from the host, or HT_CORE in case of a message from another heterogeneous tile.
On the other hand, whenever the user logic has a message to send over the network, it builds the output message message_out, storing the message body in the data field, while tying the message_type filed to HT_CORE value. Then, the destination tile (or tiles) has to be declared in the destination_valid output bitmask, each tile in the network is decoded using a bitmask, and each bit represents a different tile based on the position of the corresponding bit, e.g. bit in position 0 targets tile 0, and so on. The Network Interface connected will forward the message to each tile declared in such a mask. When both message and destinations are ready, the user first checks the availability of the network reading the network_available input bit, waiting for it if necessary. Then, the control logic might assert the message_out_valid signal forwarding both message and destinatins to the Network Interface, which will take care of the delivery from this moment onward.
Heterogeneous Dummy provided
This FSM first synchronizes with other ht in the NoC. Each dummy core in a ht tile requires a synchronization for LOCAL_BARRIER_NUMB threads (default = 4).
// Issue synchronization requests
SEND_BARRIER : begin
breq_valid <= 1'b1;
breq_barrier_id <= 42;
barrier_served <= 1'b1;
if(rem_barriers == 1)
next_state <= WAIT_SYNCH;
else
next_state <= IDLE;
end
The SEND_BARRIER state sends LOCAL_BARRIER_NUMB requests with barrier ID 42 through the Synchronization interface. It sets the total number of threads synchronizing on the barrier ID 42 equal to TOTAL_BARRIER_NUMB (=LOCAL_BARRIER_NUMB x `TILE_HT, number of heterogeneous tile in the system). When the last barrier is issued, SEND_BARRIER jumps to WAIT_SYNCH waiting for the ACK from the synchronization master.
// Synchronizes all dummy cores
WAIT_SYNCH : begin
if(&bc_release_val)
next_state <= IDLE;
end
At this point, all threads in each ht tile are synchronized, and the FSM starts all pending memory transactions.
The START_MEM_READ_TRANS performs LOCAL_WRITE_REQS read operations (default = 128), performing a LOAD_8 operation (op code = 0) each time. In the default configuration, 128 LOAD_8 operations on consecutive addresses are spread among all threads and issued to the LSU through the Memory interface. When read operations are over, the FSM starts writing operations in a similar way.
// Starting multiple read operations
START_MEM_READ_TRANS : begin
if ( rem_reads == 1 )
next_state <= DONE;
else
next_state <= IDLE;
if(lsu_het_almost_full[thread_id_read] == 1'b0) begin
read_served <= 1'b1;
req_out_valid <= 1'b1;
req_out_id <= rem_reads;
req_out_op <= 0; // LOAD_8
incr_address <= 1'b1;
req_out_thread_id <= thread_id_read;
end
end
The START_MEM_WRITE_TRANS performs LOCAL_WRITE_REQS (default = 128) write operations on consecutive addresses through the Memory interface. This time the operation performed is a STORE_8, and all ht tile are issuing the same store operation on the same addresses competing for the ownership in a transparent way. The coherence is totally handled by the LSU and CC, on the core side lsu_het_almost_full bitmap states the availability of the LSU for each thread (both writing and reading).
// Starting multiple write operations
START_MEM_WRITE_TRANS : begin
if ( pending_writes )
next_state <= IDLE;
else
next_state <= DONE;
if(lsu_het_almost_full[thread_id_write] == 1'b0 ) begin
write_served <= 1'b1;
req_out_valid <= 1'b1;
req_out_id <= rem_writes;
req_out_thread_id <= thread_id_write;
req_out_op <= 'b100000; // STORE_8
tmp_data_out[0] <= 8'hee;
incr_address <= 1'b1;
end
end
In both states, a thread first checks the availability stored in a position equal to its ID (lsu_het_almost_full[thread_id]), then performs a memory transaction.
Adding custom logic
The provided heterogeneous dummy core might be replaced with a custom accelerator or user logic, that ought to be allocated into the tile_ht replacing the following lines:
// -----------------------------------------------------------------------
// -- Tile HT - Dummy Core
// -----------------------------------------------------------------------
het_core_example #(
.TILE_ID ( TILE_ID ),
.THREAD_NUMB ( THREAD_NUMB )
) u_dummy_het_core (
.clk ( clk ),
.reset ( reset ),
/* Memory Interface */
.req_out_valid ( req_in_valid ),
.req_out_id ( req_in_id ),
.req_out_thread_id ( req_in_thread_id ),
.req_out_op ( req_in_op ),
.req_out_address ( req_in_address ),
.req_out_data ( req_in_data ),
...
/* Synchronization Interface */
.breq_valid ( breq_valid ),
.breq_op_id ( breq_op_id ),
.breq_thread_id ( breq_thread_id ),
.breq_barrier_id ( breq_barrier_id ),
.breq_thread_numb ( breq_thread_numb ),
...
);
