Position embedding
Also called position embeddings
A vector added to each token's embedding so the model knows where the token sits in the sequence.
Also called position embeddings
A vector added to each token's embedding so the model knows where the token sits in the sequence.