Other types of caches exist (which do not count towards the "cache size" of the most important caches mentioned above), such as the Translation Lookup Cache (TLB) which is one part of the Memory Management Unit (MMU) found on most processors.
When you try to read or write to a location in main memory, the processor checks to see if the data in that location is already in the cache. If so, the processor will read or write to cache instead of main memory which is much slower.
Many modern desktops, servers, and industrial processors have at least three independent caches:
Instruction Cache
Used to speed up retrieval of executable instructions
Data cache
Used to speed up data retrieval and storage; Data caches are typically organized as a hierarchy of multiple cache levels (L1, L2, etc.; see also multi-level caches below). Translation lookup cache (CTR)
Used to speed up virtual-to-physical address translation for executed instructions and data. A single TLB may be provided for both instruction and data access, or a separate instruction TLB (ITLB) and data CTR (DTLB) may be provided. However, the TLB cache is part of the memory management unit (MMU) and is not directly related to the CPU cache.
History
Early processors that used a cache had only one cache level; unlike later level 1 cache, it is not divided into L1d (for data) and L1i (for instructions). Split L1 cache started in 1976 with the IBM 801 processor,[2][3] became popular in the late 1980s, and in 1997 entered the embedded processor market with ARMv5TE. In 2015, even sub-dollar SoCs split the L1 cache. They also have L2 cache and, for larger processors, L3 cache as well. The L2 cache is usually not plugged in and acts as a common storage for the split L1 cache. Each core in a multi-core processor has its own dedicated L1 cache and is not usually shared between cores. L2 cache and higher level cache can be shared between cores. L4 caches are rare now and are usually found on (some form of) dynamic random access memory (DRAM), not static random access memory (SRAM), on a separate die or chip (unusually, the eDRAM form is used for all cache levels, up to L1). This has also been the case historically with L1, since the larger chips allow it to be integrated, and generally all levels of the cache, except perhaps the last one. Each additional cache level tends to be larger and is optimized differently.
Cache memory (as with RAM before) is typically sized to powers: 2, 4, 8, 16, etc. KiB; when up to MiB size (i.e. for larger nonL1s), very soon the pattern was broken, to allow for larger caches without being forced into the size doubling model, for example : Intel Core 2 Duo with 3 MiB of L2 cache as of April 2008. However, much later for L1 sizes, still count at small KiB, however IBM zEC12 from 2012 is one exception, to achieve an unusually large 96 KiB L1 data cache for its day, and for example the IBM z13 has a 96 KiB L1 instruction cache (and 128 KiB L1 data cache) [4] and 2018 Intel Ice Lake processor with 48 KiB L1 data cache and 48 KiB L1 instruction cache. In 2020, some Intel Atom CPUs (with up to 24 cores) have (multiple of) 4.5 MiB and 15 MiB cache sizes.
Cache Entries
Data is transferred between memory and cache in fixed-sized blocks, called cache lines or cache blocks. When a cache line is copied from memory to cache, a cache entry is created. The cache entry will include the copied data along with the requested memory location (called a tag).
When the processor needs to read or write a location in memory, it first looks for the matching entry in the cache. The cache verifies the contents of the requested memory location in all cache lines that are likely to contain this address. If the processor finds that the memory location is in the cache, a cache hit has occurred. However, if the processor can't find the memory location in the cache, a cache miss has occurred. In the event of a cache attack, the processor immediately reads or writes data to the cache line. In the event of a cache failure, the cache allocates a new entry and copies the data from main memory, after which the request is fulfilled from the cache's contents.
Policies
Replacement Policies
To make room for a new entry in the event of a cache miss, the cache may need to discard one of the existing entries. The heuristic method it uses to choose which entries to discard is called the replacement policy. The fundamental problem with any overwrite policy is that it has to predict which current cache entry is least likely to be used in the future. Predicting the future is difficult, so there's no perfect way to choose from the wide variety of alternative fonts available. A popular, least recently used (LRU) substitution policy that replaces the least recently accessed entry.
Marking some areas of memory as non-cacheable can improve performance by avoiding caching areas of memory that are rarely accessed. This avoids the cost of loading something into the cache without reusing it. Cache entries can also be disabled or locked depending on the context.
Writing Policy
Main article: Caching (computer) Writing policy
If data is written to the cache, at some point it must also be written to main memory; This recording time is called the write strategy. In a write-through buffer, each write to the buffer results in a single write to main memory. Also, in the overwrite or copy cache, writes are not copied immediately in main memory, and the cache instead keeps track of locations that have been overwritten, marking them as dirty. Data in these locations is written back to main memory only when that data is removed from the cache. For this reason, missing a read in the write-later cache can sometimes require two memory accesses to the service: one first to write the dirty location to main memory, then another. another access to read the new location from memory. In addition, writing to a main memory location that has not been mapped to the rewrite cache can discard an already dirty location, freeing that cache space for the new memory location.
There are also intermediate policies. The cache can be written over, but writes can be temporarily held in a cache data queue, usually so that multiple stores can be serviced together (which can reduce bus spin-up and improve bus memory usage).
Cached data in main memory can be modified by other entities (e.g. a device using direct memory access (DMA) or another core in a multi-core processor ), in which case the cached copy may become obsolete or obsolete. In addition, when one CPU in a multiprocessor system updates data in the cache, the copies of the data in the cache associated with the other CPUs become stale. The communication protocols between cache managers that provide data coins are known as cache coin protocols.
Cache performance
Measuring cache performance has become important in recent times as the speed gap between memory performance and CPU performance is growing exponentially. Caching was introduced to reduce this speed gap. Therefore, it becomes important to know how well caching can bridge the gap between CPU and memory speeds, especially in high-performance systems. Cache hit rate and cache miss rate play an important role in determining this performance. To improve cache performance, reducing miss rate becomes one of the necessary steps among others. Decreasing the access time to the cache also gives a boost to its performance.
CPU stalls
The time taken to fetch one cache line from memory (read latency due to a cache miss) matters because the CPU will run out of things to do while waiting for the cache line. When a CPU reaches this state, it is called a stall. As CPUs become faster compared to main memory, stalls due to cache misses displace more potential computation; modern CPUs can execute hundreds of instructions in the time taken to fetch a single cache line from main memory.
Various techniques have been employed to keep the CPU busy during this time, including outoforder execution in which the CPU attempts to execute independent instructions after the instruction that is waiting for the cache miss data. Another technology, used by many processors, is simultaneous multithreading (SMT), which allows an alternate thread to use the processor core while the first thread waits for the necessary CPU resources to become available.
You must be logged in to post a comment.