Repository navigation
Tags
Tags
[Backport] Resource Group Level CPU Control (tikv#19482) Signed-off-by: Debin <debin.jia@pingcap.com>
tikv_alloc: remove thread from thread_arena_map when thread exits (ti… …kv#19479) (tikv#19481) close tikv#19478 Fix inaccurate `tikv_allocator_thread_stats` reporting in the jemalloc-backed allocator metrics. `tikv_allocator_thread_stats` is derived from per-arena jemalloc stats grouped by trimmed thread name. The collector could over-report in two cases: 1. `THREAD_ARENA_MAP` entries were not removed when a thread exited, so stale arena stats could remain visible in the metric. 2. Arena stats were summed without deduplicating `(thread_name, arena_index)`, so the same arena could be counted more than once for one logical thread group. This patch fixes both issues by removing the current thread from `THREAD_ARENA_MAP` in `remove_thread_memory_accessor()` and deduplicating arena indices during `iterate_arena_allocation_stats()` Signed-off-by: glorv <glorvs@163.com> Co-authored-by: glorv <glorvs@163.com>
crossbeam-skiplist: fix RefRange iterator memory leak (tikv#19461) * crossbeam-skiplist: fix RefRange iterator memory leak RefRange::next() and next_back() were using clone_from() to update self.head/self.tail. Since RefEntry has no Drop implementation, the old entry was dropped without decrementing its refcount, causing permanent memory leaks. Fix by using mem::replace() and manually calling decrement() on the old entry, matching the pattern used in RefIter::next(). Signed-off-by: ekexium <eke@fastmail.com> (cherry picked from commit 043854d) Signed-off-by: ekexium <eke@fastmail.com> * fix fmt Signed-off-by: ekexium <eke@fastmail.com> * *: fix clippy by upgrading bytes crate (tikv#19346) (tikv#19349) close tikv#19345 Fix the clippy by upgrading bytes crate. Signed-off-by: ti-chi-bot <ti-community-prow-bot@tidb.io> Signed-off-by: gengliqi <gengliqiii@gmail.com> Signed-off-by: you06 <you1474600@gmail.com> Co-authored-by: you06 <you1474600@gmail.com> Co-authored-by: gengliqi <gengliqiii@gmail.com> (cherry picked from commit c4d7f12) Signed-off-by: ekexium <eke@fastmail.com> --------- Signed-off-by: ekexium <eke@fastmail.com> Co-authored-by: Ti Chi Robot <ti-community-prow-bot@tidb.io> Co-authored-by: you06 <you1474600@gmail.com> Co-authored-by: gengliqi <gengliqiii@gmail.com>
crossbeam-skiplist: fix RefRange iterator memory leak (tikv#19461) * crossbeam-skiplist: fix RefRange iterator memory leak RefRange::next() and next_back() were using clone_from() to update self.head/self.tail. Since RefEntry has no Drop implementation, the old entry was dropped without decrementing its refcount, causing permanent memory leaks. Fix by using mem::replace() and manually calling decrement() on the old entry, matching the pattern used in RefIter::next(). Signed-off-by: ekexium <eke@fastmail.com> (cherry picked from commit 043854d) Signed-off-by: ekexium <eke@fastmail.com> * fix fmt Signed-off-by: ekexium <eke@fastmail.com> * *: fix clippy by upgrading bytes crate (tikv#19346) (tikv#19349) close tikv#19345 Fix the clippy by upgrading bytes crate. Signed-off-by: ti-chi-bot <ti-community-prow-bot@tidb.io> Signed-off-by: gengliqi <gengliqiii@gmail.com> Signed-off-by: you06 <you1474600@gmail.com> Co-authored-by: you06 <you1474600@gmail.com> Co-authored-by: gengliqi <gengliqiii@gmail.com> (cherry picked from commit c4d7f12) Signed-off-by: ekexium <eke@fastmail.com> --------- Signed-off-by: ekexium <eke@fastmail.com> Co-authored-by: Ti Chi Robot <ti-community-prow-bot@tidb.io> Co-authored-by: you06 <you1474600@gmail.com> Co-authored-by: gengliqi <gengliqiii@gmail.com>
resource_metering: cherry pick topsql adding network and logical io c… …hanges (tikv#19378) close tikv#18815 Add network/io info collection for TopSQL: 1. Introduce resource-metering.enable-network-io-collection config to control whether enable this new feature. Default is disabled. 2. Collect network_in, network_out, logical_read, logical_write execution info and recorded in TopSQL: i. Since the LocalStorage of TopSQL recorder is TLS, and only be accessed inside the thread with attached tag. Here, we use GLOBAL_TRACKERS to help record network_in data size. ii. For network_out, we can only directly get resp's size for Coprocessor request. Thus we need to collect this data one by one for all requests. Since we only care about requests that potentially generate large response, we bypass some "write requests" whose response only contained "commit_ts" data. iii. Use the processed_size(https://github.com/tikv/tikv/blob/1deb3a135dc41c3ca227e3d5a29712526b492a4c/components/tikv_kv/src/stats.rs#L195) as logical read size iv. Use the scheduled tasks' write_bytes() as logical write size Signed-off-by: yibin87 <huyibin@pingcap.com>
br: add content length for multi part upload (tikv#19358) (tikv#19377) * draft Signed-off-by: Jianjun Liao <jianjun.liao@outlook.com> * add unit tests Signed-off-by: Jianjun Liao <jianjun.liao@outlook.com> * fix typos Signed-off-by: Jianjun Liao <jianjun.liao@outlook.com> * make clippy Signed-off-by: Jianjun Liao <jianjun.liao@outlook.com> * fix unit test Signed-off-by: Jianjun Liao <jianjun.liao@outlook.com> --------- Signed-off-by: Jianjun Liao <jianjun.liao@outlook.com> Co-authored-by: Jianjun Liao <jianjun.liao@outlook.com>
br: add content length for multi part upload (tikv#19358) (tikv#19377) * draft Signed-off-by: Jianjun Liao <jianjun.liao@outlook.com> * add unit tests Signed-off-by: Jianjun Liao <jianjun.liao@outlook.com> * fix typos Signed-off-by: Jianjun Liao <jianjun.liao@outlook.com> * make clippy Signed-off-by: Jianjun Liao <jianjun.liao@outlook.com> * fix unit test Signed-off-by: Jianjun Liao <jianjun.liao@outlook.com> --------- Signed-off-by: Jianjun Liao <jianjun.liao@outlook.com> Co-authored-by: Jianjun Liao <jianjun.liao@outlook.com>
storage: fix wrong commit_ts in Get/TableScan when latest write is Op A517 … …_Lock (tikv#19374) close tikv#19312 storage: fix wrong commit_ts in Get/TableScan when latest write is Op_Lock Problem Summary: - When `load_commit_ts=true` and a key has newer committed `Op_Lock` versions on top of the visible `Op_Put`, TiKV may return the lock commit_ts instead of the put commit_ts. What changed: - Fix `PointGetter` to return the commit_ts of the actual write version loaded by `last_change` shortcut, instead of the current cursor version. - Fix backward scanner to only update `loaded_commit_ts` when selecting a `Put` version, avoiding ts overwrite from `Lock/Rollback` versions. Tests: - Add `test_point_get_load_commit_ts_with_top_lock_versions` in `point_getter.rs`. - Add `test_load_commit_ts_with_top_lock_versions` in `scanner/backward.rs`. - Verify existing related tests: - `test_point_get_load_commit_ts` - `test_scan_with_load_commit_ts` Signed-off-by: Chao Wang <cclcwangchao@hotmail.com>
storage: fix wrong commit_ts in Get/TableScan when latest write is Op… …_Lock (tikv#19374) close tikv#19312 storage: fix wrong commit_ts in Get/TableScan when latest write is Op_Lock Problem Summary: - When `load_commit_ts=true` and a key has newer committed `Op_Lock` versions on top of the visible `Op_Put`, TiKV may return the lock commit_ts instead of the put commit_ts. What changed: - Fix `PointGetter` to return the commit_ts of the actual write version loaded by `last_change` shortcut, instead of the current cursor version. - Fix backward scanner to only update `loaded_commit_ts` when selecting a `Put` version, avoiding ts overwrite from `Lock/Rollback` versions. Tests: - Add `test_point_get_load_commit_ts_with_top_lock_versions` in `point_getter.rs`. - Add `test_load_commit_ts_with_top_lock_versions` in `scanner/backward.rs`. - Verify existing related tests: - `test_point_get_load_commit_ts` - `test_scan_with_load_commit_ts` Signed-off-by: Chao Wang <cclcwangchao@hotmail.com>
tici: support versioned lookup via per-key read ts (range_versions) (t… …ikv#19302) close tikv#19357 coprocessor: support versioned lookup via per-key read ts (range_versions) This introduces a special "versioned lookup" read path for coprocessor DAG requests: - When `coppb::Request.range_versions` is empty, behavior is unchanged. - When `range_versions` is provided: - `ranges.len()` must equal `range_versions.len()` - all `ranges` must be point ranges - only DAG + TableScan is allowed - each `ranges[i]` is read at `range_versions[i]` (per-key read ts), while bypassing txn-related checks: - endpoint memory lock checking / max_ts update - lock-cf checking in PointGetter - RcCheckTs newer-ts conflicts Implementation overview: - Plumb `range_versions` through Endpoint -> DagHandlerBuilder -> TableScan -> RangesScanner. - Add Store/SnapshotStore `get_entry_at_ts` backed by PointGetter with `bypass_txn_check`. - Disable coprocessor cache for versioned lookup to avoid unintended cache hits. Tests: - Add unit tests for validation/guard rails and bypass behaviors. - Add executor tests to verify per-range ts usage (including backward scan alignment). - Extend coprocessor integration tests for reading older versions. Dependencies: - Requires kvproto change adding `coppb::Request.range_versions` (kvproto PR: pingcap/kvproto#1356). The temporary `[patch]` in `Cargo.toml` can be removed after kvproto is merged. Signed-off-by: wshwsh12 <793703860@qq.com>
PreviousNext