diff --git a/docs/feature_flags.md b/docs/feature_flags.md
index 0de3d2bf7..2a5a9263f 100644
--- a/docs/feature_flags.md
+++ b/docs/feature_flags.md
@@ -131,7 +131,7 @@ Note that during this delay, the Head continues its usual operations, which incl
 
 Despite the delay in compaction, the blocks produced are time-aligned in the same manner as they would be if the delay was not in place.
 
-## Delay __name__ label removal for PromQL engine
+## Delay `__name__` label removal for PromQL engine
 
 `--enable-feature=promql-delayed-name-removal`
 
@@ -139,6 +139,18 @@ When enabled, Prometheus will change the way in which the `__name__` label is re
 
 This allows optionally preserving the `__name__` label via the `label_replace` and `label_join` functions, and helps prevent the "vector cannot contain metrics with the same labelset" error, which can happen when applying a regex-matcher to the `__name__` label.
 
+Note that evaluating parts of the query separately will still trigger the
+labelset collision. This commonly happens when analyzing intermediate results
+of a query manually or with a tool like PromLens.
+
+If a query refers to the already removed `__name__` label, its behavior may
+change while this feature flag is set. (Example: `sum by (__name__)
+(rate({foo="bar"}[5m]))`, see [details on
+GitHub](https://github.com/prometheus/prometheus/issues/11397#issuecomment-1451998792).)
+These queries are rare to occur and easy to fix. (In the above example,
+removing `by (__name__)` doesn't change anything without the feature flag and
+fixes the possible problem with the feature flag.)
+
 ## Auto Reload Config
 
 `--enable-feature=auto-reload-config`