You said the Graph Editor doesn’t work with Time, but it does. It also works with rate-of-change. And transform magnitude. What other parameters to an animation channel are there?
A constant and linear curve – a horizontal line – produces no change in the affected channel, because the Y-axis value is constant. It’s still a curve, just not one that produces “movement.”
If this line is given a slope and its start and end keypoint handle-types are made Vector (or the entire curve is made Linear) then there is change in the Y-axis value that produces “movement,” but because it is linear, it has no ease-in and -out. Its rate of change is constant, though the Y-axis values are not.
Switching to bezier handles allows the curve to curve. To ease-in, the rate of change in the Y-axis has to be faster at the beginning (steeper slope) than at the end of the curve. So it follows that compared to a linear sloped curve, the channel value at any particular frame of a non-liner curve has to be different, causing the “movement” you mention. It cannot be any other way. It is not any other way in any other animation package that uses animation curves. You cannot “change anything but time,” because time and “movement” are inextricably linked. Even with a linear curve, if you mess with the time dimension (the X-axis) you also change the slope of the curve (rate of change). The only curve that is not affected by time-axis shifts or slope changes is the constant linear curve where no “movement” takes place at all.
And this applies to any animation channel. Quats, Eulers, translations, scale, even RGB material parameters – they all change based on the curves in their anim channels, and those curves follow the same rules regardless of the effect they have in the Scene.
Maybe posting a specific example in a .blend would help clarify, since we seem to talking about the same thing from different angles.
EDIT: Here’s some specific numbers. For an animation curve that extends 10 frames in Time (the x-axis) and 10 units in value (the Y-axis, or “movement”), a linear curve will produce the following values at each frame (stated as frame/value):
1/0, 2/1/, 3/2, 4/3, 5/4, 6/5, 7/6, 8/7, 9/8, 10/9
The “movement” is positive one unit per frame, producing a linear slope and a constant rate of change.
Adding ease-in, the values might be these:
1/0, 2/2, 3/3.75, 4/5.25, 5/6.5, 6/7.5, 7/8.25, 8/8.75, 9/8.875, 10/9
Here the change between frames (rate of change ) is non-linear, as it needs to be for the “movement” to be faster at the start than at the end but wind up at the same value after 10 frames. But since the per-frame values differ, there is a “movement” relative to the two curves that is a result of adding the ease-in. It can’t work any other way.
The “time” you want to adjust is rate of change, and that will always force some sort of “movement” within the actual time segment (X-axis extent) being edited. Without that “movement” there is no easing in or out.