Skip to content

fix: properly include peering links into segments #4890

Open
katyatitkova wants to merge 24 commits into
masterfrom
peering_test
Open

fix: properly include peering links into segments #4890
katyatitkova wants to merge 24 commits into
masterfrom
peering_test

Conversation

@katyatitkova
Copy link
Copy Markdown
Contributor

The path combination algorithm was written when peering links with a core AS on either end were not allowed. Now they are allowed, and sometimes the valid paths that include such peering links are not being found. This PR fixes it by introducing one-hop segments: single AS entries (ConsIngress=0, ConsEgress=0) with peer entries for each peering link. They are stored as both Up and Down types. src == dst is used to request such one-hop segments. One-hop segments can follow an up segment.

Big thanks to @oncilla, who helped me to debug the issue and proposed an idea of one-hop segments together with @shitz.

There were a lot of corner cases, that's why peering_test.go is so big. I tried to cover every possible combination of types of ASes (core to core, core to non-core and so on). All the tests are using the graph which picture is included into this PR. Please tell me if I missed any combination either in the tests or in the graph.

Fixes #4828

Copy link
Copy Markdown
Contributor

@tzaeschke tzaeschke left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is quite a big PR. I thinnk my main concern is that we appear to break some central SCION assumptions:

  • we have at most one of each of UP, CORE, DOWN (e.g. UP is never followed by UP)
  • Between CORE ASes we have only CORE segments, no UP or DOWN.

Also, in some places we seem to have UP and DOWN for core ASes, in other only UP.

This code is new to me, and I haven't implemented peering myself yet, so I assume I misunderstand many things, maybe they could be clarified?


// validNextSeg returns whether nextSeg is a valid next segment in a path from the given currSeg.
// A path can only contain at most 1 up, 1 core, and 1 down segment.
// Exception: one-hop segments (for core peering) can follow up segments.
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this really an exception? Isn't the one-hop segment also a core segment?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After reading the rest of the PR, this seems to be correct, one-hop segments in the core don't appear to be core segments....


for _, b := range beacons[beacon.DefaultGroup] {
if w.Intfs.Get(b.InIfID) == nil {
// TODO: less hacky?
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe clarify what that TODO means?

Comment thread control/tasks.go
// registered remotely), which is what we need for one-hop segments.
writers := []*periodic.Runner{t.segmentWriter(beacon.RegPolicyTypeCore)}
if hasPeeringInterfaces(t.AllInterfaces) {
writers = append(writers, t.segmentWriter(beacon.RegPolicyTypeUp))
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This feels like a hack to me. The user of this function requests a CORE segment and instead gets an UP segment?
This relates to the comment in graph.go, where a UP segment can now be followed by another UP segment. Is there no other way of doing this?

Comment thread control/segreq/fetcher.go
}
return up, nil
case seg.TypeUp:
// Up segment requests are normally resolved locally. The exception is
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As mentioned elsewhere, I find it a bit confusing that we have UP segments here. As I understand, they are only used when peering between cores?
Why are there UP segments between cores? Is there no other way?

toRegister = append(toRegister, &seg.Meta{Type: r.Type, Segment: b.Segment})
logBeacons[b.Segment.GetLoggingID()] = b

// For one-hop segments (single AS entry with 0/0 hop field), store as both
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If I understand correctly, here it says that we have UP and DOWN segments for one-hop core segments. In other places we have to have only UP segments. Why do we have sometimes only UP and sometimes UP and DOWN?

Comment thread control/beacon/store.go

// SegmentsToRegister returns a GroupedBeacons to register at the time of the call.
// The selection is based on the configured policy for the requested segment type.
// For TypeDown or TypeUp, returns an empty map (core ASes don't have up/down beacons,
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, in other places it says that we now do have UP and DOWN (on-hop-)segments for core ASes....?

switch currSeg.Type {
case proto.PathSegType_up:
// Allow transitioning to one-hop up segments for core peering
if nextSeg.Type == proto.PathSegType_up && isOneHopSegment(nextSeg) {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there no other way than allowing UP after UP segments? This breaks a very fundamental assumption about SCION, i.e. having maximum one of each UP/CORE/DOWN.
I would be a bit concerned that many other places, including communication with endhosts or other control server implementations, may be affected by this. This assumption is probably also manifested in many places in scionproto, it feels to me like it should be documented prominently for future developers that this assumption no longer holds....

Maybe I misunderstand something fundamentally here?.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I realized that I developed a tunnel vision with this approach, so I ended up asking AI if it's possible to find another way, without UP after UP segments. It produced a solution that seems rather elegant. In it, one-hop segments are stored only as Down segments, so they are destination-side only. The source-side peering edges are instead extracted directly from the last ASEntry's PeerEntries in core segments. I made a separate pull request to make comparison in the Github UI somewhat simpler: #4905 What do you think?

Also, while writing that code, AI was able to find a minor bug in the tests in this PR, which I pushed here too.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the effort, that looks better to me.

Unfortunately, I still don't quite understand how core peering works. Is there a document that describes that?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, there's no document. The questions you asked here are mostly about this particular implementation, not the one in #4905. Do you have any concrete questions regarding #4905, or is it about the algorithm overall?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unfortunately, I still don't quite understand how core peering works. Is there a document that describes that?

To be clear, this is not something I necessarily expect from this PR, it is just a general question.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you have any concrete questions regarding #4905, or is it about the algorithm overall?

Sorry, I didn't see your answer. The question is concerning the algorithm overall. Maybe I should raise a issue.

That said, I seem to remember you discussed this with Anapaya? Wasn't there a document involved?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@katyatitkova
Copy link
Copy Markdown
Contributor Author

@oncilla What do you think about this PR vs #4905? There, one-hop segments are stored only as Down segments to address Til's concerns regarding Up + Up (One-hop) being a valid combination

@oncilla
Copy link
Copy Markdown
Contributor

oncilla commented Mar 27, 2026

@oncilla What do you think about this PR vs #4905? There, one-hop segments are stored only as Down segments to address Til's concerns regarding Up + Up (One-hop) being a valid combination

To be honest I do not really share the concern. An up and down segment are the same thing. There is no difference between them other than we call them "up" or "down" depending on the context that they are being used in.

FWIW, SCION would have worked just as well with "core" and "non-core" segments. Up just means "non-core" segment that is traversed against construction direction, Down just means "non-core" segment that is traversed in construction direction.

This PR does not change anything in that regard, it also does not change the SCION invariants regarding segment traversal.

Now, I don't have the whole code in mind anymore, and I need to read through it again. There might be an alternative way such that we do not confuse "up" and "down" segments in terminology. But I think the approach itself is sound.

Attaching peering links to core segments is strictly worse to me. Here we change the SCION invariants because we now are allowed to use peering links between core and non core segments. This will require additional dataplane checks that we do not traverse the core segment before going into a peering link. An abuse scenario which we do not even need to think about with the current approach. (I have not give an in-depth read on the other PR, so take that statement with a grain of salt.)

@tzaeschke
Copy link
Copy Markdown
Contributor

A bit more background why I was concerned about up+up. I agree, they are all just segments and differ only in the construction direction. However:

  • SCION specifically specifies that UP must be followed by CORE or DOWN (if any), e.g. (IETF dataplane draft 15, section 1.1):

    Forwarding Path: A complete end-to-end path between two SCION endpoints which is used to transmit packets in the data plane. Endpoints can create paths with a combination of up to three path segments (an up segment, a core segment, and a down segment).

    So why do we have to call the 2nd segment "up"?

  • Having the concept of up+up in the code seems to make it less obvious that we always end up with three segments or less. Is it obvious that we may not end up with up+up+core+down? I think it makes the code less clear.

  • If I don´t really understand core peering yet, but why are we not calling it up+core+down? We do we call it "up" if it is essentially replacing a core segment? I may misunderstand how this works though.

Attaching peering links to core segments is strictly worse to me. Here we change the SCION invariants because we now are allowed to use peering links between core and non core segments. This will require additional dataplane checks that we do not traverse the core segment before going into a peering link. [...]

@oncilla Now I am confused, as far as I understand that invariant is already broken because it is already allowed to use peering links between core and non core segments (the IETF spec now specifically allows peering between any nodes, including cores). This PR only fixes a problem with the code that breaks this invariant...? Sorry if I misunderstand what you are saying.

Also, could you briefly explain what the problem is if we traverse a core link before going into a peering link? If that is a problem, it would eliminate more options of what is possible (e.g. there could not be any peering between any of (B),(C),(D) or (E) with each other or with (Y1) or (Y2) in example 5 here, because they inevitably already have UP (X2->A) and CORE (A->B).

I realize I am hijacking this thread for a general discussion, so please let me know if you think it's easier if I wait for the document to be available. I am happy to wait :)

@oncilla
Copy link
Copy Markdown
Contributor

oncilla commented Mar 27, 2026

A bit more background why I was concerned about up+up. I agree, they are all just segments and differ only in the construction direction. However:

  • SCION specifically specifies that UP must be followed by CORE or DOWN (if any), e.g. (IETF dataplane draft 15, section 1.1):

    Forwarding Path: A complete end-to-end path between two SCION endpoints which is used to transmit packets in the data plane. Endpoints can create paths with a combination of up to three path segments (an up segment, a core segment, and a down segment).

    So why do we have to call the 2nd segment "up"?

I agree with you. The second segment should be referred to as "down". I will need to read through the code and see how we can amend this.

  • Having the concept of up+up in the code seems to make it less obvious that we always end up with three segments or less. Is it obvious that we may not end up with up+up+core+down? I think it makes the code less clear.

We will not end up with 4 segments no. In fact if we traverse a peering link we will always only end up with exactly 2 segments.

  • If I don´t really understand core peering yet, but why are we not calling it up+core+down? We do we call it "up" if it is essentially replacing a core segment? I may misunderstand how this works though.

I think this is what leads to the confusion. The fact that a core AS is involved in the peering link should not change what type of segments are traversed. Even in that case, we traverse a "up" and "down" segment. (With at least one of them being a single-hop segment)

Attaching peering links to core segments is strictly worse to me. Here we change the SCION invariants because we now are allowed to use peering links between core and non core segments. This will require additional dataplane checks that we do not traverse the core segment before going into a peering link. [...]

@oncilla Now I am confused, as far as I understand that invariant is already broken because it is already allowed to use peering links between core and non core segments (the IETF spec now specifically allows peering between any nodes, including cores). This PR only fixes a problem with the code that breaks this invariant...? Sorry if I misunderstand what you are saying.

It is important to distinguish between core ASes and core segments.
Peering links are allowed to involve core ASes. They are not allowed to involve core segments.

Also, could you briefly explain what the problem is if we traverse a core link before going into a peering link? If that is a problem, it would eliminate more options of what is possible (e.g. there could not be any peering between any of (B),(C),(D) or (E) with each other or with (Y1) or (Y2) in example 5 here, because they inevitably already have UP (X2->A) and CORE (A->B).

I realize I am hijacking this thread for a general discussion, so please let me know if you think it's easier if I wait for the document to be available. I am happy to wait :)

The goal of peering links is to offer connectivity between the two "cones" of downstreams that are spawned by the two ASes that peer together. It is an explicit non-goal to offer the cone transit to the SCION internet. Stitching a core segment with a down segment, or up segment with a core segment would however do exactly that.

@tzaeschke
Copy link
Copy Markdown
Contributor

tzaeschke commented Mar 30, 2026

@oncilla Thanks, that is really helpful, it clarifies some thing for me.
Also pinging @nicorusti for draft relevance.

For context, SCION IETF CP draft states (Section 1.3):
"Peering links exist between ASes with a peering relationship (settlement-free or paid).
They can be established between any two ASes (core or non-core) and can cross ISD boundaries."

This appears to be incorrect.
Except for edge cases, peering can only go from an AS on an UP segment to an AS on a DOWN segment, ASes on a CORE segment cannot be used in peering.

So as I understand it there are now two scenarios, the (simple) main scenario and the more complex edge case with up+core / core+down / core.

Main scenario:

Peering links go from any AS on an UP segment to any AS on a DOWN segment (these ASes may be CORE ASes).

Edge cases:

There are up to three scenarios outside the main scenario:

  • up + core: peering from anywhere on the UP segment to the last AS of the CORE segment
  • core + down: peering from first AS of the CORE segment to any AS on the DOWN segment
  • core: peering from the first AS to the last AS of the CORE segment

So my open questions are:

  1. Do these edge cases exist? (the slack discussion suggests they do)
  2. Are peering links advertised in the CORE segment? If they are advertised in the metadata of the CORE segment, anyone can use them, which defeats their purpose. However, if they are not advertised in the CORE segment, how can an endpoint learn about them?
  3. If an endpoint wants to use the peering link, the CORE segment is turned into an UP or DOWN segment. The proposed solution appears to be to request a 1-AS segment from the paths service.
    However, I tried it and this doesn't appear to be implemented (and also contradicts the IETF draft which requires minimum two ASes per segment). How does that currently work?
  4. Just to be sure, do these edge cases work when peering crosses an ISD border?

EDIT:

  1. Is the solution to 2. that we see a peering link in the UP and DOWN and then request a 1-AS DOWN or UP segment? That way the CORE segment doesn't need to have peering information... ?

@nicorusti
Copy link
Copy Markdown
Contributor

Peering links are allowed to involve core ASes. They are not allowed to involve core segments.

Should we update the IETF drafts to clarify this? I was about to send the draft for publication later today.. I could try to package it as part of the final review clarifications. If we do, we need to be really sure about it, as we are really at the last step

@katyatitkova
Copy link
Copy Markdown
Contributor Author

ASes on a CORE segment cannot be used in peering

I might misunderstand what a CORE segment here means, but here're my 2 cents:

They can in certain situations. For example, see the test in this PR: both 120 and 410 are core ASes, and the peering link exists. Also, see the test for 410 and 123: we can use the peering link between 410 and 120 to further go down to 123 from 120.

If these two examples are forbidden, then there's no sense in peering links in a way they already exist in the world. Peering links exist for the commercial contracts between two ASes in my understanding. Direct traffic should always flow between them. We sometimes restrict if it's the traffic that involves any other ASes.

@tzaeschke
Copy link
Copy Markdown
Contributor

tzaeschke commented Mar 30, 2026

@nicorusti This also affects DP:
SCION DP 1.1 "Peering Link: A link between two SCION border routers of different ASes that can be used as a shortcut.
Peering link information is added to segment information during the beaconing process and used to shorten
paths while assembling them from segments. It is possible to construct a path out of only two partial segments
whose top-most hops are joined by a peering link. Two peering ASes may be in different ISDs and may exist
between any ASes, including core ASes."

(also, the last sentence seems garbled)

The following is clearer, but contradicts what was discussed in the slack discussion which seem to state that up+core and core+down can also have peering links.
SCION DP 1.4 states: "If there are two path segments (one up and one down segment) that both announce the same peering link, then a shortcut via this peering link is possible."


Peering links are allowed to involve core ASes. They are not allowed to involve core segments.

This is correct, however, there is the "loophole" (the edge cases I mention above) where a up+core/core+down/core is very much allowed as long as the core segment is replaced by a 1-AS up or down segment.
This really complicates the whole thing and I think it conflicts with another part of the spec that says that a segment mast have a t least two ASes. (I can´; t find the statement now, I will keep looking).

Should we update the IETF drafts to clarify this? I was about to send the draft for publication later today.. I could try to package it as part of the final review clarifications. If we do, we need to be really sure about it, as we are really at the last step

As my questions above indicate, the point of allowing up+core and core+down is unclear to me and I am not sure how it is supposed to work.


How about rephrasing the spec to something like:

CP draft (Section 1.3): "Peering links exist between ASes with a peering relationship (settlement-free or paid).
They can be established between any AS from an UP segment to any AS on a DOWN segment, and can cross ISD boundaries. Peering ASes can be core or non-core."

DP Section 1.1:
"Peering Link: A link between two SCION border routers of different ASes that can be used as a shortcut.
Peering link information is added to segment information during the intra-AS beaconing process and used to shorten
paths while assembling them from segments. It is possible to construct a path out of only two partial segments
whose top-most hops are joined by a peering link. Peering can be established between any AS from an UP segment to any AS on a DOWN segment, and can cross ISD boundaries. Peering ASes can be core or non-core."

@tzaeschke
Copy link
Copy Markdown
Contributor

Allowing core+down and up+core (and core only?) can be done in the next iteration, but currently I think these would require more explanation than just rearranging a sentence.

@tzaeschke
Copy link
Copy Markdown
Contributor

tzaeschke commented Mar 30, 2026

@katyatitkova Thanks, could you paste a diagram of the topology?
Also, how do these tests work, do they directly test the combinator or the whole system? What I mean is, does it test how the client learns that peering is actually possible? And how is the 1-AS segment requested?

This basically is question 2. and 3. here: https://app.slack.com/client/T89RH7NCQ/C8ADA9CEP
EDIT: #4890 (comment)

@katyatitkova
Copy link
Copy Markdown
Contributor Author

@tzaeschke The diagram is here: https://github.com/scionproto/scion/pull/4890/changes#diff-2761d4ea424d2914a49640a110ec5574629756311f937b5d24306ebcd71097ba. The tests are for combinator only, but it's possible to add an integration test too if needed (it will be rather trivial, but they'll require longer times to set up and run compared to unit tests). Since it's unit tests only, we provide the segments to the test, with a helper for one-hop segments

@tzaeschke
Copy link
Copy Markdown
Contributor

tzaeschke commented Mar 30, 2026

@katyatitkova Thanks. I meant more like a graphical diagram, but it's not important.
Also, I don't really need more tests, I am more interested in how they work, I wanted to answer the questions that I mentioned above (sorry for posting a wrong link earlier): #4890 (comment)

@katyatitkova
Copy link
Copy Markdown
Contributor Author

@tzaeschke In my opinion, the peering link itself is allowed to exist between any two ASes. The reason why it exists is more about their internal agreements and contracts, and it doesn't make sense to restrict creation of the peering links. The usage might be restricted, but I think there are already enough mechanisms in the draft that explain those restrictions. The paths must be valley-free, which already disallows to use a certain peering link in a certain scenario (but it's still valid to use it in another scenario). Having multiple peering links in a path is already disallowed, or connecting two segments of the same type with a peering link. So, in my opinion, there's no need to modify the drafts. What do you think?

@oncilla
Copy link
Copy Markdown
Contributor

oncilla commented Mar 30, 2026

@oncilla Thanks, that is really helpful, it clarifies some thing for me. Also pinging @nicorusti for draft relevance.

For context, SCION IETF CP draft states (Section 1.3): "Peering links exist between ASes with a peering relationship (settlement-free or paid). They can be established between any two ASes (core or non-core) and can cross ISD boundaries."

This appears to be incorrect. Except for edge cases, peering can only go from an AS on an UP segment to an AS on a DOWN segment, ASes on a CORE segment cannot be used in peering.

I do not follow the conclusion. The draft is talking about peering relationships and AS types. It does not talk about segment types.

So as I understand it there are now two scenarios, the (simple) main scenario and the more complex edge case with up+core / core+down / core.

Main scenario:

Peering links go from any AS on an UP segment to any AS on a DOWN segment (these ASes may be CORE ASes).

Edge cases:

There are up to three scenarios outside the main scenario:

  • up + core: peering from anywhere on the UP segment to the last AS of the CORE segment
  • core + down: peering from first AS of the CORE segment to any AS on the DOWN segment
  • core: peering from the first AS to the last AS of the CORE segment

So my open questions are:

  1. Do these edge cases exist? (the slack discussion suggests they do)

No they do not. And this is exactly why we introduce these one-hop segments.
Peer links between cores are advertised via non-core segments only.

  1. Are peering links advertised in the CORE segment? If they are advertised in the metadata of the CORE segment, anyone can use them, which defeats their purpose. However, if they are not advertised in the CORE segment, how can an endpoint learn about them?

No, peering links should not be advertised in core segments.

  1. If an endpoint wants to use the peering link, the CORE segment is turned into an UP or DOWN segment. The proposed solution appears to be to request a 1-AS segment from the paths service.
    However, I tried it and this doesn't appear to be implemented (and also contradicts the IETF draft which requires minimum two ASes per segment). How does that currently work?

How did you try it? With this PR?

  1. Just to be sure, do these edge cases work when peering crosses an ISD border?

These edge cases do not exist. The main scenario allows to cross ISD borders.

EDIT:

  1. Is the solution to 2. that we see a peering link in the UP and DOWN and then request a 1-AS DOWN or UP segment? That way the CORE segment doesn't need to have peering information... ?

@tzaeschke
Copy link
Copy Markdown
Contributor

@katyatitkova Thanks you.

I don't think peering between any two ASes is possible, counterexamples:

  • (you already mention this): Peering between two ASes on the same segment is impossible
  • Peering between any core-segment-AS and any other AS is impossible (except, sometimes but not always, the first and last, see next point).
  • Depending on how you want to interpret it, the first and last AS of the CORE segment may be involved in peering, but this is only a meaning full exception in cases where either the UP or DOWN is missing, but not both. (At least that is my current understanding). Moreover, how this works is that the CORE is turned into an 1-AS UP or DOWN segment, so in the end we are again peering between an UP and DOWN segment.

This third option gives me headaches.
From the perspective of the IETF draft I have to say we should just write that only UP/DOWN are allowed. Otherwise we need to add a lot of text explaining that in some narrow scenario a CORE segment may be involved but in the end with still just stitch UP and DOWN together. At this stage, adding such an explanation is probably not possible. Also, for those who know, they can still do the CORE-based stitching and say they are compliant because in the end they are actually only stitching UP/DOWN. We also need to document that additional segment requests may be required.
Also we don't even have a documented design yet how this is supposed to work.

From the design perspective, I see several problems:

  • I still don't understand how the whole thing is useful. There seems to be an argument to prevent transit traffic, but as far as I understand, the whole thing is only useful if the origin AS or the destination AS is one of the peering ASes. At that point, by definition, we do not have transit....?
  • It seems weird that we have to do additional segment requests. And we have to introduce 1-AS segments. And we are requesting a CORE segment and then turning it into an UP or DOWN segment.
  • As I understand, a pure "core" segment (without UP/DOWN) cannot be turned into an UP or DOWN segment. This seems also weird (even if I see the technical problem).

For reference, I created two PRs:

@tzaeschke
Copy link
Copy Markdown
Contributor

@oncilla Thanks again.

For context, SCION IETF CP draft states (Section 1.3): "Peering links exist between ASes with a peering relationship (settlement-free or paid). They can be established between any two ASes (core or non-core) and can cross ISD boundaries."

This appears to be incorrect. Except for edge cases, peering can only go from an AS on an UP segment to an AS on a DOWN segment, ASes on a CORE segment cannot be used in peering.
I do not follow the conclusion. The draft is talking about peering relationships and AS types. It does not talk about segment types.

Exactly. It just says "any two ASes", which includes ASes on CORE segment, which, as I understand, is not allowed?

It is more obvious here where it talks about segment but does that it cannot be CORE segments (SCION DP 1.1):

"Peering Link: A link between two SCION border routers of different ASes that can be used as a shortcut.
Peering link information is added to segment information during the beaconing process and used to shorten
paths while assembling them from segments. It is possible to construct a path out of only two partial segments
whose top-most hops are joined by a peering link. Two peering ASes may be in different ISDs and may exist
between any ASes, including core ASes."

  1. Do these edge cases exist? (the slack discussion suggests they do)

No they do not. And this is exactly why we introduce these one-hop segments. Peer links between cores are advertised via non-core segments only.

Okay, to clarify my question: they do exist as input (it is what a an endhost receives from a path service) but not as output (the core is replaced with up or down), right?

In case you mean that they don't even exist as input, could you give an example where we need the 1-AS segment?

  1. If an endpoint wants to use the peering link, the CORE segment is turned into an UP or DOWN segment. The proposed solution appears to be to request a 1-AS segment from the paths service.
    However, I tried it and this doesn't appear to be implemented (and also contradicts the IETF draft which requires minimum two ASes per segment). How does that currently work?

How did you try it? With this PR?

Yes I did try, but not with this PR. I assumed it would be working in the production network.
I haven't fully understood the PR, and the description talks about fixing the path combinator. I hadn't realized/checked that there are changes to the path service? Also, I think the AS I tried uses an Anapaya AS, so this should already work, or not?
However, thinking about this, I assume my problem is that I asked only one core AS, and I don't know whether this has any peering links, so it can be expected that it doesn't return an 1-AS segment?

  1. Just to be sure, do these edge cases work when peering crosses an ISD border?

These edge cases do not exist. The main scenario allows to cross ISD borders.

EDIT:

  1. Is the solution to 2. that we see a peering link in the UP and DOWN and then request a 1-AS DOWN or UP segment? That way the CORE segment doesn't need to have peering information... ?

Thanks again for all the input @oncilla and @katyatitkova

@oncilla
Copy link
Copy Markdown
Contributor

oncilla commented Mar 30, 2026

@oncilla Thanks again.

For context, SCION IETF CP draft states (Section 1.3): "Peering links exist between ASes with a peering relationship (settlement-free or paid). They can be established between any two ASes (core or non-core) and can cross ISD boundaries."
This appears to be incorrect. Except for edge cases, peering can only go from an AS on an UP segment to an AS on a DOWN segment, ASes on a CORE segment cannot be used in peering.
I do not follow the conclusion. The draft is talking about peering relationships and AS types. It does not talk about segment types.

Exactly. It just says "any two ASes", which includes ASes on CORE segment, which, as I understand, is not allowed?

I might misunderstand your point, but core ASes can be on core segments and on non-core segments.
Even if an AS is on a core segment, they are allowed to have peering links (which they do not advertise in the core segments).

It is more obvious here where it talks about segment but does that it cannot be CORE segments (SCION DP 1.1):

"Peering Link: A link between two SCION border routers of different ASes that can be used as a shortcut.
Peering link information is added to segment information during the beaconing process and used to shorten
paths while assembling them from segments. It is possible to construct a path out of only two partial segments
whose top-most hops are joined by a peering link. Two peering ASes may be in different ISDs and may exist
between any ASes, including core ASes."

I don't read that sentence as contradicting to anything I said above. Could you point me to exactly what you mean.

  1. Do these edge cases exist? (the slack discussion suggests they do)

No they do not. And this is exactly why we introduce these one-hop segments. Peer links between cores are advertised via non-core segments only.

Okay, to clarify my question: they do exist as input (it is what a an endhost receives from a path service) but not as output (the core is replaced with up or down), right?

They do neither exist in input, nor in output. All the peering information comes from non-core segments.

In case you mean that they don't even exist as input, could you give an example where we need the 1-AS segment?

As mentioned above. They are used if a core AS (not core segment) is involved in the peering link.
Somehow we need to discover the core side of the link. This is done with the 1-AS segment.

Example 1:

C1 -peer- C2
          |
          N1

Involved segments:

  • up (1-AS)
  • down (C2 -> N1)

Example 2:

C1 -peer- C2
|
N1

Involved segments:

  • up (N1 -> C1)
  • down (1-AS)
  1. If an endpoint wants to use the peering link, the CORE segment is turned into an UP or DOWN segment. The proposed solution appears to be to request a 1-AS segment from the paths service.
    However, I tried it and this doesn't appear to be implemented (and also contradicts the IETF draft which requires minimum two ASes per segment). How does that currently work?

How did you try it? With this PR?

Yes I did try, but not with this PR. I assumed it would be working in the production network. I haven't fully understood the PR, and the description talks about fixing the path combinator. I hadn't realized/checked that there are changes to the path service? Also, I think the AS I tried uses an Anapaya AS, so this should already work, or not? However, thinking about this, I assume my problem is that I asked only one core AS, and I don't know whether this has any peering links, so it can be expected that it doesn't return an 1-AS segment?

Peering links are not implemented on the Anapaya appliance, and support in the open-source is also not finished.
Also, the 1-AS segment is being introduced in this PR.

Checking against production will not show you anything.

@tzaeschke
Copy link
Copy Markdown
Contributor

tzaeschke commented Mar 30, 2026

In case you mean that they don't even exist as input, could you give an example where we need the 1-AS segment?

As mentioned above. They are used if a core AS (not core segment) is involved in the peering link. Somehow we need to discover the core side of the link. This is done with the 1-AS segment.

Example 1:

C1 -peer- C2
          |
          N1

Involved segments:

* up (1-AS)

* down (C2 -> N1)

Example 2:

C1 -peer- C2
|
N1

Involved segments:

* up (N1 -> C1)

* down (1-AS)

I still don't quite understand this, so maybe we need to go through this step by step. E.g. for example 1:

  1. An endhost E1 in AS C1 decides it wants to connect to N1.
  2. E1 asks the local path server PS-C1 for a route.

I assumed the following:
3) PS-C1 returns a core-segment C1->C2 and a down segment C2->N1
4) The endhost somehow figures out that there is a peering link C1->C2, requests a 1-AS link for C1 from PS-C1 and uses this to create an UP link and DOWN link: UP= C1--peer->C2 DOWN=C2->N1

If I understand you correctly, this is wrong because there is a core segment used in 3) and 4) and this is wrong.
So how does this work?


It is more obvious here where it talks about segment but does that it cannot be CORE segments (SCION DP 1.1):

"Peering Link: A link between two SCION border routers of different ASes that can be used as a shortcut.
Peering link information is added to segment information during the beaconing process and used to shorten
paths while assembling them from segments. It is possible to construct a path out of only two partial segments
whose top-most hops are joined by a peering link. Two peering ASes may be in different ISDs and may exist
between any ASes, including core ASes."

I don't read that sentence as contradicting to anything I said above. Could you point me to exactly what you mean.

It talks about two partial segments that can be combined, but it does not specify that they cannot be core segments, instead they must be one up and one down segment (possibly 1-AS segments).
If I understand you correctly, core links cannot be used in peering?

@oncilla
Copy link
Copy Markdown
Contributor

oncilla commented Mar 30, 2026

I still don't quite understand this, so maybe we need to go through this step by step. E.g. for example 1:

  1. An endhost E1 in AS C1 decides it wants to connect to N1.
  2. E1 asks the local path server PS-C1 for a route.

I assumed the following: 3) PS-C1 returns a core-segment C1->C2 and a down segment C2->N1 4) The endhost somehow figures out that there is a peering link C1->C2, requests a 1-AS link for C1 from PS-C1 and uses this to create an UP link and DOWN link: UP= C1--peer->C2 DOWN=C2->N1

If I understand you correctly, this is wrong because there is a core segment used in 3) and 4) and this is wrong. So how does this work?

E1 notices that it is in a core AS (through the TRC), hence it sends the following requests:

  • up segment request from C1 to C1 (finding the 1-AS segment) <--- new addition in this PR
  • core segment requests from C1 to any core AS in ISD of N1 (which will be irrelevant, but required for the down segment lookup)
  • down segment requests from any core we have a core segmetn to in ISD of N1 to N1

E1 then combines the 1-AS up segment it found, with the C2 -> N1 down segment it found. Not utilizing the core segment.

Note: Here, core segments are used during the segment lookup (exactly the same as they were before). But they are not used during path construction (neither input nor output) when finding a path that traverse the peer link.

It is more obvious here where it talks about segment but does that it cannot be CORE segments (SCION DP 1.1):

"Peering Link: A link between two SCION border routers of different ASes that can be used as a shortcut.
Peering link information is added to segment information during the beaconing process and used to shorten
paths while assembling them from segments. It is possible to construct a path out of only two partial segments
whose top-most hops are joined by a peering link. Two peering ASes may be in different ISDs and may exist
between any ASes, including core ASes."

I don't read that sentence as contradicting to anything I said above. Could you point me to exactly what you mean.

It talks about two partial segments that can be combined, but it does not specify that they cannot be core segments, instead they must be one up and one down segment (possibly 1-AS segments). If I understand you correctly, core links cannot be used in peering?

Your sentence is contradicting:

"two partial segments that can be combined, but it does not specify that they cannot be core segments"
and
"they must be one up and one down segment (possibly 1-AS segments)" cannot both be true.

A core segment is neither an up nor a down segment. Hence it is not allowed to be used to peer.

If I understand you correctly, core links cannot be used in peering?

Only peering links can be used for peering. Core links cannot be used as peering links

@tzaeschke
Copy link
Copy Markdown
Contributor

E1 notices that it is in a core AS (through the TRC), hence it sends the following requests:

* up segment request from C1 to C1 (finding the 1-AS segment) <--- new addition in this PR

* core segment requests from C1 to any core AS in ISD of N1 (which will be irrelevant, but required for the down segment lookup)

* down segment requests from any core we have a core segmetn to in ISD of N1 to N1

E1 then combines the 1-AS up segment it found, with the C2 -> N1 down segment it found. Not utilizing the core segment.

Note: Here, core segments are used during the segment lookup (exactly the same as they were before). But they are not used during path construction (neither input nor output) when finding a path that traverse the peer link.

Thanks, that makes sense. I guess what confused me was the slack thread which says "I guess you'd need to actually request a down-segment to a down-stream AS".

It is more obvious here where it talks about segment but does that it cannot be CORE segments (SCION DP 1.1):

"Peering Link: A link between two SCION border routers of different ASes that can be used as a shortcut.
Peering link information is added to segment information during the beaconing process and used to shorten
paths while assembling them from segments. It is possible to construct a path out of only two partial segments
whose top-most hops are joined by a peering link. Two peering ASes may be in different ISDs and may exist
between any ASes, including core ASes."

I don't read that sentence as contradicting to anything I said above. Could you point me to exactly what you mean.

It talks about two partial segments that can be combined, but it does not specify that they cannot be core segments, instead they must be one up and one down segment (possibly 1-AS segments). If I understand you correctly, core links cannot be used in peering?

Your sentence is contradicting:

"two partial segments that can be combined, but it does not specify that they cannot be core segments" and "they must be one up and one down segment (possibly 1-AS segments)" cannot both be true.

Exactly, that is why I put the word "instead" between the two statements.

To be more clear: the spec simply specifies that segment can be combined, but fails to say that only up and down segments can be combined. As in implementer this would be very confusing because if I read that I would try to implement it for all types of segments, including core segments.

A core segment is neither an up nor a down segment. Hence it is not allowed to be used to peer.

Yes.

If I understand you correctly, core links cannot be used in peering?

Only peering links can be used for peering. Core links cannot be used as peering links

Apologies, I meant to write " core segments cannot be used in peering?". I think the answer is "yes".

@katyatitkova
Copy link
Copy Markdown
Contributor Author

The part regarding a peering link connecting up and down segment is in the draft already: https://datatracker.ietf.org/doc/html/draft-dekater-scion-dataplane#section-4.2.2 The quoted part of the same draft is intentionally high-level

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Peering links bugs

4 participants