News

Saving the Best for Last: Bell’s Network Congestion Admission

As is often the case with House of Commons committee hearings, yesterday’s Industry Committee hearing on usage based billing saved the best for last. Addressing a specific question from the chair about the separation of IPTV and Internet services on Bell’s network, Bell’s Mirko Bibic responded:

There is a copper loop that goes from our Central Office to the home and all data travels on that pipe so it’s Internet traffic, it’s television traffic, it’s actually voice traffic, long distance traffic, but that’s not where there are general congestion issues. The real issue is when you get to the Central Office and you go behind that to the general Internet, FIBE TV is completely different.

Bell’s comments are noteworthy since they confirm that there is no congestion in the “last mile” – the connection between the user and the so-called Central Office. At the moment, Bell aggregates the data from both its own retail customers and independent ISPs at this stage (which it says causes the congestion necessitating traffic shaping and UBB), though the independent ISP subscriber traffic later goes to the independent ISP before heading to the Internet.  The “congestion problem” is therefore not at the last mile nor at the Internet – it is in the intermediate stage between the two.

This raises the question of why not offer the independent ISPs access to the network at the Central Office or at other earlier points in the network so that their users’ traffic never causes congestion for Bell?  If the congestion problem occurs during the brief period when wholesale and retail traffic is aggregated (Bell said the same during the Internet traffic management hearings in 2009), why not avoid it by mandating that Bell allow independent ISPs to access their subscribers’ traffic earlier?

The answer appears to be that Bell vehemently opposed just such a solution, telling the CRTC in June 2010 that the approach would mean that it would not invest in its own network. When asked about one such proposal – known as ADSL-CO – Bibic told CRTC Chair Konrad von Finckenstein that (para 7806):

with ADSL-CO, once the independent ISP gets subsidized access to the full speed and capacity over that fibre, and the ability to fully differentiate their service from ours — and there goes usage-based billing — there is nothing more for the ISP to build. If they get a customer, they pay us. If they don’t get a customer, they don’t pay us. Their cost structure is success-based, as I mentioned yesterday, with no upfront risk capital required.

In other words, Bell recognized that ADSL-CO would mean that independent ISPs would be able to better differentiate their services (including the prospect of no traffic shaping) and eliminate Bell’s ability to implement wholesale usage based billing. The entire exchange is worth reading because it plainly recognizes the consequences of allowing independent ISPs to fully compete with speed matching and ADSL-CO. Bell describes it as “damaging” since it “undermines the ability to win the broadband home.”

Of course, the whole point is to foster competition so that Bell competes for the broadband home, rather than winning it by default since their are few other viable alternatives. Yet despite Bell making it very clear that ADSL-CO would make the market more competitive with more differentiated offerings and despite the fact that companies like Primus and TekSavvy indicated that they would invest to use such a service, the CRTC denied the application to declare it an essential service.

62 Comments

  1. This has to be repeated….over and over again
    It truly is the bottom line as to why Bell wants UBB and Throttling

  2. *chuckle* Ah Bell. Yes, how dare you actually have to compete, that would be unheard of.

  3. Logistical nightmare?
    Michael, I love reading your blog but it’s pretty obvious at this point that although you are very good at what you do when it comes to regulatory law and the likes you don’t know much about networking… đŸ™‚ Of course congestion is impossible at the last mile, it’s a dedicated point-to-point circuit from the C.O. to someone’s home. Now let me explain to you why you are oversimplifying this whole thing:

    Bell giving access to the last mile implies that independent ISPs have to co-locate their equipment (DSLAM) in Bell’s offices. That might sound simple but multiply that but hundreds of different ISPs and you can imagine what it would look like. Also consider the fact that traffic has to be hauled out of there somehow. Then you have to consider that in recent deployments of fibre to the node (and even lots of old ADSL deployments) the equipment referenced above has to be installed at the node (or curb) to reduce the distance in order to achieve the higher speeds (yes that means outside in cabinets). Can you imagine all these companies wanting to put their own DSLAM aggregators outside at the curb? We’ll have choice in Internet but it will an urbanism nightmare…


  4. And the thing that has the biggest contribution to that “congestion” is probably made by Sandvine.

    Nap.

  5. This needs to be seen b y as many politicians as possible, ASAP!
    Congratulations, once again, for cutting through Bell’s snake-oil and making a valid point. This needs to be seen by as many politicians as possible – Charlie Angus and Marc Garneau, are you online?

  6. @Tony: With Bell Fibe’s service, DSLAMs are oftentimes no longer found at the COs, but rather out on the streets in the pedestals. The DSLAMs need to be as close as possible to the home in order to get the very high rate of bandwidth required for TV and DSL.

    Inside these pedestals are the actual phone pairs (the copper loop) that connect to the Lucent Stinger (aka the OSLAM, or Optical Subscriber Line Access Multiplexer). Once connected to the Stinger, traffic is then carried to another Bell facility / Data Centre / CO.

    In the old days, CLECs (competitors to the Bell/Telus ILECs) could colocate their DSLAMs or other related gear in the central offices. That game has now changed and Bell’s new competitive advantage is there simply is NO physical space where CISPs (competitive ISPs) could install their equipment in the pedestals.

    Costs aside, we also must remember that in order to have physical equipment installed within Bell facilities, a company would need to be a registered CLEC, which many CISPs are not.

  7. Referring to DSLAM collocation + unbundled loop?
    I think what Bibic was trying to refer to wasn’t ADSL-CO but instead collocation and then using the unbundled loop. That’s possible now. However, there are large upfront fees and that doesn’t give access to remotes; so the ability to give clients higher speeds is extremely geographically limited.

    ADSL-CO was supposed to solve that remote access problem, but instead was scrapped for the reasons Bibic indicated. Bell successfully argued that its FTTN deployments were high risk and that the addition of competition would make the remotes unviable.

  8. Clayton Zekelman says:

    Presdient, MNSi Internet
    And of course, the Central Office is the easiest place to eliminate congestion. Internet Service Providers already lease on a monthly basis, fixed capacity links into Bell’s aggregation network. We cannot send our customers traffic that exceeds the capacity of these links. Bell bills us for the CAPACITY to interconnect with their network, now they want to bill us for the USAGE of that capacity. Hardly seems “fair” – to put Mr. Bibic’s spin on it.

  9. UBB vs. unbundled loop access
    Is there a good reason why UBB is appropriate to address the problem of congestion at the DSLAM? I would expect that a per-user + per-DSLAM-month fee structure might be better keyed to the problem–assuming one exists.

  10. Petitio Principii
    “This begs the question …”

    No, it raises the question, begging the question is something else entirely.


  11. @Mark: “Is there a good reason why UBB is appropriate to address the problem of congestion […]?”

    Well if capacity is limited and you want to distribute some to all of your clients, you can have caps/throttling or you can have UBB.

    Looking again at electricity/water which are UBB. People don’t let the water run in the sink and the lights on when not really needed.

    Same with internet. People will stop running 24/24h torrents that bring files of little/questionable value.

    What would be unacceptable would to have both throttling/caps and UBB at same time.

    Nap.

  12. @Napalm: But UBB doesn’t bill for capacity; it bills for usage. Water and electricity are resources that are consumed by usage. The quantity consumed in a network is capacity. They should be charging for capacity.

    The added cost of operating a switch at full capacity rather than partial capacity is negligible–similar to RAM, electrical wiring, waste systems, and so on, utilization efficiency actually goes UP the closer you are to capacity.

  13. Hi Michael,

    To Tony’s point above, “last mile” congestion is not an issue and is in fact possible. What Bell seemed to say is that there is also no congestion between the BAS and DSLAM (http://i.imgur.com/14bEw.jpg).

    http://parlvu.parl.gc.ca/ParlVu/ContentEntityDetailView.aspx?ContentEntityId=7298

    At just past 2:11 Bibic states that there is no congestion on the copper loop from the central office to the home. He claimed that there was congestion only after IPTV and Internet was separated which appears to be a lie. I don’t have the technical expertise to understand this.

    Either way, the entire meeting was a crock. Bibic repeatedly referred to total data transfer as “bandwidth”, was fixated on Bell’s retail services and not the wholesale side of things (which was actually the purpose of the whole meeting), referred to blocks of data in terms of how many minutes of viewing you would receive that was factually incorrect and misleading, spoke in blanket statements regarding investments without clearly delineating what proportion went towards their IPTV services, and so on.

  14. Meant to say “impossible” (at least in my understanding). To Tony’s point above, “last mile” congestion is not an issue and is in fact possible.

  15. @Mark
    I agree, to a point.

    However, operating the switch at near capacity has other impacts that are not so much directly monetary. A drop out of a single switch as a result of switch failure or programmed replacement to accommodate more subscribers, where heavily loaded, can mean that many more current subscribers are affected (it depends on the loading from each subscriber). This can have monetary impacts in terms of SLAs and/or bad press; certainly you’ll have more people complaining about a lack of connectivity, not just of internet but also real essential services such as 911 service. Running it near capacity means that intermittent peaks of usage are more likely to swamp the capabilities as well; the old adage of the sewer system and the first commercial break during the Superbowl applies. While the system may be operating at 100% of capacity, that is a poor justification for those people whose bathrooms have just been flooded because the system couldn’t handle any more sewage. Designing to operate at or near capacity has its risks… are you willing to live with those risks?

  16. DisgruntledCdn says:

    Bell’s DSLAM congestion — Cheap Solution Proposed: 2007
    This is a good read:

    http://arstechnica.com/old/content/2008/06/bells-p2p-traffic-issues-easily-and-inexpensively-solved.ars


  17. @Mark: “But UBB doesn’t bill for capacity; it bills for usage. Water and electricity are resources that are consumed by usage. The quantity consumed in a network is capacity. They should be charging for capacity.

    The added cost of operating a switch at full capacity rather than partial capacity is negligible–similar to RAM, electrical wiring, waste systems, and so on, utilization efficiency actually goes UP the closer you are to capacity.”

    “Capacity” costs too. They have to buy bandwidth from their upstream supplier. As with any telecom business, that bandwidth is oversold by a certain factor (you start with the assumption that not all users will be connected and downloading at full speed all the time).

    When nearly everybody starts to run unattended 24/24h bittorents, you have to reduce the “overselling” factor and buy more upstream bandwidth. This costs money. Eventually you reach the situation where you’re not “overselling” anymore but then the monthly fee wouldn’t be just $39 either. So in the end it’s about usage.

    As for the second paragraph. You don’t really want to load any network/computing resource near to 0. Anon-K has already explained it.

    Nap.

  18. correction
    “near to 0” should read “near to 100%”

  19. @Anon-K: Good points, most of which I was aware of. I was intending ‘capacity’ in the ‘nominal operating capacity’ sense, incorporating some spillover room for equipment failure, ACK thrashing, peak times and such. Kind of like ‘full speed’ versus ‘flank speed’ on a battleship.

    As a corollary, I would absolutely be behind a rate structure that steered movable usage towards off-peak hours, so people could run scheduled backups at night, or throttle their torrents during the day.

  20. Systems Architec
    All the Big ISP’s want to Place Download/Upload Caps/GB not Bandwith Caps/GB.

    They want to charge me more for Data they don’t own as well
    I can’t control the rate at which the data is deliverd thus making it easy to go over the caps.. I used to all the time with videotron I had a 20 GB Bi-directional CAP at 5mbs /sec and 4 kids all doing what they do on the internet me and a wife doing what we do.

    So be it

    I need internet. thus I will return my phone, My Dish 110.00 Dollars a month and Pay 80.00 a month in stead of 38.00/per month to my Third party ISP, Use Netflix’s and get a majic Jack or some other VOIP service with unlimited long distance in Canada for home phone. I will still save money and be happier.

    Bell and the other providers must be stopped. I urge people to drop the the big companies and go to small companies strickly for princple if for no other reason

  21. @Anon-K
    Regarding SLA’s, if you want one, you generally pay more for it. The company offering it over-provisions, uses quality equipment, monitors the system carefully, implements backup generators, and the whole shi-bang. You’ll find it often in the hosting industry. You pay more for a host that offers an SLA, and with good reason. Competitors who don’t want to offer one (very few actually need an SLA) can provide the service for less.

    As far as 911 service goes, there are plenty of ways to address the potential of congestion getting in the way. If sent as a data stream, you can prioritize those packets – essentially QoS – if you’ve maxed out the connection, drop any/all packets except for the 911 stuff. Note that in the cellular industry, there are already measures in place to prioritize 911. You might not be in range of your carrier’s tower (and see “no service” or “911 only”), but if you dial 911, you’ll instantly get a signal to another carrier’s tower. I don’t know quite how far these measures go, but it’s completely possible to “boot” somebody off a tower to open a slot for a 911 call. This sort of behavior is easily done at a technical level. Again, I don’t know if they’ve gone that far in the current implementation (just that it’s gone as far as connecting to another carrier’s tower even if it wouldn’t for a normal call), but it’s something that could (and may) be done.

    The sewer system adage doesn’t really need to apply. Remember, with the internet, you can (metaphorically speaking) essentially slow down all the bathtub or sink drains to keep basements from flooding when a bunch of toilets are flushed. The sewer system’s a pretty “dumb” system and can’t really do that. There’s no way to tell whether water is coming from your sink, your toilet, or the shower drain. Computer networks on the other hand can distinguish certain traffic, and are much more capable of dealing with worst-case-scenarios, which is one of the reasons why it’s very hard to draw analogies to a lot of physical real-world systems.

  22. Does Bell Oversell, and if so, where?
    I think many of us have long understood that there can’t be congestion on the last mile. And our third party ISP manages it’s own congestion issues with peering arrangements and by buying backhaul capacity (the AHSSPI connections).

    Here’s where it gets confusing. The AHSSPI connections *separate* the third party ISP traffic from Bell’s own traffic.

    The only way I can see congestion still occuring between Bell’s traffic and the third party ISP traffic is in the AHSSPI layer – and this is only possible if Bell is overselling the AHSSPI connections. This might mean that even though the third party ISPs have, in aggregate, paid for capacity “up to” some level, Bell doesn’t have the aggregate capacity they actually sold to ISPs.

    If this is the case, I can see why Bell would skirt around the question – AHSSPI is a tariffed service, and if Bell admitted to not fulfilling its tariff obligations, the third party ISPs would have a real case before the CRTC.

    So – does Bell oversell AHSSPI tariffed connections? And are they actually allowed to do this?


  23. @Matt: “Remember, with the internet, you can (metaphorically speaking) essentially slow down all the bathtub or sink drains to keep basements from flooding when a bunch of toilets are flushed. ”

    The effect is the same. It doesn’t matter where you plug the pipe – in the basement or right under the sink. If the water is running it will overflow anyway.

    Of course you could implement some “QoS” system with computer controlled electric valves. So when you turn on the one at the sink, the bathtub one will be turned off automatically so the total debit in the pipe doesn’t exceed its maximum capacity. Now when you flush the toilet, it will automatically turn off all the faucets in the house.

    Of course, it might be cheaper and more convenient to install a larger pipe in the first place.

    Nap.

  24. Hunter_Killers says:

    The internet can’t overflow, it’s only going to get slower or data is simply going to be discarded in favor of others.

    UBB is billing me for what I did a week ago in the middle of the night. If congestion was such a big issue for these large ISP’s they probably wouldn’t be tossing around 100mbs residental connections but they are.

    How is the lower tier users subsidizing my 100mbs connection I’m paying another $100-$125 than they are for it? The entire premise of this is backwards, at least Shaw is acting like they care rather than demonizing everyone using the service they payed for

  25. @Napalm
    When an ISP sells me a 10Mbps connection I fully expect to get that capacity. If the ISP hasn’t purchased enough upstream capacity to handle the capacity the sold it’s obviously their own fault. It’s no different than an airline overbooking. I realize that they advertised the speed as “up to 10Mbps” but there needs to be a minimum as well. A 25GB cap implies that I’m only getting 80Kbps if I want to use my internet 24/7.

  26. As usual, Bibic is a Bell mouthpiece and liar trying to defend gouging Canada as we go 3rd world into the internet age!
    Bibic only continues to dig his own grave as soon as he opens his mouth! It’s pretty hard defend the indefensible.

  27. @Matt
    Agreed, iff you are talking IPv6 (at least to my understanding of the current technologies). How many carriers/ISPs have implemented this on the backhaul? Anything else can involve something else that raises the hairs on a lot of people’s necks, Deep Packet Inspection. IPv4 doesn’t really have QoS capabilities in the protocol.

    Cellular 911 – today it isn’t as big a problem in most of Canada with so many phones being 3G capable… It is a bit more of an issue where the phone or tower isn’t, as it needs to be in range of, and visible to, a compatible tower (for instance, to my knowledge Belus does not support GSM phones… if you are not in range of a GSM equipped tower, generally Rogers, then you are SOL… the same applies to frequencies). For instance, I don’t get a 3G signal on my phone at home, when I can get a cellular signal at all; the phone downgrades to GSM.

    I in fact believe the sewer analogy is applicable; in both cases the impact is a back-up of the system. The throttling of which you speak occurs outside of the customer premises, at the customer’s end they keep trying to shove packets down the pipe but can’t because the system can’t accept it. Where the system is not capable of IPv6 (and as I indicated above I understand most are not) and is not doing DPI, either because the ISP/carrier hasn’t implemented or is prevented by regulation from doing so, the important packets get blocked as well.

  28. Bandwidth vs Usage
    I can make a network go over capacity by sending a larger data file as quickly as my computer will let me.
    I can’t cause a network to go over capacity by sending the same file, but over a month time frame.

    If Bell’s networks are congested, they should be charging for the former, not for the later. Or upgrading their networks to deal with the change in traffic, and sell smaller bandwidth plans until they do so. And yes, I realise people here have already said it.


  29. @Darren: “I realize that they advertised the speed as “up to 10Mbps” but there needs to be a minimum as well.”

    Now we’re talking. Can anyone explain how they can get away without defining any “minimum”?

    Customer: “hello, my internet speed is very slow, it takes 5 minutes to load the front http://www.google.com page”
    Rep: “Does the page load eventually?”
    Customer: “yes it does but it’s excruciatingly slow”
    Rep: “then you’re doing fine, Sir, your internet connection is working; it’s just those pesky radical extremist pirates that are creating congestion with their bittorent thing”
    Customer: “can you fix it?”
    Rep: “yes we do but we need your support to introduce UBB”
    Customer: “UBB what? what is it?”
    Rep: “we need to charge per gigabyte, that will cost pirates money and they will stop and you’ll get better speed”
    Customer: “how much will that cost me?”
    Rep: “something around $2/GB”
    Customer: “what would that mean, let’s say if I watch a movie online?”
    Rep: “around $10 per movie”
    Customer: “woot $10 per movie? @##%% off! I can buy 2 for 10 from Walmart”
    Rep: “That’s expensive Sir; may I interest you in this special offer on the Sattelite dish, you could watch as many movies as you can just for $24 per month (for the first year; special condition apply; with 1 year commitment etc)”
    Customer: “errr… urr… I just want to get on google, can you fix it?”
    Rep: I take it you will support us with UBB?
    Customer: err… no…. I don’t think it’s a good idea…. let’s just fix my High Speed Unlimited plan…
    Rep: We can’t Sir, those radical pirates are using Bittorent, we need your support…
    Customer: errr… No… eh…
    Rep:
    Customer: hey… it nearly stopped loading… it will take days to load at this speed…
    Rep: How can I help you today, Sir?
    Customer: I WANT TO GOOGLE!
    Rep: Sure Sir, will you support our UBB initiative…
    Customer: err… whatever….
    Rep: Thank you Sir!

    nap.

  30. Stanley Pain says:

    Can we please stop with the ridiculous analogies of bandwidth v utilities, cars, etc? Things such as gas, water, electricity are FINITE, where as bandwidth isn’t going to run out at the bandwidth refinery. Technically it’s an infinite resource. Capacity ISN’T a problem for the likes of Bell and Telus when you realize that their HD streams move over the same part of the network, stream 24/7 and require HUGE amounts of bandwidth and they are NO WHERE NEAR at capacity.

    The best analogy I can come up with for this is to take a look a high ways. You’ve already built the road, and now you have cars driving on it. You hit traffic (congestion) and instead of upgrading your infrastructure you dictate to drivers that because of some HEAVY users you can only drive 1000km/month. Oh crap, is this a thinly veiled car analogy? đŸ˜›

    Bandwidth is dirt cheap, more so if you’re a big ISP. Fractions of a penny/GB. I can get a 100Mbps UNMETERED port for less than it would cost me to purchase 25Mbps + $50 in overage charges from the likes of Robellus, and transfer well over a terabyte of information/month and the person selling me said port would still make money.

    To recap, capacity isn’t the issue, bandwidth isn’t the issue, heavy users CLOGGY UP THE INTERTUBES aren’t the issue. It’s GREED, plain and simple. UBB @ $0.05/GB + a nominal monthly admin fee would be FAIR, $1-$2/GB + outrageous monthly costs and caps is not FAIR, it’s highway robbery.

  31. @Hunter-Killers
    Agreed, however the issue here, on the Bell system in particular, is that the same network is being used to transport more than just internet. It also transports POTS and video (“real-time” data). These latter are not as tolerant of delay jitter. The human ear is remarkably good at noticing the jitter in digital audio playback, in fact.

    Add to this that people already whine about not getting their 6 mbps download speed (or whatever they’ve signed up for)…

    The phone companies seem to be provisioning their networks in the same way that they’ve provisioned POTS for years. The phone system is designed with the idea that normally the system won’t be more than X% busy no more than Y% of the time… so the links were designed to handle that many connections. For instance, a T1 has only 24 links. A T1 connected to a switch would service perhaps 72 customers, as 99% of the time there would be no more than 24 of then using their phone. If someone tried to call a 25th, or the 25th tried to call out, there would be a busy signal. (Take this as an example rather than a concise explanation… newer tech is a bit different). Using this model, the expectation is that no more than a percentage of the users would be online at any one time; the concept of an “always on” and potentially “always active” connection in the residence is a foreign concept. These are the connections that are problematic, as they don’t fit the provisioning model.

    Don’t forget that the rate you pay is based on a couple of things. The first is the quantity (cap), the second is the capacity (bandwidth/speed). There are large parts of the country where 100 mbps connections are not available via Bell… the profits made in those regions with help support the introduction of such services into other regions.

    There was an interesting statistic brought up last night on TVO’s The Agenda. The representative from TekSavvy indicated that Bell spent $9 Billion in the past three years on infrastructure expansion ($3B per year); he asked Bibic what the breakdown was. Bibic indicated it was about $1B on cellular and $8B on the terrestrial network (which supplies phone, TV and internet). Just so that we can put into perspective the costs involved in the network expansion…

  32. @Darren
    A residential ISP will see you an oversubscribed 10Mbps connection. They way they have traditionally priced residential internet to lower the costs. A guaranteed/dedicated 10Mbps would cost much more and Bell would make sure you always got that speed (and upgrade their pipe more ofter to accomodate). The problem is that on over subscribed during peak hours the pipe will ALWAYS be slower.

    I don’t have an issue with my connection being slighly slower and paying no cap. A minimum sounds like a nice idea. That way Bell etc really just needs to allocate the minimum speed at worst (they might already have some internal rate).

    I am sick and tired of Bell trying to rationalize a scheme that has no direct link to bandwidth usage during peak time. Bibic talked about the peak pipe cost. The thing is…once that upgrade is complete there isn’t a real large cost to it (monitoring the network, and replacing broken stuff etc is already included in their costing figures).

    For example..Say you buy a 100 Mbps switch for your home to share files or something. You also need to buy the network cable to hook it up . So something like $130 bucks you spent. If you don’t use it then you are still out your money. If you use it then really you are only paying for electricity (if it breaks you will need to buy another one and you should budget something in addition each year for this scenario) and start adding more people then the speed available per person will drop. You buy some a 1Gbps switch now to upgrade. You may need new networking cable or if you are smart you bought cabling already capable of carrying 1Gbps. Now everyone is happy and you spend an extra $100 bucks when things got bad.

    You didn’t bill your roommates $1 per GB from the start to fund the thing because they would laugh and say but your cost was only…. (in fact your roommates might try to hurt you if you try something like that)

  33. “”Capacity” costs too. They have to buy bandwidth from their upstream supplier. As with any telecom business, that bandwidth is oversold by a certain factor (you start with the assumption that not all users will be connected and downloading at full speed all the time).

    When nearly everybody starts to run unattended 24/24h bittorents, you have to reduce the “overselling” factor and buy more upstream bandwidth. This costs money. Eventually you reach the situation where you’re not “overselling” anymore but then the monthly fee wouldn’t be just $39 either. So in the end it’s about usage.”

    So you’re saying that amortization links cost to usage as well as capacity, because increased consistent usage leads to increased cost to the provider. That makes some sense, and it might be enough to shift my opinion somewhat. But in this case I would vie for a peak- and off-peak pricing regime that allows them to bring package deals closer to the overselling rate, over a usage-based billing model, because I want to encourage the end user to feel free to consume more internet, whether it be P2P services, Netflix, cloud storage/backup or what-have-you.


  34. @Mark: “So you’re saying that amortization links cost to usage as well as capacity, because increased consistent usage leads to increased cost to the provider. That makes some sense, and it might be enough to shift my opinion somewhat.”

    Yes, they’re very similar to Hydro in their cost structure:

    1. Cost of (installing and maintaining) their own infrastructure

    2. Cost of uplink bandwidth to Internet backbone

    When usage increases, you either buy more of (2) or service degrades.

    Like in Hydro draws (and pays) more power from the power plants – or not and lets brownouts happen.

  35. Network Supervisor, MCSA
    To Tony, 3rd comment in this section, “Logistical nightmare?”

    Obviously you don’t understand some of the innovations 3rd party ISP have had to come up with to compete against the Bell monopoly. Take “Quadro Communications” in the Perth County region of Ontario for example. They have had to compete several 3rd party ISPs, Bell and Rogers.
    So they invested in their own infrastructure and over time went from dial-up capacity in the entire service area to copper DSL to total Fibre internet.
    It took some time, but it happened in under 2 decades worth of investing and selling internet access. What is Bell’s excuse for capacity and infrastructure demands? It’s called being innovative!

  36. @Shawn
    Tier 2 wholesale rates are between $5-10 per Mbps per month for a one year contract depending on location.
    Let’s say it costs Bell $5 per MBps that’s $50 per month on a 10Mbps line.

    My monthly bill is $50 and currently capped at 100GB.
    I require a dedicated 300Kbps connection (24/7) to service that traffic. That 10Mbps line supports about 33 users at 300Kbps (24/7).

    Let’s do some more math, 33*$50=$1650.
    So the actual internet cost is only 3% of my bill if I use all of that 100GB.

    I would think that Bell gets better than Tier 2 pricing. Am I missing something?


  37. @Mark:

    But if Micro Fibic says the truth, then their problem is in their own NOC and their facing upgrades of their equipment (like in getting some higher capacity routers). Such a pity they spent the money on Sandvine instead. It won’t help route more.

    Nap.

  38. UBB – Ultimately Bell Bullshit!
    So there you have it kids. Bell just admitted that it’s THEIR central office infrastructure and routing techniques that are causing THEIR congestion problems NOT the last mile end-users.

    This would be like me claiming that people who drive too fast on single-lane downtown streets cause traffic jams! (:

    Ridiculous Bell bullshit!

  39. @Mark,

    so then if Bell subscribed 1,000,000 customers and put them on a contract for $39 a month, when the uplink to the backbone itself costs $42 per customer/month, who’s fault is it? You want to blame the customer for Bell’s overselling? Perhaps Bell shouldn’t have grossly oversold their internet service (as many web hosting providers do today), and then began to cry when they had offer their customers the contracted rate.

    Bandwidth it getting cheaper and cheaper every day — I have no sympthy for companies that refuse to innovate and improve their networks and offer world-class performance when they charge world-class prices.

  40. @Mark
    Why do you keep singling out bittorrent? It is a time insensitive protocol that many ISPs are already throttling (and potentially increasing congestion in their network with dropped packets).
    BTW Bittorrent usage has been dropping.
    Some new uses are by several large games to lighten peak loads for distributing patches but that doesn’t really suck bandwidth 24/7 like you say.

    Streaming/real-time applications are the ones that can drive a need for increasing speeds (Ie. video skype).

    Your remarks seem to be aimed at demonizing bt.

  41. oops
    meant to say bittorrent is a time insensitive application protocol from the point of the downloading the data of the content can be out of order (versus streaming protocols)

  42. Finally some excellent dialog on the issue
    I enjoyed the article but its the intelligent discussion in the comment thread that I have found most valuable. I am frustrated to see Bibic interchange Bandwidth (speed) and Volume (amount of data) and equally frustrated to see conversations fixate on the cost component of UBB and not around whether its an applicable way to “share the cost of sharing the load”.
    @mark(s) @napalm @anon @anon-k I truly appreciate the _conversation_ you are having because so many comment threads are a circle jerk of praise for or against the article. Thanks for helping me clarify my arguments and concerns so I can be clearer in voicing them to others.
    Oh…. and Geist is right.
    thanks

  43. @Shawn
    That is the funny thing. Those caps they are charging are arbitrarily set market based prices. They have no basis in cost Bell.

    Bell within eastern Canada probably requiring no external peering agreements. From their scale point of view its a fixed cost.

    I would guess the costs for them would be much lower too. The thing you are missing is the extent of Bell’s short term greed at the expense of everything else

  44. analogies
    I’m so tired of Bell and others insulting everyone’s intelligence with proof by analogy. If you cannot explain why Bell should be able to oversell their wholesale ISP connections *and* charge those ISP’s for each user’s individual usage, then you have no business making comparisons to sewage or hydro. Explain to us why this is not just a double-charging cash grab in a clear and rational manner and I may listen. Otherwise this is just “You wouldn’t steal a TV???” all over again.

  45. email from TekSavvy
    An email has been sent to all TekSavvy customers. Of particular note is the last paragraph.

    “Dear TekSavvy Customer,

    Because of your support and participation in reaching out to the
    various MPs and the Minister of Industry we’re pleased to announce
    that the March 1st UBB implementation date has been suspended indefinitely.

    With over 460,000 signatures having been sent to Parliament Minister
    Clement and the Industry committee requested the CRTC review its
    recent decision in full, which it has obliged and are now doing.
    Until this review has occurred UBB over DSL has been completely
    shelved. Minister Clement also added they would overturn the review
    should it resemble in any way to the previous outcome!

    So, congratulations to all who’ve spoken up about this issue, you’ve
    made a huge difference. As a Canadian, today, I can honestly say it’s
    refreshing to know politicians do listen and will affect change when
    the public at large shows concern!

    Effective March 1st we are going to move forward and make changes to
    the Usage, but we’re going to make it a positive one!

    We are reinstating the Unlimited package but the 200GB package will be
    changed… to 300GB! UBB is about Internet Costs, and as a result of
    lower costs with our providers (Peer1, Lime Light, etc…), costs
    outside our relationship with companies like Bell, we are extending
    the savings on to you, the clients… Enjoy!”

    Bell, take note – that’s how it’s done.

  46. Canadian Consumer says:

    I just want to clear something up. People keep comparing the bandwidth to electricity/roads/water. This is not a valid comparison. Bandwidth expands. Your demand for bandwidth is greater now than it was 10 years ago. This isn’t just because your usage changed. This is because more bits and bytes are used by web pages and software in general.

    The Ad companies sell you adds, this adds are so large that if it was dialup, you would be tapping your fingers for minutes waiting for them to load. Companies relay on bandwidth technology to do business. In order to have the add talk to you or display video you need to have bandwidth.

    Moore’s Law states that everything in technology doubles in speed every 18 months. By setting a USAGE tax, this law gets destroyed. There is no incentive for us to create new technology, like transferring 3d matrices in the future (which is where everything is heading). A Kilobyte seemed like a lot in the 80’s, a Megabyte was a lot in the 90’s, a GB is a lot now. This will all change as our systems get more advanced.

    Your power usage, electricity and roads do not work this way. Cars don’t get bigger, your consumption is stable. Taxing technology by the GB like we were about to do, is one of the DUMBEST, SHORTSIGHTED moves any advanced culture could make. There would be no incentive to increases our usage, and people would avoid the technology rather than improve it.

    So let’s stop these silly comparisons. They internet is a growing thing, to put a tax on its growth, within our own borders is beyond dumb.

    Tony:

    There are lots of companies that would love to lay fiber along side of Bells, they just don’t play at the same golf clubs as Bell and the people responsible for regulation in this country. Competition is itching to jump into this market because of the pricing that Bell has created. Maybe the network HUBS should be neutral ground… and Bell, only pays to rent them from the taxpayer. Owning these techs leads to cronyism like this debacle with the CRTC. The profits made were not used properly to build up the next phase of the network, instead they were used to play Monopoly by buying up CTV and other interests. Now that the users are starting to push the limits of this dinosaur network that Bell has recouped costs for over and over, Bell wants to change your habits by regulation rather than expending capacity.

    No one should own the internet! There should only be contracts for its service. Those that can’t maintain capacity and service should not have their contracts renewed. It’s ridiculous that Bell should reciever the subsidees to created these networks and own them for life stiffling their growth.

    We need to buy them out and hire them on a contractual basis.

  47. David I. McIntosh says:

    Decision makers…
    Do you know if decision makers such as Tony Clements are reading your blogs? What can be done to ensure they do? The decisions made over the last few years show a CLEAR lack of understanding of the issues by those making the decisions, so Michael, WHAT CAN WE DO TO GET THEM TO READ ARTICLES LIKE YOURS so that they understand the issues? WHAT CAN WE DO?????

  48. Let me get this straight. Bell is saying that if they allow ISP access to ADSL-CO, the ISP can install better/faster/more fiber to the CO, which Bell sees as unfair competition, or at least a level of competition they can’t compete with?

    It appears Bell may have done a decent job with curb DSLAMs and fiber to the CO from these DSLAMs. But based on their own words, that’s not where the congestion occurs. It is after aggregation/split at the CO, on the fiber to/from the CO.

    So here is a suggestion, why doesn’t Bell make a deal with the second tier ISP’s that allows Bell to route xx% (50?) of their own “congested internet bandwidth” through the ISP ADSL-CO located fiber, in exchange for the physical space required to place the equipment? The CRTC could even make this a requirement for allowing an ADSL-CO installation. This is in addition to what the second tier ISP already pays to Bell for the “last mile” access from the CO to the customer premises.

    The problem is bandwidth, in particular peak time bandwidth congestion, not usage per se. So focus on the “problem” and creative solutions to those problems. There are ways to turn a competitor into a willing partner as well.

  49. @Anon-K
    “Bibic indicated it was about $1B on cellular and $8B on the terrestrial network (which supplies phone, TV and internet)”

    It’s funny how Bell likes to blur the line between TV and internet when they’re talking about investment, yet considers them separate services when they want to justify UBB.

    I’d think their IPTV system is going to have to be engineered assuming that normal demand and peak demand are equal. Most people I know with IPTV leave their set top boxes on 24/7 and none of them would tolerate an interruption in their TV stream due to ‘congestion’. They also have no idea that it’s consuming bandwidth all night long.

    If Bell can engineer their IPTV service to deal with peak demand 24/7, why can’t they do the same for the backhaul network they use to connect third party ISPs? They’ve gotten used to overselling the network and their’s no profit in fair pricing, plain and simple.

    If anything comes out of the whole fiasco, I hope everyone learns how badly Bell and the other ISPs were preying on the ignorance of consumers before a lot of on-demand services started being delivered via the internet. I’m sure Bell only needed to buy a tiny fraction of the capacity they were selling back then. Did we hear any objections from Bell when it was consumers getting the raw end of the deal? Nope.

    Tiering internet prices based on bandwidth was an excellent tactic for Bell 10 years ago. They got to charge us more and it cost them virtually nothing. When everyone was pulling down 100kB web pages in short bursts, increasing speeds for the last mile didn’t make any difference.

    Back then, Bell could have doubled their last mile speeds from (for example) 1Mbit/s to 2Mbit/s without upgrading anything else and the worst case scenario for consumers would have been waiting .8s instead of .4s for their 100kB (800000 bits) web page to load.

    Now that our population is becoming more tech savvy and starting to use their internet service as advertised, that unscrupulous marketing tactic is coming back to haunt Bell. The _way_ people used to use the internet obfuscated the fact that the network was severely oversold.

    Now, on-demand services like Netflix and YouTube are destroying Bell’s business model of overselling the network instead of upgrading the network and they want the CRTC to regulate the industry to protect them from terrible, short sighted, profit driven business decisions they made a decade ago. I say too bad. They made their proverbial bed, let them sleep in it.

    Bell (and the other incumbent) ISPs have been price gouging Canadians for over a decade. They’ve let our infrastructure languish and jeopardized our ability to compete in a global, digital economy. They’ve demonstrated they aren’t responsible enough to handle such a critical piece of Canada’s infrastructure and we should seriously be considering taking it away from them (and nationalizing it).

  50. This says it all:

  51. Some misconceptions here…
    @Napalm: There is no ‘upstream’ provider that Bell is buying bandwidth here. That’s the ISP’s responsibility. The traffic Bell is trying to bill them for never leaves the Bell network. The ISP pays Bell for a certain capacity link into their network (both on their end and the customer’s end), and there’s no reason at all that cost shouldn’t cover providing that capacity end-to-end, regardless of usage.

    @Anon-K: IPTV is delivered in a somewhat different way to STBs than it is via standard Internet services. It’s generally a multicast service that your STB subscribes to, so while the combination of all channels that need to be delivered to a CO is still going to be a fairly substantial amount of bandwidth, it’s not like each subscriber is creating congestion on the backhaul network – they’re all getting the same stream, which is only sent once. VOD is the exception of course.

    I think another poster said it perfectly – the fact that they are offering high-capacity links like 100mbit at all indicates that there isn’t a congestion problem on their backhaul, or they’d never be able to offer service anywhere close to that capacity.


  52. “Bibic indicated it was about $1B on cellular and $8B on the terrestrial network (which supplies phone, TV and internet). Just so that we can put into perspective the costs involved in the network expansion… ”

    $8B is about the quarterly world sales of Cisco. That could have bought quite a lot of routers, how comes we have this “congestion”?

    Nap.


  53. Telus net income up 46%:

    http://www.directioninformatique.com/DI/client/fr/DirectionInformatique/Nouvelles.asp?id=61264&cid=79

    while Bell up 25.4% (missing the target):

    http://www.directioninformatique.com/DI/client/fr/DirectionInformatique/Nouvelles.asp?id=61264&cid=79

    This is interesting: “Les produits d’exploitation du secteur services sur fil de Bell devraient profiter de la croissance des produits d’exploitation des services de tĂƒÂ©lĂƒÂ©vision et Internet dĂƒÂ©coulant de la performance au chapitre du produit mensuel moyen par utilisateur (PMU)”

    translation: since they don’t get many new customers to TV and Internet services, they intend to charge more the existing ones.

    Nap.

  54. What About Bell DSL HSA
    I do not quite understand the situation that is occurring between the Bell ATM switch and the ISP premise. Perhaps someone can refer to this page from Bell and tell me what I am missing http://www.wholesale.bell.ca/dsl.asp

    From my read, it appears that Bell offers two modes of connect. The first, DSL GAS does involve a shared PVC, so it makes sense to me that congestion could occur on that link. If this is the situation, then Bell’s claim appears to be accurate.

    What I do not understand is why the congestion cannot be solved by upgrading the ISP to Bell link to a DSL HSA link. Yes, that would be more expensive because it would be a Private Virtual Circuit instead of a shared PVC, but it should effectively partition ISP demand from Bell traffic. The downside is that the ISP would need to provision one of those links to each CO (or as another post has indicated all the way to a pedestal).

    Put simple, why is DSL HSA not employed to fix this problem instead of UBB?

    Is it possible that wholesale UBB (including usage fees) is still less expensive than a dedicated PVC to each CO for the independent ISP?

    From where I sit, it looks like we are not getting the full story from either Bell or the independent ISP’s.

  55. Michael, excellent article and excellent feedback
    Keep up the fantastic work! The CRTC and big ISPs need to be put in their place.

  56. Odynet out of London Ontario has successfully bypassed this “intermediate” step in a pilot project. The packages offered through this service are of significant difference to the myopic baseline rates offered by most ISPs. What’s interesting is seeing the capacity of connections without the intermediate bottleneck (Bell Canada) and realize just how significantly Bell has stifled the marketplace with their sneaking and scheming.

    In some lights the dirty business practices employed by Bell make the blatant incompetence displayed by BP this past summer look silly. Could you imagine what our world would look like if these people were in possession of something dangerous to the public?

  57. Ramblin' Rose says:

    Traffic Shaping and Netflix
    Two days ago I signed up for the Netflix Free Month. Last night on two separate occasions (after 4:00 pm)at the beginning of an older TV feed and again later for an HD movie there was a pause…”your internet connection has slowed, netflix is making adjustments…

    I am neither a geek nor a lawyer but is my experience last night another example of whether through Usage-Based Billing, Traffic Shaping or their 50 year affair with the CRTC that Bell are operating behind a smoke screen after intercourse with everyone who is supposed to be looking after our interests?

  58. Saving the best for the LAST of Bell, Rogers’, ….
    We’ve had Fibre-Optics since the late 80’s, we(BELL) should have been slowly replacing our 100-year old copper lines to Fibre already, and to every home in Canada. Money you say ?, well sure and look at this Bell Corporation that runs it. Fat-cat CEO’s bled out that possibility many moons ago.
    Bell needs to be at the very least split-up soon, 12 separate Bells, one or more for each province, regulated where ALL profits go right back into constant “maintenence”, and “upgrades” then fine, that way we would have had Fibre direct to home, or Fibre-to-pole (wirless to home, in more rural areas… years ago, anyway whatever the logistics.
    Remember one thing, all the Bell’s, the Rogers, the tekk-savy’s, the acanac’s,…, ALL want one thing “profits” do you honestly think for one minute greed won’t step in ? Sheer Fibre-Optics simply holds the most bandwidth physically possible right now. “Traffic Shaping” would have been a non-existent phrase.
    Some things like “PUBLIC” Utilities, aka phone, Internet, Fibre, … are just NOT meant for Corporations, foreign or otherwise.
    Our own rates that we pay is what will pay this maintenance, upgrades, and more… since we will ALL own it. -simple as that. Sort-of like PBS -on a massive scale.
    Let the advertisers, and corporates also pay for that usage, based on usage.
    Then the only congestion that would ever exist would be from the (world)Internet itself which would have to try to keep up with our newly and constantly re-vamped Canadian Network.

    We(Canadians) may not want to think about all this right now, but we will be darn-well acting on it years down the road. -be it of course we truly want the best for our decsendants(aka, our children) ?

    Cheers

  59. Bell’s Network Congestion ?
    http://www.thelocal.se/7869/20070712/

    Excerpt: “…A 75 year old woman from Karlstad in central Sweden has been thrust into the IT history books – with the world’s fastest internet connection….”
    and the above was in “2007” ?
    Bell Canada, Rogers’, …, have really dropped the Internet ball onto all Canadians. Canada is now officially a 3rd world stone-age country when it comes to the Internet. Wow, it’s almost hard to believe over a decade ago we were pretty close to an innovative Numero Uno.
    But now?, shame on Bell.


  60. @Rose: I suppose you’re on Bell. Since mid January it works like this after 7PM:

    C:>ping -n 10 http://www.google.ca

    Pinging http://www.l.google.com [74.125.226.17] with 32 bytes of data:
    Reply from 74.125.226.17: bytes=32 time=202ms TTL=55
    Reply from 74.125.226.17: bytes=32 time=13ms TTL=55
    Reply from 74.125.226.17: bytes=32 time=175ms TTL=55
    Reply from 74.125.226.17: bytes=32 time=210ms TTL=55
    Reply from 74.125.226.17: bytes=32 time=390ms TTL=55
    Reply from 74.125.226.17: bytes=32 time=242ms TTL=55
    Reply from 74.125.226.17: bytes=32 time=334ms TTL=55
    Reply from 74.125.226.17: bytes=32 time=264ms TTL=55
    Reply from 74.125.226.17: bytes=32 time=658ms TTL=55
    Reply from 74.125.226.17: bytes=32 time=917ms TTL=55

    Ping statistics for 74.125.226.17:
    Packets: Sent = 10, Received = 10, Lost = 0 (0% loss),
    Approximate round trip times in milli-seconds:
    Minimum = 13ms, Maximum = 917ms, Average = 340ms

    This with no other traffic but the ping.

    Nap.

  61. Ramblin' Rose says:

    No Napalm…supposition will make an ass…whoops supp hose…:)
    @Napalm..nope with a formerly unshaped Indie…

    Wasn’t it Bell and others affeared of competition that convinced the CRTC to let them shape the unlimited small providers?

    Regardless, armed with an Analog feed from Rogers, still on my free Netflix subscription, a streaming video box…eee, I’m enjoying a plethora of Internet Digital feeds…gosh no wonder Bell and Rogers introduced caps!

    I’m reminded of the guy years ago that said, “Take the big Corporations to small claims court because there isn’t a Judge alive who hasn’t been screwed by them too and would love to rule on your behalf just to feel good”!

    Any openings for old screwed judges at the CRTC? đŸ™‚

  62. bell_legal@hotmail.com says:

    @LTR
    HSA is basically an independent ISP co-locating their own DSLAMs in a Bell CO, then Bell will offer them DS1, DS3, OC3, or OC12 transport to their own POP. You are obviously limited in your servicing area since customers need to be fed straight from CO’s and have relatively short loops for higher speeds above 5Mbps.

    With GAS, an independent ISP don’t need to co-locate their own DSLAMs since they are using access nodes which are already aggregated to Bell’s local, then regional network and terminating at the regional BAS. An ISP will pay Bell ~$20/month for this 5Mbps profile which will get their customer aggregated through Bell’s network to that BAS. From the BAS, ISPs must pay monthly AHSSPI fees to get that customer further aggregated to their own POP where they’ll provide the actual transit to the internet.

    Teksavvy alone pays Bell well over a million dollars per month for their ~60,000 GAS customers.

    Now with the case of congestion, I can assure you that the only reason there is congestion is because of Bell’s network management, or lack thereof.

    Over-subscribed ASAM 7300 access nodes with OC3 ATM links. They haven’t upgraded those access nodes in ten plus years.. and instead, they started to build their new IP network with Ethernet aggregated access nodes and won’t even allow GAS customers on that network.

    I have a lot of proof and I am seriously thinking about taking some sort of legal action. My independent ISP has lost countless customers due to this congestion and they have absolutely no recourse with Bell whatsoever, if a customer complains and wants a credit, the GAS ISP eats the costs.

    bell_legal@hotmail.com